
LLMãã¡ã€ã³ãã¥ãŒãã³ã°ã® LoRA ãš ãã«ãã©ã¡ãŒã¿ ã®æ¯èŒ
以äžã®èšäºãé¢çœãã£ãã®ã§ãããããŸãšããŸããã
ã»Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2
1. LoRA ãš ãã«ãã©ã¡ãŒã¿ ã®æ¯èŒ
ãã®èšäºã§ã¯ããLoRAããšããã«ãã©ã¡ãŒã¿ãã®LLMãã¡ã€ã³ãã¥ãŒãã³ã°ãæ¯èŒãã2ã€ã®ææ³ã®é·æãšçæãæ¢ããŸãããGSM8kããViGGOããSQLãã®3ã€ã®ããŒã¿ã»ããã§ãLlama 2ãããLoRAããšããã«ãã©ã¡ãŒã¿ãã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããŠèª¿æ»ããŸããã
1-1. ViGGO
ãViGGOãã¯ãæç« ãã颿°è¡šçŸãæœåºããã¿ã¹ã¯ãåŠç¿ããããã®ããŒã¿ã»ããã§ãããã®ã¿ã¹ã¯ã«ã¯ãé«ã¬ãã«ãªè«çãæšè«ã¯å¿ èŠãããŸããããã衚çŸããå¥ã®è¡šçŸã«ãããã³ã°ããã ãã§ãã

è©äŸ¡ã¯ã次ã®ãšããã§ãã

ãLoRAãã¯ããã«ãã©ã¡ãŒã¿ããããããã«å£ããŸãã (95% 察 97%) ãå®çšäžã¯å€§ããŠåé¡ãªããããããŸããã
1-2. GSM8k
ãGSM8kãã¯ãå°åŠæ ¡ã®ç®æ°åé¡ã«å¯ŸããŠã®æšè«èœåãåŠç¿ããããã®ããŒã¿ã»ããã§ããä»ã®ããŒã¿ã»ãããšã¯ç°ãªããæ£è§£ã«å°éããã«ã¯å€ãã®æ¹æ³ããããŸããè©äŸ¡ã¯ãæçµåçã®ã¿ãèæ ®ããŠããŸãã

è©äŸ¡ã¯ã次ã®ãšããã§ãã

ãLoRAãã¯ããã«ãã©ã¡ãŒã¿ããšæ¯èŒããŠãæããã«ããã©ãŒãã³ã¹ãå£ã£ãŠããŸãã70Bã®å·®ã¯å°ããã§ãããããŒã¹ã©ã€ã³ããã®æ¹åãå°ããã§ããä»ã®ã¿ã¹ã¯ãšæ¯èŒãããšããã«ãã©ã¡ãŒã¿ããããããŸã§ããŸãã¯æ©èœããŠããŸããã
æ°åã®äŸã䜿ã£ããã¡ã€ã³ãã¥ãŒãã³ã°ã®ã¿ã§ã¯ãæ°åŠçãªæšè«èœåãåŠç¿ããããšã¯é£ããããšãããããŸãã
1-3. SQL-create-context
ãSQL-create-contextãã¯ãèªç¶èšèªãSQLã¯ãšãªã«å€æããã¿ã¹ã¯ãåŠç¿ããããã®ããŒã¿ã»ããã§ãã

è©äŸ¡ã¯ã次ã®ãšããã§ãã

ãViGGOãã¿ã¹ã¯ãšã®é¡äŒŒæ§ããããã®ã¿ã¹ã¯ããã¡ã€ã³ãã¥ãŒãã³ã°ã§è§£æ±ºã§ããææãªåè£ã§ããçç±ãããããŸãã
2. èæ ®ãã¹ãäºé
ãLoRAãããã³ããã«ãã©ã¡ãŒã¿ãã®LLMãã¡ã€ã³ãã¥ãŒãã³ã°ãè¡ãéã«ãèæ ®ãã¹ãäºé ããããŸãã
2-1. ã¿ã¹ã¯ã®çš®é¡ãéèŠ
ãLoRAãã¯ããã¡ã€ã³ãã¥ãŒãã³ã°ã®éã«çæ³çãªéã¿ã®äžäœè¿äŒŒå€ãšããŠæ©èœããããããããã¯ãŒã¯ã®ãé©å¿èœåããäºå®äžå¶éãããŸãã
ãGSM8kãã¿ã¹ã¯ã§ã¯ãææŠçãªæ°ããã¹ãã«ãåŠç¿ããå¿ èŠããããŸããããããã¯äœã©ã³ã¯ã®è¿äŒŒã§ã¯ããŸãæããããŸããã§ãããããã«å¯ŸãããViGGOãããSQL-create-contextããªã©ã®ããã衚çŸããå¥ã®è¡šçŸã«ãããã³ã°ããã¿ã¹ã¯ã¯ããŸãåŠç¿ã§ããŸããã
2-2. åŠç¿çã«å¯Ÿãã LoRA ã®æåºŠ
ãLoRAããè¯ãæ§èœãçºæ®ããã¿ã¹ã¯ã§ãã£ãŠããåŠç¿ãå®å®ãããããã«ãåŠç¿çãã調æŽããå¿ èŠããããŸããããLoRAãã§ã¯ãã©ã¡ãŒã¿ã®æ°ãéãããŠãããããæé©åã¯ããã«ãã©ã¡ãŒã¿ããããé£ãããªããŸãã
以äžã¯ãSQL-create-contextãã®ã°ã©ãã§ãåŠç¿ã®å®å®æ§ã«å¯ŸããåŠç¿çã®åœ±é¿ãšãæ€èšŒã»ããã«å¯Ÿããè€éãã瀺ããŠããŸãã

ãã®ã¿ã¹ã¯ã§ã¯ãåŠç¿ãå®å®ãããããã«åŠç¿çã 1e-4 ãã 3e-5 ã«äžããŸãããLossã®äžå®å®ãã¯ãLoRAã¢ãã«ã®èœåãå€§å¹ ã«äœäžããå¯èœæ§ããããŸããé©åãªåŠç¿çãéžæãããšãåæã¯ã»ãŒæé©ã«ãªããŸãã
åŠç¿çãäœããšãææ¬ã®ãããªåŠç¿æ²ç·ãçæãããŸãããåŠç¿çãé«ããšäžå®å®ã«èŠããŸããåŠç¿ã³ã¹ããç¯çŽããããšããèªæã«è² ããã«ãäžå®å®ã®å¯èœæ§ãèªèããããšã倧äºã«ãªããŸãã
2-3. ããã³ããã«ã¿ã¹ã¯èª¬æã远èš
ãLoRAãã®äž»ãªå©ç¹ã®1ã€ã¯ãã¡ã¢ãªãšãµãŒãã¹ã®å¹çæ§ ã§ããããããæé©ãªæ§æãèŠã€ããããã«è€æ°ã®ãžã§ããèµ·åããã°ãªããæ€çŽ¢ãå®è¡ããå¿ èŠãããã®ã§ããã°ãããŸãé åçã§ã¯ãããŸããã
ããã§ãããã³ããã«ã¿ã¹ã¯èª¬æã远èšããããšã§ããã®åé¡ã軜æžã§ããŸããã¿ã¹ã¯èª¬æã«ãããåçå ã®ããŒã¯ã³ã®åºçŸã質åã«ååšããããŒã¯ã³ã«åºã¥ããŠæ¡ä»¶ä»ããããå¯èœæ§ãé«ããªããæé©ååé¡ãããç°¡åã«ãªãããã广çã«ãã¡ã€ã³ãã¥ãŒãã³ã°ã§ããããã«ãªããŸãã

以äžã¯ãViGGOãã®ã°ã©ãã§ãã¿ã¹ã¯èª¬æããã³ããã®åœ±é¿ã瀺ããŠããŸãã

ä»ã®ãã€ããŒãã©ã¡ãŒã¿ãåºå®ãããšãããã³ããã«ããåŠç¿ã®å®å®æ§ãå€§å¹ ã«åäžããŸããããã ããããã³ããççž®ãšãããã¡ã€ã³ãã¥ãŒãã³ã°ã®ç®çã®1ã€ãæãªãããå¯èœæ§ãããããšã«æ³šæãå¿ èŠã§ãã
3. LoRAã®ãã€ããŒãã©ã¡ãŒã¿
å®éšã«äœ¿ãããLoRAã®ãã€ããŒãã©ã¡ãŒã¿ã¯ã次ã®ãšããã§ãã
"r": 8,
"lora_alpha": 16,
"lora dropout": 0.05,
"target_modules": ["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "up_proj", "down_proj", "embed_tokens", "lm_head"],
"modules_to_save": [],â Rank: 8
åè§£è¡åã«é«ããRankããéžæãããšããLoRAãã®ããã©ãŒãã³ã¹åäžã劚ããããŠããŸããŸããäºåãã¹ãã§ã¯ã16 ã«äžããå Žåã®ããã©ãŒãã³ã¹åäžã¯æå°éã§ããããšã瀺åãããŸããããã®çµæããã§ãã¯ãã€ã³ããã¡ã€ã«ã®ãµã€ãºã®å¢å€§ãé¿ããããã8 ã«èœã¡çããŸããã
â Alpha: 16
ãAlphaãã¯åŠç¿ããéã¿ãã¹ã±ãŒãªã³ã°ããŸãããªãªãžãã«ã® LoRA è«æãå«ãæ¢åã®æç®ã§ã¯ãAlphaã調æŽå¯èœãªãã€ããŒãã©ã¡ãŒã¿ãšããŠæ±ãã®ã§ã¯ãªããAlphaãåºå® (å€ãã®å Žå 16) ããããšãæšå¥šãããŠããŸãã
â Target modules: All dense layers
ãªãªãžãã«ã® LoRA è«æã¯ããQããšãVãã®ã¢ãã³ã·ã§ã³è¡åã®ã¿ããã¡ã€ã³ãã¥ãŒãã³ã°ããããšã«çŠç¹ãåœãŠãŠããããã®ææ³ã®æå¹æ§ã蚌æãã確ããªçµæãéæããŸãããããããã®åŸã®ç ç©¶ã«ãã additional layersãããã«ã¯å
šãŠã®ã¬ã€ã€ãŒ all layers ãã¿ãŒã²ããã«ããããšã§ããã©ãŒãã³ã¹åäžããããšãããããŸããã
LoRAãããå€ãã®ã¬ã€ã€ãŒã«é©çšããããšã§ããã«ãã©ã¡ãŒã¿ã®ãã¡ã€ã³ãã¥ãŒãã³ã°ã«è¿ã¥ããšä»®èª¬ãç«ãŠãŠããŸãããããã£ãŠããã¹ãŠã®ã¬ã€ã€ãŒã«ããã£ãŠ LoRA ãå®è£ ããããšãéžæããŠããŸãã
â Base learning rate: 1e-4
ãLoRAãã§ LLMãã¡ã€ã³ãã¥ãŒãã³ã°ããå Žåã1e-4 ã®åŠç¿çãæšæºã«ãªã£ãŠããŸããåŠç¿æå€±ã®äžå®å®æ§ãæã
çºçããŸããããåŠç¿çã 3e-5 ãªã©ã®äœãå€ã«äžããããšãããã»ã¹ã®å®å®åã«å¹æçã§ããããšã倿ããŠããŸãã