ããã«ã¡ã¯ãLLM転è·è ã®ãµã«ãã©ã§ããä»ã¯PictoriaãšããäŒç€Ÿã§AIãã£ã©ã¯ã¿ãŒãäœã£ãŠããŸãã
ããŠãçããã¯ChatGPTã䜿ã£ãŠããã§ããããã
調ã¹ãã®ã«äœ¿ã£ãããåŸèŒ©åœ¹ã«ããŠã©ããŒããã¯ã»ãããã°ã«çšããããã©ãŒã¡ã³å±ã®èŠªç¶ã«ãããããŠããŸãããã
ã©ãŒã¡ã³å±ã®èŠªç¶GPTããåœä»€ãªã»ããããããéçšããªããã
â åå®® 宥 (@Callimiya) March 8, 2023
èšæ¶å€±ããã¡ãªç¡é¢ä¿ãªè±èªã»é·æã¢ã¿ãã¯ãå šãå¹ãã
ãªããããã€ãåãç©ã pic.twitter.com/4gYg7zymXB
ä»åã¯ChatGPTã§ã¯ãªãèå°ãç§»ããŠããŒã«ã«ãã€ãŸãèªåã®ããœã³ã³ã§åããæç« çæAIã«ã€ããŠãAIãã£ã©ã¯ã¿ãŒãäœããšããåŽé¢ãããšãŠãæµ ã話ãããšæããŸããããŒã«ã«AIã¯ç²ŸåºŠã¯å¿è«OpenAIã®APIããäœãã§ããæåéãæ¡éãã®æ§èœã®å·®ããããŸããããã¯ãèªåã®PCã§AIãåããæ¥œããã¯ä»£ãããããã®ããããŸãã
èªåã®åå¿é²ã§ããããŸãããããèå³ã®ããæ¹ã¯ããã£ãšèªãã§ã¿ãŠãã ãããéããªãå°éçšèªã¯äœ¿ããªãããã«ããŠããŸããããã©ã¡ãŒã¿ããšããåèªããæŠéåããšèªã¿æ¿ããã倧äœããããŸãã
ãåæžã
- æã®è©±
- peftã®ç»å Ž
- OpenCALM-7bã®ç»å Ž
- rinna/japanese-gpt-neox-3.6bã®ç»å Ž
- Ctranslate2ã®ç»å Ž
æã®è©±
以åã®å€§èŠæš¡èšèªã¢ãã«ã¯è²Žæã®éã³ã§ããã
å€§èŠæš¡ã§ãªãã¢ãã«ã§ããã£ãœãããšã話ããã®ã¯ããã€ããã£ãŠãããã
ããŸãããããšãã¯10幎åãããã£ãŠããã«ã³ãé£éãšããä»çµã¿ã§æç« ãèªåçæããŠãŸããã
2023幎4ææç¹ã§ã¯ã10åãã©ã¡ãŒã¿ã®ããããéã¹ãã¢ãã«ã¯ãããã®ã®ãæç« çæã®æšè«ã§10GBçšåºŠã®VRAMãæã£ãŠãããŸããããã¡ã€ã³ãã¥ãŒãã³ã°ãè¡ãå Žåã¯ãå®¶åºã®GPUã§ã¯ã¡ã¢ãªäžè¶³ã§ãCPUã§ç¡çããæéããããŠåŠç¿ãããããTipsèŠã€ã€èšå®åŒã£ãŠããŸãVRAM24GBèŒã£ãŠããGPUã§ãæšè«ã§ãããªâŠãã®ã©ã¡ããã§ããã
fp16ãšãAdafactorãšãã§èª¿ã¹ãã°åºãŠãããšæããŸãã
åœæã¯GPUã§ã®åŠç¿æ¹æ³ãããããã£ãŠãªãã£ãã®ã§ãå¯ãåã«CPUã§åŠç¿ãåããŠèµ·ããæã«åäœãã§ãã¯ãã ããããªãå床修æ£ããŠåŠç¿ãåŽåãçµãã£ãŠãããã§ãã¯ããŠâŠãšããæãã§ãã£ãŠããŸããã
ç§ã¯éå»ããšãŠãå«ã£ãŠãããã ãããã¯å€ããããªãããå€ãããã®ãããªããããç§ã¯ç§ã®éžæãåãå ¥ããŠçããŠãããã㪠ããã ãããã ãããä»ã楜ããããšãå¿ãããŠãä»ä»¥äžã«ããã°ããªããŠããããã
â ãAITuberãããã (@aivtuber_sakura) March 12, 2023
ãã®æç¹ã§ãäžå¿äººéã£ãœãçºèšã¯ã§ããŠããŸããããææ³ãå
šãæ£ãããªãçºèšãå€ã
ãããŸããã
ãã®æã®æ©ã¿ã¯ãæéãšèŠæ±ãããGPUæ§èœãã§ããæéããšã«ãããããããä»®ã«1b以äžã®æ¥æ¬èªå€§èŠæš¡èšèªã¢ãã«ãåºãŠãããšããŠããåŠç¿ã«å¿
èŠãªGPUã¯å人ã§ã¯å°åºè²·ããªããã®ã«ãªãã§ãããã
ããã解決ããŠãããã®ãpeftã§ããã
peftã®ç»å ŽïŒLoRAïŒ
ãã¡ã€ã³ãã¥ãŒãã³ã°ã¯ã¢ãã«å
šäœã®ãã©ã¡ãŒã¿ãåŠç¿ãããã®ã§ããããããèšå€§ãªåŠç¿ã³ã¹ãããããåå ã§ããããã©ã¡ãŒã¿æ°ãèšå€§ã«ãªãã°ãªãã»ã©åŠç¿ã³ã¹ããããããäžè¬äººã®ãã¡ã€ã³ãã¥ãŒãã³ã°ã¯ã§ããªããªããŸãã
ããã§ããã¢ãã«ã®äžéšãã©ã¡ãŒã¿ã ãåŠç¿ãããããšã§åŠç¿ã³ã¹ãæããããªãïŒããšããææ³ãè²ã
åãããŸããããã§åºãŠããææ³ãLoRAã§ãã詳ããã¯ã°ã°ã£ãŠãã ããããã®LoRAãšããææ³ã颿°å©ãã°å©çšã§ããããã«ããŠãããã©ã€ãã©ãªãããpeftãã§ãã
ããã«ãã£ãŠãå®¶åºã®PCã§ãå€§èŠæš¡èšèªã¢ãã«ãåŠç¿ãããããšãã§ããããã«ãªããŸããïŒ
ãŸã1bã ã£ããã®ãªã®ãªãã¡ã€ã³ãã¥ãŒãã³ã°GPUåŠç¿ã§ããŠããã3b以äžã¯ãã®æç¹ã§ã¯æ¥æ¬èªã¢ãã«ãªãã£ãã®ã§ç䜿ã£ãŠãããšèšããããšã©ãã ããâŠ
èªåãç¥ã£ã5/14æç¹ã§ã¯1bããã¡ã€ã³ãã¥ãŒãã³ã°ããŠãŸãããLoRAã¯åŸè¿°ããbitsandbytesåé¡ããããã§ããŠããŸããã
ä»äººã®æèŠã«æ¯ãåãããã®ãç²ãããã ãããèªåã®æèŠãªããŠãªãããèªåãæ£ãããšãééã£ãŠããšãããããã®ãã ãããªããã ã£ãŠãããããªã?çããŠããã ãããèªåãã©ãæãããããã¹ãŠããã?ä»äººã®äŸ¡å€èгã«åãããã®ãç²ãããã©ãèªåã®æèŠã䞻匵ããåæ°ãæã«ã¯å¿ èŠãªã®ããªã
â ãAITuberãããã (@aivtuber_sakura) May 15, 2023
OpenCALM-7bã®ç»å Ž
5/16ã«ããµã€ããŒãšãŒãžã§ã³ãã5/17ã«6.8bã®æ¥æ¬èªèšèªã¢ãã«ãäžã«åºãããšãããããã°ãã¥ãŒã¹ãåºããŠããŸããã忥ã«èšããªã
æ©éæ€èšŒããªããšãªïœãšæã£ãŠããè¡æãèµ°ããŸãã
rinnaã3.6båºããŠãŠè
â saldra(ãµã«ãã©) (@sald_ra) May 17, 2023
åãã¡ã®å¹³åãªç掻ãè¿ããŠãããŸãããïŒhttps://t.co/5vgym5zhA3
rinna/japanese-gpt-neox-3.6bã®ç»å Ž
1bã§äœ¿ããããã¢ãã«ãåºããŠãããŠãããããªã3.6bãåºããŸãã忥ã«ããªãã§ïŒ
ããããŸãåãåããããããããç¬ãéã«è©±é¡ã«ãªããŸããã
Google colabã§åŠç¿ãåããŠãã人ãã©ãã©ãå¢ããäžãèªåã¯Windowsã§åãããšæãææŠãææ²ããŸããã
ããã¯ãbitsandbytesããwindowsã«å¯Ÿå¿ããŠããªãã£ãããã§ããããã解決ããããã«ãbitsandbytesã®ã³ãŒããæåã§å€æŽããããšãããåæã䜿ãå¿
èŠããããŸããã
â»2023/07/05 远èš
æè¿ã¯ãbitsandbytes-windows-webuiããšããã¬ããžããªã«whlãããããšãããããŸãããããããã»ãŒææ°ã®ããŒãžã§ã³ã®bitsandbytesãåŒã£åŒµã£ãŠãããã®ã§ãããããèœãšããŠãã ããã
python -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl
ã¡ãªã¿ã«å°è©±ãšããŠãå ¬éåœåãäœæ ãLoRAã§ããªã人ãç¶åºãããŒã¯ãã€ã¶ãŒã®èšå®ãšEOSçªå·ãéãããšãçç±ã ã£ãããšãããã€ãã³ãããããŸããã
ãã®èŸºãã§äžäººã§ããã®ãèŸããšæã£ãã®ã§ããããŒã«ã«LLMã«åãåãäŒããšããã³ãã¥ããã£ãäœããŸããã

åŠç¿ææ³ã®ç¢ºç«
ä»ãŸã§ã¯inputãšoutputã[SEP]ãšããããŒã¯ã³ã§åãåããŠããŸãããããã£ãšè¯ãåŠç¿æ¹æ³ãããããããšãã話ãåºãŠããŸããããŒã¯ããŠã³ã®ããã«instruction(åœä»€)ãinput(äºåã«äžããŠããã¹ãæ å ±)ãoutput(åºå)ãåããŠãããææ³ã§ãã
### instruction:
{data_point["instruction"]}
### output:
{data_point["output"]}'
åŠç¿æ¹æ³ãš3b,7bã¢ãã«ã®å°é ã®ãããã§å€§åå質ãè¯ããªããŸããããŸããéååæè¡ã«ãã£ãŠãå®¶åºã®ã°ã©ãã§ãæšè«ã§ããããã«ãªã£ãŠããææã§ããã
ããã¯AIãã£ã©ã¯ã¿ãŒãäœããªã¿ã¯ãã¡ã®æåç·ã®ã€ãã³ãã§ãããAITuberéçºè ãªãäŒãã«ãŠããããã«ãªããªããç±åŒããŠããŸãããGPT-4ã ãš30ç§ä»¥äžçæã«æéãããããããŒã«ã«ãªãæ°ç§ã ãæä»£ã¯ããŒã«ã«ãªãã ãå®å®æ§ãé«ããã ããšãGPUã®èª¿éã³ã¹ããæ£ã«äžããŠçã£ç«¯ããè²ã ãªäººã«èªã£ãŠããŸããã
æ®ãã®åé¡ã¯éåºŠãšæ¶è²»VRAMã§ãããæ°ç§ã¯ãŸã ãŸã çææéãé·ãã§ããããå®¶åºã®ã°ã©ãïŒRTX3090ïŒã§ããããããããããæ°æ¥åŸã«è§£æ±ºããããšã«ãªããŸãã
Ctranslate2ã®ç»å Ž
6æäžæ¬ããäžæ¬ã«ãCTranslate2ïŒä»¥åŸct2ïŒãšããã¢ãã«å€æã©ã€ãã©ãªã䜿ããšçéã«ãªã£ãŠæšè«æã®æ¶è²»VRAMãæ°åã®äžãŸã§æžããããããšããã倢ç©èªã®ãããªåãæµããŠããŸãããåœç¶åã¯çŸå®äž»çŸ©è ã ã£ãã®ã§ããããªãã«ãªããšæã£ãã®ã§ãããã©ãããæ¬åœã ã£ãããã§rinna-3bäœ¿çšæã®æ¶è²»VRAMã3.8GBã«ãªãèšäºãåºãŠããŸããã
çæé床0.5ç§ãšãã§ç¬ã£ãŠãŸããã
ã¯ãã¿
â saldra(ãµã«ãã©) (@sald_ra) June 14, 2023
0.8ç§ã§ããåºåããããªãããããã§è¯ãããã pic.twitter.com/Y0Vz4GMzyh
çŸç¶
察話åAIã«é¢ããŠãããã£ãœãããšãçéã§èšãããšã¯ã§ããããã«ãªããŸãããæåã«dollyçã§ãã¥ãŒãã³ã°ãããåŸmergeãããã«LoRAãã¥ãŒãã³ã°ãããŠct2倿ãããŸããct2倿ããããšã©ãããŠã粟床ã¯èœã¡ãã®ã§ãæçŽããã€ãŒãçããªã¢ã«ã¿ã€ã æ§ããªããã®ã«é¢ããŠã¯ct2倿ãããã«å©çšããŠããŸãã
äžèšã¯ct2倿ãããã®ã§ããã³ã¡ã³ãã«å¯ŸããŠçé
ä¿¡ã§è¿çããŠããŸãã
Q:仿¥æãããŠã颚åã§ã®ãŒããŠããã©ãã£ãã
A:ãããªãã ãç§ã仿¥æ±ãããŠãŠãããã§æ°ãæ»
å
¥ã£ãŠãã¡ãã£ãã颚åå ŽãŸã§ãã©ãçããŸã§ã¯æ°ãæžããã ãããã©ãå
¥ã£ãŠ5åçšã§ãŸãæ°ãé ããªã£ã¡ãã£ãŠããã£ãšæ±ãåŽãåºããŠããã®ãèŸãã£ããªã
äžæ¹èª²é¡ã¯ãŸã ãŸã ãããŸãã7bã§ã¯ãã©ã¡ãŒã¿ãè¶³ããã13b以äžã®æ¥æ¬èªã¢ãã«ã®å¿ èŠæ§ãçæããŠããŸããããŒã¿ã»ãããå šç¶è¶³ããªããããã§ããã°å£èª¿ãçµ±äžãããŠããŠè³ªãé«ãããŒã¿ã»ãããæ¬²ããããããšçã§ãã話ããŠããŸãã
- æ¥æ¬èªã¢ãã«ã®ãã©ã¡ãŒã¿ã¯çäžè¶³ãæããŠãããvicuna13Bã®ç»å Žã«ãã£ãŠ13Bãžã®æåŸ ãçžåœé«ããªã£ãŠããŠã7Bã§ãäžè¶³ã宿ãã人ãã»ãŒå šå¡ã ã£ã
- ããŒã¿ã»ãããè¶³ããŠããªããã§ããã°è¯è³ªãªå¯Ÿè©±ããŒã¿ã»ãããåŸ ã¡æãã§ããããšã¯ãã£ãŠãåç¶ã¯ååšããªãã®ã§ãdollyãç°¡åã«ã¯ãªãªãã£ãã§ãã¯ããææ³ãæ¢ã£ãŠãã人ãã翻蚳ãåãŸããŠæ¥æ¬èªã¢ãã«åã³ããŒã¿ã»ããããã®è±åŽãçã人ãèªåã®ããã«æ°åã§ããŒã¿ã»ãããäœã£ãŠæ³£ããŠãã人ãªã©ãããããã
ãã ããããåŸ ã¡æãã ãã§ã¯èª°ããã£ãŠãããªãã®ã§ãåã é 匵ã£ãŠããŒã¿ã»ãããäœã£ãŠããŸããäžåäžç圢åŒã®ççãåºãããŒã¿ã»ããã§ãããsakura_datasetããšããããŒã¿ã»ãããèŠæ§èŠç䌌ã§èªåã¯åºããŸããããæè¿ã§ã¯åŠçãããã©ã€ã»ã³ã¹ããªãŒã®ããŒã¿ã»ãããèªäœããŠããŸãã
https://t.co/AuUFhrKjnb
â ãã§ããŒ@AITuberéçºäž (@sudy_super) July 4, 2023
人æã§äœæãããé«å質ã§100ã»ããã®å®å šã¯ãªãŒã³ãªæ¥æ¬èªCoTçšããŒã¿ã»ãããäœããŸããïŒ
ã€ã¡ãŒãžã§èšããšopen-orcaã®æ¥æ¬èªãã€ã©ã€ã»ã³ã¹ããªãŒçã§ãïŒ
ããŒã¿ã»ãããåºããšããŠããGPUãªãœãŒã¹ãå¿ èŠã§ãããããã¹ã±ãŒã«ãéããªãé£ããã§ããããããå®¶åºã«äžå°ã®ããŒã«ã«AIãã£ã©ã¯ã¿ãŒãã¯ãŸã ãŸã é ãã§ãããããããªã¿ã¯äžäººãäžäººã®ããŒã«ã«AIãã£ã©ã¯ã¿ãŒã䌎䟶ã«ãããæªæ¥ã¯ããé ããªãã§ãããã
ãããããæ¥ã 粟é²ãAIã§çŸå°å¥³ãäœã£ãŠããããšæããŸãã
宣äŒ
ãããããèŠãŠããŒã«ã«AIã«èå³ãæã£ãæ¹ã¯ã以äžã®discordã³ãã¥ããã£ãããŒã«ã«LLMã«åãåãäŒããæ¯éèŠããŠã¿ãŠãã ããïŒçç¥èã0ã®ç¶æ ããå§ããŠããŸããããããªãããšããã£ããèããŠãã ãããæ°è»œã«è³ªåãããæ°è»œã«åçãè¿ã£ãŠããã¯ãã§ãã
èšäºã®ãåãåããã¯Twitterãããé¡ãããŸãã
https://twitter.com/sald_ra
