
ã¯ããã«ïŒã¡ã¢ãª8GBã¯ããŒã«ã«LLMã®ãéçããªã®ãïŒ
è¿å¹ŽãChatGPTãClaudeãªã©ã®ã¯ã©ãŠãåLLMïŒå€§èŠæš¡èšèªã¢ãã«ïŒãæ¥éã«æ®åããæ¥ã ã®éçºæ¥åã«æ¬ ãããªããã®ãšãªã£ãŠããŸããããããéçºè ãæè¡ãªãŒããŒãšããŠã·ã¹ãã ãèšèšããéãåžžã«èæ ®ãã¹ããªã®ããããŒã¿ã»ãã¥ãªãã£ãããAPIã³ã¹ããããããŠããããã¯ãŒã¯äŸåããšãã課é¡ã§ãã
ãããã®èª²é¡ã解決ããæåãªéžæè¢ãšããŠæ³šç®ãããŠããã®ããèªèº«ã®ããŒã«ã«ç°å¢ã§LLMãåãããããŒã«ã«LLMãã§ãã
ãããŒã«ã«LLMãåããã«ã¯ãé«äŸ¡ãªGPUãç©ãã ãã€ã¹ããã¯ãªPCãå¿ èŠãªã®ã§ã¯ããšæããããããããŸããã確ãã«ã最沢ãªãªãœãŒã¹ãããã°å¿«é©ã§ãããå®ã¯ãã¡ã¢ãª8GBããšãããçŸä»£ã®äžè¬çãªããžãã¹PCããšã³ããªãŒåãã®MacïŒM1ãããæèŒã¢ãã«ãªã©ïŒã§ãã£ãŠãã工倫次第ã§ååã«å®çšçãªããŒã«ã«LLMç°å¢ãæ§ç¯ããããã«ã¯èªåŸçã«åããAIãšãŒãžã§ã³ããã皌åãããããšãå¯èœã§ãã
ä»åã¯ãéããããªãœãŒã¹ãããºã«ã®ããã«çµã¿åãããã¡ã¢ãª8GBã®éçã«æãããŒã«ã«LLMã®å°å ¥ã¢ãããŒããšããããæ¯ããäž»èŠãªããŒã«ã«ã€ããŠè§£èª¬ããŸãã
ã¡ã¢ãª8GBç°å¢ã§ã®åäœå®çžŸãšãéååãã®æè¡
å®éã«ãã¡ã¢ãª8GBã®ç°å¢ã§ããŒã«ã«LLMãAIãšãŒãžã§ã³ãã¯åäœããã®ã§ãããããçµè«ããç³ãäžãããšããã§ã«å€ãã®éçºè ããã®å¶éäžã§å®çžŸãäžããŠããŸãã
äŸãã°ãM1ãããæèŒã»ã¡ã¢ãª8GBã®Macç°å¢ã«ãããŠã1GBã®è»œéãªããŒã«ã«LLMã䜿çšãã2åºèã®ãããäžã§èªåŸçã«è¡åããAIãšãŒãžã§ã³ããåäœãããå®çžŸãå ±åãããŠããŸããAPIã³ã¹ããæãã€ã€ãããŒã«ã«å ã§ãšãŒãžã§ã³ãã®ã·ãã¥ã¬ãŒã·ã§ã³ãåããç¶ãããšãã£ãéçšãå®éã«å¯èœã§ãã
ããããéåžžãå®çšã¬ãã«ãšãããã7BïŒ70åãã©ã¡ãŒã¿ïŒãã¯ã©ã¹ã®ã¢ãã«ãåäœãããã«ã¯ãæäœã§ã8GBçšåºŠã®VRAMïŒãããªã¡ã¢ãªïŒãå¿ èŠãšãããŠããŸããã¡ã€ã³ã¡ã¢ãªã8GBã®PCã§ã¯ãOSãä»ã®ã¢ããªã±ãŒã·ã§ã³ã®æ¶è²»åãèæ ®ãããšããã®ãŸãŸã§ã¯ã¡ã¢ãªäžè¶³ã«é¥ããŸãã
ããã§éµãšãªãã®ãããéååïŒQuantizationïŒããšåŒã°ããã¢ãã«å§çž®æè¡ã§ãã
**éååïŒQuantizationïŒãšã¯**
> ã¢ãã«ã®éã¿ïŒãã©ã¡ãŒã¿ïŒã®ç²ŸåºŠïŒäŸïŒ16ãããæµ®åå°æ°ç¹æ°ïŒãã4ãããã8ããããªã©ã®ããäœããããæ°ã«å€æããæè¡ã§ãã
äŸãã°ã4ãããéååïŒQ4ïŒããªã©ãé©çšãããšãã¢ãã«ã®ãã¡ã€ã«ãµã€ãºãå®è¡æã®ã¡ã¢ãªæ¶è²»éãåçã«åæžã§ããŸããæ§èœãããçšåºŠç¶æãããŸãŸãæ¯èŒçäœã¹ããã¯ãªPCã§ãå®çšã¬ãã«ã§åäœãããããšãå¯èœã«ãªããŸãã
ããŒã«ã«LLMãåäœããåºæ¬çãªä»çµã¿
ããŒã«ã«LLMã¯ãã¯ã©ãŠãAPIãšã¯ç°ãªãããã¹ãŠã®åŠçãæå ã®ã³ã³ãã¥ãŒã¿å ã§å®çµããŸãããã®åäœããã»ã¹ã¯ä»¥äžã®éãã§ãã
ã¢ãã«ã®ããŠã³ããŒã
-Hugging Faceãªã©ã®ãã©ãããã©ãŒã ãããæ°GBãæ°åGBã®ã¢ãã«ãã¡ã€ã«ãããŠã³ããŒãããŸããããŒã«ã«ç°å¢ã§ã®å®è¡
-å°çšã®ãœãããŠã§ã¢ãçšããŠãããŠã³ããŒãããã¢ãã«ãããŒãããŸããæšè«åŠç
-ãŠãŒã¶ãŒã®å ¥åïŒããã³ããïŒã«å¯ŸããããŒã«ã«ã®CPUãGPUãçšããŠæšè«ãè¡ããŸããçµæã®åºå
å€éšãµãŒããŒãä»ããããã¹ãŠã®åŠçãèªèº«ã®ã³ã³ãã¥ãŒã¿å ã§å®çµããçµæãåºåãããŸãã
ããŒã«ã«LLMãæ¯ãã3ã€ã®äž»èŠå®è¡ããŒã«
ããŒã«ã«LLMãåããããã®ç°å¢æ§ç¯ã¯ãåªããããŒã«ã®ç»å Žã«ãã£ãŠéåžžã«å®¹æã«ãªã£ãŠããŸãã代衚çãª3ã€ã®ããŒã«ã玹ä»ããŸãã
1. OllamaïŒãªã©ãïŒ
åããŠããŒã«ã«LLMã詊ãå Žåã«ãæãæšå¥šãããããŒã«ã§ããWindowsãMacãLinuxã«å¯Ÿå¿ããŠãããã€ã³ã¹ããŒã«ã容æã§ãã³ãã³ãã©ã€ã³ããã·ã³ãã«ãªæäœã§ã¢ãã«ã®ç®¡çãèµ·åãè¡ããŸãã
2. LM StudioïŒãšã«ãšã ã»ã¹ã¿ãžãªïŒ
GUIïŒã°ã©ãã£ã«ã«ãŠãŒã¶ãŒã€ã³ã¿ãŒãã§ãŒã¹ïŒããŒã¹ã§ãçŽæçã«æäœã§ããããŒã«ã§ãïŒWindowsãMac察å¿ïŒãã¢ãã«ã®æ€çŽ¢ãããŠã³ããŒãããã£ãã圢åŒã§ã®ãã¹ãå®è¡ã容æã«è¡ããŸãã
3. text-generation-webui
ãã詳现ãªèšå®ããé«åºŠãªã«ã¹ã¿ãã€ãºãè¡ããããŠãŒã¶ãŒåãã®Webã€ã³ã¿ãŒãã§ãŒã¹ã§ãããã©ã¡ãŒã¿ã®åŸ®èª¿æŽãªã©ãå°éçãªå¶åŸ¡ãè¡ãå Žåã«é©ããŠããŸãã
ã¡ã¢ãª8GBãšããå¶çŽã¯äžèŠãããšå€§ããªå£ã§ããããéååæè¡ããšãæé©åãããããŒã«ããçµã¿åãããããšã§ãååã«çªç Žå¯èœãªããŒãã«ãšãªããŸãã
次åã¯ããã®ããŒã«ã«LLMç°å¢ãããŒã¹ã«ãèªåŸçã«ã¿ã¹ã¯ãå®è¡ãããAIãšãŒãžã§ã³ãããåããå ·äœçãªææ³ãšããã®ã¡ãªããã»ãã¡ãªããã«ã€ããŠæ·±æãããŠãããŸãã