Qwenãéãå§ããã®ã§Gemma 4ãéžãã ââGPUãªãPCã§ããŒã«ã«LLMããã¡ã€ã³ãã¥ãŒãã³ã°
ããŒã«ã«LLMããã¡ã€ã³ãã¥ãŒãã³ã°ïŒFTïŒããŠã¿ããã£ãã
ã¯ã©ãŠãã®APIã§FTããããšã¯ãããã§ãããã¯ãããŒãžããµãŒãã¹ã®è©±ãæå ã®PCã§ãèªåã ãã®LLMãè²ãŠãã®ã¯å¥ã®äžçã ãšæã£ãŠããGPUãç©ãã§ãªããã
調ã¹ãŠã¿ãããColabã§ç¡æGPUåããã°ããã§ããããã ãFTãã®ãã®ãã ã©ã®ã¢ãã«ãéžã¶ã ã®æ¹ããã£ãšæ©ãã ã
ã¢ãã«ãå€ããã
2026幎5ææç¹ã§ãFTã§ãããªãŒãã³ã¢ãã«ã¯ãã£ãšããã ãããã
| ã¢ãã« | ãã©ã¡ãŒã¿ | æšè«RAM | æ¥æ¬èª | ã©ã€ã»ã³ã¹ | åºå |
|---|---|---|---|---|---|
| Qwen 3.5-27B | 27B | ~16GB | â | Apache 2.0 | Alibaba |
| Gemma 4 E4B | 4.5B | ~6GB | â | Apache 2.0 | |
| Gemma 4-26B MoE | 26BïŒå®å¹3.8BïŒ | ~8GB | â | Apache 2.0 | |
| DeepSeek V4 Flash | 284BïŒå®å¹13BïŒ | ~26GB | â | MIT | DeepSeek |
| Phi-4 | 14B | ~9GB | â³ | MIT | Microsoft |
| Llama 3.1-8B | 8B | ~5GB | â³ | Meta Community | Meta |
| Swallow 8B | 8B | ~5GB | ââ | Llama 3.1 Community | æ±å·¥å€§ |
â» æ¥æ¬èªã® ââ ã¯æ¥æ¬èªç¹åã¢ãã«ïŒLlama 3.1ããŒã¹ãæ¥æ¬èªããŒã¿ã§è¿œå åŠç¿ãããã®ïŒ
â» ã©ã€ã»ã³ã¹: Apache 2.0ã»MITã¯åçšå©çšOKã»æ¹å€èªç±ãMeta/Llama Communityã¯åçšOKã ãå©çšèŠçŽãžã®åæãå¿
èŠ
Qwenã¯æ¥æ¬èªã匷ããŠã©ã€ã»ã³ã¹ãç·©ããæåã¯ããã§æ±ºãŸãã ãšæã£ãã
ã¯ãã ã£ãã
Qwenãéãå§ããŠã
Qwen 3.5ã䜿ãããšæã£ãŠèª¿ã¹ãŠããããäžç©ãªèšäºãèŠã€ããã
ãã¡ã¯ããã§ãã¯ããŠã¿ãããå šéšæ¬åœã ã£ããæ£çŽããããŸã§ãšã¯æããªãã£ãã
2026幎3æä»¥éã«èµ·ããããš:
- æè¡ãªãŒãïŒLin JunyangïŒãäºå®äžã®è§£ä»»ããèŸä»»ããšçºè¡šãããããããŒã ã¡ã³ããŒããèªåã®ææã§ã¯ãªãã£ãããšç€ºåããŠã
- ã³ã¢ã¡ã³ããŒãMetaãByteDanceãžæµåº
- åŸä»»ã«ã¯Google DeepMindã®Geminiéçºè ãæèã忥åã·ããã«èŠãã
- Alibaba㯠AI ã€ã³ãã©ã«3幎ã§8å åæè³ããŠããååããªããšãããªã
ããšããš Maxã·ãªãŒãºïŒæäžäœã¢ãã«ïŒã¯Qwen3ãããã£ãšãŠã§ã€ãéå ¬éã»APIéå® ã§ããªãŒãã³ãªã®ã¯äžå以äžã ããããã«äººææµåºãšåæ¥åã·ãããéãªã£ãŠãã
å°ãäžèŠæš¡ã¢ãã«ïŒ27Bã35BçïŒã¯ãŸã Apache 2.0 ã§å ¬éãããŠããä»ãã䜿ããªããªããããããªãããã æäžäœã¯ãã§ã«ã¯ããŒãºãã§ãäžå以äžããã€åãéããã©ããããããªãã
ãããã¯ãã«çµã¿èŸŒãåæãªããã¡ãã£ãšæãã
Gemma 4 ã«ããçç±
æ¶å»æ³ã«è¿ãã
- Google補 â å°ãªããšãåºå ã®ä¿¡é Œæ§ã¯é«ã
- Apache 2.0 â Gemma 3ãŸã§ã¯åçšå¶çŽããã£ããã©ã4ã§å€ãã
- Colabç¡ææ ã§åã â E2BïŒ2.3BïŒãš E4BïŒ4.5BïŒã¯ T4 16GB ã«åãŸã
- Unslothå¯Ÿå¿ â FTã®ããŒã«ãã§ãŒã³ãæŽã£ãŠã
æ¥æ¬èªãQwenãšéè²ãªããJCommonsenseQAïŒæ¥æ¬èªã®åžžèæšè«ãã³ãããŒã¯ïŒã§ã¯Gemma 4ã96.51%ãQwen 3.5ã96.16%ã§ãã»ãŒäºè§ãšãã宿ž¬ããŒã¿ãããã
FTãšRAGã®éã
Gemma 4ã«æ±ºãããšããã§ãããããFTã£ãŠäœããããã®ãªã®ãããå°éç¥èãå ¥ããããšæã£ãŠããã©ãããã¯ååééãã ã£ãã
| FT | RAG | |
|---|---|---|
| ããšã | æ°äººã«ç ä¿®ããŠå©ã蟌ã | ã«ã³ããèŠãªããçãããã |
| åããŠã | å£èª¿ã»å¿å¯Ÿãã¿ãŒã³ã»æšè«ã®ç | ææ°æ å ±ã»ç€Ÿå ããã¥ã¡ã³ã |
| æŽæ° | ååŠç¿ïŒéãïŒ | ããã¥ã¡ã³ãå·®ãæ¿ãïŒè»œãïŒ |
| ééãããšã | åãå ã ãšã€ã | ãèŠã€ãããŸããã§ããããšè¿ãã |
ææ¥ã®å€©æ°ãåºã®ç©ºãç¶æ³ã¿ããã« æ¥ã å€ããããŒã¿ ãå ¥ããããªããFTãããªããŠRAGãFTã¯ãã©ãçããããã®ã¹ã¿ã€ã«ãå€ããã®ã«åããŠãã
çŸå®çã«ã¯äž¡æ¹çµã¿åãããã®ãäž»æµãç¥èã¯RAGã§æž¡ããŠãå¿å¯Ÿã¹ã¿ã€ã«ã¯FTã§èŠããããã
LoRAã£ãŠäœ
FTã®æå€§ã®ããŒãã«ã¯GPUã¡ã¢ãªãã¢ãã«å šäœãåŠç¿ããããšãããVRAMãè¶³ããªãã
ããã§äœ¿ãã®ã LoRAïŒããŒã©ãLow-Rank AdaptationïŒãã¢ãã«å šäœãåŠç¿ããããããªããŠãå°ããªãä»ç®ãã貌ã£ãŠãä»ç®ã®éšåã ãåŠç¿ãããå ã®ã¢ãã«ã«ã¯è§Šããªãã
Gemma 4 E4B ã ãšãLoRAã§åŠç¿ããã®ã¯å šãã©ã¡ãŒã¿ã®ãã¡ ãã£ã0.23%ã500ããŒãžã®æ¬ã«ä»ç®ã1æè²Œããããã®æèŠãã ããT4 16GBã§ãåãã
Colabã§ãã£ãŠã¿ã
çå±ã¯ããã£ããæãåãããå šäœã§15ã20åã
æºå
- Google Colab ãéã
-
ãã¡ã€ã« â ããŒãããã¯ãéã â GitHub ã¿ãã§
unslothai/notebooksãæ€çŽ¢ -
nb/Gemma4_(E4B)-Text.ipynbãéžã¶ïŒAMD-ã§å§ãŸãã®ã¯AMD GPUçšãªã®ã§æ³šæïŒ - ãã©ã€ãã«ã³ã㌠ããŠãã
- ã©ã³ã¿ã€ã â ã©ã³ã¿ã€ã ã®ã¿ã€ããå€æŽ â T4 GPU ãéžæ
å®è¡ã¹ããã
ããŒãããã¯ã®ã»ã«ãäžããé ã«å®è¡ããã ãã
| ã¹ããã | ããããš | æèŠæé |
|---|---|---|
| 1. Installation | Unslothãã€ã³ã¹ããŒã« | æ°å |
| 2. ã¢ãã«èªã¿èŸŒã¿ | Gemma 4 E4B ãããŠã³ããŒãïŒçŽ10.8GBïŒ | 5ã8å |
| 3. LoRAèšå® | ä»ç®ã®å Žæãšãµã€ãºã決ãã | æ°ç§ |
| 4. Data Prep | åŠç¿ããŒã¿ãæŽããïŒãµã³ãã«: è±èªäŒè©±3000ä»¶ïŒ | 1å |
| 5. Train | åŠç¿ãåãïŒ60ã¹ãããïŒ | 5ã6å |
| 6. Inference | FTããã¢ãã«ã«è³ªåããŠã¿ã | æ°ç§ |
| 7. Save | ã¢ãã«ãä¿åãã | æ°å |
Training Lossãäžãã£ãŠãã
Trainã®ã»ã«ãå®è¡ãããšãããããæ°åãåºãŠããã
| Step | Training Loss |
|---|---|
| 1 | 10.52 |
| 4 | 9.99 |
| 8 | 9.75 |
| 10 | 7.32 |
| 11 | 7.29 |
ãã®æ°åã ã¢ãã«ã®ãééã床åãã ãäžãã£ãŠããã®ã¯ãåŠç¿ããŒã¿ã«åãããŠã¢ãã«ã調æŽãããŠã蚌æ ããã£ã60ã¹ãããã§10.52â7.29ã3岿žã£ãŠãã
èŠãŠããŠå°å³ã«ãã³ã·ã§ã³äžãããèªåãè²ãŠãŠãã¢ãã«ããç®ã®åã§è³¢ããªã£ãŠããæèŠã
ãã£ãããããèããŠã¿ã
Inferenceã®ã»ã«ã§ãFTããã¢ãã«ã«è³ªåããŠã¿ãããµã³ãã«ã®è³ªåã¯ã1, 1, 2, 3, 5, 8, ã®ç¶ãã¯ïŒãã
13, 21, 34, 55, 89, ... This is the Fibonacci sequence, where each number is the sum of the two preceding ones.
ïŒ13, 21, 34, 55, 89, ... ããã¯ãã£ããããæ°åã§ãåæ°ã¯åã®2ã€ã®æ°ã®åã§ããïŒ
æ£è§£ãæ°åã®ç¶ãã ããããªããŠãããã£ããããæ°åã§ãåã®2ã€ãè¶³ãããã®ãã£ãŠèª¬æãŸã§ããŠãããã
æã¡åž°ãããšããã
FTããã¢ãã«ãããŒã«ã«PCã§åããã«ã¯ãGGUFïŒãžãŒãžãŒãŠãŒãšãïŒãšãã圢åŒã«å€æããå¿ èŠããããCPUã ãã§æšè«ã§ãã圢åŒã§ãOllamaãLM Studioã§èªã¿èŸŒããã
ããŒãããã¯ã® Save ã»ã¯ã·ã§ã³ã«GGUF倿ã®ã³ãŒãããããif False ã if True ã«æžãæããŠå®è¡ã
倿ãå§ãŸã£ãŠãããã°ã¬ã¹ããŒãé²ãã§...
ã䜿çšå¯èœãª RAM ããã¹ãŠäœ¿çšããåŸã§ãã»ãã·ã§ã³ãã¯ã©ãã·ã¥ããŸãããã
E4BïŒ4.5BïŒã®GGUF倿ã¯ãColabã®ç¡ææ ã ãšã¡ã¢ãªãè¶³ããªãã£ããåŠç¿èªäœã¯æåããŠãã®ã«ãæã¡åž°ã段éã§èœã¡ãã
次å㯠E2BïŒ2.3BïŒã§ããããLoRAã¢ããã¿ã ãä¿åã㊠Colab äžã§äœ¿ãæ¹åã«ããã
FTã¯æã£ãããè¿ãã£ã
æ£çŽããã¡ã€ã³ãã¥ãŒãã³ã°ã£ãŠãGPUäœæãç©ãã ãµãŒããŒã§äœæ¥ãåããã€ã¡ãŒãžã ã£ããColabã§ç¡æGPUåããŠãUnslothã§è»œãããŠãLoRAã§ãã©ã¡ãŒã¿ã®0.23%ã ãåŠç¿ãããããã§15ã20åãæã£ãŠãã®ãšå šç¶éã£ãã
FTãã®ãã®ãããã©ã®ã¢ãã«ãéžã¶ãã®æ¹ããã£ãšèããããšãå€ããQwenã®ã¯ããŒãºãåã¿ããã«ãæè¡ä»¥å€ã®èŠçŽ ã倿ã«å ¥ã£ãŠããã®ãé¢çœãã£ãã
æ¬¡ã¯æ¥æ¬èªããŒã¿ã§FTããŠãGemma 4ã®æ¥æ¬èªãã©ããŸã§è¯ããªãã詊ããŠã¿ããã
Discussion