[翻蚳]LLMã§1幎ééçºããŠåŠãã ããšãLLMãããã¯ãéçºãæåã«å°ãããã®å®è·µçã¬ã€ãã
ä»ã¯å€§èŠæš¡èšèªã¢ãã«ïŒLLMïŒã䜿ã£ãéçºããšã£ãŠããšããµã€ãã£ã³ã°ãªææã§ãããã®1幎éã§ãLLMã¯å®äžçã®ã¢ããªã±ãŒã·ã§ã³ã«å¯ŸããŠãååã«è¯ãããã®ã«ãªããŸããããããŠã幎ã è¯ããªããå®ããªã£ãŠããŸãããœãŒã·ã£ã«ã¡ãã£ã¢äžã§ã®ãã¢ã®æ°ã ãšçžãŸã£ãŠã2025幎ãŸã§ã«AIãžã®æè³é¡ã¯æšå®2000åãã«ã«äžããšèšãããŠããŸããããã«ãLLMãAPIã§äœ¿ããããã«ãªã£ãŠãã身è¿ãªãã®ã«ãªããæ©æ¢°åŠç¿ãšã³ãžãã¢ãç§åŠè ã ãã§ãªãã誰ããèªåã®ãããã¯ãã«AIãçµã¿èŸŒãããšãã§ããããã«ãªããŸããã
ãšã¯ãããAIã䜿ã£ãéçºãžã®åå ¥éå£ã¯äœããªã£ããã®ã®ããã¢ã§çµãããã«å¹æçãªãããã¯ããã·ã¹ãã ãäœãããšã¯ãæ³åããŠãããããé£ããã®ãçŸç¶ã§ãã
ç§ãã¡ã¯éå»1幎éãéçºã«åãçµã¿ããã®éçšã§å€ãã®å°é£ã«çŽé¢ããŠããŸãããæ¥çå šäœã代衚ããŠèªãããšã¯ã§ããŸããããç§ãã¡ã®å€±æãé¿ããããéãã€ãã¬ãŒã·ã§ã³ãåãããããåŠãã ããšãã·ã§ã¢ããããšæããŸãããããã¯ä»¥äžã®3ã€ã®ã»ã¯ã·ã§ã³ã«åãããŠããŸãã
- æŠè¡(Tactical)ïŒããã³ãããRAGããããŒãšã³ãžãã¢ãªã³ã°ãè©äŸ¡ãã¢ãã¿ãªã³ã°ã®ããã®ããã€ãã®ãã©ã¯ãã£ã¹ãLLMã䜿ã£ãŠéçºãè¡ãå®åè ã®æ¹ã®ããã«æžãããã»ã¯ã·ã§ã³ã§ãã
- éçš(Operational)ïŒãããã¯ããéçšããéã®çµç¹çãæ¥åžžçãªèª²é¡ãšã广çãªããŒã ãäœãæ¹æ³ãæç¶çãã€ç¢ºå®ã«ãããã€ããããã«ããããã¯ãã»æè¡ãªãŒããŒã®æ¹ã«åããŠæžãããŠããŸãã
- æŠç¥(Strategic)ïŒé·æçã倧å±çãªèŠç¹ã§ããPMFåã«GPUã¯äžèŠããã¢ãã«ã§ã¯ãªãã·ã¹ãã ã«æ³šåãã¹ãããªã©ã®æèšããã€ãã¬ãŒã·ã§ã³ã®æ¹æ³ãè¿°ã¹ãŠããŸãã嵿¥è ãçµå¶è ã®æ¹ã ã念é ã«æžãããŠããŸãã
LLMã䜿ã£ãŠæåãããããã¯ããäœãããã®å®çšçãªã¬ã€ãã«ããããšèããŠãããç§ãã¡èªèº«ã®çµéšãå ã«ãæ¥çã®äºäŸã玹ä»ããŠããŸãã
ã§ã¯æ©éèŠãŠè¡ããŸãããïŒ
1 æŠè¡(Tactical)ïŒLLMã䜿ãäžã§ã®åºæ¬
ããã§ã¯ãLLMã䜿ã£ããããã¯ããéçºããŠããäžã§ã®ã³ã¢ãªèŠçŽ ãã¡ã®ãã¹ããã©ã¯ãã£ã¹ã玹ä»ããŸããå質ãšä¿¡é Œæ§ãåäžãããããã®ããã³ããã®ãã³ããåºåãè©äŸ¡ããããã®è©äŸ¡æŠç¥ãæ£ç¢ºæ§ãåäžãããããã®RAGã®ã¢ã€ãã¢ãHuman-in-the-loopãªã¯ãŒã¯ãããŒã®èšè𿹿³ãªã©ã§ããæè¡ã¯ãŸã çºå±éäžã§ããããããã®æèšã¯åºãé©çšå¯èœã§ãããå ç¢ãªLLMã¢ããªã±ãŒã·ã§ã³ã®ãªãªãŒã¹ã«åœ¹ç«ã€ãšä¿¡ããŠããŸãã
1.1 ããã³ãã
æ°ããã¢ããªã±ãŒã·ã§ã³ã®ãããã¿ã€ããäœæããéã¯ãããã³ããããå§ããããšããå§ãããŸããããã³ããã£ã³ã°ã®éèŠæ§ã¯éå°è©äŸ¡ãé倧è©äŸ¡ããããã¡ã§ããé©åãªããã³ããæè¡ãæ£ãã䜿çšããã°éåžžã«é«ã粟床ãåºããããšããæå³ã§ã¯éå°è©äŸ¡ãããŠããŸãããããã³ããããŒã¹ã®ã¢ããªã±ãŒã·ã§ã³ã§ã¯ããã³ããåšãã«å€§èŠæš¡ãªãšã³ãžãã¢ãªã³ã°ãå¿ èŠããšããæå³ã§ã¯é倧è©äŸ¡ãããŠããŸãã
1.1.1 åºæ¬çãªããã³ããæè¡ãæå€§éã«æŽ»çšãã
ããŸããŸãªã¢ãã«ãšã¿ã¹ã¯ã§ããã©ãŒãã³ã¹ãåäžãããã®ã«äžè²«ããŠåœ¹ç«ã£ãŠããããã³ããæè¡ãããã€ããããŸããn-shotããã³ãããšIn-Context LearningãCoTãé¢é£ãªãœãŒã¹ã®æäŸãªã©ã§ãã
n-shotããã³ãã(蚳蚻:Few-shotsãšããšãããåŒã°ããŸã)ã«ããIn-Context Learningã®ã¢ã€ãã¢ã¯ãLLMã«ã¿ã¹ã¯ã宿ŒããåºåãæåŸ
ã«åãããããã®äŸãæäŸããããšã§ãã
n-shotãæŽ»çšããäžã§ã®ããã€ãã®TipsïŒ
- næ°ãå°ãªããããšãã¢ãã«ã¯ãããç¹å®ã®äŸã«éå°ã«åŒãã¥ãããæ±åããèœåãæãªãããå¯èœæ§ããããŸããçµéšåãšããŠãnâ§5ãç®æããŠãã ãããæ°åãŸã§å¢ãããŠãæ§ããŸããã
- äŸã¯ãæ¬çªã®ååžã代衚ãããã®ã§ãªããã°ãªããŸãããäŸãã°æ ç»èŠçŽåšãæ§ç¯ããå Žåã¯ãå®éã«äºæ³ãããã®ãšã»ãŒåãå²åã§ãç°ãªããžã£ã³ã«ã®ãµã³ãã«ãå«ããŸãã
- åžžã«å ¥å-åºåã®ãã¢ãæäŸããå¿ èŠã¯ãããŸãããæãŸããåºåã®äŸã ãã§ååãªå ŽåããããŸãã
- LLMã«ããŒã«ã䜿çšãããäºå®ã®å Žåã¯ããããã®ããŒã«ã䜿çšããäŸãå«ããŠãã ããã(蚳蚻:Function Calling ãªã©ã§äœ¿ã Tool ã®è©±)
Chain-of-ThoughtïŒCoTïŒããã³ããã§ã¯ãæçµçãªçããè¿ãåã«ãLLMã«æèããã»ã¹ã説æããããä¿ããŸããLLMã«ã¡ã¢åž³ãæž¡ããŠããã¹ãŠãã¡ã¢ãªã«ä¿æããªããŠå€§äžå€«ãªããã«ãããããªãã®ãšèããŠãã ããã
å (蚳蚻: CoTãæåã«æå±ãããè«æã®ããš)ã®ã¢ãããŒãã¯ãåã«ã段éçã«èããŠã¿ãŸããã(Letâs think step by step)ããšãããã¬ãŒãºãæç€ºã®äžéšãšããŠè¿œå ããããšã§ããããCoTãããå ·äœçã«ããããšã圹ç«ã€ããšãä»ã§ã¯åãã£ãŠããŸãã1ã2æè¿œå ããŠå ·äœæ§ãé«ãããšããã°ãã°ãã«ã·ããŒã·ã§ã³çãå€§å¹ ã«æžå°ããŸãã
äŸãã°ãäŒè°ã®æåèµ·ãããèŠçŽããããLLMã«æ±ããå Žåãæ¬¡ã®ããã«æé ãæç€ºããããšãã§ããŸãã
- ãŸããè°äºé²ã«äž»èŠãªæ±ºå®äºé ããã©ããŒã¢ããé ç®ãããã³é¢é£ããã¹ããŒã¯ãã«ããŒãåæããŸãã
- 次ã«ãè°äºé²ã®è©³çްãæåèµ·ãããšäºå®äžäžèŽããŠããããšã確èªããŸãã
- æåŸã«ãéèŠãªãã€ã³ããç°¡æœãªèŠçŽã«ãŸãšããŸãã
ãã ãæè¿ãã®ææ³ãä¿¡ããããŠããã»ã©åŒ·åãã©ããã«ã€ããŠçåãæããããããŠããããšã«æ³šæããŠãã ããããŸããCoTã䜿çšãããŠããéã®æšè«äžã«æ£ç¢ºã«äœãèµ·ãã£ãŠãããã«ã€ããŠãã倧ããªè°è«ããããŸãããšã¯ããããã®ææ³ã¯å¯èœãªéãå®éšãã䟡å€ããããŸãã
é¢é£ãªãœãŒã¹ãæäŸããããšã¯ãã¢ãã«ã®ç¥èããŒã¹ãæ¡åŒµãããã«ã·ããŒã·ã§ã³ãæžããããŠãŒã¶ãŒã®ä¿¡é Œãé«ããããã®åŒ·åãªã¡ã«ããºã ã§ããå€ãã®å ŽåãRAGã«ãã£ãŠå®çŸãããŸãããã¢ãã«ã«çŽæ¥å©çšã§ããããã¹ãã®ã¹ãããããæäŸããããšã¯äžå¯æ¬ ãªææ³ã§ãã
é¢é£ãªãœãŒã¹ãæäŸããéãåã«ããã³ããã«å«ããã ãã§ã¯äžååã§ããã¢ãã«ã«ãããã®äœ¿çšãåªå ããããäŒãããåã³ãªãœãŒã¹ãäžååãªå Žåã«ãã®æšãäŒããããšãå¿ããªãã§ãã ããã
1.1.2 å ¥åãšåºåãæ§é åãã
æ§é åãããå ¥åãšåºåã¯ãã¢ãã«ãå ¥åãããè¯ãçè§£ããäžæµã®ã·ã¹ãã ãšç¢ºå®ã«çµ±åã§ããåºåãè¿ãã®ã«åœ¹ç«ã¡ãŸããå ¥åã«ã·ãªã¢ã©ã€ãŒãŒã·ã§ã³åœ¢åŒ(蚳蚻:MarkdownãªãJSONãªãã®æ§é åããããã©ãŒããã)ã远å ãããšãã³ã³ããã¹ãå ã®ããŒã¯ã³éã®é¢ä¿ãç¹å®ã®ããŒã¯ã³ã«é¢ãã远å ã®ã¡ã¿ããŒã¿ããŸãã¯ãªã¯ãšã¹ããã¢ãã«ã®ãã¬ãŒãã³ã°ããŒã¿ã®é¡äŒŒã®äŸã«é¢é£ä»ããããã®æããããã¢ãã«ã«æäŸããã®ã«åœ¹ç«ã¡ãŸãã
äŸãã°ãã€ã³ã¿ãŒãããäžã®SQLã®æžãæ¹ã«é¢ãã質åã®å€ãã¯ãæåã«SQLã¹ããŒããæå®ããŠããŸãããããã£ãŠãText-to-SQLã®å¹æçãªããã³ããã«ã¯ãæ§é åãããã¹ããŒãå®çŸ©ãå«ããå¿ èŠããããšäºæ³ã§ããŸãã
æ§é åãããå ¥åã¯ãã¿ã¹ã¯ãæç¢ºã«è¡šçŸãããã¬ãŒãã³ã°ããŒã¿ã®æžåŒã«é¡äŒŒããŠãããããããè¯ãåºåãåŸãããå¯èœæ§ãé«ããªããŸããæ§é åãããåºåã¯ãã·ã¹ãã ã®äžæµã®ã³ã³ããŒãã³ããžã®çµ±åãç°¡åã«ããŸããæ§é åãããåºåã«ã¯ãInstructorãšOutlinesãé©ããŠããŸããïŒLLM API SDKãã€ã³ããŒãããå Žåã¯Instructorããã»ã«ããã¹ãã¢ãã«çšã«Huggingfaceãã€ã³ããŒãããå Žåã¯Outlinesã䜿çšããŸããïŒ
æ§é åãããå
¥åã䜿çšããå ŽåãåLLMã®ã¢ãã«ã«ã¯ããããã®å¥œã¿ãããããšã«æ³šæããŠãã ãããClaudeã¯<xml>ã奜ã¿ãŸãããOpenAIã®GPTã¯MarkdownãšJSONã奜ã¿ãŸããXMLã䜿çšãããšã次ã®ããã«<response>ã¿ã°ãæå®ããããšã§ãClaudeã®å¿çãäºåã«å
¥åããããšãã§ããŸãã
messages=[
{
"role": "user",
"content": """Extract the <name>, <size>, <price>, and <color> from this product description into your <response>.
<description>The SmartHome Mini is a compact smart home assistant available in black or white for only $49.99. At just 5 inches wide, it lets you control lights, thermostats, and other connected devices via voice or appâno matter where you place it in your home. This affordable little hub brings convenient hands-free control to your smart devices.
</description>"""
},
{
"role": "assistant",
"content": "<response><name>"
}
]
1.1.3 1ã€ã®ããšãããŸãè¡ãå°ããªããã³ãããäœã
ãœãããŠã§ã¢ã«ãããäžè¬çãªã¢ã³ããã¿ãŒã³/ã³ãŒãã¹ã¡ã«ã¯ããç¥ãªããžã§ã¯ããã§ããããã¯ããã¹ãŠãè¡ãåäžã®ã¯ã©ã¹ã颿°ãæã£ãŠããç¶æ ã§ããããã¯ããã³ããã«ãåœãŠã¯ãŸããŸãã
ããã³ããã¯éåžžãã·ã³ãã«ã«å§ãŸããŸããæ°æã®æç€ºãš2ã3ã®äŸãããã°ãããã«äœ¿ããŸããããããããã©ãŒãã³ã¹ãåäžãããããå€ãã®ãšããžã±ãŒã¹ãåŠçããããšãããšãè€éããå¿ã³å¯ã£ãŠããŸãã
ããå€ãã®æç€ºã倿®µéã®æšè«ãæ°åã®n-shots...ã
æåã¯ã·ã³ãã«ã ã£ãããã³ããããä»ã§ã¯2,000ããŒã¯ã³ã®ãã©ã³ã±ã³ã·ã¥ã¿ã€ã³ã«ãªã£ãŠããŸããããã«æ²ããããšã«ãããåçŽãªå
¥åãšæ¯èŒããŠããã©ãŒãã³ã¹ãæªåããŠããã®ã§ãïŒGoDaddyã¯ãLLMã䜿ã£ãæ§ç¯ããåŠãã æèšã®äžã§ããã®èª²é¡ã第1äœã«æããŠããŸãã
ã·ã¹ãã ãã³ãŒããã·ã³ãã«ã«ä¿ã€ããã«å¥®éããã®ãšåãããã«ãããã³ãããã·ã³ãã«ã«ãã¹ãã§ããäŸãã°ãäŒè°ã®æåèµ·ããã®èŠçŽã®ããã«åäžã®å æ¬çãªããã³ãããçšæããã®ã§ã¯ãªãã以äžã®ããã«ã¹ãããã«åå²ããããšãã§ããŸãã
- äž»èŠãªæ±ºå®äºé ãã¢ã¯ã·ã§ã³é ç®ãææè ãæ§é åããã圢åŒã§æœåºãã
- æœåºããã詳现ãå ã®æåèµ·ãããšç §åããŠæŽåæ§ããã§ãã¯ãã
- æ§é åããã詳现ããç°¡æœãªèŠçŽãçæãã
ãã®çµæã1ã€ã®ããã³ãããè€æ°ã®ããã³ããã«åå²ããŸããããããããã·ã³ãã«ã§ãç®çãæç¢ºã§ãçè§£ãããããªã£ãŠããŸãããŸããåå²ããããšã§ãåããã³ãããåå¥ã«å埩ããè©äŸ¡ããããšãã§ããããã«ãªããŸããã
1.1.4 ã³ã³ããã¹ãããŒã¯ã³ãæ éã«äœã
å«ããå¿ èŠãããã³ã³ããã¹ãã®éã«ã€ããŠæ éã«èãçŽããŠã¿ãŠãã ããã
ãã±ã©ã³ãžã§ãã®ããã«ãã³ã³ããã¹ãã®åœ«å»ãäœãäžããã®ã§ã¯ãªããäœåãªçŽ æãåãèœãšããŠãã ãããRAGã¯ãæœåšçã«é¢é£ããã§ããããã¹ãŠã®æ å ±ãååŸããããã®äžè¬çãªæ¹æ³ã§ããããã®äžããå¿ èŠãªãã®ãæœåºããããã«ã¯ã©ãããããããšæããŸããïŒ
æçµçãªããã³ãããçœçŽã«æžãåºããŠèªãã ãã§ãã³ã³ããã¹ãã®æ§ç¯ãã¡ã¿ããã³ããã£ã³ã°ãRAGã®çµæãªã©ãã¢ãã«ã«éä¿¡ãããããã³ãããèŠçŽãã®ã«éåžžã«åœ¹ç«ã€ããšãããããŸãããããããããšã«ãã£ãŠåé·æ§ãèªå·±ççŸããå 容ãäžé©åãªæžåŒãªã©ãªã©ãèŠã€ãããŸãã
ãã1ã€ã®éèŠãªæé©åã¯ãã³ã³ããã¹ãã®æ§é ã§ããäžããããã³ã³ããã¹ãæ å ±ã人éã«ãçè§£ãé£ãå ŽåããšãŒãžã§ã³ãã«ãšã£ãŠãè¯ããšã¯éããŸãããã³ã³ããã¹ãã®æ§é ãæ éã«æ€èšããŠããã®éšåéã®é¢ä¿ã匷調ããæœåºãã§ããã ãç°¡åã«ããŸãã
ããå€ãã®ããã³ããã£ã³ã°ã®ã¡ã³ã¿ã«ã¢ãã«ãäºåå ¥åãã³ã³ããã¹ãã®é 眮ãªã©ãããã³ããã®åºæ¬ã«ã€ããŠã¯ãã¡ãããåç §ãã ããã
1.2 æ å ±æ€çŽ¢ / RAG
ããã³ãã以å€ã®LLMã®æ§èœãåŒãåºã广çãªæ¹æ³ã¯ãããã³ããã®äžéšãšããŠç¥èãæäŸããããšã§ãããã®ç¥èã¯In-Context Learningã«äœ¿çšãããããã¯ãretrieval-augmented generationïŒRAGïŒãšåŒã°ããŠããŸããRAGã¯åºåã®æ¹åã«å¹æçã§ããããã¡ã€ã³ãã¥ãŒãã³ã°ãšæ¯èŒããŠãã¯ããã«å°ãªãåŽåãšã³ã¹ãã§æ±ãããšãã§ããŸãã
1.2.1 RAGã®ç²ŸåºŠã¯æ€çŽ¢ãããããã¥ã¡ã³ãã®ãé¢é£æ§ãå¯åºŠã詳现ãã«ãã£ãŠæ±ºãŸã
RAGã®åºåã®å質ã¯ãæ€çŽ¢ãããããã¥ã¡ã³ãã®å質ã«äŸåããŠãããããã€ãã®èŠå ã«æ²¿ã£ãŠèããããšãã§ããŸãã
æãæçœãªææšã¯ãé¢é£æ§ã§ããããã¯éåžžãå¹³åéé äœïŒMRRïŒãæ£èŠåãããå²åŒçޝç©å©åŸïŒNDCGïŒãªã©ã®ã©ã³ãã³ã°ææšã§å®éåãããŸããMRRã¯ãã·ã¹ãã ãã©ã³ã¯ä»ãããããªã¹ãã§æåã®é¢é£çµæãã©ãã ãããŸãé
眮ããããè©äŸ¡ããNDCGã¯ãã¹ãŠã®çµæãšãã®äœçœ®ã®é¢é£æ§ãèæ
®ããŸãã
ãããã¯ãã·ã¹ãã ãé¢é£æ§ã®é«ãããã¥ã¡ã³ããé«ãã©ã³ã¯ä»ãããé¢é£æ§ã®äœãããã¥ã¡ã³ããäœãã©ã³ã¯ä»ãããèœåãæž¬å®ããŸããäŸãã°ãæ ç»ã¬ãã¥ãŒã®èŠçŽãçæããããã«ãŠãŒã¶ãŒã®èŠçŽãæ€çŽ¢ããå Žåãç¹å®ã®æ ç»ã®ã¬ãã¥ãŒãé«ãã©ã³ã¯ä»ãããä»ã®æ ç»ã®ã¬ãã¥ãŒãé€å€ããããšèããã§ãããã
åŸæ¥ã®æšèŠã·ã¹ãã ãšåæ§ã«ãæ€çŽ¢çµæã®é äœã¯ãLLMãäžæµã®ã¿ã¹ã¯ã§ã©ã®ãããªããã©ãŒãã³ã¹ãçºæ®ãããã«å€§ããªåœ±é¿ãäžããŸãããã®åœ±é¿ã枬å®ããããã«ãRAGããŒã¹ã®ã¿ã¹ã¯ãå®è¡ããŸãããæ€çŽ¢çµæãã·ã£ããã«ããŠé çªãå€ããŠã¿ãŠãã ããããããããšãRAGã®åºåã¯ã©ã®ãããªããã©ãŒãã³ã¹ã瀺ãã§ããããïŒ
第äºã«ãæ å ±ã®å¯åºŠãèæ ®ããŸãã2ã€ã®ããã¥ã¡ã³ããçããé¢é£ããŠããå Žåãããç°¡æœã§ãç¡é¢ä¿ãªè©³çްãå°ãªãã»ããéžã¶ã¹ãã§ããæ ç»ã®äŸã«æ»ããšãæ ç»ã®æåèµ·ãããšãã¹ãŠã®ãŠãŒã¶ãŒã¬ãã¥ãŒã¯ãåºãæå³ã§é¢é£ããŠãããšèãããããããŸãããããã§ããé«ãè©äŸ¡ãããã¬ãã¥ãŒããšãã¹ããŒãã®ã¬ãã¥ãŒã¯ãæ å ±ãããå¯ã§ãããšèããããŸãã
æåŸã«ãããã¥ã¡ã³ãã§æäŸããã詳现ã®ã¬ãã«ãèæ
®ããŠãã ããã
äŸãšããŠãèªç¶èšèªããSQLã¯ãšãªãçæããRAGã·ã¹ãã ãæ§ç¯ããŠãããšæ³åããŠã¿ãŠãã ãããåçŽã«ååãå«ãããŒãã«ã¹ããŒãã®æ
å ±ãæž¡ãããšãã§ããŸãããããã«å ããŠåã®èª¬æãšããã€ãã®ä»£è¡šçãªå€ãå«ãããã©ãã§ããããïŒè¿œå ã®è©³çްæ
å ±ã¯ãLLMãããŒãã«ã®ã»ãã³ãã£ã¯ã¹ãããè¯ãçè§£ããããæ£ç¢ºãªSQLãçæããã®ã«åœ¹ç«ã€å¯èœæ§ããããŸãã
1.2.2 ããŒã¯ãŒãæ€çŽ¢ãå¿ããã«ããããããŒã¹ã©ã€ã³ãšããŠãã€ããªããæ€çŽ¢ãæ€èšãã
EmbeddingããŒã¹ã®RAGã䜿ã£ããã¢ããšãŠãããã¥ã©ãŒãªã®ã§ãæ å ±æ€çŽ¢ã®åéã«ãããäœå幎ãã®ç ç©¶ãšè§£æ±ºçãå¿ãããèŠèœãšãããããã¡ã§ãã
Embeddingã¯ééããªã匷åãªããŒã«ã§ãããäžèœãªè§£æ±ºçã§ã¯ãããŸããã
第äžã«ãé«ã¬ãã«ã®æå³çé¡äŒŒæ§ãæããããšã«åªããŠããŸãããååïŒäŸïŒIlyaïŒãé åèªïŒäŸïŒRAGïŒãIDïŒäŸïŒclaude-3-sonnetïŒãªã©ã®ãããå ·äœçãªããŒã¯ãŒãããŒã¹ã®ã¯ãšãªã§ã¯èŠæŠãããããããŸãããBM25ãªã©ã®ããŒã¯ãŒãããŒã¹ã®æ€çŽ¢ã¯ããã®ç®çã®ããã«èšèšãããŠããŸãã
ãŸãããŠãŒã¶ã¯ããŒã¯ãŒãæ€çŽ¢ã®äœéšã«æ £ããã£ãŠããŠãæ€çŽ¢å¯Ÿè±¡ã®ããã¥ã¡ã³ããè¿ãããŠããªãå Žåã«ãã©ã¹ãã¬ãŒã·ã§ã³ãæãããããããŸããã
ãã¯ãã«åã蟌ã¿ã¯ãæ€çŽ¢ãéæ³ã®ããã«è§£æ±ºããããã§ã¯ãããŸãããå®éãæã倧å€ãªãšããã¯æå³çé¡äŒŒæ§æ€çŽ¢ã§åã©ã³ã¯ä»ãããåã®ã¹ãããã«ãããŸããBM25ãå šææ€çŽ¢ãæ¹åããããšã¯æ¬åœã«é£ããã®ã§ããâ Aravind Srinivas, Perplexity.ai CEO
ç§ãã¡ã¯äœã¶æãåãããã®ããšã顧客ãããŒãããŒã«äŒããŠããŸãããã€ãŒããªåã蟌ã¿ã䜿çšããæè¿åæ€çŽ¢ã§ã¯ãéåžžã«ãã€ãºã®å€ãçµæãåŸãããŸããããŒã¯ãŒãããŒã¹ã®ã¢ãããŒãããå§ããã»ããããã§ããããâ Beyang Liu, Sourcegraph CTO
第äºã«ãããŒã¯ãŒãæ€çŽ¢ã§ã¯ããªãããã¥ã¡ã³ããæ€çŽ¢ãããã®ããçè§£ããã®ãããç°¡åã§ã--ç§ãã¡ã¯ã¯ãšãªã«äžèŽããããŒã¯ãŒããèŠãããšãã§ããŸããå¯Ÿç §çã«ãEmbeddingããŒã¹ã®æ€çŽ¢ã¯è§£éãã«ããã§ãããŸããLuceneãOpenSearchãªã©ã®äœå幎ã«ãããã£ãŠæé©åããã宿Šã§æ€èšŒãããã·ã¹ãã ã®ãããã§ãããŒã¯ãŒãæ€çŽ¢ã¯éåžžãèšç®å¹çãé«ããªããŸãã
ã»ãšãã©ã®å Žåããã€ããªãããæé©ã«æ©èœããŸããæãããªäžèŽã«ã¯ããŒã¯ãŒããããã³ã°ããå矩èªãäžäœèªãã¹ãã«ãã¹ããã«ãã¢ãŒããªãã£ïŒç»åãšããã¹ããªã©ïŒã«ã¯Embeddingã䜿çšããŸãã
åèãŸã§ã«Shortwaveã¯ãã¯ãšãªã®æžãæããããŒã¯ãŒããšåã蟌ã¿ã®æ€çŽ¢ãã©ã³ãã³ã°ãå«ããRAGãã€ãã©ã€ã³ã®æ§ç¯æ¹æ³ãæžããŠãããŠããã®ã§ãåç §ãã ããã
1.2.3 æ°ããç¥èã®è¿œå ã«ã¯ãã¡ã€ã³ãã¥ãŒãã³ã°ãããŸãã¯RAG
RAGãšãã¡ã€ã³ãã¥ãŒãã³ã°ã¯ã©ã¡ãããLLMã«æ°ããæ å ±ãçµã¿èŸŒã¿ãç¹å®ã®ã¿ã¹ã¯ã§ã®ããã©ãŒãã³ã¹ãåäžãããããã«äœ¿çšã§ããŸããããããã©ã¡ããåªå ãã¹ãã§ããããïŒ
æè¿ã®ç ç©¶ã§ã¯ãRAGãåªäœã«ç«ã€å¯èœæ§ã瀺åãããŠããŸããããç ç©¶ã§ã¯ãRAGãæåž«ãªããã¡ã€ã³ãã¥ãŒãã³ã°ïŒå¥åãcontinued pretrainingïŒãšæ¯èŒããäž¡è ãMMLUã®ãµãã»ãããšææ°ã®åºæ¥äºã®æ å ±ã§è©äŸ¡ããŸããã
ãã®çµæãåŠç¿äžã«ééããç¥èãšå šãæ°ããç¥èã®äž¡æ¹ã«ãããŠãRAGãäžè²«ããŠãã¡ã€ã³ãã¥ãŒãã³ã°ãäžåãããšãããããŸãããå¥ã®è«æã§ã¯ã蟲æ¥ããŒã¿ã»ããã«ãããRAGãšæåž«ãããã¡ã€ã³ãã¥ãŒãã³ã°ãæ¯èŒããŸãããåæ§ã«ãRAGã«ããããã©ãŒãã³ã¹ã®åäžã¯ãç¹ã«GPT-4ã«ãããŠããã¡ã€ã³ãã¥ãŒãã³ã°ãäžåããŸããïŒè¡š20åç §ïŒã
ããã©ãŒãã³ã¹ã®åäžä»¥å€ã«ããRAGã«ã¯ä»ã®å®çšçãªå©ç¹ããããŸãããŸããç¶ç¶çäºååŠç¿ããã¡ã€ã³ãã¥ãŒãã³ã°ãšæ¯èŒããŠãæ€çŽ¢ã€ã³ããã¯ã¹ãææ°ã®ç¶æ ã«ä¿ã€ããšã容æã§ãã³ã¹ããå®ããªããŸããæ¬¡ã«ãæ€çŽ¢ã€ã³ããã¯ã¹ã«æå®³ãªã³ã³ãã³ããåã£ãã³ã³ãã³ããå«ãåé¡ã®ããããã¥ã¡ã³ããããå Žåãç°¡åã«ãã®ããã¥ã¡ã³ããåé€ãŸãã¯ä¿®æ£ã§ããŸãã
ããã«ãRAGã®Rã¯ãããã¥ã¡ã³ãã®æ€çŽ¢æ¹æ³ããããã现ããå¶åŸ¡ã§ããŸããäŸãã°ãè€æ°ã®çµç¹åãã«RAGã·ã¹ãã ããã¹ãããŠããå Žåãæ€çŽ¢ã€ã³ããã¯ã¹ãåå²ããããšã§ãåçµç¹ãèªèº«ã®ã€ã³ããã¯ã¹ããã®ã¿ããã¥ã¡ã³ããæ€çŽ¢ã§ããããã«ããããšãã§ããŸããããã«ãããããçµç¹ã®æ å ±ãå¥ã®çµç¹ã«äžæ³šæã«å ¬éããŠããŸãããšãé²ããŸãã
1.2.4 é·ãã³ã³ããã¹ãã®ã¢ãã«ã¯RAGãç¡çšã®é·ç©ã«ããªã
Gemini 1.5ã¯æå€§1000äžããŒã¯ã³ã®ã³ã³ããã¹ããŠã£ã³ããŠãæäŸããŠãããããRAGã®å°æ¥ã«ã€ããŠçåãæã¡å§ãã人ãããŸãã
ç§ã¯ãGemini 1.5ãSoraã«ãã£ãŠå€§å¹ ã«èªåŒµãããŠãããšèããåŸåããããŸãã1000äžããŒã¯ã³ã®ã³ã³ããã¹ããŠã£ã³ããŠã¯ãæ¢åã®RAGãã¬ãŒã ã¯ãŒã¯ã®ã»ãšãã©ãäžèŠã«ããŸããããŒã¿ãã³ã³ããã¹ãã«å ¥ããŠãéåžžã©ããã¢ãã«ãšå¯Ÿè©±ããã ãã§ããã»ãšãã©ã®ãšã³ãžãã¢ãªã³ã°ã®åªåãRAGã«è²»ããããŠããã¹ã¿ãŒãã¢ããããšãŒãžã§ã³ããlangchainãããžã§ã¯ãã«ã©ã®ãããªåœ±é¿ãäžãããæ³åããŠã¿ãŠãã ããð ã€ãŸãã1000äžã®ã³ã³ããã¹ããRAGãæ®ºãã®ã§ããGeminiã®çŽ æŽãããä»äºã§ãããâ Yao Fu
é·å€§ãªã³ã³ããã¹ããè€æ°ã®ããã¥ã¡ã³ãã®åæãPDFãšã®ãã£ãããªã©ã®ãŠãŒã¹ã±ãŒã¹ã«ãšã£ãŠç»æçãªãã®ã«ãªãã®ã¯äºå®ã§ãããRAGã®æ¶æ» ãèããã®ã¯å€§ããã§ãã
ãŸãã1000äžããŒã¯ã³ã®ã³ã³ããã¹ããµã€ãºã§ãã£ãŠããé¢é£ããã³ã³ããã¹ããéžæããæ¹æ³ãå¿ èŠã§ããæ¬¡ã«ãNeedle-in-a-haystackãšããè©äŸ¡åºæº(蚳蚻: é·å€§ãªã€ã³ãããã®äžããç¹å®ã®æ å ±ãæãåºãæ§èœãè©äŸ¡ãããã³ãããŒã¯)ã«ãããŠãã¢ãã«ã倧ããªã³ã³ããã¹ããµã€ãºã§å¹æçã«æšè«ã§ãããšãã説åŸåã®ããããŒã¿ã¯ãŸã èŠãããŸããããããã£ãŠãé©åãªæ€çŽ¢ïŒããã³ã©ã³ãã³ã°ïŒããªããã°ãã¢ãã«ã«äœèšãªæ å ±ãäžãéãããªã¹ã¯ããããŸããå®å šã«ç¡é¢ä¿ãªæ å ±ã°ããã§ã³ã³ããã¹ããŠã£ã³ããŠãåãŸã£ãŠããŸãå¯èœæ§ãããããŸãã
æåŸã«ãã³ã¹ãã®åé¡ããããŸããæšè«äžãTransformerã®æéèšç®éã¯ã³ã³ããã¹ãé·ã«æ¯äŸããŠæ¡å€§ããŸããå質åã«çããåã«çµç¹å šäœã®Googleãã©ã€ãã®ã³ã³ãã³ããèªãããšãã§ããã¢ãã«ãååšãããããšãã£ãŠããããè¯ãã¢ã€ãã¢ã ãšã¯éããŸãããRAMã®äœ¿ç𿹿³ãšã®é¡æšãèããŠã¿ãŠãã ãããæ°åãã©ãã€ãã®RAMãæèŒããã³ã³ãã¥ãŒãã€ã³ã¹ã¿ã³ã¹ãååšããŸãããããããã£ã¹ã¯ããèªã¿æžãããŠããŸãã
ã§ããããRAGããŽãç®±ã«æšãŠãã®ã¯ãŸã æ©ãã§ããã³ã³ããã¹ããµã€ãºã倧ãããªã£ãŠããåŒãç¶ã圹ã«ç«ã€ã§ãããã
1.3 ã¯ãŒã¯ãããŒã®ãã¥ãŒãã³ã°ãšæé©å
ããã³ããã¯ã»ãã®å§ãŸãã«éããŸãããLLMãæå€§éã«æŽ»çšããã«ã¯ãåäžã®ããã³ãããè¶ ããŠèããã¯ãŒã¯ãããŒãèããå¿ èŠããããŸãã
äŸãã°ã1ã€ã®è€éãªã¿ã¹ã¯ãè€æ°ã®ã·ã³ãã«ãªã¿ã¹ã¯ã«åå²ããã«ã¯ã©ãããã°ããã§ããããïŒãã¡ã€ã³ãã¥ãŒãã³ã°ããã£ãã·ã³ã°ã¯ãããã©ãŒãã³ã¹ã®åäžãã¬ã€ãã³ã·ãŒ/ã³ã¹ãã®åæžã«åœ¹ç«ã€ã®ã¯ãã€ã§ãããã?
ããã§ã¯ãå®èšŒæžã¿ã®æŠç¥ãšå®éã®äŸã玹ä»ããä¿¡é Œæ§ã®é«ãLLMã¯ãŒã¯ãããŒãæ§ç¯ããã®ã«åœ¹ç«ãŠãŠããã ããŸãã
1.3.1 ã¹ããããã€ã¹ãããã®å€æ®µéã®ããããŒãã¯å€§ããªå¹æããããã
1ã€ã®å€§ããªããã³ãããè€æ°ã®å°ããªããã³ããã«åè§£ããããšã§ãããè¯ãçµæãåŸããããšããããšã¯äžè¬çã«ç¥ãããŠããŸããäŸãã°ãAlphaCodiumã§ã¯ã1ã€ã®ããã³ãããã倿®µéã®ã¯ãŒã¯ãããŒã«åãæ¿ããããšã§ãCodeContestsã«ãããGPT-4ã®ç²ŸåºŠïŒpass@5ïŒã19ïŒ ãã44ïŒ ã«åäžãããŸããã
ãã®ã¯ãŒã¯ãããŒã«ã¯ä»¥äžã®ãããªãã®ãå«ãŸããŸãã
- åé¡ã«ã€ããŠèãã
- å ¬éãã¹ãã«ã€ããŠæšè«ãã
- å¯èœãªè§£æ±ºçãçæãã
- å¯èœãªè§£æ±ºçãã©ã³ã¯ä»ããã
- åæãã¹ããçæãã
- å ¬éãã¹ããšåæãã¹ãã§è§£æ±ºçãç¹°ãè¿ã
æç¢ºãªç®çãæã€å°ããªã¿ã¹ã¯ã¯ããšãŒãžã§ã³ãããããŒã®ããã³ããã«æé©ã§ãããã¹ãŠã®ãšãŒãžã§ã³ãããã³ãããæ§é åãããåºåãèŠæ±ããå¿ èŠã¯ãããŸããããæ§é åãããåºåã¯ããšãŒãžã§ã³ããšç°å¢ãšã®çžäºäœçšã調æŽããã·ã¹ãã ãšã®ã€ã³ã¿ãŒãã§ãŒã¹ã«å€§ãã«åœ¹ç«ã¡ãŸãã
å°ããããã³ãããäœãæã«è©ŠããŠã¿ã䟡å€ã®ãããã®ã¯ä»¥äžã®éãã§ãã
- æç¢ºã§å ·äœçãªèšç»ã®ã¹ããããå«ããããšããããããçšæããèšç»ã®äžããéžæã§ããããã«ããããšãæ€èšããŠã¿ãŠãã ããã
- ãŠãŒã¶ãŒãæåã«å ¥åããããã³ãããããšãŒãžã§ã³ããçè§£ããããããã³ããã«æžãæããããšããã ãããã®éçšã§æ å ±ã倱ãããå¯èœæ§ãããããšã«æ³šæãå¿ èŠã§ãïŒ
- ãšãŒãžã§ã³ãã®åäœããåçŽãªé£éãæåéå·¡åã°ã©ãïŒDAGïŒãã¹ããŒããã·ã³ãªã©ã®åœ¢ã§è¡šçŸããããšãã¿ã¹ã¯ã®èŠæš¡ã«å¿ããŠãç°ãªãäŸåé¢ä¿ãããžãã¯ã®é¢ä¿ãé©åãªå ŽåããããŸããã¿ã¹ã¯ã®æ§é ã工倫ããããšã§ãããã©ãŒãã³ã¹ã®æé©åãå³ãããããããŸããã
- èšç»ã®åŠ¥åœæ§ãæ€èšŒããããšãä»ã®ãšãŒãžã§ã³ãããã®å¿çãè©äŸ¡ããæçµçãªçµã¿ç«ãŠãé©åã«æ©èœããããã®æç€ºãèšç»ã«çã蟌ãããšãã§ããŸãã
- ãšãŒãžã§ã³ãããã³ãããèšèšããéã¯ããã®ããã³ãããåŒã³åºãããåã«èµ·ããããæ§ã ãªç¶æ³ãæ³å®ãããããã®ç¶æ³ã«å¯ŸããŠé©åã«æ©èœããããã«ããããšã
1.3.2 決å®è«çãªã¯ãŒã¯ãããŒãåªå ãã
AIãšãŒãžã§ã³ãã¯ãŠãŒã¶ãŒã®ãªã¯ãšã¹ããç°å¢ã«åçã«åå¿ã§ããŸããããã®é決å®è«çãªæ§è³ªã«ãã£ãŠãããã¯ãã«çµã¿èŸŒãããšãé£ãããªããŸãã
ãšãŒãžã§ã³ããå®è¡ããåã¹ãããã«ã¯å€±æã®å¯èœæ§ãããããšã©ãŒããå埩ããå¯èœæ§ã¯äœãã§ãããããã£ãŠããšãŒãžã§ã³ãããã«ãã¹ãããã®ã¿ã¹ã¯ãæ£åžžã«å®äºããå¯èœæ§ã¯ãã¹ãããæ°ã®å¢å ãšãšãã«ææ°é¢æ°çã«äœäžããŸãããã®ããããšãŒãžã§ã³ããæ§ç¯ããããŒã ã¯ä¿¡é Œæ§ã®é«ããšãŒãžã§ã³ãããããã€ããããšã«èŠåŽããŠããŸãã
èããããã¢ãããŒãã¯ããšãŒãžã§ã³ãã·ã¹ãã ã«æ±ºå®è«çãªèšç»ãçæããããããæ§é åãããæ¹æ³ã§å®è¡ããããšã§ãã
ãŸãã倧ãŸããªç®æšãããã³ãããäžããŠãšãŒãžã§ã³ãã«èšç»ãçæããŠããããæ¬¡ã«ããã®èšç»ã決å®è«çã«å®è¡ãããŸããããã«ãããåã¹ããããããäºæž¬å¯èœã§ä¿¡é Œæ§ã®é«ããã®ã«ãªããŸãã
å©ç¹ã¯ä»¥äžã®éãã§ãã
- çæãããèšç»ã¯ããšãŒãžã§ã³ãã«ããã³ãããåºããããã¡ã€ã³ãã¥ãŒãã³ã°ãããããããã®Few-shotsãµã³ãã«ãšããŠæ©èœããŸãã
- 決å®è«çãªå®è¡ã«ãããã·ã¹ãã ã®ä¿¡é Œæ§ãé«ãŸãããã¹ããšãããã°ã容æã«ãªããŸããããã«ããšã©ãŒãã©ã®ã¹ãããã§èµ·ãããããã¬ãŒã¹ããããšãã§ããŸã
- çæãããèšç»ã¯ãæåéå·¡åã°ã©ãïŒDAGïŒãšããŠè¡šçŸã§ããŸããããã¯ãéçãªããã³ãããšæ¯èŒããŠãçè§£ãããããæ°ããç¶æ³ã«é©å¿ãããããªããŸãã
æãæåããŠãããšãŒãžã§ã³ãéçºè ã¯ãæ°äººãšã³ãžãã¢ã®ç®¡çã«é¢ããè±å¯ãªçµéšãæã£ãŠãã人ãããããŸããããªããªããèšç»ãçæããããã»ã¹ã¯ãæ°äººã«æç€ºãåºããã管çãããããæ¹æ³ã«äŒŒãŠããããã§ããæ°äººã«ã¯ããããŸãã§çµããã®ãªãæç€ºã§ã¯ãªããæç¢ºãªç®æšãšå ·äœçãªèšç»ãäžããŸãããšãŒãžã§ã³ãã«å¯ŸããŠãåãããã«ãã¹ãã§ãã
ã€ãŸããšãããä¿¡é Œæ§ãé«ããæ©èœãããšãŒãžã§ã³ããžã®éµã¯ãããæ§é åãããæ±ºå®è«çãªã¢ãããŒããæ¡çšããããã³ãããæ¹è¯ãã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããããã®ããŒã¿ãåéããããšã«ããã§ããããããããªããã°ãæã ã¯éåžžã«ããŸãæ©èœãããããããŸããããåºæ¬çã«ã¯ãŠãŒã¶ãŒã倱æããããšãŒãžã§ã³ããæ§ç¯ããããšã«ãªããŸãã
1.3.3 temperature以å€ã®æ¹æ³ã§å€æ§ãªåºåãåŸã
ããã¿ã¹ã¯ã§LLMã®åºåã«å€æ§æ§ãæ±ãããããšããŸããäŸãã°ããŠãŒã¶ãŒã以åã«è³Œå ¥ãã補åã®ãªã¹ããåºã«ãã«ã¿ãã°ããè³Œå ¥ãã補åãææ¡ããLLMãã€ãã©ã€ã³ãäœæããŠãããšããŸããããã³ãããè€æ°åå®è¡ãããšãçµæãšããŠåŸãããã¬ã³ã¡ã³ããŒã·ã§ã³ãé¡äŒŒããããŠããããšã«æ°ã¥ããããããŸãããããã§ãLLMãªã¯ãšã¹ãã®temperatureãã©ã¡ãŒã¿ãäžããŠã¿ããšããã§ãããã
æçã«èšããšãtemperatureãã©ã¡ãŒã¿ãäžãããšLLMã®å¿çããã倿§ã«ãªããŸãããµã³ããªã³ã°æã«ã次ã®ããŒã¯ã³ã®ç¢ºçååžãå¹³åŠã«ãªããéåžžã¯ããŸãéžã°ããªããããªããŒã¯ã³ãé »ç¹ã«éžã°ãããããªããŸãã
ããã§ããtemperatureãäžãããšãåºåã®å€æ§æ§ã«é¢é£ããããã€ãã®æ¬ é¥ã«æ°ã¥ããããããŸãããäŸãã°ãã«ã¿ãã°ã®äžã«ã¯ãã£ãããããããããªã補åãããã®ã«ãLLMã§ã¯åºåãããªããããããŸãããLLMããã¬ãŒãã³ã°æã«åŠç¿ããããšã«åºã¥ããŠããã³ããã«ç¶ãå¯èœæ§ãé«ãå Žåãåãäžæ¡ãã®è£œåãåºåã«éå°ã«è¡šçŸããããããããŸãããtemperatureãé«ããããšãååšããªã補åãåç §ããåºåãåŸããããããããŸããã
èšãæããã°ãtemperatureãäžããŠããLLMãããªãã®æåŸ ãã確çååžããåºåããµã³ããªã³ã°ããããšã¯ä¿èšŒãããŸãããããã§ããåºåã®å€æ§æ§ãé«ããããã®ä»ã®ããªãã¯ããããŸãã
æãç°¡åãªæ¹æ³ã¯ãããã³ããå ã®èŠçŽ ã調æŽããããšã§ããäŸãã°ãããã³ãããã³ãã¬ãŒãã«éå»ã®è³Œå ¥å±¥æŽãªã©ã®ã¢ã€ãã ã®ãªã¹ããå«ãŸããŠããå Žåãããã³ããã«æ¿å ¥ãããã³ã«ãããã®ã¢ã€ãã ã®é åºãã·ã£ããã«ãããšã倧ããªéããçãŸããŸãã
ããã«ãçŽè¿ã®åºåã®ãªã¹ããä¿æããŠãããšåé·æ§ãé²ãã®ã«åœ¹ç«ã¡ãŸãã
äŸãã°ããã®ä¿æãããªã¹ãããã¢ã€ãã ãææ¡ããªãããLLMã«æç€ºããããæè¿ã®ææ¡ãšäŒŒãŠããåºåãæåŠããŠåãµã³ããªã³ã°ãããããããšã§ãå¿çãããã«å€æ§åã§ããŸãã
ãã1ã€ã®å¹æçãªæŠç¥ã¯ãããã³ããã§äœ¿çšãããã¬ãŒãºãå€ããããšã§ãã
äŸãã°ããŠãŒã¶ãŒã宿çã«äœ¿çšããããªãã¢ã€ãã ãéžãã§ãã ããããããŠãŒã¶ãŒãå人ã«ãå§ãããããªè£œåãéžãã§ãã ããããšãã£ããã¬ãŒãºãåãå
¥ããããšã§ãéç¹ãç§»ãããã®çµæãããããååã®çš®é¡ã«åœ±é¿ãäžããããšãã§ããŸãã
1.3.4 ãã£ãã·ã¥ã¯éå°è©äŸ¡ãããŠãã
ãã£ãã·ã¥ã¯ãåãå ¥åã«å¯Ÿããå¿çã®åèšç®ããªããããšã§ãã³ã¹ããç¯çŽããçæã®ã¬ã€ãã³ã·ãŒãæé€ããŸããããã«ãå¿çã以åã«ã¬ãŒãã¬ãŒã«ãããŠããå Žåããããã®æ€èšŒæžã¿ã®å¿çãæäŸããæå®³ãŸãã¯äžé©åãªã³ã³ãã³ããæäŸãããªã¹ã¯ãæžããããšãã§ããŸãã
ãã£ãã·ã³ã°ã®ç°¡åãªã¢ãããŒãã®1ã€ã¯ãæ°ããèšäºã補åã¬ãã¥ãŒãèŠçŽããå Žåã®ããã«ãåŠçäžã®ã¢ã€ãã ã«äžæã®IDãä»äžããããšã§ãããªã¯ãšã¹ããå±ãããããã£ãã·ã¥ã«èŠçŽããã§ã«ååšãããã©ããã確èªããŸããååšããå Žåã¯ããã«è¿ããååšããªãå Žåã¯çæããã¬ãŒãã¬ãŒã«ããŠãå°æ¥ã®ãªã¯ãšã¹ãã®ããã«ãã£ãã·ã¥ã«ä¿åããŸãã
ããèªç±åœ¢åŒã®ã¯ãšãªã«ã€ããŠã¯ããªãŒãã³ãšã³ãã®å ¥åã«ããã£ãã·ã³ã°ã掻çšããæ€çŽ¢åéã®æè¡ã掻çšããããšãã§ããŸãããªãŒãã³ã³ããªãŒããã¹ãã«ãã§ãã¯ãææ¡ã¯ãšãªãªã©ã®æ©èœã¯ããŠãŒã¶ãŒå ¥åãæ£èŠåãããã£ãã·ã¥ãããçãé«ããã®ã«ã圹ç«ã¡ãŸãã
1.3.5 ãã¡ã€ã³ãã¥ãŒãã³ã°ã®ã¿ã€ãã³ã°
ç·»å¯ã«èšèšãããããã³ããã§ãäžååãªã¿ã¹ã¯ããããããããŸããã
倧å¹
ãªããã³ãããšã³ãžãã¢ãªã³ã°ãè¡ã£ãåŸã§ããã·ã¹ãã ãä¿¡é Œæ§ãé«ãé«å質ãªåºåãè¿ããŸã§ã«ã¯ãŸã ãŸã æéãããããããããŸããã
ãã®å Žåãç¹å®ã®ã¿ã¹ã¯ã®ããã«ã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããå¿
èŠãåºãŠããå¯èœæ§ããããŸãã
ãã¡ã€ã³ãã¥ãŒãã³ã°ã®å®éã®æåäŸã¯ä»¥äžã®éãã§ãã
- Honeycombã®èªç¶èšèªã¯ãšãªã¢ã·ã¹ã¿ã³ã: åœåããããã°ã©ãã³ã°ããã¥ã¢ã«ãã¯ãIn-Context Learningã®ããã®n-shotã®äŸãšãšãã«ããã³ããã§æäŸãããŠããŸãããããã¯ããŸãæ©èœããŸããããã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããããšã§ããã¡ã€ã³åºæã®èšèªã®æ§æãšã«ãŒã«ã«é¢ããåºåãæ¹åãããŸããã
- Rechatã®Lucy: LLMã¯ãããã³ããšã³ããæ£ããã¬ã³ããªã³ã°ããããã«ãæ§é åããŒã¿ãšéæ§é åããŒã¿ãçµã¿åãããéåžžã«ç¹æ®ãªåœ¢åŒã§å¿çãçæããå¿ èŠããããŸãããäžè²«ããŠæ©èœãããã«ã¯ããã¡ã€ã³ãã¥ãŒãã³ã°ãäžå¯æ¬ ã§ããã
ãšã¯ããããã¡ã€ã³ãã¥ãŒãã³ã°ã¯å¹æçã§ããã倧ããªã³ã¹ããããããŸãã
ãã¡ã€ã³ãã¥ãŒãã³ã°ããŒã¿ã«ã¢ãããŒã·ã§ã³ãä»ããã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããŠè©äŸ¡ããæçµçã«ããããã»ã«ããã¹ãããªããã°ãªããŸããã
ãããã£ãŠãããé«ãåæã³ã¹ãã«èŠåããã©ãããæ€èšããŠãã ãããããã³ããã§90ç¹ãŸã§ããããªãããã¡ã€ã³ãã¥ãŒãã³ã°ãžã®æè³ã¯å²ã«åããªããããããŸããããããããã¡ã€ã³ãã¥ãŒãã³ã°ããããšã«æ±ºãããªãã人éãã¢ãããŒã·ã§ã³ããããŒã¿ãåéããã³ã¹ããåæžããããã«ãåæããŒã¿ãçæããŠãã¡ã€ã³ãã¥ãŒãã³ã°ããããOSSããŒã¿ã§ããŒãã¹ãã©ãããããããããšãã§ããŸãã
1.4 è©äŸ¡ãšã¢ãã¿ãªã³ã°
LLMã®è©äŸ¡ã¯å°é·åã§ãããAnthropicã®ãããªæå€§ã®ç ç©¶æã§ããå°é£ã«çŽé¢ããŠããŸããLLMã¯èªç±åœ¢åŒã®åºåãè¿ããç§ãã¡ãå®è¡ããŠã»ããã¿ã¹ã¯ã¯å€å²ã«ããããŸããããã§ãã峿 Œã§ææ ®æ·±ãè©äŸ¡ã¯éåžžã«éèŠã§ããOpenAIã®ãã¯ãã«ã«ãªãŒããŒã§ãè©äŸ¡ã«åãçµã¿ãåã ã®è©äŸ¡ã«ãã£ãŒãããã¯ãè¡ãã®ã¯å¶ç¶ã§ã¯ãããŸããã
LLMã¢ããªã±ãŒã·ã§ã³ã®è©äŸ¡ã«ã¯ã倿§ãªå®çŸ©ããããŸããåãªããŠããããã¹ãã§ãã£ããããªãã¶ãŒãããªãã£ã«è¿ãã£ããããããã¯åã«ããŒã¿ãµã€ãšã³ã¹ã§ãã£ããããŸããç§ãã¡ã¯ãããããã¹ãŠã®èŠç¹ãæçšã§ããããšã«æ°ã¥ããŸããããã®ã»ã¯ã·ã§ã³ã§ã¯ãè©äŸ¡ãšã¢ãã¿ãªã³ã°ã®ãã€ãã©ã€ã³ãæ§ç¯ããäžã§éèŠãªç¹ã«ã€ããŠããã€ãã®æèšãæäŸããŸãã
1.4.1 å®éã®å ¥å/åºåãµã³ãã«ããå°æ°ã®ã¢ãµãŒã·ã§ã³ããŒã¹ã®ãŠããããã¹ããäœæãã
æ¬çªç°å¢ããã®å ¥åãšåºåã®ãµã³ãã«ã§æ§æããããŠããããã¹ãïŒã€ãŸããã¢ãµãŒã·ã§ã³ïŒãäœæããŸããåºåã«å¯ŸããæåŸ å€ã¯ãå°ãªããšã3ã€ã¯è©äŸ¡åºæºãå¿ èŠã§ãã"3ã€"ã®åºæºã¯æ£æçã«èŠãããããããŸãããããšããããå§ããã®ã«å®çšçãªæ°ã§ããããããå°ãªããšãã¿ã¹ã¯ãååã«å®çŸ©ãããŠããªãããæ±çšãã£ãããããã®ããã«èªç±åœ¢åŒãããããšã瀺ããŠããå¯èœæ§ããããŸãã
ãããã®ãŠããããã¹ããã€ãŸãã¢ãµãŒã·ã§ã³ã¯ãããã³ããã®ç·šéãRAGãä»ããæ°ããã³ã³ããã¹ãã®è¿œå ããã®ä»ã®å€æŽãªã©ãäœãããã®ãã€ãã©ã€ã³ãžã®å€æŽã«å¯ŸããŠå®è¡ãããŸããäžèšèšäºã«ã¯ãå®éã®ãŠãŒã¹ã±ãŒã¹ã«ãããã¢ãµãŒã·ã§ã³ããŒã¹ã®ãã¹ãã®äŸã瀺ãããŠããŸãã
å¿çãå«ããã¹ãããŸãã¯é€å€ãã¹ããã¬ãŒãºãæå®ããã¢ãµãŒã·ã§ã³ããå§ããããšãæ€èšããŠã¿ãŠãã ããããŸããåèªæ°ãã¢ã€ãã æ°ãææ°ãç¯å²å ã«ããããšã確èªãããã§ãã¯ã詊ããŠã¿ãŠãã ããã
ã³ãŒãçæã®ãããªãã®ä»ã®çš®é¡ã®çæã§ã¯ãã¢ãµãŒã·ã§ã³ã¯ç°ãªã圢ã«ãªãå¯èœæ§ããããŸããå®è¡ããŒã¹ã®è©äŸ¡ã¯ãã³ãŒãçæãè©äŸ¡ãã1ã€ã®æ¹æ³ã§ããçæãããã³ãŒããå®è¡ããã©ã³ã¿ã€ã ã®ç¶æ ã»çµæããŠãŒã¶ãŒã®èŠæ±ã«ååã§ãããã©ããã確èªããŸãã
äŸãšããŠããŠãŒã¶ãŒãfooãšããååã®æ°ãã颿°ãèŠæ±ããå ŽåããšãŒãžã§ã³ãã®çæããã³ãŒããå®è¡ããåŸãfooãåŒã³åºãå¯èœã§ãªããã°ãªããŸããã
ãããããšãŒãžã§ã³ããçæããã³ãŒãã¯ã人éãæžãã³ãŒããšã¯å°ãç°ãªã圢åŒã«ãªãããšããããããŸããäŸãã°ã倿°åã颿°åãç°ãªã£ãŠããããã³ãŒãã®æ§é ãå°ãéã£ãŠãããããå¯èœæ§ããããŸãã
ãã®ãããªå Žåãå³å¯ã«ãfooãšããååã®é¢æ°ãååšããåŒã³åºãå¯èœã§ããããšããã¢ãµãŒã·ã§ã³ã§ã¯ããšãŒãžã§ã³ããçæããã³ãŒããäžå¿
èŠã«äžåæ Œãšãªã£ãŠããŸãå¯èœæ§ããããŸãã
ããã§ããèŠæ±ãããæ©èœãæäŸãã颿°ãååšããåŒã³åºãå¯èœã§ããããšããããã«ãã¢ãµãŒã·ã§ã³ãå°ãç·©åããããšã广çãªå ŽåããããŸããããã«ãããã³ãŒãã®çްéšãå°ã
ç°ãªã£ãŠããŠããå
šäœãšããŠã®æ©èœãæºããããŠããã°åæ Œãšã¿ãªãããšãã§ããŸãã
æåŸã«ããããã¯ãããŠãŒã¶ãå®éã«äœ¿ãããã«äœ¿çšããããšïŒã€ãŸãããããã°ããŒãã£ã³ã°ãïŒã¯ãå®äžçã®ããŒã¿ã§ã®ãšã©ãŒã»èª²é¡ã«é¢ããæŽå¯ãåŸãããŸãããã®ã¢ãããŒãã¯ãæœåšçãªèª²é¡ãç¹å®ããã®ã«åœ¹ç«ã€ã ãã§ãªããè©äŸ¡ã«å€æã§ããæ¬çªãµã³ãã«ã®æçšãªæ å ±æºã«ããªããŸãã
1.4.2 LLM-as-Judgeã¯ïŒããçšåºŠïŒæ©èœããããäžèœã§ã¯ãªã
LLM-as-Judgeã¯ã匷åãªLLMã䜿çšããŠä»ã®LLMã®åºåãè©äŸ¡ããææ³ã§ããããã°ãã°æççãªèŠæ¹ããããŠããŸããïŒç§ãã¡ã®äžã«ããåœåã¯å€§ããªæçè«è ãããŸãããïŒããã§ããé©åã«å®è£ ãããLLM-as-Judgeã¯ã人éã®å€æãšãŸããŸãã®çžé¢ãéæããå°ãªããšãæ°ããããã³ãããææ³ãã©ã®ããã«æ©èœãããã«ã€ããŠã®äºåã®ç¥èãæ§ç¯ããã®ã«åœ¹ç«ã¡ãŸãã
å ·äœçã«ã¯ããã¢ã¯ã€ãºæ¯èŒïŒcontrol vs. treatmentïŒãè¡ãéãLLM-as-Judgeã¯éåžžãåã¡/è² ãã®å€§ããããã€ãºã«ãªãå¯èœæ§ããããŸãããæ¹åæ§ã¯æŠãæ£ãã倿ããŸãã
LLM-as-Judgeãæå€§éã«æŽ»çšããããã®ããã€ãã®ææ¡ã以äžã«ç€ºããŸãã
- ãã¢ã¯ã€ãºæ¯èŒã䜿çšããïŒLLMã«ãªãã«ãŒã尺床ã§åäžã®åºåãè©äŸ¡ããããæ±ãã代ããã«ã2ã€ã®ãªãã·ã§ã³ãæç€ºããããè¯ãã»ããéžæããããæ±ããŸããããã«ãããããå®å®ããçµæãåŸãããåŸåããããŸãã
- äœçœ®ãã€ã¢ã¹ãå¶åŸ¡ããïŒæç€ºããããªãã·ã§ã³ã®é åºã«ãã£ãŠãLLMã®æ±ºå®ã«ãã€ã¢ã¹ããããå¯èœæ§ããããŸããããã軜æžããããã«ãåãã¢ã¯ã€ãºæ¯èŒã2åè¡ãããã¢ã®é åºãæ¯åå ¥ãæ¿ããŸãã
- åŒãåãã蚱容ããïŒå Žåã«ãã£ãŠã¯ãäž¡æ¹ã®ãªãã·ã§ã³ãåçã«è¯ãå ŽåããããŸãããããã£ãŠãLLMãæ£æçã«åè ãéžã°ãªããŠãæžãããã«ãåŒãåãã宣èšã§ããããã«ããŸãã
- CoTã䜿çšããïŒæçµçãªçããåºãåã«LLMã«è©äŸ¡ã®çç±ã説æããããæ±ããããšã§ãè©äŸ¡ã®ä¿¡é Œæ§ãé«ããããšãã§ããŸããããŒãã¹ãšããŠãCoTã䜿ãããšã«ããããã匱ããé«éãªLLMã§ãåæ°Žæºã®è©äŸ¡çµæãåŸãããšãã§ããŸãããã€ãã©ã€ã³ã®ãã®éšåã¯éåžžãããã§å®è¡ããããããCoTã«ããäœåãªã¬ã€ãã³ã·ãŒã¯åé¡ã«ãªããŸããã
- å¿çã®é·ããå¶åŸ¡ããïŒLLMã¯é·ãå¿çã«ãã€ã¢ã¹ããããåŸåããããŸããããã軜æžããããã«ãå¿çãã¢ã®é·ããåçšåºŠã«ãªãããã«ããŸãã
LLM-as-Judgeã®æçšãªå¿çšäŸã¯ãæ°ããããã³ããæŠç¥ãæ¢åã®ãã®ãšç §ããåãããŠãã§ãã¯ããããšã§ããæ¬çªã§ã®å ¥åºåã®ããŒã¿ãããå Žåããããã®æ¬çªäŸãæ°ããããã³ããæŠç¥ã§åå®è¡ããLLM-as-Judgeã䜿çšããŠæ°ããæŠç¥ã®ã©ã®éšåã«åé¡ãããããè¿ éã«è©äŸ¡ã§ããŸãã
ãã¡ãã®èšäºã§ã¯ãLLMã®å¿çãå€æã®æ¹è©ïŒã€ãŸããCoTïŒãæçµçãªåºåçµæããèšé²ããªãããã¹ããŒã¯ãã«ããŒã«ã¬ãã¥ãŒããŠãããããšã«ããå埩çã«LLM-as-Judgeã®ç²ŸåºŠãäžããããã»ã¹ã玹ä»ãããŠããŸãã
å®éã«ãã®ããã»ã¹ã«ãã£ãŠãããã3åã®å埩ãçµãŠäººéãšLLMã®äžèŽçã¯68ïŒ
ãã94ïŒ
ã«åäžããŸããã

ãã ããLLM-as-Judgeã¯äžèœã§ã¯ãããŸãããæã匷åãªã¢ãã«ã§ãããèšèªã®åŸ®åŠãªåŽé¢ã確å®ã«è©äŸ¡ã§ããªãå ŽåããããŸããããã«ãåŸæ¥ã®åé¡åšãšå ±é ¬ã¢ãã«ã¯ãLLM-as-Judgeãããé«ã粟床ãéæã§ããã³ã¹ããšã¬ã€ãã³ã·ãŒãäœãããšãããããŸããããŸãã³ãŒãçæã«ãããŠã¯ãLLM-as-Judgeã¯ãå®è¡è©äŸ¡ã®ãããªããçŽæ¥çãªè©äŸ¡æŠç¥ããã粟床ãäœãå¯èœæ§ããããŸãã
1.4.3 çæç©ãè©äŸ¡ããããã®ãã€ã³ã¿ãŒã³ãã¹ãã
çæç©ãè©äŸ¡ããéãæ¬¡ã®ãããªãã€ã³ã¿ãŒã³ãã¹ãããšããæèå®éšãããŠã¿ãã®ãç§ã®å¥œã¿ã§ãã
èšèªã¢ãã«ãžã®å
¥åïŒã³ã³ããã¹ããå«ãïŒãååŸããé¢é£ããå°æ»ã®å¹³åçãªå€§åŠçã«ã¿ã¹ã¯ãšããŠäžããå Žåã圌ãã¯æåã§ããã§ãããããã©ã®ãããã®æéããããã§ããããã
- çããããŒã§ãLLMã«å¿ èŠãªç¥èãäžè¶³ããŠããå Žåã¯ãã³ã³ããã¹ããè±ãã«ããæ¹æ³ãæ€èšããŠãã ããã
- çããããŒã§ãã³ã³ããã¹ããæ¹åããŠãä¿®æ£ã§ããªãå Žåã¯ãçŸä»£ã®LLMã«ãšã£ãŠé£ããããã¿ã¹ã¯ã«ã¶ã€ãã£ãå¯èœæ§ããããŸãã
- çããã€ãšã¹ã ããæéããããå Žåã¯ãã¿ã¹ã¯ã®è€éããæžããããšã詊ã¿ãããšãã§ããŸããåè§£å¯èœã§ãããããã¿ã¹ã¯ã®äžéšããã ãã³ãã¬ãŒãåã§ããåŽé¢ã¯ãããŸããã
- çããã€ãšã¹ã§ãããã«çè§£ã§ããå Žåã¯ãããŒã¿ãæãäžããææã§ããã¢ãã«ã¯äœãééããŠããã®ã§ããããã倱æã®ãã¿ãŒã³ãèŠã€ããããšã¯ã§ããŸãããå¿çã®ååŸã«ã¢ãã«ã«èªèº«ã説æããããæ±ããŠãå¿ã®çè«(蚳蚻: ä»è ã®èããäºæž¬ããèœåã®ããš)ãæ§ç¯ããã®ã«åœ¹ç«ãŠãŠãã ããã
1.4.4 ç¹å®ã®è©äŸ¡ãé床ã«éèŠãããšãå šäœçãªããã©ãŒãã³ã¹ãäœäžããå¯èœæ§ããã
ãããææšãã¿ãŒã²ããã«ãªããšãããã¯è¯ãææšã§ã¯ãªããªããâ ã°ããããŒãã®æ³å
ãã®äŸãšããŠãNeedle-in-a-HaystackïŒNIAHïŒè©äŸ¡ããããŸãã
ãã®è©äŸ¡ã§ã¯ãã³ã³ããã¹ããµã€ãºã®å¢å€§ã«äŒŽãã¢ãã«ã®ãªã³ãŒã«ã枬å®ããã³ã³ããã¹ãå
ã§ã®ç®æšæ
å ±ã®äœçœ®ããªã³ãŒã«ã«ã©ã®ãããªåœ±é¿ãäžãããã調ã¹ãŸããããããããã®è©äŸ¡ææ³ã¯é床ã«åŒ·èª¿ãããããåŽé¢ããããGemini 1.5ã®ã¬ããŒãã®å³1ã§ãåãäžããããŠããŸãã
ãã®è©äŸ¡ã§ã¯ãããŒã«ã»ã°ã¬ã¢ã ã®ãšãã»ã€ãç¹°ãè¿ãé·ææžã«ç¹å®ã®ãã¬ãŒãºïŒâThe special magic {city} number is: {number}âïŒãæ¿å
¥ããã¢ãã«ã«ãã®ããžãã¯ãã³ããŒãæãåºãããã«ä¿ããŸãã
äžéšã®ã¢ãã«ã¯ã»ãŒå®ç§ãªãªã³ãŒã«ãéæããŸãããNIAHãå®äžçã®ã¢ããªã±ãŒã·ã§ã³ã«å¿ èŠãªæšè«ãšãªã³ãŒã«èœåãçã«æž¬å®ããŠãããã©ããã¯çåã§ãã
ããå®çšçãªã·ããªãªãèããŠã¿ãŸãããã1æéã®äŒè°ã®æåèµ·ãããäžããããå ŽåãLLMã¯äž»èŠãªæ±ºå®ã𿬡ã®ã¹ããããèŠçŽããåé ç®ãé¢é£ããäººã«æ£ããåž°å±ãããããšãã§ããã§ããããããã®ã¿ã¹ã¯ã¯ãæ©æ¢°çãªæèšã ãã§ãªããè€éãªè°è«ãè§£æããé¢é£æ å ±ãç¹å®ããèŠçŽãåæããèœåãèæ ®ã«å ¥ããããçŸå®çãªãã®ã§ãã
äžèšèª²é¡ã®è§£æ±ºã®ããã«äœããããã³ãããŒã¯ã§ããpractical NIAH evalã§ã¯ãå»åž«ãšæ£è ã®æåèµ·ããã䜿çšããŠLLMã«æ£è ã®è¬ã«ã€ããŠã®è³ªåãããããããé£ããã¿ã¹ã¯ãšããŠã¯ãå®ç§ãªãã¶ãäœãããã«å¿ èŠãªç§å¯ã®ææã¯æ¬¡ã®ãšããã§ãïŒãšã¹ãã¬ããœã«æµžããããŒããã¬ã¢ã³ãã€ã®ã®ããŒãºããªã©ã®ãã¶ã®ãããã³ã°ããã©ã³ãã ãªææã®ãã¬ãŒãºãæ¿å ¥ããããšããã¿ã¹ã¯ããããŸãã
è¬ã®ã¿ã¹ã¯ã§ã®ãªã³ãŒã«ã¯çŽ80ïŒ ããã¶ã®ã¿ã¹ã¯ã§ã¯çŽ30ïŒ ã§ããã

äœè«ã§ãããNIAHè©äŸ¡ãé床ã«éèŠãããšãæœåºãšèŠçŽã®ã¿ã¹ã¯ã®ããã©ãŒãã³ã¹ãäœäžããå¯èœæ§ããããŸãããããã®LLMã¯ãã¹ãŠã®æã«æ³šæãæãããã«ãã¥ãŒãã³ã°ãããŠãããããç¡é¢ä¿ãªè©³çްãéèŠãªãã®ãšããŠæ±ãå§ããæçµçãªåºåã«å«ããŠããŸãå¯èœæ§ããããŸãïŒå«ããã¹ãã§ã¯ãªãã®ã«ïŒïŒã
ããã¯ãä»ã®è©äŸ¡ããŠãŒã¹ã±ãŒã¹ã«ãåœãŠã¯ãŸãå¯èœæ§ããããŸããäŸãã°ãèŠçŽã§ããäºå®ã®äžè²«æ§ãéèŠãããšãèŠçŽãå ·äœæ§ã«æ¬ ãïŒãããã£ãŠäºå®ãšççŸããå¯èœæ§ãäœãïŒãé¢é£æ§ãäœããªãå¯èœæ§ããããŸããéã«ãæç« ã®ã¹ã¿ã€ã«ãšéåŒããéèŠãããšãäºå®ã®ççŸãåŒãèµ·ããå¯èœæ§ã®ãããããè¯ãããªããŒã±ãã£ã³ã°çšã®è¡šçŸã«ã€ãªããå¯èœæ§ããããŸãã
1.4.5 ã¢ãããŒã·ã§ã³ããã€ããª(Yes/Noã§çãããã)ã¿ã¹ã¯ãŸãã¯ãã¢ã¯ã€ãºæ¯èŒã«åçŽåãã
ã¢ãã«åºåã«å¯ŸãããªãŒãã³ãšã³ãã®ãã£ãŒãããã¯ããªãã«ãŒã尺床ã§ã®è©äŸ¡ãæäŸããããšã¯ã人éã®èªç¥çã«èŠæ±ãå³ãããã®ã§ãããã®çµæãåéãããããŒã¿ã¯ãã€ãºãå€ããªãïŒäººéã®è©äŸ¡è éã®ã°ãã€ãã®ããïŒãããŸã圹ã«ç«ã¡ãŸããããã广çãªã¢ãããŒãã¯ãã¿ã¹ã¯ãåçŽåããã¢ãããŒã¿ãŒã®èªç¥çè² æ ã軜æžããããšã§ãããã€ããªåé¡ãšãã¢ã¯ã€ãºæ¯èŒããã°ãã°ããŸããããŸãã
ãã€ããªåé¡ã§ã¯ãã¢ãããŒã¿ãŒã«ã¢ãã«ã®åºåã«å¯ŸããåçŽãªã€ãšã¹ãããŒã®å€æãæ±ããŸããçæãããèŠçŽããœãŒã¹ææžãšäºå®äžäžèŽããŠãããã©ãããææ¡ãããå¿çãé¢é£ããŠãããã©ãããæå®³æ§ãå«ãŸããŠãããã©ããããªã©ãå°ããããšãã§ããã§ãããããªãã«ãŒãå°ºåºŠãšæ¯èŒããŠããã€ããªã®æ±ºå®ã¯ããæ£ç¢ºã§ãè©äŸ¡è éã®äžè²«æ§ãé«ããã¹ã«ãŒããããé«ããªããŸããããã¯ãDoordashãã€ãšã¹ãããŒã®è³ªåã®ããªãŒãéããŠã¡ãã¥ãŒé ç®ã«ã¿ã°ä»ãããããã®ã©ããªã³ã°ãã¥ãŒãèšå®ããæ¹æ³ãšåãã§ãã
ãã¢ã¯ã€ãºæ¯èŒã§ã¯ãã¢ãããŒã¿ãŒã«ã¢ãã«ã®å¿çã®ãã¢ãæç€ºãããã©ã¡ããåªããŠããããå°ããããŸãã人éã«ãšã£ãŠãAãŸãã¯Bã«åå¥ã«ã¹ã³ã¢ãå²ãåœãŠããããããAã¯BããåªããŠããããšèšãæ¹ãç°¡åãªãããã¢ãããŒã·ã§ã³ãããéãããä¿¡é Œæ§ã®é«ããã®ã«ãªããŸãïŒãªãã«ãŒã尺床ãããïŒã
Llama2ã®ããŒãã¢ããã§ãLlama2è«æã®èè ã®1人ã§ããThomas Scialomæ°ã¯ããã¢ã¯ã€ãºæ¯èŒããæžé¢ã§ã®åçãªã©ã®æåž«ãã埮調æŽããŒã¿ã®åéãããé«éãã€å®äŸ¡ã§ããããšã確èªããŸãããåè ã®ã³ã¹ãã¯1ãŠãããããã3.5ãã«ã§ãåŸè ã®ã³ã¹ãã¯1ãŠãããããã25ãã«ã§ãã
ã©ããªã³ã°ã®ã¬ã€ãã©ã€ã³ãäœæããå Žåã¯ãGoogleãšBingæ€çŽ¢ã®äŸã®ã¬ã€ãã©ã€ã³ãåç §ããŠãã ããã
1.4.6 ïŒãªãã¡ã¬ã³ã¹ããªãŒã®ïŒè©äŸ¡ãšã¬ãŒãã¬ãŒã«ã¯äº€æå¯èœã§ãã
ã¬ãŒãã¬ãŒã«ã¯äžé©åãŸãã¯æå®³ãªã³ã³ãã³ããæ€åºããã®ã«åœ¹ç«ã¡ãè©äŸ¡ã¯ã¢ãã«ã®åºåã®åè³ªãšæ£ç¢ºæ§ã枬å®ããã®ã«åœ¹ç«ã¡ãŸãããããŠããªãã¡ã¬ã³ã¹ããªãŒã®è©äŸ¡ã仿§ããå Žåãããããã¬ãŒãã¬ãŒã«ãšããŠã掻çšã§ããŸãããªãã¡ã¬ã³ã¹ããªãŒã®è©äŸ¡ãšã¯ã人éãæžããåçãªã©ã®ããŽãŒã«ãã³ããªãã¡ã¬ã³ã¹ã«äŸåãããå ¥åããã³ãããšã¢ãã«ã®å¿çã®ã¿ã«åºã¥ããŠåºåã®å質ãè©äŸ¡ã§ãããã®ã§ãã
äŸãã°ãèŠçŽã®è©äŸ¡ã§ã¯ãå ¥åææžã®ã¿ãèæ ®ããŠãäºå®ã®äžè²«æ§ãšé¢é£æ§ã«ã€ããŠèŠçŽãè©äŸ¡ããŸãããããã®ææšã§èŠçŽã®è©äŸ¡ãäœãå ŽåããŠãŒã¶ãŒã«è¡šç€ºããªãããšãéžæã§ããå®è³ªçã«è©äŸ¡ãã¬ãŒãã¬ãŒã«ãšããŠäœ¿çšããããšãã§ããŸããåæ§ã«ããªãã¡ã¬ã³ã¹ããªãŒã®ç¿»èš³è©äŸ¡ã§ã¯ã人éã翻蚳ãããªãã¡ã¬ã³ã¹ãå¿ èŠãšããã«ç¿»èš³ã®å質ãè©äŸ¡ã§ããããããããã¬ãŒãã¬ãŒã«ãšããŠäœ¿çšã§ããŸãã
1.4.7 LLMã¯ããããã¹ãã§ãªãå Žåã§ãåºåãè¿ã
LLMãæ±ãäžã§ã®éèŠãªèª²é¡ã®äžã€ã¯ãåºåãã¹ãã§ãªãå Žåã§ããã°ãã°åºåãçæããŠããŸãããšã§ããããã¯ç¡å®³ã ãæå³äžæãªå¿çããæå®³æ§ãå±éºãªã³ã³ãã³ããšãã£ãããæ·±å»ãªæ¬ é¥ã«ã€ãªããå¯èœæ§ããããŸããäŸãã°ãææžããç¹å®ã®å±æ§ãã¡ã¿ããŒã¿ãæœåºããããæ±ããããå ŽåãLLMã¯ãããã®å€ãå®éã«ã¯ååšããªããŠãèªä¿¡ãæã£ãŠå€ãè¿ãããšããããŸãããŸããã³ã³ããã¹ãã«è±èªä»¥å€ã®ææžãæäŸããããã«ãã¢ãã«ãè±èªä»¥å€ã®èšèªã§å¿çããããšããããŸãã
LLMã«ã該åœãªããããäžæããšããå¿çãè¿ãããä¿ãããšã¯ã§ããŸããããããå®å šã§ã¯ãããŸããã察æ°ç¢ºç(蚳蚻:èšèªã¢ãã«ãç¹å®ã®åèªãæãçæãã確çã察æ°ã¹ã±ãŒã«ã§è¡šãããã®ãã¢ãã«ã®äºæž¬ã®ç¢ºä¿¡åºŠãç€ºãææšãšããŠäœ¿ãããããšããããŸã)ãå©çšå¯èœãªå Žåã§ããããã¯åºåã®å質ã瀺ãè¯ãææšãšã¯èšããŸããã察æ°ç¢ºçã¯ããŒã¯ã³ãåºåã«çŸããå¯èœæ§ã瀺ããŸãããå¿ ãããçæãããããã¹ãã®æ£ç¢ºæ§ãåæ ãããã®ã§ã¯ãããŸããã
ããã©ããããã¯ãšãªã«çããŠäžè²«æ§ã®ããå¿çãçæããããèšç·Žãããæç€ºèª¿æŽã¢ãã«ã®å Žåã察æ°ç¢ºçã¯é©åã«èª¿æŽãããŠããªãå¯èœæ§ããããŸãããããã£ãŠãé«ã察æ°ç¢ºçã¯åºåãæµæ¢ã§äžè²«æ§ãããããšã瀺ãå¯èœæ§ã¯ãããŸããããããæ£ç¢ºãŸãã¯é¢é£æ§ãããããšãæå³ããããã§ã¯ãããŸããã
æ éãªããã³ãããšã³ãžãã¢ãªã³ã°ã¯ãããçšåºŠåœ¹ç«ã¡ãŸãããæãŸãããªãåºåãæ€åºããŠãã£ã«ã¿ãªã³ã°/åçæããå ç¢ãªã¬ãŒãã¬ãŒã«ãšçµã¿åãããå¿ èŠããããŸããäŸãã°ãOpenAIã¯ããã€ãã¹ããŒããèªå·è¡çºãæ§çãªåºåãªã©ã®å®å šã§ãªãå¿çãç¹å®ã§ããã³ã³ãã³ãã¢ãã¬ãŒã·ã§ã³APIãæäŸããŠããŸãã
åæ§ã«ãå人ãç¹å®ã§ããæ å ±ãæ€åºããããã®ããã±ãŒãžã¯æ°å€ããããŸããã¬ãŒãã¬ãŒã«ã®å©ç¹ã®1ã€ã¯ããŠãŒã¹ã±ãŒã¹ã«ã»ãšãã©äŸåãããç¹å®ã®èšèªã®ãã¹ãŠã®åºåã«åºãé©çšã§ããããšã§ããããã«ãæ£ç¢ºãªæ€çŽ¢ã«ãããé¢é£ããããã¥ã¡ã³ãããªãå Žåãã·ã¹ãã ã¯æ±ºå®è«çã«ãããããŸããããšå¿çã§ããŸãã
ããã§ã®ä»éçãªç¹ãšããŠãLLMãæåŸ ãããŠããå Žåã«åºåãçæã§ããªãããšããããŸããããã¯APIãããã€ããŒããã®é·ãã¬ã€ãã³ã·ãŒã®ãããªåçŽãªåé¡ãããã³ã³ãã³ãã¢ãã¬ãŒã·ã§ã³ãã£ã«ã¿ãŒã«ãã£ãŠåºåããããã¯ããããããªè€éãªåé¡ãŸã§ãããŸããŸãªçç±ã§èµ·ããåŸãŸãããã®ããããããã°ãšã¢ãã¿ãªã³ã°ã®ããã«ãå ¥åãšïŒå Žåã«ãã£ãŠã¯æ¬ åŠããŠããïŒåºåãäžè²«ããŠãã°ã«èšé²ããããšãéèŠã§ãã
1.4.8 ãã«ã·ããŒã·ã§ã³ã¯æ ¹åŒ·ãåé¡ã§ãã
泚ç®åºŠãé«ãããã£ãã«çºçããªãã³ã³ãã³ãã»ãŒããã£ãŒãPIIã®æ¬ é¥ãšã¯ç°ãªããäºå®ã®ççŸã¯é åºã«ååšããæ€åºãããå°é£ã§ããäºå®ã®ççŸã¯ããäžè¬çã§ã5ã10ïŒ ã®åºæºçã§çºçããŸããèŠçŽã®ãããªåçŽãªã¿ã¹ã¯ã§ããã2ïŒ ä»¥äžã«æããã®ã¯é£ãããšãããŠããŸãã
ããã«å¯ŸåŠããããã«ãããã³ãããšã³ãžãã¢ãªã³ã°ïŒçæã®äžæµïŒãšäºå®ã®ççŸãæ€åºããã¬ãŒãã¬ãŒã«ïŒçæã®äžæµïŒãçµã¿åãããããšãã§ããŸãã
ããã³ãããšã³ãžãã¢ãªã³ã°ã§ã¯ãCoTã®ãããªææ³ã䜿çšããŠãLLMã«æçµçãªåºåãè¿ãåã«æšè«ã説æãããããšã§ãã«ã·ããŒã·ã§ã³ãæžããããšãã§ããŸãããããŠãäºå®ã®ççŸãæ€åºããã¬ãŒãã¬ãŒã«ãé©çšããŠèŠçŽã®äºå®æ§ãè©äŸ¡ãããã«ã·ããŒã·ã§ã³ãå«ãåºåããã£ã«ã¿ãªã³ã°ããããåçæãä¿ãããããããšãå¯èœã§ãã
ç¶æ³ã«ãã£ãŠã¯ããã«ã·ããŒã·ã§ã³ã決å®è«çã«æ€åºããããšãå¯èœã§ããäŸãã°ãRetrieval-Augmented Generation (RAG) ãçšããŠæ€çŽ¢çµæããã®æ å ±ãæŽ»çšããå Žåãåºåãæ§é åãããŠããŠãªãœãŒã¹ã®åºæãç¹å®ãããŠããã°ãããããå ¥åã³ã³ããã¹ãããååŸããããã®ã§ããããšãæåã§ç¢ºèªã§ããã¯ãã§ãã
2 éçš(Operational)ïŒæ¥ã ã®æ¥åãšçµç¹ã®èª²é¡
2.1 ããŒã¿
飿ã®å質ãæçã®å³ã決å®ããã®ãšåæ§ã«ãå ¥åããŒã¿ã®åè³ªãæ©æ¢°åŠç¿ã·ã¹ãã ã®ããã©ãŒãã³ã¹ã決å®ã¥ããŸãããŸããåºåããŒã¿ã¯ãããã¯ããæ©èœããŠãããã©ããã倿ããå¯äžã®æ¹æ³ã§ããç§ãã¡ã¯ããŒã¿ã«æ³šç®ããããŒã¿ååžïŒãã®ã¢ãŒãããšããžã±ãŒã¹ãã¢ãã«ã®å¶éïŒãããããçè§£ããããã«ãé±ã«æ°æéãå®éã«èµ·ããŠããå ¥åãšåºåã確èªããŠããŸãã
2.1.1 éçºç°å¢ãšæ¬çªç°å¢ã®ä¹é¢ã確èªãã
åŸæ¥ã®æ©æ¢°åŠç¿ãã€ãã©ã€ã³ã«ããããšã©ãŒã®äžè¬çãªåå ã¯ãåŠç¿æãšæšè«æã®ä¹é¢ã§ããããã¯ããã¬ãŒãã³ã°ã«äœ¿çšãããããŒã¿ããã¢ãã«ãæ¬çªç°å¢ã§ééãããã®ãšç°ãªãå Žåã«çºçããŸããLLMã¯ãã¬ãŒãã³ã°ããã¡ã€ã³ãã¥ãŒãã³ã°ãªãã§äœ¿çšã§ããããããã¬ãŒãã³ã°ã»ããã¯ãããŸããããéçºç°å¢ãšæ¬çªç°å¢ã®ããŒã¿ã®ä¹é¢ãšããåæ§ã®åé¡ãçºçããŸããéçºäžã«ã·ã¹ãã ããã¹ãããããŒã¿ã¯ãã·ã¹ãã ãæ¬çªç°å¢ã§çŽé¢ãããã®ãåæ ããŠããå¿ èŠããããŸããããã§ãªãå Žåãæ¬çªç°å¢ã§ã®ç²ŸåºŠãäœäžããå¯èœæ§ããããŸãã
LLMã®éçºç°å¢ãšæ¬çªç°å¢ã®ä¹é¢ã¯ãæ§é çãªä¹é¢ãšã³ã³ãã³ãããŒã¹ã®ä¹é¢ã®2çš®é¡ãååšããŸãã
æ§é çãªä¹é¢ãšã¯ãããŒã¿åœ¢åŒã®äžäžèŽãæããŸããäŸãã°ããªã¹ãåã®å€ãæã€JSONãã£ã¯ã·ã§ããªãšJSONãªã¹ãã®éããäžè²«æ§ã®ãªãã±ãŒã¹ãã¿ã€ããã¹ãæã®æçãªã©ã®ãšã©ãŒãå«ãŸããŸããLLMã¯ç¹å®ã®ããŒã¿åœ¢åŒã§ãã¬ãŒãã³ã°ãããŠãããããã³ãããäºçްãªå€æŽã«éåžžã«ææã§ããããããããã®ãšã©ãŒã¯äºæž¬äžå¯èœãªã¢ãã«ããã©ãŒãã³ã¹ã«ã€ãªããå¯èœæ§ããããŸããäžæ¹ãã³ã³ãã³ãããŒã¹ãŸãã¯ãæå³çããªä¹é¢ãšã¯ãããŒã¿ã®æå³ãã³ã³ããã¹ãã®éããæããŸãã
åŸæ¥ã®æ©æ¢°åŠç¿ãšåæ§ã«ãLLMã«ãããŠãå ¥å/åºåãã¢ã®ä¹é¢ã宿çã«æž¬å®ããããšãéèŠã§ããå ¥åãšåºåã®é·ãããJSONãXMLãªã©ã®ç¹å®ã®æžåŒèŠä»¶ãšãã£ãåçŽãªææšãçšããããšã§ã倿Žã远跡ããããã®ç°¡åãªæ¹æ³ãšãªããŸããããã«é«åºŠãªããªããæ€åºã§ã¯ãå ¥å/åºåãã¢ã®åã蟌ã¿ãã¯ã©ã¹ã¿ãªã³ã°ããããšã§ããŠãŒã¶ãŒãè°è«ããŠãããããã¯ã®ã·ãããªã©ã®æå³çãªããªãããæ€åºããããšãã§ããŸããããã¯ãã¢ãã«ããããŸã§ã«è§Šããããšã®ãªãé åãæ¢çŽ¢ããŠããããšã瀺åããŠããå¯èœæ§ããããŸãã
ããã³ãããšã³ãžãã¢ãªã³ã°ãªã©ã®å€æŽããã¹ãããéã¯ãããŒã«ãã¢ãŠãããŒã¿ã»ãã(蚳蚻: åŠç¿ããŒã¿ãšãã¹ãããŒã¿ãåãããããŒã¿ã»ããã®ããš)ãææ°ã®ãã®ã§ãããæãæè¿ã®ãŠãŒã¶ãŒã€ã³ã¿ã©ã¯ã·ã§ã³ã®çš®é¡ãåæ ããŠããããšã確èªããŠãã ãããäŸãã°ãæ¬çªç°å¢ã®å ¥åã§ã¿ã€ããã¹ãäžè¬çãªå ŽåãããŒã«ãã¢ãŠãããŒã¿ã«ãã¿ã€ããã¹ãååšããå¿ èŠããããŸãã
ãŸããåãªãæ°å€çãªä¹é¢ã®æž¬å®ãè¶ ããŠãåºåã«å¯ŸããŠå®æ§çãªè©äŸ¡ãè¡ãããšã倧äºã§ããã¢ãã«ã®åºåã宿çã«ã¬ãã¥ãŒããããšïŒéç§°ããã€ããã§ãã¯ãïŒã«ãããçµæãæåŸ ãšäžèŽãããŠãŒã¶ãŒã®ããŒãºã«ããããç¶ããŠããããšã確èªã§ããŸãã
æåŸã«ãä¹é¢ãã§ãã¯ã«éæ±ºå®æ§ãçµã¿èŸŒãããšãæçšã§ãããã¹ãããŒã¿ã»ããã®åå ¥åã«å¯ŸããŠãã€ãã©ã€ã³ãè€æ°åå®è¡ãããã¹ãŠã®åºåãåæããããšã§ãçšã«ããçºçããªãç°åžžãæ€åºããå¯èœæ§ãé«ããªããŸãã
2.1.2 LLMã®å ¥åãšåºåã®ãµã³ãã«ãæ¯æ¥ç¢ºèªãã
LLMã¯åçã§åžžã«é²åããŠããŸãããŒãã·ã§ããèœåãåªããŠãããå€ãã®å Žåãåºåãçºããã®ã¯æ¥œãããã®ã§ãããLLMãèµ·ãããšã©ãŒã¯éåžžã«äºæž¬äžå¯èœã§ããç¬èªã®ã¿ã¹ã¯ã§ã¯ãLLMãã©ã®ããã«æ©èœããããçŽæçã«çè§£ããããã«ãããŒã¿ãµã³ãã«ã宿çã«ã¬ãã¥ãŒããããšãäžå¯æ¬ ã§ãã
æ¬çªç°å¢ããã®å ¥å-åºåãã¢ã¯ãLLMã¢ããªã±ãŒã·ã§ã³ã®ãçŸå°çŸç©ãã§ããã代æ¿ããããšã¯ã§ããŸãããæè¿ã®ç ç©¶ã§ã¯ãéçºè ãããå€ãã®ããŒã¿ãšã€ã³ã¿ã©ã¯ã·ã§ã³ããã«ã€ããŠããè¯ããåºåãšãæªããåºåã®èªèãå€åããããšã匷調ãããŠããŸãïŒcriteria driftãšåä»ããããŠããŸãïŒã
éçºè ã¯ãLLMã®åºåãè©äŸ¡ããããã®åºæºãããçšåºŠäºåã«èããããšãã§ããŸããããããã®äºåã«å®çŸ©ãããåºæºã¯å€ãã®å Žåäžå®å šã§ããäŸãã°ãéçºã®éçšã§ãè¯ãå¿çã®ç¢ºçãé«ããæªãå¿çã®ç¢ºçãäžããããã«ããã³ãããæŽæ°ãããããããŸããããã®ãè©äŸ¡ãåè©äŸ¡ãåºæºã®æŽæ°ãã®å埩ããã»ã¹ã¯ã©ãããŠãå¿ èŠã«ãªã£ãŠããŸãããªããªããåºåãçŽæ¥èгå¯ããã«ãLLMã®åäœã人éã®å¥œã¿ãäºæž¬ããããšã¯é£ããããã§ãã
LLMã®æ§èœã广çã«ç®¡çããããã«ã¯ãå ¥åãšåºåããã°ã«èšé²ããããšãäžå¯æ¬ ã§ããæ¯æ¥ãããã®ãã°ã®ãµã³ãã«ããã§ãã¯ããããšã§ãæ°ãããã¿ãŒã³ã課é¡ããã¡æ©ãçºèŠããçŽ æ©ã察å¿ããããšãã§ããŸããæ°ããåé¡ãèŠã€ãã£ãå Žåã¯ãããã«ã¢ãµãŒã·ã§ã³ãè©äŸ¡ãæžãããšãã§ããŸããåæ§ã«ã課é¡ã®å®çŸ©ãæŽæ°ãããå Žåã¯ãè©äŸ¡åºæºã«ããã®å€æŽãåæ ãããå¿ èŠããããŸãããããã®ããã€ããã§ãã¯ãã¯ãã¢ãã«ã®åºåã«åé¡ãããå¯èœæ§ã瀺ãéèŠãªã·ã°ãã«ã§ããã³ãŒããšã¢ãµãŒã·ã§ã³ã䜿ã£ãŠããããã®åé¡ãå®çšçã«ç®¡çããããšãã§ããŸãã
æåŸã«ããã®åãçµã¿ãçµç¹å šäœã§æµžéãããããšã倧åã§ããäŸãã°ããªã³ã³ãŒã«å¯Ÿå¿ã®ããŒããŒã·ã§ã³ã«ãå ¥åãšåºåã®ã¬ãã¥ãŒãã¢ãããŒã·ã§ã³ã远å ãããªã©ã®æ¹æ³ãèããããŸãããã®ããã«ãLLMã®æ§èœç®¡çãæ¥åžžçãªã¿ã¹ã¯ã«çµã¿èŸŒãããšã§ãã¢ãã«ã®å質ãç¶ç¶çã«é«ããŠããããšãã§ããã§ãããã
2.2 ã¢ãã«ã®æäœ
LLM APIãå©çšããããšã§ãå°æ°ã®ãããã€ããŒãæäŸããAIã«é Œãããšãã§ããŸããããã«ã¯å€§ããªã¡ãªãããããäžæ¹ã§ãããã©ãŒãã³ã¹ãã¬ã€ãã³ã·ãŒãã¹ã«ãŒããããã³ã¹ããªã©ã«é¢ãããã¬ãŒããªããååšããŸãããŸããã»ãŒæ¯æã®ããã«æ°ããåªããã¢ãã«ãç»å ŽããŠãããããå€ãã¢ãã«ã廿¢ããæ°ããã¢ãã«ãžç§»è¡ããéã«ã¯ã補åãã¢ããããŒãããæºåãå¿ èŠã§ãã
ãã®ã»ã¯ã·ã§ã³ã§ã¯ãã¢ãã«ãã»ã«ããã¹ãããããèªåã§ç®¡çãããããããšãã§ããªãç°å¢ã«ããããå®å šã«ã³ã³ãããŒã«ã§ããªãæè¡ã䜿ãäžã§ã®æèšãã·ã§ã¢ããŠãããŸãã
2.2.1 äžæµã§ã®æŽ»çšã容æã«ããããã«æ§é åãããåºåãçæãã
ã»ãšãã©ã®å®éã®ãŠãŒã¹ã±ãŒã¹ã§ã¯ãLLMã®åºåã¯ãäœããã®ãã·ãŒã³ãªãŒããã«ãªåœ¢ãä»ããŠäžæµã®ã¢ããªã±ãŒã·ã§ã³ã§äœ¿ãããŸãã
äŸãã°ãäžåç£CRMã®Rechatã§ã¯ãããã³ããšã³ãããŠã£ãžã§ãããã¬ã³ããªã³ã°ããããã«æ§é åãããã¬ã¹ãã³ã¹ãå¿ èŠã§ãããåæ§ã«ãè£œåæŠç¥ã®ã¢ã€ãã¢ãçæããããã®ããŒã«ã§ããBobaã§ã¯ãã¿ã€ãã«ãèŠçŽãåŠ¥åœæ§ã¹ã³ã¢ãæé軞ã®ãã£ãŒã«ããæã€æ§é åãããåºåãå¿ èŠã§ãããæåŸã«ãLinkedInã¯ãLLMã«YAMLãçæãããããšã«ã€ããŠæžããŠããŸããçæãããyamlã¯ãã©ã®ã¹ãã«ã䜿çšããããæ±ºå®ããããã¹ãã«ãåŒã³åºãããã®ãã©ã¡ãŒã¿ãæäŸããããã«äœ¿çšããããããŸãã
ãã®ãããªã¢ããªã±ãŒã·ã§ã³ãã¿ãŒã³ã¯ãPostelã®æ³åã®æ¥µç«¯ãªããŒãžã§ã³ãšåŒã¹ãã§ããã: å ¥åã«ã¯å¯å®¹ã«ïŒä»»æã®èªç¶èšèªïŒãåºåã¯å³å¯ã«ïŒåä»ãããããã·ãŒã³ãªãŒããã«ãªãªããžã§ã¯ãïŒã
çŸåšãInstructorãšOutlinesãšããã©ã€ãã©ãªãã¡ã¯ãLLMããæ§é åãããåºåãåŒãåºãããã®äºå®äžã®æšæºæè¡ãšãªã£ãŠããŸããLLM APIïŒAnthropicãOpenAIãªã©ïŒã䜿çšããŠããå Žåã¯Instructorããã»ã«ããã¹ãã¢ãã«ïŒHuggingfaceãªã©ïŒã䜿çšããŠããå Žåã¯Outlinesã䜿çšããŠãã ããã
2.2.2 ã¢ãã«éã§ã®ããã³ããã®ç§»è¡ã¯ãã®ãããé¢å
æ éã«äœæããããã³ããããããã¢ãã«ã§ã¯éåžžã«ããŸãæ©èœãããã®ã®ãå¥ã®ã¢ãã«ã§ã¯æ©èœããªãããšããããŸããããã¯ãããŸããŸãªã¢ãã«ãããã€ããŒéã§åãæ¿ããå Žåã ãã§ãªããåãã¢ãã«ã®ããŒãžã§ã³éã§ã¢ããã°ã¬ãŒãããå Žåã«ãçºçããå¯èœæ§ããããŸãã
äŸãã°ãVoiceflowã¯ãgpt-3.5-turbo-0301ããgpt-3.5-turbo-1106ã«ç§»è¡ãããšãæå³åé¡ã¿ã¹ã¯ã§10ïŒ ã®ç²ŸåºŠäœäžãçºçããããšãçºèŠããŸãããïŒè©äŸ¡ãçšæããŠãã£ããããã§äºãªããåŸãŸããïŒïŒåæ§ã«ãGoDaddyã¯ãããŒãžã§ã³1106ã«ã¢ããã°ã¬ãŒãããããšã§ãgpt-3.5-turboãšgpt-4ã®ããã©ãŒãã³ã¹ã®å·®ãçž®å°ãããšãããããžãã£ããªæ¹åã®ãã¬ã³ãã芳å¯ããŸãããïŒãããã¯ãç©äºãååãã«æãã人ãªããæ°ããã¢ããã°ã¬ãŒãã«ãã£ãŠgpt-4ã®åªäœæ§ãçž®ãŸã£ãããšãæ®å¿µã«æããããããŸããïŒ
ãããã£ãŠãã¢ãã«éã§ããã³ãããç§»è¡ããå¿ èŠãããå Žåã¯ãåã«APIãšã³ããã€ã³ããå ¥ãæ¿ããã ãã§ã¯æžãŸããæ³å®ããæéããããããšãèŠæããŠãã ããããåãããã³ããã䜿ãã°åçšåºŠãŸãã¯ããè¯ãçµæãåŸãããããšæåŸ ããªãã§ãã ããã
ãŸããä¿¡é Œæ§ã®é«ãèªåè©äŸ¡ãè¡ãããšã§ãç§»è¡ã®ååŸã§ã¿ã¹ã¯ã®ããã©ãŒãã³ã¹ã枬å®ããæåæ€èšŒã«å¿ èŠãªåŽåãæžããããšãã§ããŸãã
2.2.3 ã¢ãã«ã®ããŒãžã§ã³ã管çããåºå®ãã
æ©æ¢°åŠç¿ãã€ãã©ã€ã³ã§ã¯ããäœãã倿Žãããšããã¹ãŠã倿Žããããã®ãåžžã§ããããã¯ãèªåãã¡ã§ãã¬ãŒãã³ã°ããŠããªãLLMã®ãããªã³ã³ããŒãã³ãã«äŸåããŠããŠãç¥ããªããã¡ã«å€æŽãããå¯èœæ§ãããå Žåã«ç¹ã«åœãŠã¯ãŸããŸãã
幞ããªããšã«ãå€ãã®ã¢ãã«ãããã€ããŒã¯ãç¹å®ã®ã¢ãã«ããŒãžã§ã³ïŒgpt-4-turbo-1106ãªã©ïŒã**ãåºå®ã**ãããªãã·ã§ã³ãæäŸããŠããŸããããã«ãããã¢ãã«ã®éã¿ã®ç¹å®ã®ããŒãžã§ã³ã䜿çšã§ãããããã倿Žãããªãããã«ããããšãã§ããŸããæ¬çªç°å¢ã§ã¢ãã«ã®ããŒãžã§ã³ãåºå®ããããšã§ãã¢ãã«ã®åäœãäºæãã倿Žãããã®ãé¿ããããšãã§ããŸããããã«ãããã¢ãã«ã亀æãããšãã«çºçããå¯èœæ§ã®ãããåé·ãªåºåããã®ä»ã®äºæããªããšã©ãŒã«ãã顧客ããã®èŠæ ãé²ãããšãã§ããŸãã
ããã«ãæ¬çªç°å¢ããã©ãŒãªã³ã°ãã€ã€ãææ°ã®ã¢ãã«ããŒãžã§ã³ã䜿çšããç°å¢ãäœãããšãæ€èšããŠãã ãããããã«ãããæ°ãããªãªãŒã¹ãå®å šã«å®éšããã³ãã¹ãã§ããŸãããããã®æ°ããã¢ãã«ããã®åºåã®å®å®æ§ãšåè³ªãæ€èšŒããããæ¬çªç°å¢ã®ã¢ãã«ããŒãžã§ã³ãèªä¿¡ãæã£ãп޿°ã§ããŸãã
2.2.4 ã¿ã¹ã¯ãå®äºã§ããæå°ã®ã¢ãã«ãéžæãã
æ°ããã¢ããªã±ãŒã·ã§ã³ã«åãçµãéã¯ãå©çšå¯èœãªäžã§æåŒ·ã®ã¢ãã«ã䜿çšããããªããŸããããããã¿ã¹ã¯ãæè¡çã«å®çŸå¯èœã§ããããšã確èªããããããå°ããªã¢ãã«ã§åçã®çµæãéæã§ãããã©ãããå®éšãã䟡å€ããããŸãã
å°ããªã¢ãã«ã®å©ç¹ã¯ãã¬ã€ãã³ã·ãŒãšã³ã¹ããäœãããšã§ãã
粟床ãäœããªãå¯èœæ§ããããŸãããCoTãn-shotããã³ãããIn-Context Learningãªã©ã®ææ³ã¯ãå°ããªã¢ãã«ããã®èœå以äžã®ããã©ãŒãã³ã¹ãçºæ®ããã®ã«åœ¹ç«ã¡ãŸããLLM APIã䜿ãã ãã«éãããç¹å®ã®ã¿ã¹ã¯ã«å¯ŸããŠèªåãã¡ã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããããšã§ãããã©ãŒãã³ã¹ãåäžãããããšãã§ããŸãã
ãŸãšãããšãå°ããªã¢ãã«ã䜿çšããŠæ éã«äœæãããã¯ãŒã¯ãããŒã¯ãå€ãã®å Žåãåäžã®å€§ããªã¢ãã«ã®åºåå質ã«å¹æµããããããã¯ãããäžåãããšãã§ããããé«éã§ã³ã¹ããå®ããªããŸãã
äŸãã°ããã®ãã€ãŒãã§ã¯ãHaiku + 10-shotããã³ããããŒãã·ã§ããã®OpusãšGPT-4ãäžåããšãããšããœãŒããå ±æãããŠããŸãã
é·æçã«ã¯ãåºåå質ãã¬ã€ãã³ã·ãŒãã³ã¹ãã®æé©ãªãã©ã³ã¹ãšããŠãããå°ããªã¢ãã«ã䜿çšãããããŒãšã³ãžãã¢ãªã³ã°ã®äŸãããå€ãèŠãããããã«ãªãã®ã§ã¯ãªãããšæåŸ ããŠããŸãã
å¥ã®äŸãšããŠãäžèŠã·ã³ãã«ãªåé¡ã¿ã¹ã¯ãèããŠã¿ãŸããããDistilBERTïŒ6700äžãã©ã¡ãŒã¿ïŒãªã©ã®è»œéã¢ãã«ã¯ãé©ãã»ã©åŒ·åãªããŒã¹ã©ã€ã³ãšãªããŸãã4åãã©ã¡ãŒã¿ã®DistilBARTã¯ããã1ã€ã®åªãããªãã·ã§ã³ã§ãããªãŒãã³ãœãŒã¹ããŒã¿ã§ãã¡ã€ã³ãã¥ãŒãã³ã°ãããšãã»ãšãã©ã®LLMãäžåã0.84ã®ROC-AUCã§ãã«ã·ããŒã·ã§ã³ãç¹å®ã§ããã¬ã€ãã³ã·ãŒãšã³ã¹ãã¯5ïŒ æªæºã§ãã
éèŠãªã®ã¯ãå°ããªã¢ãã«ã軜èŠããªãããšã§ãã巚倧ãªã¢ãã«ããã¹ãŠã®åé¡ã«æå ¥ããã®ã¯ç°¡åã§ãããåµé æ§ãšå®éšãéããããšã§ãããå¹ççãªè§£æ±ºçãèŠã€ããããšãã§ããŸãã
2.3 ãããã¯ã
æ°ããæè¡ã¯æ°ããå¯èœæ§ãæäŸããŸãããåªãããããã¯ããæ§ç¯ããååã¯äžå€ã§ãããããã£ãŠãæ°ããåé¡ãåããŠè§£æ±ºããå Žåã§ãããããã¯ãèšèšã®æé ãåçºæããå¿ èŠã¯ãããŸããã
LLMã¢ããªã±ãŒã·ã§ã³ã®éçºãå å®ãªãããã¯ãéçºã®åºæ¬ã«åºã¥ããŠè¡ãããšã§ãå€ãã®å©çãåŸãããŸããããã«ããçã®äŸ¡å€ããŠãŒã¶ãã¡ã«æäŸã§ããŸãã
2.3.1 ãã¶ã€ã³ãæ©æãã€é »ç¹ã«åãå ¥ãã
ãããã¯ãéçºã®æ©ã段éãããã¶ã€ããŒã«é¢äžããŠãããããšã«ã¯å€§ããªæçŸ©ããããŸãããã¶ã€ããŒã¯ããããã¯ãã®æ§ç¯æ¹æ³ããŠãŒã¶ãŒãžã®æç€ºæ¹æ³ã«ã€ããŠæ·±ãçè§£ããèãããã£ãããäžããŠãããŸãããã¶ã€ããŒã¯åã«èŠãç®ãè¯ãããã ãã®ååšã§ã¯ãããŸããããŠãŒã¶ãŒã€ã³ã¿ãŒãã§ãŒã¹ã®æ¹åã¯ãã¡ãããæ¢åã®ã«ãŒã«ãåºå®èŠ³å¿µãæã¡ç Žã£ãŠã§ãããŠãŒã¶ãŒãšã¯ã¹ããªãšã³ã¹ãã©ã®ããã«åäžãããããããèããŸãã
ãã¶ã€ããŒã®åŒ·ã¿ã¯ããŠãŒã¶ãŒã®ããŒãºãæ§ã ãªåœ¢ã§åæ§æã§ããããšã§ãããã®äžã«ã¯ãä»ã®åœ¢åŒããã解決ãããããã®ããããAIãœãªã¥ãŒã·ã§ã³ã®é©çšæ©äŒã®å€å¯¡ã«ã圱é¿ããŸããAIãããã¯ãã®éçºã«ãããŠããæè¡ãããã§ã¯ãªãããŠãŒã¶ãŒãæ±ããæ©èœãäœéšãäžå¿ã«æ®ããã¹ãã§ãã
èªåèªèº«ã«åããããŠã¿ãŸããããããŠãŒã¶ãŒã¯ãã®ãããã¯ãã«äœãæ±ããŠããã®ãããã®æ©èœã¯ãã£ãããããã«é©ããŠããã®ãããããšããªãŒãã³ã³ããªãŒããªã®ãããããã¯ããŸã£ããå¥ã®ãã®ãããããªãããæ¢åã®ãã¶ã€ã³ãã¿ãŒã³ãåèã«ããããããŠãŒã¶ãŒã®ããŒãºãšã©ãé¢é£ããŠããããèããŠã¿ãŠãã ãããããããç¥èŠã¯ããã¶ã€ããŒãããŒã ã«æäŸã§ãã䟡å€ããè³ç£ãªã®ã§ãã
2.3.2 Human-in-the-Loopçšã®UXããã¶ã€ã³ãã
質ã®é«ãã¢ãããŒã·ã§ã³ãåŸã1ã€ã®æ¹æ³ã¯ãHuman-in-the-LoopïŒHITLïŒããŠãŒã¶ãŒãšã¯ã¹ããªãšã³ã¹ïŒUXïŒã«çµ±åããããšã§ãããŠãŒã¶ãŒããã£ãŒãããã¯ãšä¿®æ£ãç°¡åã«æäŸã§ããããã«ããããšã§ã峿ã®åºåãæ¹åããã¢ãã«ãæ¹åããããã®è²ŽéãªããŒã¿ãåéã§ããŸãã
ãŠãŒã¶ãŒã補åãã¢ããããŒãããŠåé¡ããEã³ããŒã¹ãã©ãããã©ãŒã ã®UXãèšèšããéãããã€ãã®ã¢ãããŒããèããããŸãã
- ãŠãŒã¶ãŒãæåã§æ£ãã補åã«ããŽãªãéžæããLLMãããã¯ãšã³ãã§å®æçã«æ°è£œåããã§ãã¯ããŠèª€ã£ãåé¡ãä¿®æ£ããæ¹æ³ã
- ãŠãŒã¶ãŒãã«ããŽãªãéžæãããLLMãããã¯ãšã³ãã§å®æçã«è£œåãåé¡ããæ¹æ³ïŒãšã©ãŒã®å¯èœæ§ããïŒã
- LLMããªã¢ã«ã¿ã€ã ã§è£œåã«ããŽãªãææ¡ãããŠãŒã¶ãŒãå¿ èŠã«å¿ããŠãããæ€èšŒã»æŽæ°ã§ããæ¹æ³ã
ãããã®3ã€ã®ã¢ãããŒãã¯ããããLLMãæŽ»çšããŠããŸããããŠãŒã¶ãŒäœéšã¯å€§ããç°ãªããŸããæåã®ã¢ãããŒãã§ã¯ããŠãŒã¶ãŒã«åæã®è² æ ãããããLLMã¯åŸåŠçã®ãã§ãã¯ãšããŠæ©èœããŸãã2çªç®ã®ã¢ãããŒãã§ã¯ããŠãŒã¶ãŒã®æéã¯å šãå¿ èŠãããŸããããéææ§ãå¶åŸ¡æ©èœã«æ¬ ããŸãã3çªç®ã®ã¢ãããŒãã¯ãäž¡è ã®ãã©ã³ã¹ãé©åã«åã£ãŠããŸããLLMãäºåã«ã«ããŽãªãææ¡ããããšã§ããŠãŒã¶ãŒã®èªç¥è² è·ã軜æžããåé¡äœç³»ãåŠã¶å¿ èŠããªããªããŸããåæã«ããŠãŒã¶ãŒãææ¡ã確èªã»ç·šéã§ããããã«ããããšã§ãæçµçãªå顿¹æ³ã®æ±ºå®æš©ããŠãŒã¶ãŒã«å§ãããã£ãããšã³ã³ãããŒã«ã§ããããã«ããŠããŸãã
ããã«ã3çªç®ã®ã¢ãããŒãã«ã¯ãã¢ãã«æ¹åã®ããã®èªç¶ãªãã£ãŒãããã¯ã«ãŒããçµã¿èŸŒãŸããŠãããšããå©ç¹ããããŸããé©åãªææ¡ã¯åãå ¥ãããïŒããžãã£ãã©ãã«ïŒãäžé©åãªææ¡ã¯ä¿®æ£ãããŸãïŒãã¬ãã£ãã©ãã«ã®åŸã«ããžãã£ãã©ãã«ïŒããã®ãããªãã£ãŒãããã¯ãéããŠãLLMã¯ç¶ç¶çã«åŠç¿ã»æ¹åããŠããããšãã§ããã®ã§ãã
ãŠãŒã¶ãŒã®ææ¡ã®åãå ¥ããæ€èšŒãããã«åºã¥ãããŒã¿åéã®ãã¿ãŒã³ã¯ãæ§ã ãªã¢ããªã±ãŒã·ã§ã³ã§äžè¬çã«èŠãããŸãã
- ã³ãŒãã£ã³ã°ã¢ã·ã¹ã¿ã³ãïŒãŠãŒã¶ãŒã¯ææ¡ãåãå ¥ããïŒåŒ·ãããžãã£ãïŒãææ¡ãåãå ¥ããäžã§èª¿æŽãå ããïŒããžãã£ãïŒããããã¯ææ¡ãç¡èŠããïŒãã¬ãã£ãïŒããšãã§ããŸãã
- MidjourneyïŒãŠãŒã¶ãŒã¯çæãããç»åãæ¡å€§ããŠããŠã³ããŒãããããšïŒåŒ·ãããžãã£ãïŒãç»åã倿ŽããããšïŒããžãã£ãïŒããããã¯æ°ããç»åã»ããã®çæãæ±ããããšïŒãã¬ãã£ãïŒãã§ããŸãã
- ãã£ãããããïŒãŠãŒã¶ãŒã¯ãããã®å¿çã«å¯ŸããŠèŠªæãäžããïŒããžãã£ãïŒãŸãã¯èŠªæãäžããïŒãã¬ãã£ãïŒããšãã§ããŸãããŸããå¿çãéåžžã«äžé©åã ã£ãå Žåã«ã¯ãå¿çã®åçæãèŠæ±ããããšãã§ããŸãïŒåŒ·ããã¬ãã£ãïŒã
(蚳蚻: äžå¿ã®è£è¶³ãšããŠã芪æãäžãã/äžããããšããã®ã¯ Good/Badãã®ãã£ãŒãããã¯ãããæã®ããããUIã®ããš)
ãããã®ãã£ãŒãããã¯ã«ã¯ãæç€ºçãªãã®ãšæé»çãªãã®ããããŸããæç€ºçãã£ãŒãããã¯ãšã¯ããŠãŒã¶ãŒããããã¯ãã®èŠæ±ã«å¿ããŠæå³çã«æäŸããæ å ±ã®ããšã§ããäžæ¹ãæé»çãã£ãŒãããã¯ãšã¯ããŠãŒã¶ãŒãæç€ºçã«ãã£ãŒãããã¯ãæäŸããªããŠãããã®ã€ã³ã¿ã©ã¯ã·ã§ã³ããåŠç¿ã§ããæ å ±ãæããŸããã³ãŒãã£ã³ã°ã¢ã·ã¹ã¿ã³ããMidjourneyã¯æé»çãã£ãŒãããã¯ã®äŸã§ããã芪æãäžãäžãããã®ã¯æç€ºçãã£ãŒãããã¯ã®äŸã§ãã
ã³ãŒãã£ã³ã°ã¢ã·ã¹ã¿ã³ããMidjourneyã®ããã«ããŠãŒã¶ãŒäœéšãé©åã«èšèšããããšã§ã補åãã¢ãã«ã®æ¹åã«åœ¹ç«ã€å€§éã®æé»çãã£ãŒãããã¯ãåéããããšãã§ããŸãããŠãŒã¶ãŒã«ãšã£ãŠèªç¶ã§è² æ ã®å°ãªã圢ã§ãã£ãŒãããã¯ãåéããããã广çã«æŽ»çšããããšããAIãæŽ»çšããã¢ããªã±ãŒã·ã§ã³ã®éèŠãªéµãšãªãã§ãããã
2.3.3 ããŒãºã®éå±€ã容赊ãªãåªå é äœä»ããã
ãã¢ãæ¬çªç°å¢ã«ç§»è¡ããéã«ã¯ã以äžã®ãããªèŠä»¶ã«ã€ããŠæ€èšãå¿ èŠã§ãã
ä¿¡é Œæ§ïŒ99.9ïŒ
ã®çšŒåæéã確ä¿ããæ§é åãããåºåã«æºæ ããããš
ç¡å®³æ§ïŒæ»æçãNSFWããŸãã¯ãã®ä»ã®æå®³ãªã³ã³ãã³ããçæããªãããš
äºå®ã®äžè²«æ§ïŒæäŸãããã³ã³ããã¹ãã«å¿ å®ã§ãããã§ã£ã¡äžããè¡ããªãããš
æçšæ§ïŒãŠãŒã¶ãŒã®ããŒãºãšãªã¯ãšã¹ãã«é¢é£ããŠããããš
ã¹ã±ãŒã©ããªãã£ïŒã¬ã€ãã³ã·ãŒSLAãæºãããå¿
èŠãªã¹ã«ãŒãããããµããŒãã§ããããš
ã³ã¹ãïŒç¡å¶éã®äºç®ãããããã§ã¯ãªããããã³ã¹ããèæ
®ããããš
ãã®ä»ïŒã»ãã¥ãªãã£ããã©ã€ãã·ãŒãå
¬å¹³æ§ãGDPRãDMAãªã©ãžã®å¯Ÿå¿
ãããã®èŠä»¶ãäžåºŠã«ãã¹ãŠæºããããšãããšããªããªã補åããªãªãŒã¹ã§ããªããªã£ãŠããŸããŸãããã®ãããåªå é äœä»ããéèŠã«ãªããŸãã劥åã§ããªãèŠä»¶ïŒä¿¡é Œæ§ãç¡å®³æ§ãªã©ïŒãæç¢ºã«ããããããæºããããªããã°è£œåã¯æ©èœãããå®çŸäžå¯èœã ãšèªèããããšã倧åã§ãã
ãã€ã³ãã¯ãæå°éã®æããã補åïŒMinimum Lovable ProductïŒãç¹å®ããããšã§ããæåã®ããŒãžã§ã³ã¯å®ç§ã§ãªããŠãæ§ããŸããããšã«ããç«ã¡äžããŠããã£ãŒãããã¯ãåŸãªããæ¹åãéããŠããããšãéèŠãªã®ã§ãã
2.3.4 ãŠãŒã¹ã±ãŒã¹ã«åºã¥ããŠãªã¹ã¯èš±å®¹åºŠã調æŽãã
èšèªã¢ãã«ãšã¢ããªã±ãŒã·ã§ã³ã®ç²Ÿæ»ã¬ãã«ã決å®ããéã¯ããŠãŒã¹ã±ãŒã¹ãšã¿ãŒã²ãããªãŒãã£ãšã³ã¹ãèæ ®ããããšãéèŠã§ããäŸãã°ãå»çãéèã®ã¢ããã€ã¹ãæäŸãã顧客åãã®ãã£ãããããã®å Žåãå®å šæ§ãšæ£ç¢ºæ§ã«ã€ããŠã¯éåžžã«é«ãåºæºãæ±ããããŸãããã¹ãäžé©åãªåºåã¯ãå®éã®è¢«å®³ãåŒãèµ·ãããä¿¡é Œãæãªãå¯èœæ§ãããããã§ãã
äžæ¹ã§ãã¬ã³ã¡ã³ãã·ã¹ãã ãªã©ã®ããã»ã©éèŠã§ã¯ãªãã¢ããªã±ãŒã·ã§ã³ããããã¯ã³ã³ãã³ãåé¡ãèŠçŽãªã©ã®ç€Ÿå åãã¢ããªã±ãŒã·ã§ã³ã§ã¯ãé床ã«å³ããèŠä»¶ãèšãããšã倧ããªäŸ¡å€ãä»å ããããšãªã鲿©ã劚ããã ãã«ãªã£ãŠããŸããããããŸããã
ãã®åŸåã¯ãæè¿ã®a16zã¬ããŒãã§ãææãããŠããŸãïŒäžã®ç»ååç §ïŒãå€ãã®äŒæ¥ããå€éšåãã®LLMã¢ããªã±ãŒã·ã§ã³ãšæ¯ã¹ãŠãå éšåãã®LLMã¢ããªã±ãŒã·ã§ã³ã®å°å ¥ãããè¿ éã«é²ããŠãããšããã®ã§ãã瀟å ã®çç£æ§åäžã®ããã«AIã詊ãããšã§ãçµç¹ã¯ããã³ã³ãããŒã«ãããç°å¢ã§ãªã¹ã¯ç®¡çã®æ¹æ³ãåŠã³ãªããã䟡å€ãåŸãããããã«ãªããŸãããããŠãèªä¿¡ãã€ãããã顧客åãã®ãŠãŒã¹ã±ãŒã¹ã«ãæ¡åŒµããŠãããã§ãããã

äŒæ¥ã«ãããLLMã®å
éšããã³å€éšåãã®ãŠãŒã¹ã±ãŒã¹ã®å²åïŒåºå
žïŒa16zã¬ããŒãïŒ
2.4 ããŒã ãšåœ¹å²
AIã®æŽ»çšãé²ãäžã§ãããŒã ã®åœ¹å²ã責任ã®åæ ãæç¢ºã«ããããšã¯å®¹æã§ã¯ãããŸãããç¹ã«ãAIãšã³ãžãã¢ãšããæ°ãã圹å²ã®äœçœ®ã¥ãã«ã€ããŠã¯ãååãªè°è«ãå¿ èŠã§ãããã ããããã§ã¯è·åã®å®çŸ©ããžã§ããã£ã¹ã¯ãªãã·ã§ã³ã®ææ¡ã«ã¯ç«ã¡å ¥ããããããããŒã å šäœãšããŠã®è²¬ä»»ã®å²ãåœãŠæ¹ã«çŠç¹ãåœãŠãããšæããŸãã
2.4.1 ããŒã«ã§ã¯ãªãããã»ã¹ã«æ³šç®ãã
LLMã®ãããªæ°ãããã©ãã€ã ã«çŽé¢ãããšããœãããŠã§ã¢ãšã³ãžãã¢ã¯ã€ãããŒã«ã«ç®ãè¡ããã¡ã§ãããããããã®ããã«ããŒã«ã§è§£æ±ºãã¹ãåé¡ãããã»ã¹ãèŠèœãšããŠããŸããšãäžå¿ èŠãªè€éããæ±ã蟌ãããšã«ãªããŸããããã¯é·æçã«ã¯ããŒã ã®çç£æ§ãäžããèŠå ãšãªããŸãã
äŸãã°ããã®èšäºã§ã¯ãç¹å®ã®ããŒã«ã䜿ã£ãŠLLMçšã®ããã³ãããèªåçæããæ¹æ³ã玹ä»ãããŠããŸããããããåé¡è§£æ±ºã®æ¹æ³è«ãããã»ã¹ãçè§£ããã«ããããããŒã«ã䜿ããšã³ãžãã¢ã¯ãäœèšãªæè¡çè² åµãæ±ããããšã«ãªããšææãããŠããŸãã
äžå¿ èŠãªè€éãã«å ããŠãããŒã«ã®å€ãã¯ååã«ç¹åãããŠããªããšããåé¡ããããŸããäŸãã°ãäžè¬çãªæ¯æ§ãç°¡æœããããŒã³ãªã©ã®æ±çšçãªè©äŸ¡åšãåãããLLMè©äŸ¡ã€ã³ã»ã¢ã»ããã¯ã¹ããæäŸããLLMè©äŸ¡ããŒã«ã®æ¥çãæé·ããŠããŸããå€ãã®ããŒã ããèªåã®ãã¡ã€ã³ã«ç¹æã®èª²é¡ã«ã€ããŠæ¹å€çã«èããããšãªãããããã®ããŒã«ãæ¡çšããŠããã®ãèŠãŠããŸããã
ãããšã¯å¯Ÿç §çã«ãEvalGenã¯ãåºæºã®èšå®ããããŒã¿ã®ã©ãã«ä»ããè©äŸ¡ã®ãã§ãã¯ãŸã§ãåã¹ãããã§ãŠãŒã¶ãŒãæ·±ãå·»ã蟌ã¿ãªããããã¡ã€ã³åºæã®è©äŸ¡ãäœæããããã»ã¹ããŠãŒã¶ãŒã«æããããšã«éç¹ã眮ããŠããŸãããã®ãœãããŠã§ã¢ã¯ããŠãŒã¶ãŒã以äžã®ãããªã¯ãŒã¯ãããŒã«å°ããŸãã

Shankar, S., et al. (2024). Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences. https://arxiv.org/abs/2404.12272 ããååŸ
EvalGenã¯ãLLMè©äŸ¡ãäœæããããã®ãã¹ããã©ã¯ãã£ã¹ããŠãŒã¶ãŒã«ç€ºããŸãã
- ã³ãŒãã䜿ã£ãã¢ãµãŒã·ã§ã³ãLLM-as-a-JudgeãæŽ»çšããŠããã¡ã€ã³åºæã®ãã¹ããå®çŸ©ããŸãïŒããã³ããããèªåçã«ããŒãã¹ãã©ãããããŸãïŒã
- ãã¹ãããŠãŒã¶ãŒã®å€æãšäžèŽããããšãéèŠã§ããããã«ããããŠãŒã¶ãŒã¯ãã¹ããæå®ãããåºæºãé©åã«ãã£ããã£ããŠããããšã確èªã§ããŸãã
- ã·ã¹ãã ïŒããã³ãããªã©ïŒã«å€æŽããã£ãå Žåã¯ããã¹ããç¹°ãè¿ãè¡ããŸãã
EvalGenã¯ãç¹å®ã®ããŒã«ã«äŸåããããšãªããè©äŸ¡æ§ç¯ããã»ã¹ã®ã¡ã³ã¿ã«ã¢ãã«ãéçºè ã«æäŸããŸãããã®ã³ã³ããã¹ããçè§£ããAIãšã³ãžãã¢ã¯ããã軜éãªããŒã«ãéžæããããç¬èªã®ããŒã«ãæ§ç¯ãããããããšãããéžæããŸãã
ããã³ããã®äœæãšè©äŸ¡ä»¥å€ã«ããLLMã®ã³ã³ããŒãã³ãã¯éåžžã«å€å²ã«ããããããããã§ãã¹ãŠãç¶²çŸ çã«åæããããšã¯ã§ããŸãããããããAIãšã³ãžãã¢ãããŒã«ãæ¡çšããåã«ããŸãããã»ã¹ãçè§£ããããåªããããšãèèŠã§ãã
2.4.2 åžžã«å®éšãè¡ã
MLãããã¯ãã®éçºã«ãããŠãå®éšã¯éåžžã«éèŠãªåœ¹å²ãæãããŸããA/Bãã¹ããã©ã³ãã 忝èŒè©Šéšã ãã§ãªããã·ã¹ãã ã®æå°éã®æ§æèŠçŽ ãé »ç¹ã«å€æŽãããªãã©ã€ã³ã§è©äŸ¡ãè¡ãããšãå®éšã®äžç°ã§ãã誰ããè©äŸ¡ã«ç±å¿ãªçç±ã¯ãå®éã«ã¯ä¿¡é Œæ§ãšèªä¿¡ãåŸãããã§ã¯ãããŸãããããã¯å®éšãå¯èœã«ãããããªã®ã§ãïŒè©äŸ¡ãåªããŠããã°ããã»ã©ãå®éšã®å埩é床ãäžããããã®çµæãã·ã¹ãã ã®æè¯ã®ããŒãžã§ã³ã«ããæ©ãå°éã§ããŸãã
å®éšã®ã³ã¹ãã倧å¹
ã«äžãã£ãããšã§ãåãåé¡ã«å¯ŸããŠç°ãªãã¢ãããŒãã詊ãããšãäžè¬çã«ãªããŸããã
ããŒã¿åéãšã¢ãã«ã®ãã¬ãŒãã³ã°ã«ãããé«ãã³ã¹ãã¯æå°éã«æããããŸããããã³ãããšã³ãžãã¢ãªã³ã°ã®ã³ã¹ãã¯ã人éã®æéããããããã«é«ãã ãã§ããããŒã ã®å
šå¡ãããã³ãããšã³ãžãã¢ãªã³ã°ã®åºæ¬ãåŠã¹ããããªäœå¶ãæŽããŸããããããã«ãããå
šå¡ãå®éšãè¡ãããšã奚å±ãããçµç¹å
šäœãã倿§ãªã¢ã€ãã¢ãåŒãåºãããšãã§ããŸãã
ããã«ãæ¢çŽ¢ã®ããã ãã§ãªããæŽ»çšã®ããã«ãå®éšãè¡ããŸãããïŒ
æ°ããã¿ã¹ã¯ã®ããéäžã®ããŒãžã§ã³ããããŸããïŒããŒã ã®ä»ã®ã¡ã³ããŒã«ãå¥ã®æ¹æ³ã§ã¢ãããŒãããŠãããããšãæ€èšããŠã¿ãŸããããããé«éã«å®è¡ã§ããå¥ã®æ¹æ³ã詊ããŠã¿ãã®ãè¯ãã§ããããCoTãFew shotsãªã©ã®ããã³ããæè¡ã詊ããŠãå質ãé«ããŠãã ãããããããŒã«ãå®éšã®é害ã«ãªã£ãŠããã®ã§ããã°ããããåæ§ç¯ããããæ¹åã®ããã®æè³ãè¡ãããšãæ€èšããŸãããã
æåŸã«ããããã¯ã/ãããžã§ã¯ãã®èšç»æã«ã¯ãè©äŸ¡ã®æ§ç¯ãšè€æ°ã®å®éšã®å®è¡ã«æéãå²ããŠãã ããã
ãšã³ãžãã¢ãªã³ã°è£œåã®ä»æ§ã«ã€ããŠèããéã«ã¯ãè©äŸ¡ã®æç¢ºãªåºæºã远å ããããã«ããŠãã ããããŸããããŒããããã³ã°ã®éã«ã¯ãå®éšã«å¿
èŠãªæéãéå°è©äŸ¡ããªãããæ³šæãå¿
èŠã§ããæ¬çªç°å¢ãžã®ãªãªãŒã¹ãèš±å¯ããããŸã§ã«ãéçºãšè©äŸ¡ãè€æ°åç¹°ãè¿ãå¿
èŠããããšäºæ³ããŠããã¹ãã§ãããã
2.4.3 å šå¡ãæ°ããAIæè¡ã䜿çšã§ããããã«ãã
çæAIã®æ®åã«äŒŽããå°éå®¶ã ãã§ãªããããŒã å šäœããã®æ°ããæè¡ãçè§£ããæŽ»çšã§ããããã«ãªãããšãæãã§ããŸããLLMã®åäœåçïŒã¬ã€ãã³ã·ãŒããšã©ãŒãUXãªã©ïŒã«ã€ããŠçŽæçã«çè§£ããã«ã¯ãå®éã«äœ¿ã£ãŠã¿ãã®ãäžçªã®æ¹æ³ã§ãã幞ããLLMã¯æ¯èŒçã¢ã¯ã»ã¹ãããããã®ã§ãããã€ãã©ã€ã³ã®ããã©ãŒãã³ã¹åäžã®ããã«ã³ãŒãã£ã³ã°ããå¿ èŠã¯ãããŸããã誰ããããã³ãããšã³ãžãã¢ãªã³ã°ãšè©äŸ¡ãéããŠè²¢ç®ã§ããã®ã§ãã
ãããå®çŸããã«ã¯ãæè²ã倧ããªåœ¹å²ãæãããŸããn-shotããã³ããã£ã³ã°ãCoTãªã©ã®ææ³ãã¢ãã«ãæãŸããåºåã«èª¿æŽããã®ã«åœ¹ç«ã€ããã³ãããšã³ãžãã¢ãªã³ã°ã®åºæ¬ããå§ããããšãã§ããŸããç¥èã®ãã人ã¯ãLLMãåºåãçæããéã«èªå·±ååž°ã§ãããªã©ã®ãããæè¡çãªåŽé¢ã«ã€ããŠãæããããšãã§ããŸããã€ãŸããå ¥åããŒã¯ã³ã¯äžŠååŠçãããŸãããåºåããŒã¯ã³ã¯é 次çæãããã®ã§ãããã®çµæãã¬ã€ãã³ã·ãŒã¯å ¥åã®é·ããããåºåã®é·ãã«äŸåããããšã«ãªããŸããããã¯ãUXã®èšèšãããã©ãŒãã³ã¹ã®æåŸ å€ãèšå®ããäžã§éèŠãªèæ ®äºé ãšãªããŸãã
ããã«ãå®è·µçãªå®éšã𿢿±ã®æ©äŒãæäŸããããšãæå¹ã§ããäŸãã°ãããã«ãœã³ãéå¬ããã®ã¯ã©ãã§ãããããããŒã ã«æ°æ¥éã詊éšçãªãããžã§ã¯ãã«åãçµãã§ãããã®ã¯é«ãã€ãããã«èŠãããããããŸãããããã®ææã«ã¯é©ããããã¯ãã§ããããããŒã ã§ã¯ãããã«ãœã³ãéããŠã3幎éã®ããŒãããããããã1幎以å ã«ã»ãŒå®äºãããå€§å¹ ã«å éãããããã§ããå¥ã®ããŒã ã§ã¯ãããã«ãœã³ããã£ããã§ãLLMã«ãã£ãŠå®çŸå¯èœã«ãªã£ããã©ãã€ã ã·ããçãªUXã«çç®ããä»å¹Žä»¥éã®åªå äºé ã«äœçœ®ã¥ãããšãããŸãã
2.4.4 ãAIãšã³ãžãã¢ãªã³ã°ããããã°ããããšããèœãšã穎ã«é¥ããªãã§ãã ãã
æ°ããè·åãäœããããšããã®åœ¹å²ã«é¢é£ããèœåã¯æåã¯é倧è©äŸ¡ãããã¡ã§ãããããã®ä»äºã®å®éã®ç¯å²ãæããã«ãªããšãå€ãã®å Žåãçã¿ã䌎ãä¿®æ£ãè¡ãããŸãããã®åéã®æ°åè ãæ¡çšæ åœè ã¯ãèªåŒµããã䞻匵ãããããéåºŠãªæåŸ ãæ±ããããããããããŸãããéå»10幎éã®é¡èãªäŸãšããŠã¯ãããŒã¿ãµã€ãšã³ãã£ã¹ããæ©æ¢°åŠç¿ãšã³ãžãã¢ïŒMLãšã³ãžãã¢ïŒããããŸãã
- ããŒã¿ãµã€ãšã³ãã£ã¹ãïŒãã©ã®ãœãããŠã§ã¢ãšã³ãžãã¢ãããçµ±èšåŠã«é·ããŠãããã©ã®çµ±èšåŠè ããããœãããŠã§ã¢ãšã³ãžãã¢ãªã³ã°ã«é·ããŠãã人ã
- æ©æ¢°åŠç¿ãšã³ãžãã¢ïŒMLãšã³ãžãã¢ïŒïŒæ©æ¢°åŠç¿ã®ãœãããŠã§ã¢ãšã³ãžãã¢ãªã³ã°äžå¿ã®èŠæ¹
åœåãå€ãã®äººãããŒã¿ãµã€ãšã³ãã£ã¹ãã ãã§ããŒã¿äž»å°ã®ãããžã§ã¯ããéè¡ã§ãããšèããŠããŸããããããå®éã«ã¯ãããŒã¿ãµã€ãšã³ãã£ã¹ãããœãããŠã§ã¢ãšã³ãžãã¢ãããŒã¿ãšã³ãžãã¢ãšååããŠãããŒã¿ãããã¯ãã广çã«éçºã»å±éããå¿ èŠãããããšãæããã«ãªããŸããã
ãã®èª€è§£ã¯ãAIãšã³ãžãã¢ãšããæ°ãã圹å²ã§ãåã³è¡šé¢åããŠãããäžéšã®ããŒã ã§ã¯AIãšã³ãžãã¢ããããã°ãããšèããŠããŸããå®éã«ã¯ãæ©æ¢°åŠç¿ãAIãããã¯ããæ§ç¯ããã«ã¯ãå¹ åºãå°éçãªåœ¹å²ãå¿ èŠã§ããAIãããã¯ãã«é¢ããŠåæ°ç€Ÿã®äŒæ¥ã«ã³ã³ãµã«ãã£ã³ã°ãè¡ã£ãŠããçµéšããããAIãšã³ãžãã¢ãªã³ã°ããããã°ããããšããèœãšã穎ã«é¥ã£ãŠããã±ãŒã¹ãäžè²«ããŠèгå¯ããŠããŸããããã®çµæã補åéçºã®éã«éèŠãªåŽé¢ãèŠèœãšããã¡ã§ããã¢ä»¥äžã«ãããã¯ããæ¡åŒµããã®ã«èŠåŽããããšãå°ãªããããŸããã
äŸãã°ããã€ããã§ãã¯ã§åŸãããã€ã³ãµã€ã以äžã«ãããã¯ããæ¡åŒµããã«ã¯ãè©äŸ¡ãšæž¬å®ãäžå¯æ¬ ã§ãã广çãªè©äŸ¡ã®ããã®ã¹ãã«ã¯ãæ©æ¢°åŠç¿ãšã³ãžãã¢ã«äŒçµ±çã«èŠããã匷ã¿ã®äžéšãšéãªã£ãŠããŸããAIãšã³ãžãã¢ã®ã¿ã§æ§æãããããŒã ã¯ããããã®ã¹ãã«ã«æ¬ ããŠããå¯èœæ§ããããŸããèè ã®äžäººã§ããHamel Husainã¯ãæè¿ã®ããŒã¿ããªããã®æ€åºãšãã¡ã€ã³åºæã®è©äŸ¡ã®èšèšã«é¢ããç ç©¶ã§ããããã®ã¹ãã«ã®éèŠæ§ã瀺ããŠããŸãã
AIãããã¯ããæ§ç¯ããéçšã§å¿ èŠãšãªã圹å²ã®çš®é¡ãšãããããæ±ããããã¿ã€ãã³ã°ã®å€§ãŸããªæµãã¯ä»¥äžã®éãã§ãã
- ãŸãããããã¯ãã®æ§ç¯ã«æ³šåããŸããããã«ã¯AIãšã³ãžãã¢ãå«ãŸãããããããŸããããå¿ é ã§ã¯ãããŸãããAIãšã³ãžãã¢ã¯ãããã¯ãïŒUXããã€ãã©ã€ã³ãªã©ïŒã®ãããã¿ã€ãã³ã°ãšè¿ éãªå埩ã«åœ¹ç«ã¡ãŸãã
- 次ã«ãã·ã¹ãã ã®äœ¿ããæ¹ãèšæž¬ããŠããŒã¿ãåéããããšã§ãé©åãªåºç€ãäœããŸããããŒã¿ã®çš®é¡ãšèŠæš¡ã«å¿ããŠããã©ãããã©ãŒã ãšã³ãžãã¢ãããŒã¿ãšã³ãžãã¢ãå¿ èŠã«ãªãå ŽåããããŸãããŸããåé¡ããããã°ããããã«ãã®ããŒã¿ãç §äŒããã³åæããã·ã¹ãã ãå¿ èŠã§ãã
- æçµçã«ã¯ãAIã·ã¹ãã ã®æé©åã«åãçµãããšã«ãªãã§ããããããã«ã¯ãã¢ãã«ã®ãã¬ãŒãã³ã°ã¯å¿ ãããå«ãŸããŸãããåºæ¬çãªæé ã«ã¯ãã¡ããªã¯ã¹ã®èšèšãè©äŸ¡ã·ã¹ãã ã®æ§ç¯ãå®éšã®å®è¡ãRAGæ€çŽ¢ã®æé©åã確ççã·ã¹ãã ã®ãããã°ãªã©ããããŸããMLãšã³ãžãã¢ã¯ããã«éåžžã«é©ããŠããŸãïŒãã ããAIãšã³ãžãã¢ãããããç¿åŸã§ããŸãïŒãéåžžãåææ¡ä»¶ãšãªãã¹ããããå®äºããŠããªãéããMLãšã³ãžãã¢ãæ¡çšããŠãæå³ããããŸããã
ãããšã¯å¥ã«ãåžžã«ãã¡ã€ã³ã®å°éå®¶ãå¿ èŠã§ããå°ããªäŒæ¥ã§ã¯ãçæ³çã«ã¯åµæ¥ããŒã ããã®åœ¹å²ãæ ãã¹ãã§ãããå€§äŒæ¥ã§ã¯ãããã¯ããããŒãžã£ãŒããã®åœ¹å²ãæããããšãã§ããŸãã
圹å²ã®é²è¡ãšã¿ã€ãã³ã°ãèªèããããšãéèŠã§ããééã£ãã¿ã€ãã³ã°ã§äººæãæ¡çšããïŒäŸãã°ãMLãšã³ãžãã¢ãæ©ãããææã«æ¡çšããïŒããšããŸãã¯ééã£ãé åºã§ãããã¯ããæ§ç¯ããããšã¯ãæéãšãéã®ç¡é§ã§ãããé¢è·ã«ã€ãªãããŸããããã«ããã§ãŒãº1ãš2ã®éã¯ãMLãšã³ãžãã¢ã«å®æçã«ãã§ãã¯ã€ã³ããŠãããããšïŒãã ãããã«ã¿ã€ã ã§æ¡çšããªãããšïŒã§ãäŒæ¥ãé©åãªåºç€ãæ§ç¯ããã®ã«åœ¹ç«ã¡ãŸãã
3 æŠç¥ïŒLLMã䜿ã£ããããã¯ãéçºã§åºãæãããªãããã«
æåãããããã¯ãã«ã¯ãçèã«åºã¥ãèšç»ãšåªå é äœä»ããå¿ èŠã§ãããçµããã®ãªããããã¿ã€ãã³ã°ãææ°ã®ã¢ãã«ãªãªãŒã¹ã®ãã¬ã³ãã远ãããšã¯å¿ èŠãããŸããã
ãã®æåŸã®ã»ã¯ã·ã§ã³ã§ã¯ãåªããAIãããã¯ããæ§ç¯ããããã®æŠç¥çãªäºæã«ã€ããŠèããŸãããŸããããŒã ãçŽé¢ããéèŠãªãã¬ãŒããªãïŒãã€æ§ç¯ãããã€äœã«ãéã䜿ãããªã©ïŒãæ€èšããåæã®LLMã¢ããªã±ãŒã·ã§ã³éçºæŠç¥ã®ããã¬ã€ããã¯ããææ¡ããŸãã
3.1 PMFã®åã«GPUã¯äžèŠ
åªãããããã¯ããéçºããããã«ããããã¯ãã¯åã«ä»ç€Ÿã®APIã®èãã©ãããŒä»¥äžã®ãã®ã§ããå¿ èŠããããŸããããããå察æ¹åã®ééãã¯ããã«ã³ã¹ãããããå¯èœæ§ããããŸããæšå¹Žã¯ãæç¢ºãªãããã¯ãããžã§ã³ãã¿ãŒã²ããåžå ŽããªããŸãŸãã¢ãã«ã®ãã¬ãŒãã³ã°ãšã«ã¹ã¿ãã€ãºã«60åãã«ã®ã·ãªãŒãºAãå«ã倧éã®ãã³ãã£ãŒãã£ãã¿ã«ãè²»ããããŸããããã®ã»ã¯ã·ã§ã³ã§ã¯ãããã«ç¬èªã®ã¢ãã«ã®ãã¬ãŒãã³ã°ã«é£ã³ã€ãããšãééãã§ããçç±ã説æããã»ã«ããã¹ãã£ã³ã°ã®åœ¹å²ã«ã€ããŠèããŸãã
3.1.1 ãŒãããã®ãã¬ãŒãã³ã°ã¯ïŒã»ãšãã©ïŒæå³ããªã
ã»ãšãã©ã®çµç¹ã«ãšã£ãŠãLLMããŒãããäœãããšã¯ããããã¯ããæ§ç¯ããäžã§åççãªéžæè¢ã«ãªãåŸãŸããã
ãããã©ãã»ã©ãšããµã€ãã£ã³ã°ã§ãä»ã®èª°ãããããè¡ã£ãŠããããã«èŠãããšããŠããæ©æ¢°åŠç¿ã€ã³ãã©ã®éçºãšä¿å®ã«ã¯å€ãã®ãªãœãŒã¹ãå¿ èŠã§ããããã«ã¯ãããŒã¿ã®åéãã¢ãã«ã®ãã¬ãŒãã³ã°ãšè©äŸ¡ãããã³ãããã®ãããã€ãå«ãŸããŸãããŸã PMFãæ€èšŒããŠããæ®µéã§ããã°ããããã«æéã䜿ãããšã¯äžæ žè£œåã®éçºãããªãœãŒã¹ãéžãããŠããŸããŸãã
ãŸããããšãèšç®ãªãœãŒã¹ãããŒã¿ãæè¡çãªèœåããã£ããšããŠããäºååŠç¿ãããLLMã®éçºã¯æ°ãæã§æä»£é
ãã«ãªãå¯èœæ§ããããŸãã
éèã¿ã¹ã¯çšã«ç¹å¥ã«ãã¬ãŒãã³ã°ãããLLMã§ããBloombergGPTã®äŸãèŠãŠã¿ãŸãããããã®ã¢ãã«ã¯ãAIãšã³ãžãã¢ãªã³ã°ãã4åãML補åãšç ç©¶ãã5åã®åèš9åã®ãã«ã¿ã€ã åŸæ¥å¡ã«ããè±éçãªåªåã«ããã3630åããŒã¯ã³ã§ããªãã¬ãŒãã³ã°ãããŸãããããã«ããããããã1幎以å
ã«gpt-3.5-turboãšgpt-4ã«ãããã®ã¿ã¹ã¯ã§è¿œãè¶ãããŠããŸããŸããã
ãã®è©±ã¯ãã»ãšãã©ã®å®çšçãªã¢ããªã±ãŒã·ã§ã³ã§ã¯ããã¡ã€ã³åºæã®ããŒã¿ã§ãã£ãŠãããŒãããLLMãäºååŠç¿ããããšããªãœãŒã¹ã®æè¯ã®äœ¿ãæ¹ã§ã¯ãªãããšã瀺åããŠããŸãããã®ä»£ããã«ãããŒã ã¯å©çšå¯èœãªæåŒ·ã®ãªãŒãã³ãœãŒã¹ã¢ãã«ãç¹å®ã®ããŒãºã«åãããŠãã¡ã€ã³ãã¥ãŒãã³ã°ããããšããå§ãããŸãã
ãã¡ããäŸå€ã¯ãããŸããèŒãããäŸã®1ã€ã¯ãã³ãŒãçæãšçè§£ã®ããã«ç¹å¥ã«ãã¬ãŒãã³ã°ãããReplitã®ã³ãŒãã¢ãã«ã§ããããªãã¬ãŒãã³ã°ã«ãããReplitã¯CodeLlama7bãªã©ã®ãã倧ããªãµã€ãºã®ä»ã®ã¢ãã«ãäžåãããšãã§ããŸãããããããä»ã®ããæ§èœã®é«ãã¢ãã«ããªãªãŒã¹ãããã«ã€ããŠãæçšæ§ãç¶æããã«ã¯ç¶ç¶çãªæè³ãå¿ èŠã§ããã
3.1.2 å¿ èŠæ§ã蚌æããããŸã§ãã¡ã€ã³ãã¥ãŒãã³ã°ãè¡ããªã
ã»ãšãã©ã®çµç¹ã«ãšã£ãŠããã¡ã€ã³ãã¥ãŒãã³ã°ã¯æç¢ºãªæŠç¥çæèãããFOMOïŒFear of Missing Out, åãæ®ãããããšãžã®ææïŒã«é§ãç«ãŠãããŠããŸãã
çµç¹ã¯ãããã ã®ã©ãããŒããšããéé£ã«æã¡åãšããšããŠãæ©ãããæ®µéã§ãã¡ã€ã³ãã¥ãŒãã³ã°ã«æè³ããŸããå®éã«ã¯ããã¡ã€ã³ãã¥ãŒãã³ã°ã¯éæ©ã§ãããä»ã®ã¢ãããŒãã§ã¯äžååã§ããããšãçŽåŸãããååãªäŸãåéããåŸã«ã®ã¿èæ ®ãããã¹ããã®ã§ãã
1幎åãå€ãã®äŒæ¥ããã¡ã€ã³ãã¥ãŒãã³ã°ã«è奮ããŠãããšèšã£ãŠããŸãããPMFãèŠã€ããããŒã ã¯ã»ãšãã©ãªããã»ãšãã©ããã®æææ±ºå®ãåŸæããŠããŸãã
ãã¡ã€ã³ãã¥ãŒãã³ã°ãè¡ãã€ãããªããããŒã¹ã¢ãã«ãæ¹åããããã³ã«ç¹°ãè¿ãè¡ãæºåãã§ããŠããããšãæ¬åœã«ç¢ºä¿¡ããå¿ èŠããããŸãã詳ããã¯ãã®èšäºäžã®ãã¢ãã«ã¯ãããã¯ãã§ã¯ãªãããããåãå·»ãã·ã¹ãã ããããã¯ãã§ããããšãLLMOpsãæ§ç¯ããããæ£ããçç±ã§æ§ç¯ããïŒããéãå埩ããåç §ããŠãã ããã
å®éã«ãã¡ã€ã³ãã¥ãŒãã³ã°ãæ£ããéžæãšãªãã®ã¯ãã€ã§ããããïŒ
ããã®ãŠãŒã¹ã±ãŒã¹ããæ¢åã®ã¢ãã«ã®ãã¬ãŒãã³ã°ã«äœ¿çšãããŠããã»ãšãã©ãªãŒãã³ãªãŠã§ãã¹ã±ãŒã«ã®ããŒã¿ã»ããã§ã¯å©çšã§ããªãããŒã¿ãå¿
èŠãšããå ŽåãããããŠãæ¢åã®ã¢ãã«ã§ã¯äžååã§ããããšãå®èšŒããMVPããã§ã«æ§ç¯ããŠããå Žåãã§ãããã ãåãçµãããšã«ãªã£ããšããŠã泚æãå¿
èŠã§ããåªãããã¬ãŒãã³ã°ããŒã¿ãã¢ãã«éçºè
ãããã«å©çšã§ããªãå Žåãã©ããããããå
¥æããã®ã§ããããïŒ
LLMãå©çšããã¢ããªã±ãŒã·ã§ã³ã¯ç§åŠã®èŠæ¬åžã®ãããžã§ã¯ãã§ã¯ãããŸãããããããžã®æè³ã¯ãããžãã¹ã®æŠç¥çç®æšãšç«¶äºäžã®å·®å¥åã«å¯Ÿããè²¢ç®ã«èŠåã£ããã®ã§ãªããã°ãªããŸããã
3.1.3 APIã䜿ãããšããå§ããããã»ã«ããã¹ãã£ã³ã°ãæããªã
LLM APIã«ãããã¹ã¿ãŒãã¢ããã¯ç¬èªã®ã¢ãã«ããŒããããã¬ãŒãã³ã°ããããšãªããèšèªã¢ããªã³ã°æ©èœãæ¡çšããŠçµ±åããããšã以åããã容æã«ãªããŸãããAnthropicã OpenAIãªã©ã®ãããã€ããŒã¯ããããæ°è¡ã®ã³ãŒãã§ãããã¯ãã«AIãçµã¿èŸŒãããšãã§ããäžè¬çãªAPIãæäŸããŠããŸãã
ãããã®ãµãŒãã¹ã䜿çšããããšã§ããããåŽåãæžãã代ããã«é¡§å®¢ã«äŸ¡å€ãæäŸããããšã«éäžã§ããŸããããã«ãããã¢ã€ãã¢ãæ€èšŒããPMFã«åããŠããè¿ éã«å埩æ€èšŒãã§ããŸãã
ãã ããããŒã¿ããŒã¹ãšåæ§ã«ããããŒãžããµãŒãã¹ã¯(ç¹ã«èŠæš¡ãšèŠä»¶ãå¢å ããã«ã€ã)ãã¹ãŠã®ãŠãŒã¹ã±ãŒã¹ã«é©ããŠããããã§ã¯ãããŸããã
å»çãéèãªã©ã®èŠå¶æ¥çã§å¿
èŠãšãããããŸãã¯å¥çŽäžã®çŸ©åãæ©å¯ä¿æèŠä»¶ã«ãã£ãŠå¿
èŠãšããããæ©å¯/ãã©ã€ããŒãããŒã¿ããããã¯ãŒã¯å€ã«éä¿¡ããã«ã¢ãã«ã䜿çšããå¯äžã®æ¹æ³ã¯ãã»ã«ããã¹ãã£ã³ã°ã ããããããŸããã
ããã«ãã»ã«ããã¹ãã£ã³ã°ã¯ãæšè«ãããã€ããŒã«ãã£ãŠèª²ããããã¬ãŒãå¶éãã¢ãã«ã®deprecationã䜿çšå¶éãªã©ã®å¶éãåé¿ããŸããããã«ãã»ã«ããã¹ãã£ã³ã°ã§ã¯ã¢ãã«ãå®å šã«å¶åŸ¡ã§ãããããå·®å¥åãããé«å質ã®ã·ã¹ãã ãæ§ç¯ãããããªããŸãã
ãŸãããã¡ã€ã³ãã¥ãŒãã³ã°ã®ã»ã«ããã¹ãã£ã³ã°ã¯ãå€§èŠæš¡ãªå Žåã«ã³ã¹ããåæžã§ããŸããäŸãã°ãBuzzfeedã¯ããªãŒãã³ãœãŒã¹ã®LLMããã¡ã€ã³ãã¥ãŒãã³ã°ããããšã§ãã³ã¹ãã80ïŒ åæžãããšè¿°ã¹ãŠããŸãã
3.2 å埩ããŠå質ãé«ãã
é·æçã«ç«¶äºåã®ããåªäœæ§ãç¶æããã«ã¯ãLLMãã®ãã®ãè¶ ããŠèãããããã¯ããéç«ãããèŠçŽ ãèŠåºãå¿ èŠããããŸããã¹ããŒãã¯éèŠã§ããããããå¯äžã®åŒ·ã¿ã§ãã£ãŠã¯ãªããŸããã
3.2.1 ã¢ãã«ã¯ãããã¯ãã§ã¯ãªãããããåãå·»ãã·ã¹ãã ããããã¯ãã§ãã
ã¢ãã«ãèªç€Ÿã§æ§ç¯ããŠããªãããŒã ã«ãšã£ãŠãã€ãããŒã·ã§ã³ã®æ¥éãªããŒã¹ã¯æ©æµãšãªããŸããã³ã³ããã¹ããµã€ãºãæšè«èœåãäŸ¡æ Œå¯Ÿæ§èœæ¯ã®åäžã远æ±ããããåªãããããã¯ããæ§ç¯ããããã«ãããSOTAã¢ãã«ããæ¬¡ã®ã¢ãã«ãžãšç§»è¡ããŠããã®ã§ãããã®é²æ©ã¯ãããçšåºŠäºæž¬å¯èœã§ãããªããããšããµã€ãã£ã³ã°ãªãã®ã§ããã€ãŸããã¢ãã«ã¯ã·ã¹ãã å ã§æãèä¹ æ§ã®äœãã³ã³ããŒãã³ãã§ããå¯èœæ§ãé«ããšããããšã§ãã
ãã®ããã代ããã«æ¬¡ã®ãããªæç¶çãªäŸ¡å€ãæäŸãããã®ã«æ³šåããŠãã ããã
- è©äŸ¡ïŒã¢ãã«å šäœã§ã¿ã¹ã¯ã®ããã©ãŒãã³ã¹ã確å®ã«æž¬å®ãããã
- ã¬ãŒãã¬ãŒã«ïŒã¢ãã«ã«é¢ä¿ãªãæãŸãããªãåºåãé²ããã
- ãã£ãã·ã³ã°ïŒã¢ãã«ã®æšè«ãå®å šã«åé¿ããããšã§ãã¬ã€ãã³ã·ãŒãšã³ã¹ããåæžãããã
- ããŒã¿ãã©ã€ãã€ãŒã«ïŒäžèšãã¹ãŠã®å埩çãªæ¹åãä¿é²ãããã
ãããã®ã³ã³ããŒãã³ãã¯ãã¢ãã«èªäœã®æ©èœããããããã¯ãå質ã®ã¢ãŒããäœããŸãã
ããããããã¯ã¢ããªã±ãŒã·ã§ã³å±€ã§ã®æ§ç¯ã«ãªã¹ã¯ããªãããšãæå³ããããã§ã¯ãããŸãããå®éã«äŸ¡å€ãç£ããšã³ã¿ãŒãã©ã€ãºãœãããŠã§ã¢ãæäŸãããå Žåã«ãOpenAIããã®ä»ã®ã¢ãã«ãããã€ããŒã解決ããå¿ èŠã®ããåé¡ã«ãèªãåãçµãã¹ãã§ã¯ãããŸããã
äŸãã°ãäžéšã®ããŒã ã¯ãç¬èªã¢ãã«ããã®æ§é åãããåºåãæ€èšŒããããã®ã«ã¹ã¿ã ããŒã«ã®æ§ç¯ã«æè³ããŸãããããã§ã®æå°éã®æè³ã¯éèŠã§ãããæ·±ãæè³ã¯æéã®è¯ãäœ¿ãæ¹ã§ã¯ãããŸãããOpenAIã¯ããŠãŒã¶ãããã³ããã®äžã§é¢æ°ã®åŒã³åºããæ±ãããšãã«ãæå¹ãªé¢æ°åŒã³åºã(Function Calling)ãåŸãããããã«ããå¿ èŠããããŸãããªããªãããã¹ãŠã®é¡§å®¢ããããæãã§ããããã§ããããã§ã¯ãæŠç¥çãªå å»¶ã°ãããæ¡çšãã絶察ã«å¿ èŠãªãã®ã ããæ§ç¯ãããããã€ããŒããã®æ©èœã¢ããããŒããåŸ ã€ããšãè³¢æã§ãã
3.2.2 å°ããªããšããå§ããŠä¿¡é Œãç¯ã
ãã¹ãŠã®äººãæºè¶³ããããããã¯ããæ§ç¯ããããšããããšã¯ãå¹³å¡ããçã¿åºãã¬ã·ãã§ããé åçãªãããã¯ããäœãã«ã¯ãäŒæ¥ã¯ãŠãŒã¶ãŒãåŒãä»ãç¶ããäœéšãæ§ç¯ããå¿ èŠããããŸãã
ãŠãŒã¶ãŒãå°ããå¯èœæ§ã®ãããããã質åã«çããããšãç®çãšããæ±çšã®RAGã·ã¹ãã ãèããŠã¿ãŸããããç¹å®ã®é åãžã®ç¹åã®æ¬ åŠã¯ãã·ã¹ãã ãææ°ã®æ å ±ã«åªå é äœãä»ãããããã¡ã€ã³åºæã®åœ¢åŒãè§£æããããç¹å®ã®ã¿ã¹ã¯ã®ãã¥ã¢ã³ã¹ãçè§£ãããã§ããªãããšãæå³ããŸãããã®çµæããŠãŒã¶ãŒã¯æµ ãä¿¡é Œã§ããªãäœéšãçµéšããããŒãºãæºããããšãã§ããããã£ãŒã³ã«ã€ãªãããŸãã
ããã«å¯ŸåŠããã«ã¯ãç¹å®ã®ãã¡ã€ã³ãšãŠãŒã¹ã±ãŒã¹ã«çŠç¹ãåœãŠãŠãã ãããåºãæµ ãã§ã¯ãªããæ·±ãæãäžããããšã§ç¯å²ãçµããŸããããã«ããããŠãŒã¶ãŒã«é¿ããã¡ã€ã³åºæã®ããŒã«ãäœæãããŸããç¹å®ã®é åãžã®ç¹åã«ãããã·ã¹ãã ã®æ©èœãšå¶éã«ã€ããŠççŽã«èª¬æããããšãã§ããããã«ãªããŸããã·ã¹ãã ãã§ããããšãšã§ããªãããšã«ã€ããŠéææ§ãæã€ããšã¯ããŠãŒã¶ãŒãã©ãã«æå€§ã®äŸ¡å€ãä»å ã§ããããçè§£ããã®ã«åœ¹ç«ã¡ããããã£ãŠåºåã«å¯Ÿããä¿¡é Œãšèªä¿¡ãç¯ããŸãã
3.2.3 LLMOpsãæ§ç¯ããããæ£ããçç±ã§æ§ç¯ããïŒããéãå埩
DevOpsã¯ãæ ¹æ¬çã«åçŸå¯èœãªã¯ãŒã¯ãããŒãã·ããã¬ããã2ãã¶ããŒã ã®ãšã³ãã¯ãŒã¡ã³ãã«é¢ãããã®ã§ã¯ãããŸãããYAMLãã¡ã€ã«ãæžãããšã«é¢ãããã®ã§ããããŸããã
DevOpsã¯ãäœæ¥ãšãã®çµæã®éã®ãã£ãŒãããã¯ãµã€ã¯ã«ãççž®ããããšã§ããšã©ãŒã§ã¯ãªãæ¹åãèç©ãããããã«ããããšã«é¢ãããã®ã§ãããã®èµ·æºã¯ããªãŒã³ã¹ã¿ãŒãã¢ããéåãçµãŠãéå亀æã®åäžåéãšã«ã€ãŒã³ãéèŠãããªãŒã³çç£ãšããšã¿çç£æ¹åŒã«ããã®ãŒããŸãã
MLOpsã¯ãDevOpsã®åœ¢åŒãMLã«é©å¿ãããŸãããåçŸå¯èœãªå®éšãšãã¢ãã«éçºè ã«ãªãªãŒã¹ãå¯èœã«ãããªãŒã«ã€ã³ã¯ã³ã¹ã€ãŒãããããŸãããããŠãYAMLãã¡ã€ã«ããããŸãã
ããããæ¥çãšããŠMLOpsã¯DevOpsã®æ©èœãæ¡çšããŸããã§ãããã¢ãã«ãšãã®æšè«ãæ¬çªç°å¢ã§ã®ã€ã³ã¿ã©ã¯ã·ã§ã³ã®éã®ãã£ãŒãããã¯ã®ã£ãããçž®ããŸããã§ããã
å¿åŒ·ãããšã«ãLLMOpsã®åéã¯ãããã³ãã管çã®ãããªå°ããªåé¡ãããè©äŸ¡ã«ãã£ãŠé£çµãããå埩ãé»å®³ããããé£ãã課é¡ãžãšçŠç¹ãç§»ããŠããŸãã
ãã§ã«ããã£ãããšã³ãŒãã£ã³ã°ã¢ãã«ã®äžç«çã§ã¯ã©ãŠããœãŒã¹ã®è©äŸ¡ã®ããã®ã€ã³ã¿ã©ã¯ãã£ããªã¢ãªãŒãããããŸã(蚳蚻:ããããChatbot Arenaã®ããš)ãéåçã§å埩çãªæ¹åã®ã«ãŒãã§ãã
LangSmithãLog10ãLangFuseãW&B WeaveãHoneyHiveãªã©ã®ããŒã«ã¯ãæ¬çªç°å¢ã§ã®ã·ã¹ãã ã®çµæã«é¢ããããŒã¿ãåéããŠç §åããã ãã§ãªããéçºãšæ·±ãçµ±åããããšã§ãããã¯ãã®æ¹åã«ç¹ããããšãã§ããŸãããããã®ããŒã«ãå©çšããããç¬èªã®ããŒã«ãæ§ç¯ããŠã¿ãŠãã ããã
3.2.4 ãéã§æã«å ¥ãLLMã®æ©èœã¯èªåã§æ§ç¯ããªã
ã»ãšãã©ã®æåããããžãã¹ã¯LLMããžãã¹ã§ã¯ãããŸãããäžæ¹ã§ãã»ãšãã©ã®ããžãã¹ã«ã¯LLMã«ãã£ãŠæ¹åãããæ©äŒããããŸãã
ãã®ããšã¯ããªãŒããŒããã°ãã°èª€è§£ãããã³ã¹ããå¢å ããå質ãäœäžãããŠã·ã¹ãã ã«LLMãæ§æ¥ã«åŸä»ãããããŸãæããããŠããã¹ããŒã¯ã«ã¢ã€ã³ã³ãå®åããåœç©ã®èæ å¿ã®ãAIãæ©èœãšããŠãªãªãŒã¹ããŸãã
ããè¯ãæ¹æ³ããããŸãããããã¯ãã®ç®æšã«çã«åèŽããã³ã¢æ¥åã匷åããLLMã¢ããªã±ãŒã·ã§ã³ã«æ³šåããŠãã ããã
ããŒã ã®æéãç¡é§ã«ããããã€ãã®èª€ã£ãåãçµã¿ãèããŠã¿ãŸãããã
- ããžãã¹çšã®ã«ã¹ã¿ã ããã¹ãããSQLãžã®æ©èœãæ§ç¯ããã
- ããã¥ã¡ã³ããšå¯Ÿè©±ãããã£ããããããæ§ç¯ããã
- 顧客ãµããŒããã£ãããããã«äŒæ¥ã®ãã¬ããžããŒã¹ãçµ±åããã
äžèšã¯LLMã¢ããªã±ãŒã·ã§ã³ã®ãããŒã¯ãŒã«ãã§ããããããã¯ããéçºããŠããäŒç€Ÿãèªãæ§ç¯ããæå³ã¯ãããŸããã
ãããã¯å€ãã®ããžãã¹ã«ãšã£ãŠäžè¬çãªåé¡ã§ãããææãªãã¢ãšä¿¡é Œã§ããã³ã³ããŒãã³ãã®éã«å€§ããªã®ã£ããããããŸããçŸåšã®Yã³ã³ãããŒã¿ãŒã®æè³å
äŒæ¥ãã¡ã«ãã£ãŠåãçµãŸããŠããäžè¬çãªåé¡ã«è²ŽéãªRïŒDãªãœãŒã¹ãæè³ããããšã¯ç¡é§ã§ãã
ãããé³è ãªããžãã¹ã¢ããã€ã¹ã®ããã«èãããã®ã¯ãçŸåšã®ãã€ãã®æ³¢ã®æ³¡ç«ã€ãããªè奮ã®äžã§ããLLMãã«é¢ãããã®ã¯äœã§ãæå 端ã§ãããã®ããã«æããã©ã®ã¢ããªã±ãŒã·ã§ã³ããã§ã«ãããµãããã®ã§ãããã容æã«èª€è§£ããŠããŸãããã§ãã
3.2.5 AI in the loop; Humans at the center
çŸæç¹ã§ã¯ãLLMãå©çšããã¢ããªã±ãŒã·ã§ã³ã¯è匱ã§ããä¿¡ããããªãã»ã©ã®å®å
šå¯Ÿçãšé²åŸ¡çãªãšã³ãžãã¢ãªã³ã°ãå¿
èŠã§ãããäºæž¬ã¯é£ãããŸãŸã§ãã
ããããå³å¯ã«ç¯å²ãéå®ãããŠããå Žåããããã®ã¢ããªã±ãŒã·ã§ã³ã¯éåžžã«æçšã«ãªãå¯èœæ§ããããŸããã€ãŸããLLMã¯ãŠãŒã¶ãŒã®ã¯ãŒã¯ãããŒãå éããããã®åªããããŒã«ã«ãªããŸãã
LLMããŒã¹ã®ã¢ããªã±ãŒã·ã§ã³ãã¯ãŒã¯ãããŒãå®å šã«çœ®ãæããããšãæ³åããã®ã¯é åçãããããŸãããã仿¥ãæã广çãªãã©ãã€ã ã¯äººéãšã³ã³ãã¥ãŒã¿ã®ã±ã³ã¿ãŠãã¹ïŒã±ã³ã¿ãŠãã¹ãã§ã¹ïŒã§ãã
人éããã¥ãŒãã³ã°ãããLLMæ©èœãšãã¢ã«ãªããšãã¿ã¹ã¯ãè¡ãçç£æ§ãšå¹žçŠåºŠãå€§å¹ ã«åäžããŸããLLMã®ãã©ã°ã·ããã¢ããªã±ãŒã·ã§ã³ã®1ã€ã§ããGitHub CoPilotã¯ããã®ã¯ãŒã¯ãããŒã®åãå®èšŒããŸããã
ãå šäœãšããŠãéçºè ã¯GitHub CopilotãšGitHub Copilot Chatã䜿çšãããšãã³ãŒãã£ã³ã°ãããç°¡åããšã©ãŒãå°ãªããèªã¿ããããåå©çšå¯èœãç°¡æœãä¿å®å¯èœãããã³å埩åããããšæããããããªãã§ã³ãŒãã£ã³ã°ãããããèªä¿¡ãæã£ãŠãããšèšããŸãããã- Mario Rodriguez, GitHub
é·ãéMLã«æºãã£ãŠãã人ã¯ããHuman-in-the-loopãã®ã¢ã€ãã¢ã«é£ã³ã€ããããããŸããããããæ¥ãã§ã¯ãããŸãããHITLæ©æ¢°åŠç¿ã¯ã人éã®å°éå®¶ãMLã¢ãã«ãäºæž¬éãã«åäœããããšãä¿èšŒãããã©ãã€ã ã§ããé¢é£ã¯ããŠããŸãããããã§ã¯ãã£ãšç¹çްãªãã¥ã¢ã³ã¹ã®ããšãææ¡ããŠããŸãã
LLMäž»å°ã®ã·ã¹ãã ã¯ã«ãããŠã¯ãã»ãšãã©ã®ã¯ãŒã¯ãããŒã®äžã§ã¯AIã¯åãªããªãœãŒã¹ã«ãªãã¹ãã§ãã£ãŠãäž»èŠãªãã©ã€ããŒã§ãã£ãŠã¯ãªããŸããã
人éãäžå¿ã«çœ®ããLLMãã©ã®ããã«ãã®ã¯ãŒã¯ãããŒããµããŒãã§ããããå°ããããšã§ããããã¯ããšèšèšã®æ±ºå®ã倧ããå€ãããŸããæçµçã«ããã¹ãŠã®è²¬ä»»ãLLMã«è¿ éã«ãªãã·ã§ã¢ããããšããç«¶åä»ç€Ÿãšã¯ç°ãªããããã¯ããæ§ç¯ããããšã«ãªãã§ããããããã¯ããåªãããããæçšãªããããŠãªã¹ã¯ã®å°ãªããããã¯ãã«ãªããŸãã
3.3 ããã³ãããè©äŸ¡ãããŒã¿åéããå§ãã
åã®ã»ã¯ã·ã§ã³ã§ã¯ãæè¡ãšã¢ããã€ã¹ã®æ°ã ãæäŸããŸãããäžåºŠã«åžåããã®ã¯å€§å€ãªã®ã§ãæå°éã®æçšãªã¢ããã€ã¹ãèããŠã¿ãŸããããããŒã ãLLMãããã¯ããæ§ç¯ãããå Žåãã©ãããå§ããã¹ãã§ããããã
ãã®1幎éã§ãæã ã¯æåããLLMã¢ããªã±ãŒã·ã§ã³ããã©ãäžè²«ããè»è·¡ã確信ããã®ã«ååãªçµéšãç©ã¿ãŸããããã®ã»ã¯ã·ã§ã³ã§ã¯ããã®åºæ¬çãªãå ¥éããã¬ã€ããã¯ã説æããŸãã
æ žå¿ãšãªãã¢ã€ãã¢ã¯ãã·ã³ãã«ã«å§ããå¿ èŠã«å¿ããŠè€éãã远å ããŠããããšã§ããçµéšåãšããŠãæŽç·Žãã®ã¬ãã«ãäžããããšã«ããã®åã®ã¬ãã«ãããå°ãªããšã1æ¡å€ãã®åŽåãå¿ èŠã«ãªããŸãã
ããã§ã¯ããããã念é ã«çœ®ããŠ...
3.3.1 ãŸãã¯ããã³ãããšã³ãžãã¢ãªã³ã°
ããã³ãããšã³ãžãã¢ãªã³ã°ããå§ããŸããããåã®æŠè¡(Tactical)ã®ã»ã¯ã·ã§ã³ã§èª¬æãããã¹ãŠã®æè¡ã䜿çšããŠãã ãããCoTãn-shotã®äŸãæ§é åãããå ¥åãšåºåããæ¡çšããããšã¯ã»ãšãã©ã®å Žåã§è¯ãã¢ã€ãã¢ã«ãªããŸãããã匱ãã¢ãã«ããããã©ãŒãã³ã¹ãçµãåºãããšããåã«ãæã髿§èœãªã¢ãã«ã§ãããã¿ã€ãã³ã°ããŠãã ããã
ããã³ãããšã³ãžãã¢ãªã³ã°ã§ç®çã®ããã©ãŒãã³ã¹ã¬ãã«ãéæã§ããªãå Žåã«ã®ã¿ããã¡ã€ã³ãã¥ãŒãã³ã°ãæ€èšããŠãã ããããã®ã»ã«ããã¹ãã£ã³ã°ã®å¯èœæ§ã¯ããããã€ããŒã®ã¢ãã«ã®äœ¿çšãã§ããªããªãéæ©èœèŠä»¶ïŒããŒã¿ãã©ã€ãã·ãŒãå®å šãªå¶åŸ¡ãã³ã¹ããªã©ïŒãããå Žåã«ããé«ãŸããŸãããã ããåããã©ã€ãã·ãŒèŠä»¶ããã¡ã€ã³ãã¥ãŒãã³ã°çšã®ãŠãŒã¶ãŒããŒã¿ã®äœ¿çšããããã¯ããªããæ³šæããŠãã ããã
3.3.2 è©äŸ¡åºç€ãæ§ç¯ããããŒã¿ãã©ã€ãã€ãŒã«ãéå§ãã
LLMãããã¯ãéçºã«å ¥éããã°ããã®ããŒã ã§ãè©äŸ¡ãå¿ èŠã§ããããããªããšãããã³ãããšã³ãžãã¢ãªã³ã°ã§ååãã©ããããŸãã¯ãã¡ã€ã³ãã¥ãŒãã³ã°ããã¢ãã«ãããŒã¹ã¢ãã«ã«çœ®ãæããæºåãã§ããŠãããã©ãããããããŸããã
广çãªè©äŸ¡ã¯ã¿ã¹ã¯ã«åºæã®ãã®ã§ãããæå³ããããŠãŒã¹ã±ãŒã¹ãåæ ããŠããŸãã
ç§ãã¡ãæšå¥šããæåã®ã¬ãã«ã®è©äŸ¡ã¯ãŠããããã¹ãã§ãããããã®åçŽãªã¢ãµãŒã·ã§ã³ã¯ãæ¢ç¥ãŸãã¯ä»®èª¬ã®èª²é¡ã»ãšã©ãŒãæ€åºããåæã®ãã¶ã€ã³ã®æææ±ºå®ãèããããšã«åœ¹ç«ã¡ãŸãããŸããåé¡ãèŠçŽãªã©ã®ãã®ä»ã®ã¿ã¹ã¯åºæã®è©äŸ¡ãåç §ããŠãã ããã
ãŠããããã¹ããšã¢ãã«ããŒã¹ã®è©äŸ¡ã¯åœ¹ç«ã¡ãŸããã人éã«ããè©äŸ¡ã®å¿ èŠæ§ã«åã£ãŠä»£ãããã®ã§ã¯ãããŸããã人ã ã«ã¢ãã«/ãããã¯ãã䜿çšããŠãããããã£ãŒãããã¯ãæäŸããŠãããããããããã¯ãå®äžçã§ã®ããã©ãŒãã³ã¹ãšãšã©ãŒçãæž¬å®ãããšåæã«ãå°æ¥ã®ã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããããã«äœ¿çšã§ããé«å質ã®ã¢ãããŒã·ã§ã³ä»ãããŒã¿ãåéãããšããäºéã®ç®çãæãããŸããããã«ãããæéãšãšãã«è€å©çã«å¢å ããããžãã£ããªãã£ãŒãããã¯ã«ãŒããã€ãŸãããŒã¿ãã©ã€ãã€ãŒã«ãäœæãããŸãã
- 人éã«ããè©äŸ¡ã«ããã¢ãã«ã®ããã©ãŒãã³ã¹ã®è©äŸ¡ããã³/ãŸãã¯æ¬ é¥ã®çºèŠ
- ã¢ãããŒã·ã§ã³ä»ãããŒã¿ã䜿çšããŠã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ããããããã³ãããæŽæ°ãã
- äžèšãç¹°ãè¿ãð
äŸãã°ãLLMã§çæãããèŠçŽã®æ¬ é¥ãæ€æ»ããéãäºå®ã®ççŸãç¡é¢ä¿ãã¹ã¿ã€ã«ã®æªããç¹å®ãã詳现ãªãã£ãŒãããã¯ã§åæã«ã©ãã«ãä»ããå ŽåããããŸãããããŠããããã®äºå®ã®ççŸã®ã¢ãããŒã·ã§ã³ã䜿çšããŠãã«ã·ããŒã·ã§ã³åé¡åšããã¬ãŒãã³ã°ããããé¢é£æ§ã®ã¢ãããŒã·ã§ã³ã䜿çšããŠé¢é£æ§å ±é ¬ã¢ãã«ããã¬ãŒãã³ã°ãããã§ããŸããå¥ã®äŸãšããŠãLinkedInã¯ããã«ã·ããŒã·ã§ã³ã責任ããAIã®éåãäžè²«æ§ãªã©ãæšå®ããããã®ã¢ãã«ããŒã¹ã®è©äŸ¡åšã®äœ¿çšã«ãããæåã«ã€ããŠä»¥äžã®èšäºã§ç޹ä»ããŠããŸãã
æéãšãšãã«ãã®äŸ¡å€ãè€å©çã«å¢å ããè³ç£ãäœæããããšã§ãè©äŸ¡ã®æ§ç¯ãçŽç²ã«éçšäžã®è²»çšããæŠç¥çæè³ãžãšã¢ããã°ã¬ãŒããããã®éçšã§ããŒã¿ãã©ã€ãã€ãŒã«ãæ§ç¯ããŸãã
3.4 äœã³ã¹ãã®ç¥æ§ãåºãããã¬ã³ã
1971幎ãXerox PARCã®ç ç©¶è ãã¡ã¯ãç§ãã¡ãä»çããŠãããããã¯ãŒã¯ã§ã€ãªãã£ãããŒãœãã«ã³ã³ãã¥ãŒã¿ã®äžçãšããæªæ¥ãäºæž¬ããŸãããã€ãŒãµããããã°ã©ãã£ãã¯ã¹ã¬ã³ããªã³ã°ããããŠã¹ããŠã£ã³ããŠã«è³ããŸã§ããã®æªæ¥ãå¯èœã«ããæè¡ã®çºæã«éèŠãªåœ¹å²ãæãããæªæ¥ã®èªçã«è²¢ç®ããŸããã
ãããã圌ãã¯ã·ã³ãã«ãªåãçµã¿ãè¡ããŸãããéåžžã«æçšã ãïŒäŸãã°ããããªãã£ã¹ãã¬ã€ïŒããŸã çµæžçã§ã¯ãªãïŒã€ãŸãããããªãã£ã¹ãã¬ã€ãé§åããã®ã«ååãªRAMã¯æ°åãã«ãããïŒã¢ããªã±ãŒã·ã§ã³ã調ã¹ãã®ã§ãããããŠããã®æè¡ã®æŽå²çãªäŸ¡æ ŒååïŒã ãŒã¢ã®æ³åã«å£ã£ãŠïŒã調ã¹ããããã®æè¡ãçµæžçã«ãªãææãäºæž¬ããŸããã
LLMæè¡ã«ã€ããŠãããã©ã³ãžã¹ã¿åœããã®äŸ¡æ Œã»ã©æç¢ºãªææšã¯ãããŸããããåãããšãã§ããŸããMassively-Multitask Language UnderstandingããŒã¿ã»ããã®ãããªäººæ°ã®é·å¹Žã®ãã³ãããŒã¯ãšãäžè²«ããå ¥åã¢ãããŒãïŒfive-shotããã³ããã£ã³ã°ïŒãçšããŠã¹ã³ã¢ãç®åºããŸãããããŠããã®ãã³ãããŒã¯ã§ããŸããŸãªããã©ãŒãã³ã¹ã¬ãã«ã®èšèªã¢ãã«ãå®è¡ããã³ã¹ããæéã®çµéãšãšãã«æ¯èŒããŸãã

äžå®ã®ã³ã¹ãã§ã¯ãæ§èœãæ¥éã«åäžããŠããŸããäžå®ã®æ§èœã¬ãã«ã§ã¯ãã³ã¹ããæ¥éã«äžãã£ãŠããŸããå
±èè
ã®Charles Fryeã2024幎5æ13æ¥ã«å
¬éããŒã¿ã䜿çšããŠäœæ
OpenAIã®davinciã¢ãã«ãAPIãšããŠçºè¡šãããŠãã4幎éã§ã100äžããŒã¯ã³ïŒãã®ããã¥ã¡ã³ãã®çŽ100åïŒã®èŠæš¡ã§ãã®ã¿ã¹ã¯ã§åçã®ããã©ãŒãã³ã¹ãçºæ®ããã¢ãã«ãå®è¡ããã³ã¹ãã¯ã$20ãã10ã»ã³ãæªæºã«äžãããŸãããåæžæã¯ããã6ãæã§ããåæ§ã«ã2024幎5æã®æç¹ã§ãAPIãããã€ããŒãŸãã¯èªåèªèº«ãä»ããŠMeta's LLaMA 38Bãå®è¡ããã³ã¹ãã¯ã100äžããŒã¯ã³ãããããã20ã»ã³ãã§ãããChatGPTãå¯èœã«ããã¢ãã«ã§ããOpenAIã®text-davinci-003ãšåçã®ããã©ãŒãã³ã¹ãçºæ®ããŸãã
ãã®ã¢ãã«ãã2023幎11ææ«ã«ãªãªãŒã¹ãããæç¹ã§ã¯ã100äžããŒã¯ã³ãããçŽ$20ã®ã³ã¹ããããããŸãããããã¯ãã ãŒã¢ã®æ³åãåãªãåå¢ãäºæž¬ããåãæéã§ãã18ãæã§2æ¡ã®æžå°ã§ãã
ã§ã¯ãéåžžã«æçšã§ããïŒPark et alã®ãããªãžã§ãã¬ãŒãã£ããªãããªã²ãŒã ãã£ã©ã¯ã¿ãŒãå®çŸããïŒããŸã çµæžçã§ã¯ãªãïŒãã¡ãã§ã¯ãã®ã³ã¹ãã1æéããã$625ãšèŠç©ããããŠããïŒLLMã®ã¢ããªã±ãŒã·ã§ã³ã®äŸãèŠãŠã¿ãŸãããããã®è«æã2023幎8æã«å ¬éãããŠä»¥æ¥ãã³ã¹ãã¯çŽ1æ¡äžããã1æéããã$62.50ã«ãªããŸãããããã«9ãæåŸã«ã¯ã1æéããã$6.25ã«äžãããšäºæ³ãããŸãã
äžæ¹ãããã¯ãã³ã1980幎ã«çºå£²ãããæã仿¥ã®1ãã«ã§ãæ°åããæ°ååãã¬ã€ã§ããã¯ã¬ãžãããè²·ãããšãã§ããŸããã1æéããã6ã²ãŒã ãšããŠèãããšãããã1æéããã6ãã«ã§ãããã®å€§ãŸããªèšç®ãããé åçãªLLMã䜿ã£ãã²ãŒã äœéšã2025幎ã®ããæç¹ã§çµæžçã«å®çŸå¯èœã«ãªãããšã瀺åãããŸãã
ãããã®ãã¬ã³ãã¯æ°ããããããæ°å¹Žåã«å§ãŸã£ãã°ããã§ããããããä»åŸæ°å¹Žéã§ãã®ããã»ã¹ãé ããªãããšãæåŸ ããçç±ã¯ã»ãšãã©ãããŸãããã¢ã«ãŽãªãºã ãšããŒã¿ã»ããã®æ¹åã«ããäœã³ã¹ãã®ææã䜿ãå°œããããšããŠãïŒäŸãã°ãããã³ãã©æ¯ã*ãè¶ ãããããªã¹ã±ãŒãªã³ã°ïŒãããŒã¿ã»ã³ã¿ãŒãšã·ãªã³ã³å±€ã§ã®æ·±å±€ã®é©æ°ãšæè³ãããã®å·®ãåããããšãæåŸ ã§ããŸãã
*蚳蚻ïŒããã³ãã©æ¯ããšã¯ãèšèªã¢ãã«ã®ãã©ã¡ãŒã¿æ°ã«å¯ŸããŠãåŠç¿ã«çšããããŒã¯ã³æ°ã®æ¯çã®ããšããã®æ¯çãçŽ20ããŒã¯ã³/ãã©ã¡ãŒã¿ãè¶ ãããšãã¢ãã«ã®æ§èœåäžãéåãããšèšãããŠããã
ãããŠããããããããæãéèŠãªæŠç¥çäºå®ã§ãã仿¥ã®å®å šã«å®çŸäžå¯èœãªãã¢ãç ç©¶è«æã§ããæ°å¹ŽåŸã«ã¯ãã¬ãã¢ã æ©èœã«ãªãããã®åŸããã«äžè¬çãªãã®ã«ãªãã§ããããç§ãã¡ã¯ããã®ããšã念é ã«çœ®ããŠãã·ã¹ãã ãšçµç¹ãæ§ç¯ãã¹ãã§ãã
4 0ãã1ãžã®ãã¢ã¯ããååãä»ãã1ããNãžã®è£œååã®æ
ããããŸããLLMã®ãã¢ãäœãã®ã¯ãšãŠã楜ããããšã§ããã»ãã®æ°è¡ã®ã³ãŒãããã¯ã¿ãŒããŒã¿ããŒã¹ããããŠæ éã«äœæãããããã³ããã§ãç§ãã¡ã¯âšéæ³âšãäœãåºããŸãããããŠéå»1幎éããã®éæ³ã¯ã€ã³ã¿ãŒããããã¹ããŒããã©ã³ãããã«ã¯å°å·æ©ã«å¹æµãããšèšãããŠããŸããã
æ®å¿µãªãããå®äžçã®ãœãããŠã§ã¢ã®éçšã«æºãã£ãããšããã人ãªã誰ã§ãç¥ã£ãŠããããã«ãå¶åŸ¡ãããç°å¢ã§æ©èœãããã¢ãšãå€§èŠæš¡ã«å®å®ããŠåäœãããããã¯ããšã®éã«ã¯å€§ããªéããããããŸãã
æ³åãããã¢ãäœãããšã¯ç°¡åã«ã§ãããã補ååããã®ã¯éåžžã«é£ããåé¡ããããŸããäŸãã°ãèªåé転ãè»ã1ãããã¯ãèªåé転ãããã¢ã¯ç°¡åã§ãããããã補åã«ããã«ã¯10幎ããããŸãã- Andrej Karpathy
äŸãã°ãèªåé転è»ãèŠãŠã¿ãŸããããæåã«ãã¥ãŒã©ã«ãããã¯ãŒã¯ã§è»ãé転ãããã®ã¯1988幎ã®ããšã§ããã25幎åŸãAndrej Karpathyã¯ãWaymoã§æåã®ãã¢ä¹è»ãããŸããã
æŽã«ãã®10幎åŸãå瀟ã¯ç¡äººé転ã®èš±å¯ãååŸããŸããããããã¿ã€ãããåçšè£œåã«ãªããŸã§ã«35幎ãã®å³ãããšã³ãžãã¢ãªã³ã°ããã¹ããæ¹è¯ãèŠå¶ããã²ãŒã·ã§ã³ãè¡ãããã®ã§ãã
ç£æ¥çãšåŠè¡çã§ã¯ããã®1幎éã®æµ®ãæ²ã¿ã芳å¯ããŠããŸãããæ¥çãšããŠã®LLMã¢ããªã±ãŒã·ã§ã³éçºã®1幎ç®ã§ããç§ãã¡ãåŠãã æèšããè©äŸ¡ãããã³ãããšã³ãžãã¢ãªã³ã°ãã¬ãŒãã¬ãŒã«ãªã©ã®æŠè¡ãããéçšæè¡ãããŒã æ§ç¯ããããŠã©ã®æ©èœãå éšã§æ§ç¯ããããªã©ã®æŠç¥çèŠç¹ãŸã§ã2幎ç®ä»¥éã®çããã®å©ãã«ãªãããšãé¡ã£ãŠããŸããç§ãã¡å šå¡ãããã®ãšããµã€ãã£ã³ã°ãªæ°ããæè¡ãäžç·ã«æ§ç¯ããŠããããã«ã
5 Stay In Touch
ãã®èšäºã圹ã«ç«ã¡ãèšäºãã³ãŒã¹ã掻åã®ã¢ããããŒãã®éç¥ã欲ããå Žåã¯ã以äžã«ãµãã¹ã¯ã©ã€ãããŠãã ããã
èè ã®é£çµ¡å ã¯ãaboutããŒãžã§ãã芧ããã ããŸãã
6 è¬èŸ
ãã®ã·ãªãŒãºã¯ãã°ã«ãŒããã£ããã§ã®äŒè©±ããå§ãŸããŸãããBryanããAIãšã³ãžãã¢ãªã³ã°ã®1幎ããæžããããšæã£ãããšããã£ããã§ãããããŠãâšéæ³âšãèµ·ãããç§ãã¡å šå¡ãä»ãŸã§åŠãã§ããããšãå ±æããããšã«ãªã£ãã®ã§ãã
ç§ãã¡ã¯ãææžã®å€§éšåã®çµ±åãšå šäœçãªæ§æã«å ããå€ãã®æèšããªãŒãããŠãããEugeneã«æè¬ã®æã衚ããããšæããŸããããã«ãäž»èŠãªç·šéè²¬ä»»ãšææžã®æ¹åæ§ã«ã€ããŠãæè¬ããŠããŸãã
ãŸãããã®èšäºã®ãã£ãããäœã£ãŠãããBryanã«ãæè¬ããŠããŸããæŠè¡çãéçšçãæŠç¥çãªã»ã¯ã·ã§ã³ãšãã®å°å ¥éšåã«èšäºãåæ§æããç§ãã¡ãã©ã®ããã«ããŠã³ãã¥ããã£ã«æãå·®ã䌞ã¹ãå©ããããšãã§ãããã«ã€ããŠããã倧ããèããããç§ãã¡ãåŸæŒãããŠãããŸããã
ã³ã¹ããšLLMOpsã«ã€ããŠæ·±ãæãäžããæèšãç¹ã亀ããããšã§ãããäžè²«æ§ããããããã³ã³ãã¯ããªãã®ã«ããŠãããCharlesã«ãæè¬ããŠããŸããããã40ããŒãžã§ã¯ãªã30ããŒãžã§æžãã ã®ã¯ã圌ã®ãããã§ãã
ã¯ã©ã€ã¢ã³ãã«ã¢ããã€ã¹ããæåç·ã«ç«ã£ãŠããHamelãšJasonã®æŽå¯åãã¯ã©ã€ã¢ã³ãããåŸãå¹ åºãäžè¬åã§ããåŠã³ããããŠããŒã«ã«é¢ããæ·±ãç¥èã«ãæè¬ã®æã衚ããŸãã
ãããŠæåŸã«ãè©äŸ¡ãšå³æ Œãªè£œé å®åã®éèŠæ§ãæãåºããã圌女ã®ç ç©¶ãšãªãªãžãã«ã®çµæãæã£ãŠããŠãããShreyaã«ãæè¬ããŠããŸãã
æåŸã«ããã®ã·ãªãŒãºãéããŠåç §ãããŠããã ãããçæ§èªèº«ã®èª²é¡ãšæèšãå¯å€§ã«ãå ±æããŠãã ãã£ããã¹ãŠã®ããŒã ã®çæ§ããããŠãã®ã°ã«ãŒãã«æŽ»çºã«åå ããé¢ãã£ãŠãã ãã£ãAIã³ãã¥ããã£ã®çæ§ã«æè¬ã®æã衚ããŸãã
6.1 èè ã«ã€ããŠ
èè
ã®è©³çްã«ã€ããŠã¯ã玹ä»ããŒãžãã芧ãã ããã
ãã®èšäºã圹ã«ç«ã£ãå Žåã¯ã以äžã®ããã«åŒçšããŠãã ããã
Yan, Eugene, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, and Shreya Shankar. 2024. âApplied LLMs - What Weâve Learned From A Year of Building with LLMsâ. Applied LLMs. 8 June 2024. https://applied-llms.org/.
or
@article{AppliedLLMs2024,
title = {What We've Learned From A Year of Building with LLMs},
author = {Yan, Eugene and Bischof, Bryan and Frye, Charles and Husain, Hamel and Liu, Jason and Shankar, Shreya},
journal = {Applied LLMs},
year = {2024},
month = {Jun},
url = {https://applied-llms.org/}
}
Discussion