
ChatGPTã®ãTemperatureãã¯ãæ¬ç©ã®æž©åºŠã ã£ã â çµ±èšååŠã§ãããçæAIã®ä»çµã¿ãšãã¿ã¹ã¯å¥ã»æé©èšå®ã¬ã€ã
åã質åãããã®ã«ãçããæ¯åéãã®ã¯ãªã?
ChatGPTã«åã質åã2åããããéãçããè¿ã£ãŠããããããªçµéšã¯ãããŸããã?
ãAIã®ã¬ãã£ããæ©å«ãããã®ããªãââããæã£ãŠãã人ã¯å€ãã§ããã§ãå®ã¯ããã®æåã®è£åŽã§ã¯ãç©çåŠã®æç§æžã«èŒã£ãŠããåŒããã®ãŸãŸåããŠããŸãã
ç§ã¯å€§åŠã§éåååŠãšçµ±èšååŠããã£ãŠãããããŸã¯ITäŒæ¥ã§åãäŒç€Ÿå¡ã§ããç§ã®åŠãã§ããç¥èŠãšåèæç®ãåºã«ãããã¥ããAIãšéåããããããã解説ããããšæããŸãã
ãã®èšäºã§ã¯ã
AIã®ãTemperatureããTop-pããäœãããŠããã®ãããå®éã«æ°å€ãèšç®ããªããå³ãšã¢ãã¡ãŒã·ã§ã³ã§çè§£ãã
ãã®çè§£ãããšã«ãããã®ã¿ã¹ã¯ãªãããã®èšå®ã»ãã®æç€ºããèªåã§å€æã§ããããã«ãªã
ãšãããŸã§ããããŸãã髿 ¡æ°åŠã®ææ°é¢æ°ãåããã°èªããŸããåãããªããŠããå³ãçºããã ãã§çŽæã¯ã€ãããããã«äœããŸããã
AIã¯ã次ã®åèªã®ããåŒãããç¹°ãè¿ãæ©æ¢°
çæAIããã£ãŠããããšã¯ãçªãè©°ãããšããã ãã§ãã
ãããŸã§ã®æç« ã«ç¶ããæ¬¡ã®åèªãã®åè£ããããã«ã確çãå²ãæ¯ãããããŠããã®ç¢ºçã§ããåŒããããã
ããšãã°ãåŸèŒ©ã¯ãã«ç¶ãåèªãAIã®å éšã§ã¯ãåè£ããšã«ãã¹ã³ã¢ããèšç®ãããŸããä»®ã«ããã ã£ããšããŸãããã
ç«: ã¹ã³ã¢ 4.0
ç¬: ã¹ã³ã¢ 2.5
人é: ã¹ã³ã¢ 2.0
å ç: ã¹ã³ã¢ 1.5
ç³: ã¹ã³ã¢ 0.5
ãã®ã¹ã³ã¢ã確çã«å€æããŠãããåŒãã§1èªéžã¶ãããã1åèªãã€ç¹°ãè¿ããšæç« ãã§ããŸã(æ£ç¢ºã«ã¯AIãæ±ãåäœã¯ãããŒã¯ã³ããšãããåèªãã现ããåã端ã§ãããæ¬èšäºã§ã¯åããããããåèªããšåŒã³ãŸã)ã
éèŠãªã®ã¯ãAIã®åºåã¯ãå¯äžã®æ£è§£ãã§ã¯ãªãã確çååžããã®æœéžçµæãã ãšããããšãåã質åã§çããå€ããã®ã¯ãæ éã§ã¯ãªã仿§ã§ãã
ãããŠãã¹ã³ã¢ã確çã«å€æãããšããååžã®ãå°ãå ·åããæ±ºãããããã Temperature ã§ãã
çŸèã¯äžèŠã«ããã:枩床ã§ååžãããå€ãã
èšèã§èª¬æããåã«ãèŠãŠãã ãããäžã®5ã€ã®åè£ã«ã€ããŠã枩床ãå€ãããšç¢ºçååžãã©ãå€åœ¢ãããã®ã·ãã¥ã¬ãŒã·ã§ã³ã§ãã

枩床ãäžãããšã1äœã®ãç«ãã«ç¢ºçãéäžããŠãã(T=0.2ã§99.9%)
枩床ãäžãããšãååžãå¹³ãã«ãªãããç³ãã®ãããªå€ãªåè£ã«ãåœéžãã£ã³ã¹ãçãŸãã
ãäœæž©=å å®ã§æ¯ååããã髿ž©=倿§ã ãã©è±ç·ããããšããçµéšåã®æ£äœã¯ããã®ååžã®å€åœ¢ã§ãã
Temperatureã®æ£äœ:ãã«ããã³ååž
ãããããããã®èšäºã®çæ¿ã§ãã
çµ±èšååŠã«ã¯ããç±å¹³è¡¡ã«ããç³»ãããšãã«ã®ãŒEã®ãã1ã€ã®ç¶æ ãåã確çãã衚ãæåãªåŒããããŸãããã«ããã³ååžã§ãã
ç¶æ ã®ç¢ºç â exp(âE/T)
æ°Žãåããšååããã£ã¡ãæŽåã(äœæž©=1ã€ã®å®å®é 眮ã«éäž)ãæ²žéš°ãããšååãæŽãåã(髿ž©=倿§ãªé 眮ã«åæ£)ããã®ãäœæž©ã»ã©å®å®ã«éäžã髿ž©ã»ã©ä¹±éããšããåŸåã®ãåå°ã«ããåŒã§ã(ãªããæ°Žã0âã§ããããšåºãŸããããªæ¥å€ã¯ãååå士ã®ååã§èµ·ãããçžè»¢ç§»ããšããçŸè±¡ã§ãAIã®æž©åºŠãããã§ã¯èµ·ããŸãããå€åã¯ãªãããã§ã)ã
äžæ¹ãçæAIãã¹ã³ã¢ã確çã«å€æããåŒ(ãœããããã¯ã¹é¢æ°)ã¯ããã§ãã
åèªã®ç¢ºç â exp(ã¹ã³ã¢/T)

ããšãã«ã®ãŒã®ç¬Šå·ãã²ã£ããè¿ãããã®ãã¹ã³ã¢ããšèªã¿æ¿ããã°ã2ã€ã¯å®å šã«åãåŒã§ããAIã®èšå®ç»é¢ã®ã枩床ãã¯ãç©çããã®çŽèŒžå ¥ãªãã§ããããã¯å¶ç¶ã§ã¯ãªããAIã®æŽå²ã«ã¯ç©çåºèº«ã®éå ·ããŽããŽããããŸã(ãã«ããã³ãã·ã³ãçŒããªãŸãæ³ããããŠ2024幎ã®ããŒãã«ç©çåŠè³ã¯ãã¥ãŒã©ã«ãããç ç©¶ã®ããããã£ãŒã«ããšãã³ãã³ã«åºãŸãã)ã
ã€ãŸãã
Temperatureãäžãã = ç³»ãå·ãã = æå®å®ç¶æ (æé«ã¹ã³ã¢ã®åèª)ã«ç¢ºçãéäž
Temperatureãäžãã = ç³»ãæž©ãã = é«ãšãã«ã®ãŒç¶æ (äœã¹ã³ã¢ã®åèª)ã«ããã£ã³ã¹ãåã
ãåµé çã«ããããã°æž©åºŠãäžããããšããçµéšåã¯ãç©çã§èšãã°ãååãæŽããããããã°å ç±ãããããã ã®ç±ååŠã§ãã
åŒãã¡ãããšæžããš(æ°åŒãèŠæãªãèªã¿é£ã°ãOK)
ãâ(æ¯äŸ)ãã§ããŸãããã«ã忝ãŸã§æžããšãããªããŸããåè£iãéžã°ãã確çã¯ã
p(i) = exp(s(i)/T) ÷ Σ exp(s(j)/T)

忝ã¯ãå šåè£ã®éã¿ã®åèšãã確çã¯å šéšè¶³ããŠ1ã«ãªããªããšãããªãã®ã§ãåèšã§å²ã£ãŠããŸã(æ£èŠå)ãç©çã§ã¯ãã®åæ¯ãåé 颿°ZãšåŒã³ãŸããåŠéšã®çµ±èšååŠã¯ãããã«Zãèšç®ããããã«å€§åã®æéã䜿ããããããã䞻圹ã®éã§ãã
AIã®ãœããããã¯ã¹ãšãã«ããã³ååžã®éãã¯ã(1)ãšãã«ã®ãŒã®ç¬Šå·ãã¹ã³ã¢ã§ã¯éåããªããšã(2)ç©çã§ã¯ææ°ã®äžèº«ãE/(kBT)ãšãããç¡æ¬¡å ã®æ¯ããªã®ã«å¯ŸããAIã§ã¯ã¹ã³ã¢ã枩床ããã ã®æ°ãªã®ã§ãåäœãããããä¿æ°kBãèŠããªãããš(kB=1ãšçœ®ããã®ãšåãã§ã)ããã®2ç¹ã ãã§ã圢ã¯å®å šã«åãã§ããã ããç©çã§å¹ã£ãçŽæââãäœæž©ã§åããã髿ž©ã§ä¹±ããããåé 颿°ãå šãŠãç¥ã£ãŠãããââãããã®ãŸãŸAIã®æåã«äœ¿ããŸãã
察å¿ã®çŸããããã1ã€ãsoftmaxã¯ãå šåè£ã®ã¹ã³ã¢ã«åãæ°ãäžæã«è¶³ããŠãã確çã1ããªãå€ãããŸãã(忝ãšååã§ãã£ã³ã»ã«ããŸã)ãããã¯ç©çã®ããšãã«ã®ãŒã®åç¹(ãŒãç¹)ã¯ã©ãã«åã£ãŠãããããšãŸã£ããåãæ§é ã§ããã€ãŸãã¹ã³ã¢ã¯çµ¶å¯Ÿå€ã§ã¯ãªããå·®ã ãã«æå³ããããŸãã
ãããŸã§ããTemperatureã®æ£äœãã§ãããããããå ã¯ããã®çè§£ãå®åã«èœãšããŸãã
å®éã«æ°å€ãèšç®ããŠã枩床ã§ç¢ºçãäœ%ããäœ%ã«åãããå šéšèŠã(å³ã€ã)
ãæž©åºŠ0ãªãæ£ç¢ºããå±éºãªèª€è§£ã§ããçç±
Top-p/Top-kãååžã®äœãåã£ãŠããã®ã(å³ã€ã)
ååžã®ãä¹±éããããšã³ããããŒã§å®éåãã(ãããç©çåºèº«ã®æ¬é ã§ã)
ã¿ã¹ã¯å¥ã»èšå®ããŒãã·ãŒã(ä¿åçšã®è¡šç»åã€ã)
èšå®ãããããªãChatGPT/Claudeã®ç»é¢ã§ãå®è³ªçã«æž©åºŠãæããèšãåãéãšã5åã§ã§ããèªè å®éš
ã²ãšã€ã ãå åºããããšââæž©åºŠã1/5ã«ããã ãã§ã1äœåè£ã®ç¢ºçã¯68%ãã99.9%ãŸã§è·³ãäžãããŸãããŸãã¯ãã®ãå šæ°å€ããããèŠãããŸãã