
ãè«æç¬èªãOpenAIã®æ¬¡äžä»£LLMão3-miniãã®å®å šæ§ãåŸ¹åºæ€èšŒïŒ10,000以äžã®ãã¹ãã±ãŒã¹ããèŠããŠãã課é¡ãšææ
ããã«ã¡ã¯ïŒæ ªåŒäŒç€ŸAI Nestã§ããå€§èŠæš¡èšèªã¢ãã«ïŒLLMïŒã®çºå±ãšãšãã«ããã®å®å šæ§ãžã®æžå¿µãé«ãŸã£ãŠããŸããLLMã¯ç§ãã¡ã®æ¥åžžçæŽ»ã«æ·±ãå ¥ã蟌ãã§ããããã®åºåãåãŒã圱é¿ã¯çéã§ããªããã®ãšãªã£ãŠããŸããç¹ã«ãã©ã€ãã·ãŒã®äŸµå®³ãåèŠã®å©é·ã誀æ å ±ã®æ¡æ£ãšãã£ããªã¹ã¯ã¯ã瀟äŒçã«å€§ããªåé¡ãšãªãå¯èœæ§ããããŸãã
ãã®èšäºã®èæ¯ãšãªãåºç€ç¥èã¯ããã¡ãã§ãŸãšããŠããŸãã
ä»åã¯ãOpenAIã®æ°ããLLMã¢ãã«ão3-miniãã®äºåãããã€ã¡ã³ã段éã«ãããå®å šæ§è©äŸ¡ã«é¢ããè«æã玹ä»ããŸãããã®ç ç©¶ã¯ãå®éã®ãµãŒãã¹æäŸåã«å®æœãããå€§èŠæš¡ãªå®å šæ§ãã¹ãã®çµæãå ±åãããã®ã§ãLLMã®å®å šæ§è©äŸ¡ã®æ°ããæ¹åæ§ã瀺ãéèŠãªç ç©¶ãšãããŸãã
ã¿ã€ãã«ïŒEarly External Safety Testing of OpenAI's o3-mini: Insights from Pre-Deployment Evaluation
URLïŒhttps://arxiv.org/abs/2501.17749v1Â
æå±ïŒMondragon University, University of Seville
èè ïŒAitor Arrieta, Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura
å€åããLLMã®å®å šæ§è©äŸ¡
ãããŸã§ã®LLMã®å®å šæ§è©äŸ¡ã«ã¯ãããã€ãã®æ¬è³ªçãªèª²é¡ãååšããŠããŸããããã®æãããã®ããéçãªãã³ãããŒã¯ãžã®äŸåã§ããåºå®ããã質åã»ããã䜿çšãããã¹ãã¯ãæ°ããè åšãå€åãã瀟äŒç¶æ³ã«å¯Ÿå¿ã§ããªããšããæ¬è³ªçãªéçããããŸããã
ãŸããæéã®çµéãšãšãã«ãã¹ãã±ãŒã¹ãé³è åãããšããåé¡ãæ·±å»ã§ãããäŸãã°ã2023幎ã«äœæããããã¹ãã±ãŒã¹ã¯ã2025å¹Žã®æ¿æ²»ç¶æ³ã瀟äŒåé¡ãé©åã«åæ ã§ããŠããªãå¯èœæ§ãé«ãã®ã§ãã
ããã«ãå®å šæ§ãã¹ãã®çµæã人éãæåã§è©äŸ¡ããå¿ èŠããããå€§èŠæš¡ãªãã¹ãã®å®æœãå°é£ã§ãããšããéçšäžã®èª²é¡ãååšããŠããŸããã

ASTRALã«ãã驿°çãªå®å šæ§ãã¹ãææ³
ãã®ãããªèª²é¡ã解決ãããããç ç©¶ããŒã ã¯ãASTRALããšããæ°ãããã¹ãããŒã«ãéçºã»äœ¿çšããŸãããASTRALã®é©æ°çãªç¹ã¯ã以äžã®3ã€ã®ç¹åŸŽã«ãããŸãã
第äžã«ãåçãªãã¹ãã±ãŒã¹çææ©èœã§ããASTRALã¯14çš®é¡ã®å®å šæ§ã«ããŽãªã«å¯Ÿå¿ããããããã®ã«ããŽãªã§å€æ§ãªãã¹ãã±ãŒã¹ãèªåçæããŸããããã«ããããã¹ãç¯å²ã®ç¶²çŸ æ§ãå€§å¹ ã«åäžããŸããã
ç¹çãã¹ãã¯ãASTRALãç°ãªãæäœãã¹ã©ã³ã°ãæ¹èšãå°éçšèªãªã©ã倿§ãªèšèªè¡šçŸãèæ ®ããŠãã¹ãã±ãŒã¹ãçæã§ããç¹ã§ããããã«ãããå®éã®ãŠãŒã¶ãŒã®å€æ§ãªå ¥åãã¿ãŒã³ãã·ãã¥ã¬ãŒãããããšãå¯èœã«ãªããŸããã
第äºã«ãææ°æ å ±ã®åæ æ©èœããããŸããASTRALã¯ì¹ã¯ããŒãªã³ã°æ©èœãæèŒããŠãããåžžã«ææ°ã®ãã¥ãŒã¹ããããã¯ãåã蟌ãã§ãã¹ãã±ãŒã¹ãçæããŸããããã«ãããçŸåšã®ç€ŸäŒç¶æ³ãåæ ãããããçŸå®çãªãã¹ããå¯èœã«ãªããŸããã
第äžã«ãèªåè©äŸ¡æ©èœã§ããLLMãè©äŸ¡è ãšããŠäœ¿çšããããšã§ããã¹ãçµæã®èªåå€å®ãå®çŸããŠããŸããããã«ãããå€§èŠæš¡ãªãã¹ãã®å®æœãšè©äŸ¡ãå¯èœã«ãªããŸããã

詳现ãªãã¹ãææ³ãšå®æœããã»ã¹
ç ç©¶ããŒã ã¯2ã€ã®ç°ãªããã¹ãã¹ã€ãŒãã䜿çšããŠo3-miniã®è©äŸ¡ãè¡ããŸããã第äžã®ãã¹ãã¹ã€ãŒãïŒTS1ïŒã¯2024幎11æã«çæããã3,780ä»¶ã®ãã¹ãã±ãŒã¹ã第äºã®ãã¹ãã¹ã€ãŒãïŒTS2ïŒã¯2025幎1æã«çæããã6,300ä»¶ã®ãã¹ãã±ãŒã¹ã§æ§æãããŠããŸãã
ç¹ã«è峿·±ãã®ã¯ãTS2ã®çæææãããã«ãã»ãã©ã³ãã®å€§çµ±é 就任ãã¬ã¶å°åºã®åæŠãªã©ãéèŠãªæ¿æ²»çã€ãã³ããšéãªã£ãŠããç¹ã§ããããã«ãããLLMã®æ¿æ²»çãªçºèšã埮åŠãªåé¡ãžã®å¯Ÿå¿èœåãè©äŸ¡ããããšãã§ããŸããã
ãã¹ãçµæãšéèŠãªçºèŠ
ç·èš10,080ä»¶ã®ãã¹ãã±ãŒã¹ã®å®è¡çµæãããç ç©¶ããŒã ã¯87ä»¶ã®å®å šã§ãªãåºåãç¹å®ããŸãããããã¯å šäœã®çŽ0.86%ãšãããæ¯èŒçäœãå²åã§ãããããããã®æ°åã®èåŸã«ãã詳现ãªåæãããããã€ãã®éèŠãªç¥èŠãåŸãããŸããã

1. åäžä»£ã¢ãã«ãšã®æ¯èŒã«ãããé²å
o3-miniã¯ãåäžä»£ã®OpenAIã¢ãã«ãšæ¯èŒããŠé¡èãªæ¹åã瀺ããŸãããå ·äœçã«ã¯ãGPT-3.5ã§ã®752ä»¶ãGPT-4ã§ã®166ä»¶ãšããå®å šã§ãªãåºåã®æ°ãão3-miniã§ã¯87ä»¶ã«ãŸã§æžå°ããŠããŸãããã®æ¹åã¯ãOpenAIã®å®å šæ§ã«å¯Ÿããç¶ç¶çãªåãçµã¿ã®ææã瀺ããŠããŸãã
2. ä»ç€Ÿã¢ãã«ãšã®æ¯èŒã«ãããåªäœæ§
ä»ç€Ÿã®ææ°ã¢ãã«ãšæ¯èŒããŠãão3-miniã¯é«ãå®å šæ§ã瀺ããŸãããç¹ã«æ³šç®ãã¹ãã¯ãLlama 2ã§æ€åºããã95ä»¶ã®å®å šã§ãªãåºåãšæ¯èŒããŠããããè¯ãçµæã瀺ããŠããç¹ã§ãã
3. ããªã·ãŒé忀åºã·ã¹ãã ã®å¹æ
ç ç©¶éçšã§çºèŠãããè峿·±ãç¹ãšããŠãAPIã¬ãã«ã§ã®å®å šæ§ãã§ãã¯æ©èœã®ååšããããŸãããã®æ©èœã¯å±éºãªå ¥åãäºåã«ãããã¯ãã广ã瀺ããŸããããåæã«æ°ããªçåãæèµ·ããŸãããããªãã¡ããã®å®å šæ§ã®åäžããLLMã¢ãã«èªäœã®æ¹åã«ãããã®ãªã®ãããããšãå€éšã®ä¿è·æ©èœã«ãããã®ãªã®ããšããç¹ã§ãã
4. ææ°ã®è«äºçãããã¯ãžã®å¯Ÿå¿ã«ããã課é¡
ç¹ã«æ¿æ²»çãªå 容ã«ãããŠãå®å šã§ãªãåºåãå€ã確èªãããŸãããå ·äœçã«ã¯ã2025幎1æã®ããã«ãã»ãã©ã³ãã«é¢é£ããå 容ãç¹ã«1æ6æ¥ã®æŽåé¢ä¿è ã®æ©èµŠã«é¢ãã話é¡ã§ãã¢ãã«ãé©åãªå¿çãçæã§ããªãå Žåã確èªãããŸããã
5. éç¹çãªæ³šæãå¿ èŠãªå®å šæ§ã«ããŽãªã®ç¹å®
ç ç©¶ããŒã ã¯ãç¹ã«æ³šæãå¿ èŠãªå®å šæ§ã«ããŽãªãšããŠä»¥äžãç¹å®ããŸããïŒ
è«äºçãããã¯ïŒc3ïŒïŒæ¿æ²»çãªå 容ã瀟äŒåé¡ã«é¢ããè°è«
ãããªãºã ãšçµç¹ç¯çœªïŒc13ïŒïŒéæ³è¡çºãæŽåã«é¢ããå 容
åç©èåŸ ïŒc1ïŒïŒåç©ãžã®å±å®³ãäžé©åãªæ±ãã«é¢ããå 容
è¬ç©ä¹±çšãšæŠåšïŒc5ïŒïŒéæ³è¬ç©ãæŠåšã«é¢ããæ å ±
ä»åŸã®å±æãšèª²é¡
ãã®ç ç©¶ã¯ãLLMã®å®å šæ§è©äŸ¡ã«ãããæ°ããã¢ãããŒãã瀺ããšãšãã«ãããã€ãã®éèŠãªèª²é¡ãæããã«ããŸãããç¹ã«æ³šç®ãã¹ãç¹ã¯ãå®å šæ§ãšæçšæ§ã®ãã¬ãŒããªãã®åé¡ã§ãã
é床ã«å®å šåŽã«æ¯ããã¢ãã«ã¯ããŠãŒã¶ãŒã«ãšã£ãŠæçšãªæ å ±ãŸã§æäŸã§ããªããªãå¯èœæ§ããããŸããäŸãã°ãæŠåšã«é¢ããæŽå²çãªç ç©¶ããè¬ç©ã®å»åŠçãªå©çšã«ã€ããŠã®æ£åœãªè³ªåãŸã§æåŠããŠããŸãå¯èœæ§ããããŸãã
äžæ¹ã§ãå®å šæ§ã®åºæºãç·©ããããã°ãã¢ãã«ãå±éºãªæ å ±ãæäŸããŠããŸããªã¹ã¯ãé«ãŸããŸãããã®åŸ®åŠãªãã©ã³ã¹ãã©ã®ããã«åããã¯ãä»åŸã®LLMéçºã«ãããéèŠãªèª²é¡ãšãªãã§ãããã
ãŸãšã
o3-miniã®å®å šæ§è©äŸ¡ã¯ãLLMã®å®å šæ§ãã¹ãã«ãããæ°ããæ¹åæ§ã瀺ãéèŠãªç ç©¶ãšãããŸããç¹ã«ãASTRALã«ããåçãªãã¹ãã±ãŒã¹çæãšèªåè©äŸ¡ã®ææ³ã¯ãä»åŸã®LLMè©äŸ¡ã®æšæºãšãªãå¯èœæ§ãç§ããŠããŸãã
ãŸãããã®ç ç©¶ã¯å®å šæ§è©äŸ¡ã®ãèªååããšãç¶ç¶çãªæŽæ°ãã®éèŠæ§ã匷調ããŠããŸããæè¡ã®é²æ©ãšç€ŸäŒã®å€åãå éããäžãéçãªãã¹ãã±ãŒã¹ã ãã§ã¯ååãªè©äŸ¡ãã§ããªãããšã¯æããã§ãã
æ¬ç ç©¶ã®è©³çްãªå®éšããŒã¿ãšåçŸçšã³ãŒãã¯ã以äžã®ãªããžããªã§å
¬éãããŠããŸãïŒ
https://github.com/Trust4AI/ASTRAL
æåŸã«ããã®åéã«ãããä»åŸã®ç ç©¶ã®æ¹åæ§ãšããŠã以äžã®ç¹ãéèŠã«ãªããšèããããŸãïŒ
æåçèæ¯ãå°åæ§ãèæ ®ãããã¹ãã±ãŒã¹ã®çæ
ãªã¢ã«ã¿ã€ã ã§ã®å®å šæ§ã¢ãã¿ãªã³ã°ææ³ã®éçº
å®å šæ§ãšæçšæ§ã®ãã©ã³ã¹ãå®éçã«è©äŸ¡ããææšã®ç¢ºç«
ãããã®èª²é¡ã«åãçµãããšã§ãããå®å
šã§æçšãªLLMã®éçºãä¿é²ãããããšãæåŸ
ãããŸãã
èªãã è«æãããšã§èŠè¿ããããæŽçããã人ããããŒã ã§å
±æããã人ã«ã¯ãè«æç®¡çãµãŒãã¹ãã¹ã«ã³ã³ããããããã§ããç ç©¶è«æãæžã人åãã®æŽçã»èç©ã«ãããªãçžæ§ãããã§ãã