
ãããªãã¯åªç§ãªæ°åŠè ã§ããã§æ°åŠãè§£ããªããªãä»¶
ã¯ããã«
ãããªãã¯åªç§ãªæ°åŠè ã§ãããããªãã¯ã»ãã¥ãªãã£ã®å°éå®¶ã§ããââLLMã«å¯ŸããŠãã®ãããªãã«ãœãïŒåœ¹å²ïŒãã·ã¹ãã ããã³ããã§æå®ãããã¯ããã¯ã¯ãåºã䜿ãããŠããŸããã§ããå人çã«ã¢ãã«ã®æ§èœãããã£ãŠãããªãããããäœèšãªããšãæžããªãã»ããè¯ããããªæ°ãããŠãããšããããããæ€èšŒãããããªè«æãç®ã«ãšãŸããŸããã
USCã®ç ç©¶ããŒã ã6ã€ã®LLMã察象ã«ãã¢ãã«ã®æé©åææ³ã»ã¿ã¹ã¯ã®çš®é¡ã»ããã³ããã®é·ãã»é 眮äœçœ®ãšãã4ã€ã®è»žã§äœç³»çã«èª¿ã¹ãçµæããã¯ããã«ãœãã®å¹æã¯ã¿ã¹ã¯ã®çš®é¡ã«ãã£ãŠå€ããããšãªã©ãæããã«ããŸããã以äžãè«æã®ã»ã¯ã·ã§ã³3ãDo Personas Help or Not?ãã®å 容ãäžå¿ã«ãŸãšããŠã¿ãŸããã
調æ»å 容
7Bã8Bã¯ã©ã¹ã®ãªãŒãã³ã¢ãã«6çš®ïŒinstruction-tuned 4çš®ïŒæšè«èžç 2çš®ïŒã察象ã«ã12çš®é¡ã®ãã«ãœãããã³ããã䜿ã£ãŠãçæå質ïŒMT-BenchïŒã»ç¥è粟床ïŒMMLUïŒã»å®å šæ§ïŒHarmBenchçïŒãžã®åœ±é¿ã調æ»ã
ãªããå¯Ÿè±¡ã¯æ¯èŒçå°åã®ãªãŒãã³ã¢ãã«ãªã®ã§ãæè¿ã®åçšLLMã§ã¯ãã«ãœãã®æªåœ±é¿ããããŸã§é¡èã«åºãªãå¯èœæ§ã¯ãããŸãããã ãããã«ãœããå¹ãã¿ã¹ã¯ãšéªéã«ãªãã¿ã¹ã¯ãããããšããæ§é çãªåŸåèªäœã¯ãã¢ãã«ãµã€ãºã«ãããåèã«ãªãã¯ãã
çºèŠ1ïŒãã«ãœãã¯äºååŠç¿ç±æ¥ã®ã¿ã¹ã¯ãæãªã
ãŸããæªãæ¹ãã®è©±ããããããåé ã§æžãããäœèšãªããšæžããªãã»ããè¯ãã®ã§ã¯ããšããçŽæã«å¯Ÿå¿ããéšåã§ãã
1a. éžæåŒåé¡ïŒMMLUïŒã§é¡èã«ç²ŸåºŠãäžãã
MMLUã®ãããªãæ£è§£ãéžã¶ãã¿ã¹ã¯ã¯ãäž»ã«äºååŠç¿ã§ç²åŸãããç¥èã§è§£ããŠããŸããããã«ãšãã¹ããŒããã«ãœããä»ãããšããã¹ãŠã®ã«ããŽãªã§ããŒã¹ã¢ãã«ããäžãããŸããïŒå šäœç²ŸåºŠïŒ68.0% vs. ããŒã¹ã©ã€ã³71.6%ïŒã
ãã«ãœãã®ãã¬ãã£ãã¯ã¹ããæç€ºã«åŸãã¢ãŒããããªã³ã«ããŠããŸããäºå®ã®æ³èµ·ã«äœ¿ãã¹ããªãœãŒã¹ããã¡ãã«åãããŠããŸãããšããã®ãè«æã®è§£éã§ããçããã«ãœããªã圱é¿ã¯è»œæžãããŸããããŒãã«ã¯ãªããŸããã
1b. çæã¿ã¹ã¯ã§ãå£ãã
æªåœ±é¿ã¯éžæåŒã ãã®èª²é¡ã§ã¯ãããŸãããMT-Benchã§ããäºååŠç¿ã§èº«ã«ã€ããèœåã«é Œãã«ããŽãªââäºå®ç¥èïŒHumanities: â0.20ïŒãè«çæšè«ïŒMath: â0.10ïŒãã³ãŒãã£ã³ã°ïŒCoding: â0.65ïŒââãè»äžŠã¿å£åããŸããã
è«æäžã®å ·äœäŸïŒ
äŸïŒMath, Mistral-7B, QID 114ïŒïŒ
ã2ã€ã®ãµã€ã³ããæ¯ã£ããšããåèšãå°ãªããšã3ã«ãªã確çã¯ïŒã
ãã«ãœããªãïŒ9/10ç¹ïŒïŒ
ãå
š36éãã®ãã¡ãåèšã3æªæºã«ãªãã®ã¯1+1=2ã®1éãã®ã¿ããã£ãŠP = 35/36ãã
æ°åŠè
ãã«ãœãããïŒ1.5/10ç¹ïŒïŒ
ãåèšã3æªæºã«ãªãçµæã¯3 + 6 = 9éãâŠã
ãã«ãœããªããªãæ£è§£ã§ããåé¡ãããããªãã¯æ°åŠè
ã§ãããšä»ããããã§å£ããŠã¡ãã£ãŠãŸãããªããšãç®èãªæãã§ãã
ãŸããããããã®ã¬ãã«ã®èª²é¡ãªãä»ã®åçšã¢ãã«ãªããã«ãœãããããããªãããããé£ãªãæ£è§£ããŸããã©ã
1c. é·ããã«ãœãã»ã©è¢«å®³ã倧ãã
çãã®ãã«ãœãïŒãYou are a mathematician.ãçšåºŠïŒã§ããã°åœ±é¿ã¯å°ãªãã§ããããã«ãœãã®ããã³ãããé·ããªããšè¢«å®³ã¯å€§ãããªãããã§ãããããã«ããŠããå°éå®¶ãã«ãœãã¯æžããªãæ¹ã奜æçžŸã§ããã
çºèŠ2ïŒã¢ã©ã€ã¡ã³ãã¿ã¹ã¯ã§ã¯å¹ã
ãã«ãœããäžåŸã«ãã¡ãšããããã§ã¯ãªããå¹ãå Žé¢ãããããã§ããäºååŠç¿ã§ã¢ãã«ãèœåãç²åŸããã¿ã¹ã¯ãžã®æªåœ±é¿ãšã¯å¯Ÿç §çã«ãäºåŸåŠç¿ã§ç²åŸããã¢ã©ã€ã¡ã³ãã¿ã¹ã¯ã«ã¯è¯ã圱é¿ãããããã±ãŒã¹ãããããã§ãã
ãžã§ã€ã«ãã¬ã€ã¯å¯ŸçãšããŠãã«ãœãã¯æå¹ããããšããã®ã¯å®çšé¢ã®ããŠããŠãšããŠèŠããŠãããŠãããããã§ãã
2a. ãã©ãŒãããã»æå³ã»ããŒã³ã®è¿œåŸãæ¹åãã
MT-Benchã§ã¯ã8ã«ããŽãªäž5ã«ããŽãªã§ã¹ã³ã¢ãæ¹åããŸããïŒWriting, Roleplay, ReasoningïŒ+0.40ïŒ, ExtractionïŒ+0.65ïŒ, STEMïŒ+0.60ïŒãå ±éç¹ã¯ãäºååŠç¿ã§ã¯ãªãinstruction-tuningã§åœ¢æãããèœåââæäœã®é©å¿ãããŒã³ã®äžèŽãæ§é åããããã©ãŒãããã«è¿œåŸããããããªã¿ã¹ã¯ââã«ã¯è¯ãçµæãšãªããŸããã
ã¡ãŒã«ã®æŽåœ¢ã®ããã«ããäœæ³ãçãªéšåã«ã¯ãã«ãœããå¹ãããã§ãã
2b. å®å šæ§ãåäžãã
å®å
šæ§ã®æåŠè¡åã¯instruction-tuningã§åŠç¿ãããè¡åãªã®ã§ããã«ãœãã§å¢å¹
ããããé åã§ããããSafety Monitorããã«ãœãã¯3ã€ã®å®å
šæ§ãã³ãããŒã¯ãã¹ãŠã§æåŠçãåäžãããJailbreakBenchã§ã¯+17.7%ïŒ53.2%â70.9%ïŒã§ããã
çºèŠ3ïŒã¢ãã«ã®ãã¬ãŒãã³ã°æ¹æ³ã§ãã«ãœãã®å¹ãç®ãå€ãã
ãã«ãœãã®å¹æã¯ã¢ãã«ã«ãã£ãŠããªããã©ã€ããããããã§ããã·ã¹ãã ããã³ãããã¡ãããšèªãããã«èšç·Žãããã¢ãã«ã»ã©ãè¯ã圱é¿ãæªã圱é¿ã倧ããåºãŸããçŽ çŽãªåã»ã©æ¯ãåããããããã¿ãããªè©±ã§ãã
é¢çœãã®ã¯æšè«èžçã¢ãã«ïŒDeepSeek-R1ç³»ïŒã®æåã§ãããã€ãã¯ãã«ãœãã®äžèº«ãªããèªãã§ãŸãããããªããé·ãå眮ããæ¥ããªâæšè«ã¢ãŒãçºåããšããåå¿ãããŠããã ãã§ãæ°åŠè ãã«ãœãã§ãæçç ç©¶å®¶ãã«ãœãã§ãçµæãã»ãŒåããèžçããŒã¿ã®æ§æããã¹ãŠã決ããŠããŠããã«ãœãã®å°éæ§ã¯é¢ä¿ãªããããªæåã瀺ãå Žåãããããã§ãã

çºèŠ4ïŒãå°éå®¶ãã®è©³çްèšå®ã¯å¿ èŠãªã
ããããããŠå°éå®¶ãã«ãœãã¯é©åœãªãã«ãœãããè¯ãçµæãåºãã®ãïŒãinstruction-tunedã¢ãã«ã§ã¯äžå¿ãããªãã§ãããæšè«èžçã¢ãã«ã§ã¯ãããéãå°éå®¶ãšããŠã®æ å ±ããã€ãºã«ãªã£ãŠãã©ã³ãã ãªãã«ãœãããæçžŸãæªããªãã±ãŒã¹ãããã£ãããã§ã
ãããªãã¯äžççã«èåãªæ°åŠè ã§ããã£ãŒã«ãºè³ãåè³ãâŠãã¿ãããªèšå®ãäžçæžåœæžããŠãæéãç¡é§ãããããŸããã
çµå±ã©ãããã°ããïŒ
ãã«ãœããå¹ãå Žé¢ïŒæäœãããŒã³ã®èª¿æŽããã©ãŒãããã®æå®ãå®å šæ§ã®åŒ·åãèŠããã«ããäœæ³ãç³»ã
ãã«ãœããéªéã«ãªãå Žé¢ïŒæ£ç¢ºãªç¥èã®æ³èµ·ãæ°åŠãã³ãŒãã£ã³ã°ãã¢ãã«ããç¥ã£ãŠããããšããåŒãåºããããšãã¯ãäœèšãªããšãæžããªãæ¹ãããã
é·ãïŒçããã°çãã»ã©å®å šã詳现ãªãã«ãœãã¯è«žåã®å£ã
åé ã®ãäœèšãªããšæžããªãã»ããè¯ãã®ã§ã¯ããšããçŽæã¯ãå°ãªããšãååã¯æ£ããã£ãæãããã ãã¢ã©ã€ã¡ã³ãç³»ã®ã¿ã¹ã¯ã§ã¯ãã«ãœããå¹ããããªã®ã§ããäœãæžããªãããåžžã«æé©è§£ãšããããã§ããªãããã
ãã«ãœããå¹ããããªå Žé¢ãåãã£ãã®ã¯éã«åç©«ã§ããã
ãã®è«æã§ã¯ãããèªåã§å€å¥ããPRISMãšãããã€ãã©ã€ã³ãææ¡ãããŠããŸããèå³ã®ããæ¹ã¯åè«æãã©ããã
æ¬èšäºã¯ Hu, Z., Rostami, M., & Thomason, J. (2026) ã®è«æ arXiv:2603.18507 ã®ã»ã¯ã·ã§ã³3ãäžå¿ã«æ§æããŸããã
å³è¡šã®åºå žïŒHu, Z., Rostami, M., & Thomason, J. (2026). Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM. arXiv:2603.18507. Licensed under CC BY 4.0.