
ããŒã«ã«LLMã«åãå±¥æŽæžã200åæ¡ç¹ãããŠãtemperature 0ã®å¹ãç®ã枬ã£ãŠã¿ã
LLMã«å±¥æŽæžãæž¡ããŠã10ç¹æºç¹ã§æ¡ç¹ããŠããšé Œããšãç¹æ°ãšããã£ãšããããçç±ãè¿ã£ãŠããŸããããäžåºŠé Œããšãç¹æ°ãå€ããããšããããŸããããèšããã察çã¯ãtemperatureã0ã«ãããã§ãããã¯æ£è§£ã§ãããã ãããã ãã ãš2ã€ã®çåãæ®ããŸããããã©ã«ãèšå®ã®ãŸãŸã ãšç¹æ°ã¯å®éã©ã®ãããæºããã®ãããããŠãtemperature 0ã«ããŠãçŽããªããã®ã¯äœãã
ã©ã¡ããããŒãPCã§æ°åããã°æž¬ããŸããç¹æ°ã¯1ããŒã¯ã³ãªã®ã§ãã¢ãã«ããã®ç¢ºçããã®ãŸãŸåºããŠãããããã§ãã
ç°å¢ã¯ 8GB ã® M1 Macãmlx-lm 0.32.2ãã¢ãã«ã¯ Qwen2.5-3B-InstructïŒ4bitãMLXçïŒã§ããããã³ããã¯ããã¹ã¯ãããã¢ã㪠osFoundry ã®ãã¢ãã«ãâããã¬ã€ã°ã©ãŠã³ããã§äœ¿ã£ãæ¡ç¹ãã¢ãšåããã®ã«ããŸããããžã¥ãã¢ããŒã¿ã¢ããªã¹ãè·ã«å¿åãã6è¡ã®å±¥æŽæžãšãããžã¥ãã¢ããŒã¿ã¢ããªã¹ãè·ãšããŠããã®å±¥æŽæžã10ç¹æºç¹ã§æ¡ç¹ããŠãæåã«ç¹æ°ã次ã«çç±ã1ã€ãããšãã質åã§ãïŒè±èªçãåãå 容ã§çšæããŸããïŒããã®ã¢ããªã¯ããŒã«ã«ã¢ãã«ã llama.cpp ã® llama-server ã§åãããŠããã®ã§ããµã³ããªã³ã°ã®èšå®ãããããæ¥ãŸãã
äœãæå®ããªããšãã®ããã©ã«ã
ãªã¯ãšã¹ãã« temperature ãå ¥ã£ãŠããªããšãllama-server ã¯èªåã®ããã©ã«ãã§ãµã³ããªã³ã°ããŸããã¢ããªã«å梱ãããŠãããã«ãïŒversion 10358ïŒã® --help ã«ã¯ããåºãŸãã
--temp, --temperature N temperature (default: 0.80)
--top-k N top-k sampling (default: 40, 0 = disabled)
--top-p N top-p sampling (default: 0.95, 1.0 = disabled)
--min-p N min-p sampling (default: 0.05, 0.0 = disabled)
-s, --seed SEED RNG seed (default: -1, use random seed for -1)ã¢ããªåŽãããµãŒããŒã® temperature ãèšå®ãããŠããªããã° --temp ãä»ããã« llama-server ãèµ·åããŸãïŒlocal_inference.rsïŒãã€ãŸã temperature ãæå®ããªããªã¯ãšã¹ãã¯ãtemperature 0.8ã»ã©ã³ãã ãªã·ãŒãã§çæãããŸããllama.cpp ã䜿ãããã³ããšã³ãã®å€ãããäœãããªããã°ãã®ç¶æ ã§ãã
ç¹æ°ããæ¬¡ã®1ããŒã¯ã³ãã«ãã
èªç±ã«çãããããšããScore: 7ãã7/10ãã10ç¹æºç¹ã§è©äŸ¡ãããšâŠãã®ããã«æžãåºãããã©ãã©ã§ãç¹æ°ãã©ã®ããŒã¯ã³ãªã®ãç¹å®ãã«ãããªããŸããããã§è¿çã®é ãå ã«åããŠããïŒãã¬ãã£ã«ïŒãã¢ãã«ã次ã«åºãããŒã¯ã³ãç¹æ°ã®æ°åã«ãªãããã«ããŸããæ¥æ¬èªã§ã¯ãç¹æ°ïŒããè±èªã§ã¯ãScore: ããå ¥ããŸããã
import mlx.core as mx
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("mlx-community/Qwen2.5-3B-Instruct-4bit")
DIGITS = {d: tok.encode(str(d))[0] for d in range(1, 10)}
def ids_for(cv, question, prefill="ç¹æ°ïŒ"):
msgs = [{"role": "user", "content": cv + "\n\n" + question}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)
return tok.encode(text + prefill)
def digit_probs(ids, T=1.0):
logits = model(mx.array([ids]))[0, -1].astype(mx.float32)
p = mx.softmax(logits / T)
return {d: float(p[t]) for d, t in DIGITS.items()}Qwen ã®ããŒã¯ãã€ã¶ãŒã¯æ°åã1æ¡ãã€ã«åããã®ã§ãã7ãã¯1ããŒã¯ã³ãã10ãã¯ã1ãã®ããŒã¯ã³ããå§ãŸããŸããããã³ããã1åéãã ãã§ãç¹æ°ã®ååžããŸãããšæã«å ¥ããŸãã
ç¹æ°ã¯1ã€ã®æ°åã§ã¯ãªãååž
æ¥æ¬èªã®å±¥æŽæžïŒ222ããŒã¯ã³ïŒã§ãtemperature 1ïŒã¢ãã«ã®çã®ç¢ºçïŒãèŠããšãããªããŸããã

ã¢ãã«ã¯ããã®å±¥æŽæžã¯8ç¹ã ããšèããŠããããã§ã¯ãããŸãããããã¶ã8ç¹ãã§ã9ç¹ã7ç¹ãååããããããšèããŠããŸãã
ããããå®éã«ãµã³ããªã³ã°ããŠã¿ãŸããmlx-lm ã®ãµã³ãã©ãŒã llama-server ã®ããã©ã«ããšåã4ã€ã®å€ã«ããŠïŒmake_sampler(temp=0.8, top_p=0.95, min_p=0.05, top_k=40)ïŒãåãããã³ããã§200åçæããŸããã

ãã¡ã°ã確çã®é«ã8ç¹ãåºãã®ã¯çŽååã§ã200åã§4çš®é¡ã®ç¹æ°ãåºãŸãããåé ã®ãåãå±¥æŽæžãªã®ã«ç¹æ°ãå€ãããã¯ããã®ååžãããåŒãããŠããã ãã§ãã
è±èªçã¯ãã£ãšã決ãŸã£ãŠããã
åãå±¥æŽæžãšè³ªåã®è±èªçïŒ173ããŒã¯ã³ïŒã§ã¯ãã¢ãã«ã®è¿·ãæ¹ããŸã£ããéããŸãããtemperature ããšã®ç¢ºçã¯æ¬¡ã®ãšããã§ããtemperature ã¯ãœããããã¯ã¹ã®åã«ããžãããå²ãå€ãªã®ã§ãå°ããããã»ã©ååžãå°ããŸãã

ãã ã llama-server 㯠temperature ã ãããããããã§ã¯ãããŸãããããã©ã«ãã§ã¯ top-k 40 â top-p 0.95 â min-p 0.05 ã®é ã«åè£ãçµããtemperature ã¯æåŸã«ããããŸãããã®ååžã ãš7ç¹ãš6ç¹ã ãã§95%ãè¶ ããã®ã§ãtop-p ã®æç¹ã§ã»ãã®ç¹æ°ã¯æ¶ãã2æã«ãªããŸãã
def llamacpp_default(logits, temp=0.8, top_k=40, top_p=0.95, min_p=0.05):
order = mx.argsort(-logits)[:top_k]
kept = logits[order]
p = mx.softmax(kept)
n = int(mx.sum(mx.cumsum(p) < top_p).item()) + 1
idx = [i for i in range(n) if p[i].item() >= min_p * p[0].item()]
q = mx.softmax(kept[mx.array(idx)] / temp)
return {tok.decode([int(order[i])]): float(q[j]) for j, i in enumerate(idx)}
# -> {'7': 0.8093, '6': 0.1907}èšç®äžã¯7ç¹ã80.9%ã6ç¹ã19.1%ã§ããå®éã«200åãµã³ããªã³ã°ãããšã7ç¹ã162åïŒ81%ïŒã6ç¹ã38åïŒ19%ïŒã§ãã»ãŒãŽã£ããäžèŽããŸãããããã©ã«ãã®ãŸãŸã ãšã5åã«1åãããã¯ãã®å±¥æŽæžã6ç¹ã«ãªããŸãã2人ã®å¿åè ã1åãã€æ¡ç¹ããŠ6ç¹ãš7ç¹ã«åããããããã®å·®ãããã®çµæã§ããå¯èœæ§ã¯ååãããŸãã
æºãæ¹ã®å€§ããã¯ã¢ãã«ã§æ±ºãŸãæ§è³ªã§ã¯ãªãããã®å ¥åã§äžäœã®åè£ãã©ãã ãç«¶ã£ãŠãããã§æ±ºãŸããŸããããã¯ç¢ºçãèŠãªããšåãããŸããã
temperature 0ã§æããã®
temperature 0ïŒgreedyãæ¯åãã¡ã°ã確çã®é«ãããŒã¯ã³ãéžã¶ïŒã§ã40ããŒã¯ã³ã®åçå šäœãèšèªããšã«20åçæããŸãããç°ãªãæç« ã¯åèšèª1çš®é¡ã§ã1æåãéããŸããã§ãããæ¥æ¬èªã¯æ¯åã8ç¹ãã§å§ãŸããçç±ã®æãåãã§ãã
ã·ãŒããåºå®ããŠãåãããšãã§ããŸããmx.random.seed(42) ãæ¯åãªã»ããããŠ10åãµã³ããªã³ã°ãããšãåºãŠããæç« ã¯1çš®é¡ã§ãããå¹ ã®ããåºåãã»ããããã©ç¹å®ã®çµæãåçŸãããããšãããšãã«ã¯äŸ¿å©ã§ãããã ãå ¬å¹³ãã®éå ·ã«ã¯ãªããŸãããã·ãŒããåãã§ããå±¥æŽæžãéãã°ããã®åŒãæ¹ãå€ããã®ã§ã2人ã¯çµå±å¥ã ã®éã§æ¡ç¹ãããŸãã
ãªããããã§ã®èšæž¬ã¯ãã¹ãŠãã»ãã«è² è·ã®ãªãã¢ãã«ã«1ãªã¯ãšã¹ããã€éã£ããã®ã§ããè€æ°ã®ãªã¯ãšã¹ãããŸãšããŠåŠçãããµãŒããŒã§ã¯ãæµ®åå°æ°ç¹ã®çްããªå·®ãåºãããšããããäžäœ2ã€ã®ããŒã¯ã³ãã»ãŒåç¹ã®ãšãã ãçµæã«é¿ããŸããããã¯ä»å枬ã£ãŠããŸããã
temperature 0ã§ãæããªããã®
greedy ã«ãããšãåºåã¯å ¥åã§æ±ºãŸããŸãããã ãå ¥åã®ã¡ãã£ãšããéãã«å¯ŸããŠå®å®ããŠãããšã¯éããŸãããç¹æ°ãå€ããã¹ãã§ã¯ãªãç·šéã4çš®é¡å ããŠãtemperature 0 ã®ç¹æ°ãšç¢ºçãèŠãŸããã

èšãåããå€ãã質åã¯ãRate this CV for a junior data analyst role on a 1 to 10 scale. Give the score first, then one reason.ãã§ãããã£ãŠã»ããããšãå±¥æŽæžãåããªã®ã«ãtemperature 0 ã®ç¹æ°ã¯7ç¹ãã6ç¹ã«äžãããŸãããæ¥æ¬èªçã§ã¯5ãã¿ãŒã³ãšã8ç¹ã®ãŸãŸã§ãããã1äœã®ç¢ºçã¯æå€§ã§ã44%ã§ã2äœã¯ç·šéã«ãã£ãŠ9ç¹ã«ãªã£ãã7ç¹ã«ãªã£ããããŸããã
ã€ãŸã temperature 0 ãæ¶ãã®ã¯ãå®è¡ããšã®ãããã§ããããã³ãããžã®ææããã¯æ®ããŸããå±¥æŽæžãæ¯ã¹ããªããæ¬¡ã®3ã€ãèœãšã穎ãé¿ããã³ãã§ãã
temperature 0 ã«ããŠãæ¯ã¹ããã®ãã¹ãŠã«äžåäžå¥åã質åã䜿ãã質åãå€ãããå šéšæ¡ç¹ãçŽãã
ç¹æ°ã ãã§ãªã確çãæ®ããllama-server ã® /completion 㯠n_probs ãæž¡ããšããŒã¯ã³ã®ç¢ºçãè¿ãã®ã§ãäžã®ãã¬ãã£ã«ãšçµã¿åãããã°æåã®ããŒã¯ã³ããã®ãŸãŸç¹æ°ã®ç¢ºçã«ãªããŸãã74%ã®7ç¹ãš40%ã®8ç¹ã¯ãåãéã¿ã®çãã§ã¯ãããŸããã
確信床ãäœã1ç¹å·®ã¯åç¹ãšããŠæ±ããã©ã¡ãã®ç¹æ°ããã¢ãã«ãæžããçç±ãèªãã§ãã䜿ãã
ã¢ããªã§è©Šããªã
ãã¹ã¯ãããã¢ã㪠osFoundry ã§ã¯ããã¢ãã«ãâããã¬ã€ã°ã©ãŠã³ããã§åãããŒã«ã«ã¢ãã«ã®ããã«ã3ã€äžŠã¹ãã°åãå®éšãã§ããŸãããŸãããã©ã«ãã®ãŸãŸå®è¡ããŠæºããèŠãŠãèšå®ã®è¡ã§ã枩床ãã0ã«ããŠããäžåºŠå®è¡ããŸãã

èšå®ã®æå³ã¯çšèªéã®æž©åºŠïŒtemperatureïŒãštop-pã«è©³ãããããŸããã¢ããªã¯osFoundryã®ããŠã³ããŒãããŒãžããç¡æã§å ¥æã§ããŸãã