
nanochat#7 8ๅ้ใใชใฃใฆใๆ็ธพใฎๅทฎใฏ 0.000047 ใ ใฃใ
้ฃ่ผใๆฌ็ฉใฎLLMใใๅใใใชใใ่ชญใฟ่งฃใใ
็ฌฌ7ๅใ8ๅ้ใใชใฃใฆใๆ็ธพใฎๅทฎใฏ 0.000047 ใ ใฃใ
ใใใ 6GBใฎใฒใผใใณใฐใใผใใงใๅใใณใใณใใๅฉใใ ใ
ใฉใใใงใ๏ผใใใใใงใใ
ๅๅใMac mini M4(16GB)ใง2ๆ้15ๅใใใฆๅญฆ็ฟใๅใใพใใใ
ไปๅใฏใใพใฃใใๅใใณใใณใใ2ๅฐ็ฎใงๅฉใใพใใ
ๆๅ
ใซใใใฎใฏใWindowsใใผใPC(32GB)ใRTX 3060 ใฎ6GBใงใใ
ใใผใฟใปใณใฟใผใงไฝฟใใใGPUใฏ80GBใ13ๅใฎ1ใใใใใพใใใ
็ตๆใๅ ใซๆธใใพใใ14ๅ57็งใง็ตใใใพใใใ
ๅใใณใใณใใใๅฅใฎPCใง
ๅฉใใใฎใฏๅๅใจ1ๆๅญใ้ใใพใใใ
python -m scripts.base_train \
--depth=6 --head-dim=64 --window-pattern=L \
--max-seq-len=512 --device-batch-size=32 \
--total-batch-size=16384 \
--eval-every=100 --sample-every=100 \
--num-iterations=5000ใใผใฏใใคใถใใMacใงไฝใฃใใใฎใใใฎใพใพๆใฃใฆใใใพใใใ
ไธญ่บซใๅใใงใใใใจใฏใใใกใคใซใฎๆ็ด๏ผmd5๏ผใง็ขบใใใฆใใใพใใ
tokenizer.pkl c3250d6ff04efc56c7121ad9ec42a802 ๏ผไธกๆนใจใๅใ๏ผๅใๆนใ้ใใจ็นๆฐใๆฏในใใใชใใใใงใใๅใ็ฉๅทฎใใงๆธฌใใใใฎไธๆบๅใงใใ
่ตทๅใใ็ฌ้ใซใ้ใใๅบใ
ๅใใณใใณใใชใฎใซใ็ซใกไธใใใฎใญใฐใ้ใใพใใใไธฆในใพใใ
Mac mini
Autodetected device type: mps
COMPUTE_DTYPE: torch.float32 (auto-detected: no CUDA (CPU/MPS))
WARNING: Flash Attention 3 not available, using PyTorch SDPA fallback RTX 3060
Autodetected device type: cuda
COMPUTE_DTYPE: torch.bfloat16 (auto-detected: CUDA SM 86 (bf16 supported))
โ Using Flash Attention 3: efficient, new and awesome.ใณใผใใฏ1ๆๅญใๅคใใฆใใพใใใ
ใใใงใใๆฐใฎๆใกๆนใ float32ย ใใ bfloat16ย ใซๅใๆฟใใใพใใใ
ใใญใฐใฉใ ใ็ฐๅขใ่ฆใฆใๅๆใซๆฑบใใฆใใพใใ
nanochat/common.pyย ใซใใใฎๅคๆญใๆธใใฆใใใพใใ
bf16 requires SM 80+ (Ampere: A100, A10, etc.)ใSM 80ไปฅไธใชใ bfloat16ใใRTX 3060 ใฏ SM 86 ใชใฎใงใใใฎ็ทใ่ถใใฆใใพใใใ
ใใใฒใจใคใใโใใไปใใฆใใ่กใใใใพใใFlashAttention 3 ใไฝฟใใพใใใ
ใใใฏไบๆณใๅคใใใจใใใงใใ
็งใฏใใฎ้ฃ่ผใฎๆงๆใ็ซใฆใใจใใใFlashAttention 3 ใฏๆๆฐใฎHopperไธไปฃๅใใ ใใใ
3060ใงใฏไฝฟใใใPyTorchๆจๆบใฎใใๆนใซ่ฝใกใใฏใใ ใใจๆธใใฆใใพใใใ
flash_attention.pyย ใ่ชญใฟ็ดใใจใใใใใใพใใใ
FA3 kernels are currently compiled for Hopper (sm90), Ada (sm89)
and Ampere (sm80/sm86)Ampere ใๅฏพ่ฑกใซๅ ฅใฃใฆใใพใใใ3060ใฏใใใใๆปใ่พผใใงใใพใใ
ใคใพใMacใจใฎๅทฎใฏใ2ใคใใฃใใใจใซใชใใพใใ
ๆฐใฎๆใกๆนใๅคใใใAttentionใฎ่จ็ฎใฎใใๆนใพใงๅ
ฅใๆฟใใฃใใ
ๆฐใฎๆใกๆนใๅคใใใใจใใใฎใฏ
ใใใ ใใๅฐใ่ชฌๆใๆใฟใพใใ
ใณใณใใฅใผใฟใฏๅฐๆฐใใๆฑบใพใฃใ้ทใใฎ็ฎฑใซๅ
ฅใใฆๆใกใพใใ
float32ย ใฏ1ๅใใใ32ใใใใฎ็ฎฑใbfloat16ย ใฏ16ใใใใฎ็ฎฑใงใใ
็ฎฑใๅๅใซใชใใพใใ
ๅๅใซใชใใฐใๆธใ่พผใใๆกๆฐใๆธใใพใใ
ๅๅMacใงไฝใฃใใขใใซใฎ้ใฟใใๅฎ้ใซๅฐใใ็ฎฑใธ็งปใใฆใฟใพใใใ
1ๅฑค็ฎใฎMLPใฎใๅ
้ ญ5ๅใงใใ
float32 bfloat16
0.03140263 0.03149414
-0.00918143 -0.00915527
0.02737478 0.02734375
0.01616270 0.01611328
0.04142616 0.04150391ๅฐๆฐ็ฌฌ4ไฝใใไธใใๅใใฎใใใจใใใธๅฏใใใใฆใใพใใ
ๅคงใใใชใฃใใๅฐใใใชใฃใใใจใๆนๅใฏๆใฃใฆใใพใใใ
ใใฎๅฑคใฎ้ใฟ58ไธ9,824ๅใๅ จ้จ็งปใใฆใใใใๆธฌใใพใใใ
ใใใฎๅนณๅ 0.00003486
ใใใฎๆๅคง 0.00048733
ใใจใฎๅคใซๅฏพใใฆ 0.141%
ใดใฃใใไธ่ด 9ๅ / 589,824ๅ58ไธๅใฎใใกใใใฎใพใพๅ
ฅใฃใใฎใฏ9ๅใ ใใงใใใ
ๆฎใใฏๅ
จ้จใใฉใใใใๅใใใฆใใพใใ
้ใซใชใใพใใใใฎใถใใไธๅบฆใซ้ในใๆฐใๅใซใชใใ่จ็ฎใ้ใใชใใพใใ
็ฒใใใฆใ้ใใใใ ใใใใใฎๅใฎใใผใใงใใ
ๅใฎ้ฃ่ผใฎๆ็ตๅใงใ็งใฏ Q4_K_M ใจใใไป็ตใฟใ่ชฌๆใงใใใซ็ตใใใพใใใ
ใใใๅใ่ฉฑใงใใ16ใใใใฎใฏใใฎใใฎใ4.92ใใใใพใงๅใฃใฆใ
ไฝใ่ตทใใใฎใใๅใใใชใใฃใใ
ไปๅใฏใใใฎๆๅใฎ1ๆฎต้ใๅฎ้ใซๆธฌใใพใใ
32ใใใใ16ใใใใซใใใจใ็ตๆใฏใฉใใ ใๆชใใชใใฎใใ
้ใ
ใพใๆ้ใงใใๅใ5,000ในใใใใงใใ
Mac mini (M4) RTX 3060
1ในใใใ 1,377 ms 173 ms
tok/sec 11,901 94,473
ๅญฆ็ฟๆ้ 119.26ๅ 14.57ๅ7.94ๅใ
ๅๅใMacใงๅใใฆใใ้ใซใณใผใใผใๆทนใใฆ้ฃฒใฟ็ตใใฆๆดใ็ฉใพใงๆธใพใใฆใ
10ๅใฎ1ใ้ฒใพใชใใใจๆธใใพใใใไปๅใฏใๆทนใใฆใใใใกใซๅๅ็ตใใใพใใ
ใใผใฏใงไฝฟใฃใใกใขใชใๅบใพใใใ
Peak memory usage: 3002.09MiB3,002MiBใๆญ่ผใใฆใใใฎใฏ6,144MiBใชใฎใงใๅๅใไฝฟใฃใฆใใพใใใ
ใใใไบๆณใๅคใใใจใใใงใใ
ๆบๅใฎใจใใซใใใผใPCใฏ็ป้ขใฎๆ็ปใซ0.5ใ1GBๅใใใใฎใงใ
ๅฎ่ณช5GBๅผทใใไฝฟใใชใใใจ่ฆ็ฉใใฃใฆใใพใใใๅฎ้ใซๆธฌใใจใใใใงใใใ
8MiB / 6144MiB8MiBใ ใใผใPCใฎ็ป้ขใฏๅ
่ตใฎใฐใฉใใฃใใฏในๆฉ่ฝใๆใใฆใใฆใ
3060ใฏ่จ็ฎๅฐ็จใซใชใฃใฆใใพใใใ6GBใพใใใจไฝฟใใพใใ
ใงใ็ฒใใใใถใใ็ตๆใฏๆชใใชใฃใใฎใ
ใใใๆฌ้กใงใใ
100ๅใใจใซๆธฌใฃใใๅฝใฆใซใใใ๏ผๅๅ่ชฌๆใใใๅฐใใใปใฉ่ฏใ็นๆฐ๏ผใใ
2ๅฐใถใไธฆในใพใใ
ในใใใ Mac(32ใใใ) 3060(16ใใใ) ๅทฎ
0 3.195800 3.195800 +0.000000
100 1.940628 1.946345 +0.005717
500 1.530011 1.534937 +0.004926
1000 1.371891 1.373959 +0.002068
2000 1.279500 1.279913 +0.000413
3000 1.224675 1.222672 -0.002003
4000 1.187887 1.186090 -0.001797
5000 1.165885 1.165166 -0.000719ๅบ็บ็นใฎ 3.195800 ใฏใๅฐๆฐ็ฌฌ6ไฝใพใงๅใใงใใใ
ๅใใใผใฏใใคใถใๅใๅๆๅใชใฎใงใ็ซใฃใฆใใๅ ดๆใฏๅฎๅ
จใซๅใใงใใ
ใใใใๅ
ใ่ฆใใจใๆๅใฎใใกใฏ3060ใฎใปใใๅฐใใ ใๆชใใ
ใจใใใ2,500ในใใใใใใใง็ฌฆๅทใใฒใฃใใ่ฟใใๆๅพใฏ3060ใฎใปใใ่ฏใใ
50ๅใฎๆธฌๅฎ็นใใใใถ้่จใใพใใใ
ๅทฎใฎๅนณๅ +0.000047
ใใกใฐใ้ใใๅทฎ +0.008260 ๏ผ200ในใใใ็ฎ๏ผ
ใใกใฐใ้ใซ้ใใๅทฎ -0.002316 ๏ผ3200ในใใใ็ฎ๏ผ
3060ใๅใฃใๅๆฐ 28 / 50ๅนณๅใฎๅทฎใฏ 0.000047ใ
50ๅใฎใใก28ๅใฏ3060ใฎใปใใ่ฏใๆ็ธพใงใใใใปใผใณใคใณใในใงใใ
็ฎฑใๅๅใซใใฆใ8ๅ้ใใใฆใๅคฑใฃใใใฎใฏๆธฌใใพใใใงใใใ
ใใ ใใๆธใๆ็ซ ใฏๅฅ็ฉใซใชใฃใ
็นๆฐใใปใผๅใใชใฎใงใๅบใฆใใๆ็ซ ใๅใใใจใใใจใ้ใใพใใใ
2,000ในใใใ็ฎใฎๅใ่ณชๅใซใใใ็ญใใฆใใพใใ
Mac If yesterday was Friday, then tomorrow will be the first time
in the world to get to the next level.
3060 If yesterday was Friday, then tomorrow will be Friday, then Friday,
then Saturday, then Saturday, then Saturday, then Saturday3060ใฎใปใใฏใๆๆฅใฎๅๅใๅบใใฆใใพใใ
้ ็ชใ็ญใใ้้ใฃใฆใใพใใใใใใใฏๆๆฅใฎ่ณชๅใ ใใจๅใใฃใใใใงๆๆฅใไธฆในใฆใใใ
Macใฎใปใใฏใไธ็ใงๅใใฆๆฌกใฎใฌใใซใซๅฐ้ใใๆใใจใใพใใง้ใ่ฉฑใใใฆใใพใใ
ใใใฒใจใคใ
Mac The capital of France is the first of the 20th century.
3060 The capital of France is a major factor in the development of
the French French French French French French French FrenchใFrenchใใฎ้ฃๅผใฏใMacใงใฏไธๅบฆใๅบใชใใฃใๅฃใๆนใงใใ
ๅใใใผใฟใๅใ้ ็ชใง่ชญใพใใฆใใใฎใซใไธธใๆนใๅฐใ้ใใ ใใงใ
้ธใถๅ่ชใๅใใใฆใใใพใใ็นๆฐใฏๅใใงใใๆธใใใฎใฏๅฅ็ฉใซใชใใ
ใใใฆ5,000ในใใใๅพใ2ๅฐใฏๅใๅ ดๆใซใใฉใ็ใใพใใใ
Mac The capital of France is the capital of France, which is
the capital of France. The capital of France
3060 The capital of France is the capital of France, and the capital
of France is the capital of France.้ใ็ฐๅขใงใ้ใ็ฒพๅบฆใงใ้ใ้ใ้ใฃใฆใๅใๅ ใ ๅทกใใงใใ
6GBใซใไฝใ่ผใใฎใ
ใใใใใฏใใใฃใใGPUใใใใฎใงๆธฌใฃใฆใใใใใฃใใใจใงใใ
ๅๅใใไฝฟใฃใฆใใชใใฎใชใใใใฃใจๅคงใใใขใใซใ่ผใใฏใใงใใ
--depthย ใไธใใชใใใใฉใใง้็ใๆฅใใใ่ฉฆใใพใใใ
ๅ่จญๅฎใ3ใ12ในใใใใ ใๅใใฆใใใผใฏใฎใกใขใชใจ้ใใๆกใใพใใ
่จญๅฎ ใใฉใกใผใฟๆฐ ใใผใฏ tok/sec
d6 seq512 bs32 73,531,646 3,002 MiB 91,161
d8 seq1024 bs8 125,829,546 2,721 MiB 44,869
d12 seq1024 bs8 286,262,162 5,471 MiB 17,224d12 ใพใง่ผใใพใใใGPT-1ใจๅใ่ฆๆจก๏ผ2ๅ8,626ไธๅ๏ผใใ
6GBใฎใฒใผใใณใฐใใผใใซๅใพใฃใฆใใพใใ
้ไธญใงใใใใใชใใจใ่ตทใใฆใใพใใ
d6 73,531,646ๅ 3,002 MiB
d8 125,829,546ๅ 2,721 MiB โ ใขใใซใฏๅคงใใใฎใซใใกใขใชใฏๆธใฃใใใฉใกใผใฟใ1.7ๅใซใชใฃใใฎใซใไฝฟใฃใใกใขใชใฏๆธใใพใใใ
็็ฑใฏๅๅใฎใใพใใซๆธใใใจใใใงใใๅ ดๆใ้ฃใฃใฆใใใฎใฏใขใใซใงใฏใชใใ
่จ็ฎใฎ้ไธญใงไฝใใใใ็นๆฐ่กจใใฎใปใใงใใใ
d6 ใใใ32 ร 512ใใผใฏใณ 32 ร 512 ร 32,768 ร 4ใใคใ = 2.00 GB
d8 ใใใ8 ร 1024ใใผใฏใณ 8 ร 1024 ร 32,768 ร 4ใใคใ = 1.00 GB็นๆฐ่กจใๅๅใซใชใฃใใฎใงใใขใใซใ5,200ไธๅๅขใใฆใ็ท้ใฏไธใใใ
ใๅฑคใๅขใใใจ้ใใชใใใฎใงใฏใชใใใไธๅบฆใซๅฆ็ใใ้ใ้ใใใฎใงใใใ
ๅฃใ่ถใใฆใใ่ฝใกใชใใฃใ
ใใใใใใไปๅใใกใฐใ้ฉใใใจใใใงใใ
ใใฃใจๅคงใใใใฆใใใพใใ6,144MiBใ่ถ
ใใใใ
ใใกใขใชใ่ถณใใพใใใใจใใใจใฉใผใงๆญขใพใใฏใใงใใ
d12 seq1024 bs16 286,262,162ๅ 8,698 MiB 468 tok/sec8,698MiBใ
ๆญ่ผใใฆใใใฎใฏ6,144MiBใงใใ็ฉ็็ใซๅญๅจใใชใ้ใไฝฟใฃใฆใใพใใ
ใใใฆๆญขใพใใพใใใงใใใ ใจใฉใผใๅบใพใใใ
่ถณใใชใใถใใฏใใใฝใณใณๆฌไฝใฎใกใขใชใซ้ใใใใฆใใพใใใ
Windowsใซใใใใไป็ตใฟใใใใพใใใใ ใใไปฃๅใ้ๅบฆใงใใใ
6GBใซๅใพใฃใใจใ 17,224 tok/sec
6GBใ่ถ
ใใใจใ 468 tok/sec
โโโโโโโโโโโโโโโโโโโโโโโโโโโโ
36.8ๅใฎๆธ้ๅๅใMacใๆฏ็ง450MBๅใฎใใผใฟใ็ณใใงใฏๅบใ็ดใใฆ็ฒใฃใฆใใ่ฉฑใๆธใใพใใใ
ๅใใใจใใใใกใใงใ่ตทใใฆใใพใใใ
ๅๅใฎไบๅใงใ็งใฏใใๆธใใพใใใ
Macใฏๅง็ธฎใใฆใงใ็ฒใฃใฆใใใพใใใใใฐใฉใใฃใใฏใใผใใฏ็ฒใใพใใใ
็ฒใใพใใใ่จๆญฃใใพใใ
ใใฃใจๅคงใใใใฆใใใพใ ่ฝใกใพใใใ
d16 seq1024 bs16 536,872,506ๅ 14,826 MiB 174 tok/sec
d20 seq1024 bs4 896,534,946ๅ 13,100 MiB 37 tok/sec5ๅ3,687ไธๅใ8ๅ9,653ไธๅใ6GBใฎใซใผใใงใ14.8GBใไฝฟใฃใฆใใพใใ
้ใใฎใปใใไธฆในใใจใใใใชใใพใใ
d6 ๏ผ6GBใซๅใพใ๏ผ 91,161 tok/sec
d12 ๏ผๅฐใ่ถ
ใใ๏ผ 468
d16 174
d20 37
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
d6ใจd20ใฎๅทฎ 2,464ๅ37 tok/sec ใจใใใฎใฏใไปๆฅใฎ5,000ในใใใใ็ตใใใฎใซ25ๆฅใใใ้ ใใงใใ
ใจใฉใผใๅบใชใใปใใใใใกใๆชใใ
่ฝใกใฆใใใใฐ่จญๅฎใ็ดใใพใใใใใ ้
ใใ ใใ ใจใ
ไฝใใพใใใฎใๆฐไปใใฎใซๆ้ใใใใใพใใ
ใใใใๆธใใใฆใใๅฉใใใชใๅ ดๅใใใ
ๆบๅใฎใจใใ็งใฏ่ชๅ็จใฎใกใขใซใใๆธใใฆใใพใใใ
OOMใใใ --device-batch-sizeย ใ 8 โ 4 โ 2 ใจๅๅใซใใใ
ๅพ้ ็ดฏ็ฉใ่ชๅใงๅนใใฎใงใ็ตๆใฏๅคใใใชใใไผธใณใใฎใฏๆ้ใ ใใ
ไธๅบฆใซ้ใถ้ใๆธใใใฆใๅพๅพฉใๅขใใใ้ใถ็ท้ใฏๅคใใใชใใ
ใใๆใฃใฆใใพใใใ
่ฉฆใใฆใฟใพใใ
d16 seq1024 bs16 14,826 MiB 174 tok/sec
d16 seq1024 bs4 7,378 MiB 155 tok/sec โ 4ๅใฎ1ใซใใใใใใ4ๅใฎ1ใซใใฆใใกใขใชใฏๅๅใซใชใใพใใใ
ใงใ 7,378MiB ใงใใพใ 6,144 ใ่ถ
ใใฆใใพใใ้ๅบฆใๆปใใพใใใ
ใชใใใจใใใจใใใใใๆธใใใฆ็ฏ็ดใงใใใฎใฏ็นๆฐ่กจใจ้ไธญใฎ่จ็ฎ็ตๆใ ใใ ใใใงใใ
ใขใใซใฎ้ใฟใจใๅญฆ็ฟใซไฝฟใไป้ใใผใฟใฏใใใใใ1ใงใๅใๅ ดๆใๅใใพใใ
ใใใ ใใ่จ็ฎใใฆใฟใพใใใ
ๆทฑใ ใใฉใกใผใฟๆฐ ้ใฟใจไป้ใใผใฟใ ใใง 6,144MiBใซๅใพใใ
d6 73,531,646 705 MiB ๅใพใ
d8 125,829,546 1,184 MiB ๅใพใ
d12 286,262,162 2,616 MiB ๅใพใ
d16 536,872,506 4,800 MiB ๅใพใ๏ผๆฎใ1.3GB๏ผ
d20 896,534,946 7,880 MiB ๅใพใใชใ
d24 1,384,123,850 12,000 MiB ๅใพใใชใd16 ใฏใ็ฝฎใใ ใใง4,800MiBใๆฎใใฏ1.3GBใใใใใพใใใ
ใใใซ็นๆฐ่กจใจ่จ็ฎใฎ้ไธญ็ต้ใๅ
จ้จๅ
ฅใใใฎใฏ็ก็ใงใใใ
d20ใใไธใฏใใใใใใฉใใ ใๅใฃใฆใ่ผใใพใใใ
ใขใใซใ็ฝฎใใๆ็นใงใใใๅ ดๆใ็กใใใใงใใ
ใไธๅบฆใซ้ใถ้ใๆธใใใฐใใใใฏๆญฃใใใฎใงใใใ
ใใใใ่ท็ฉใๅๅบซใซๅ
ฅใใชใๅ ดๅใใใใพใใ
ใใใฆใใใใใ่ฝใกใ
ๆๅพใซใH100ใ8ๆๅใใฆๅใๆฌ็ชใฎ่จญๅฎ๏ผ็ฌฌ12ๅใงใใใพใ๏ผใใ
ใใฎใใผใPCใซๆใใฆใฟใพใใใ
d24 seq2048 bs4 1,384,123,850ๅ14ๅๅใ้ใฟใ็ฝฎใใ ใใง12GBใงใใ
ใพใใใขใใซใฎ็ตใฟ็ซใฆ่ชไฝใฏ้ใใพใใใ
Parameter counts:
wte : 50,331,648
value_embeds : 603,979,776
lm_head : 50,331,648
transformer_matrices : 679,480,704
scalars : 74
total : 1,384,123,850็ฌฌ5ๅใงๆใงๆฐใใใฎใจๅใๅ ่จณใงใใ14ๅๅใถใใฎๅ ดๆใฏใใชใใจใ็ขบไฟใงใใพใใใ
่ฝใกใใฎใฏใใใฎๆฌกใงใใใ
File "nanochat/optim.py", line 454, in step
self._compute_muon(group, info, gather_list, rank)
torch.AcceleratorError: CUDA error: out of memory
Search for `cudaErrorMemoryAllocation' in
https://docs.nvidia.com/cuda/cuda-runtime-api/...ๅใใฆใจใฉใผใๅบใพใใใ
optim.pyย ใฏใๅญฆ็ฟใใ็ตๆใไฝฟใฃใฆ้ใฟใๆธใๆใใ้จๅใงใใ
ใขใใซใฏ็ฝฎใใใ่ชญใฟ่พผใฟใ้ใฃใใ1ๅ็ฎใฎๆธใๆใใงใๅๅฐฝใใพใใใ
็ฒใฃใฆใ็ฒใฃใฆใใทในใใ ใฎใกใขใชใพใง้ฃใๅฐฝใใใฆใใใใง็ตใใใงใใ
็ฌฌ5ๅใงๆฐใใใจใใใใใฎd24ใฎใขใใซใฏใ
ไธญ่บซใฎๅๅใ่ชๅฝใฎ่กจ๏ผ32,768่ก ร 1,536ๅใ14ๅ๏ผใงใใ
่กจใ14ๅๆฑใใใพใพใ6GBใฎใซใผใใซๅบงใใใจใใฆใๅบงใใชใใฃใใ
ใใฎๅใฎๆฐๅญ
0.000047
32ใใใใ16ใใใใซใใฆใ8ๅ้ใใใฆใ
50ๅๆธฌใฃใๅฝใฆใซใใใฎๅนณๅใฎๅทฎใใใใใงใใใ
็งใฏใใฎๅใใใ็ฒใใใใจใใใฎใถใๆชใใชใใใจใใ่ฉฑใซใชใใจๆใฃใฆๅงใใพใใใ
็ฎฑใๅๅใซใใใฎใ ใใใไฝใใๅคฑใใฏใใ ใจใ
ๅฎ้ใซใฏใๅคฑใฃใใใฎใๆธฌใใพใใใงใใใ
50ๅใฎใใก28ๅใฏใ็ฒใใปใใ่ฏใๆ็ธพใงใใใ
็ฌฌ13ๅใงใใ Q4_K_M ใฏใใใฎๅ
ใฎ่ฉฑใงใใ
16ใใใใ4.92ใใใใพใงๅใใพใใไปๆฅใฎๅๅใฎใใใฎใพใ3ๅใฎ1ใ
ใใใพใงๅใฃใฆใๅนณๆฐใชใฎใใไปๆฅใฎ็ตๆใ่ฆใใใจใ ใจใ
ๅนณๆฐใใใใใชใใจๆใใฆใใใฎใใใใกใฐใใฎๅ็ฉซใงใใใ
ใใใฆใใใฒใจใคใ
6GBใฎใฐใฉใใฃใใฏใใผใใฏใ6GBใ่ถ
ใใฆใๆญขใพใใพใใใ้ปใฃใฆ2,464ๅ้
ใใชใใพใใ
Macใๅใใงใใใๆฏ็ง450MBใ็ณใใงใฏๅบใใฆใ2ๆ้่ตฐใๅใใพใใใ
ๆฉๆขฐใฏใ้็ใๆใใฆใใใพใใใ ๆธฌใใชใใจๅใใใพใใใ
ใใพใ๏ผWindowsใฎไธญใฎLinuxใซใๅคใใๅ ฅใ
ไปๅใMacใใ3060ใฎใใผใPCใซๅ
ฅใฃใฆไฝๆฅญใใพใใใ
WindowsใฎไธญใซLinuxใ1ๅ็ซใฆใฆ๏ผWSL2ใจใใใพใ๏ผใใใใซๅคใใใคใชใๅฝขใงใใ
nanochat ใLinuxใๅๆใซๆธใใใฆใใใใใงใใ
torch.compileย ใจใใไป็ตใฟใจใWindowsใซๅญๅจใใชใ้ๅ
ทใ3ใๆใงไฝฟใใใฆใใพใใ
ใคใชใใ ็ดๅพใGPUใ่ฆใใพใใใงใใใ
nvidia-smi: command not foundWindowsใฎไธญใฎLinuxใฏใๆฎๆฎตใฏWindowsๅดใฎ่จญๅฎใๅผใ็ถใใงใใพใใ
ใจใใใๅคใใใคใชใใจใใใใๅผใ็ถใใใพใใใ
ใฐใฉใใฃใใฏใใผใใ่งฆใใใใฎ้ๅ
ทใ็ฝฎใใฆใใๅ ดๆใใ่ฆใใชใใชใฃใฆใใพใใใ
1่ก่ถณใใฆ็ดใใพใใใ
export PATH="$PATH:/usr/lib/wsl/lib"WindowsใฎไธญใฎLinuxใซใMacใใๅ
ฅใใ ใใฎใจใใ ใ่ตทใใ็พ่ฑกใงใใ
ๅใใใจใใใใใจใใไบบใๅฑ
ใใใใใใง30ๅๆบถใใใใซๆธใฟใพใใ
ๆฌกๅ
ๆฐๅญใฎๆๅณใใใใใใ็ไปใใพใใ
ไปๅใๅฝใฆใซใใใ 1.165885 ใจ 1.165166 ใงใใปใผๅใใใจๆธใใพใใใ
ใงใฏใใใฎ 1.16 ใจใใๆฐๅญใฏใ่ฏใใฎใงใใใใใๆชใใฎใงใใใใใ
ๅใฎ้ฃ่ผใงใฏใๆๅคฑใ4.60ใใ0.03ใพใงไธใใฃใฆๅคงๅใณใใพใใใ
ใใฎใใจใๆก็นใซไฝฟใฃใๆ็ซ ใใใใฎใพใพๅญฆ็ฟใซใไฝฟใฃใฆใใใจๆฐใฅใใพใใ
ใขใใซใฏไธธๆ่จใใๆ็ซ ใ็ญใใฆใใใ ใใงใใใ
็ญใใ่ฆใชใใใในใใๅใใใใฆใๆบ็นใ ใจๅใใงใใใใใงใใ
ๅญฆ็ฟใซไฝฟใฃใๆ็ซ ใๅคใใฆๆธฌใ็ดใใใใ4.59ใ9.65ใพใง่ทณใญไธใใใพใใใ
ใงใใใใซ็ญใใใจใใใๆชใ็นใงใใ
ไปๅใฎ 1.16 ใฏใใใฎใจใใฎๆฐๅญใจๅไฝใ้ใใพใใ
ใใคใใใใไฝใใใใจใใๆธฌใๆนใงใๅใๆนใฎ้ใใขใใซๅๅฃซใงใๆฏในใใใใ
ใใใฆ nanochat ใซใฏใใใใฒใจใค็ฉๅทฎใใใใใพใใ
GPT-2 ใฎ็นๆฐใ 0.256525 ใจๆธใใฆใใฃใฆใใใใ่ถ
ใใใใจใ็ฎๆจ็ทใซใชใฃใฆใใพใใ
ๆฌกๅใฏใไปๆฅใงใใใขใใซใซใใฎ็ฉๅทฎใใๅฝใฆใพใใ
GPT-2ใพใงใใใจใฉใใใใใใ
ใงใฏใงใฏใ
ๆณจ๏ผใใฎ่จไบใฎๅท็ญใซใฏAIใไฝฟใฃใฆใใพใใๆงๆใจๆ็ต็ใชๆ็ซ ใฏ่ชๅใฎๆใซใใใใฎใงใใ