Exl2
mistral-cookbookmethodsquantizationconcept-deep-dive
Export
EXL2
A more recent format based on the GPTQ optimization method but with mixed quantization levels. It achieves an average desired bitrate with lower errors than GPTQ while keeping the same or similar bitrate. Can have a slightly higher VRAM usage but better inference speed and quality.
Quantizing with exllamav2
Lets do a short demo and quantize Mistral 7B!
let's install exllamav2 and all dependencies required.
[1]
Cloning into 'exllamav2'... remote: Enumerating objects: 6958, done. remote: Counting objects: 100% (1276/1276), done. remote: Compressing objects: 100% (550/550), done. remote: Total 6958 (delta 909), reused 1042 (delta 721), pack-reused 5682 (from 1) Receiving objects: 100% (6958/6958), 19.88 MiB | 24.76 MiB/s, done. Resolving deltas: 100% (4908/4908), done. Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 1)) (2.1.4) Collecting ninja (from -r requirements.txt (line 2)) Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl.metadata (5.3 kB) Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (0.44.0) Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (71.0.4) Collecting fastparquet (from -r requirements.txt (line 5)) Downloading fastparquet-2024.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB) Requirement already satisfied: torch>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (2.3.1+cu121) Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (0.4.4) Requirement already satisfied: sentencepiece>=0.1.97 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (0.1.99) Requirement already satisfied: pygments in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (2.16.1) Collecting websockets (from -r requirements.txt (line 10)) Downloading websockets-13.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB) Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 11)) (2024.5.15) Requirement already satisfied: numpy~=1.26.4 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (1.26.4) Requirement already satisfied: tokenizers in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (0.19.1) Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (13.7.1) Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 1)) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 1)) (2024.1) Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 1)) (2024.1) Collecting cramjam>=2.3 (from fastparquet->-r requirements.txt (line 5)) Downloading cramjam-2.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB) Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from fastparquet->-r requirements.txt (line 5)) (2024.6.1) Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from fastparquet->-r requirements.txt (line 5)) (24.1) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (3.15.4) Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (4.12.2) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (1.13.2) Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (3.3) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (3.1.4) Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting nvidia-cufft-cu12==11.0.2.54 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting nvidia-curand-cu12==10.3.2.106 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB) Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) Collecting nvidia-nccl-cu12==2.20.5 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl.metadata (1.8 kB) Collecting nvidia-nvtx-cu12==12.1.105 (from torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB) Requirement already satisfied: triton==2.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->-r requirements.txt (line 6)) (2.3.1) Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch>=2.2.0->-r requirements.txt (line 6)) Using cached nvidia_nvjitlink_cu12-12.6.20-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB) Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers->-r requirements.txt (line 13)) (0.23.5) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->-r requirements.txt (line 14)) (3.0.0) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (6.0.2) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (2.32.3) Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (4.66.5) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich->-r requirements.txt (line 14)) (0.1.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->-r requirements.txt (line 1)) (1.16.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.2.0->-r requirements.txt (line 6)) (2.1.5) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.2.0->-r requirements.txt (line 6)) (1.3.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (2.0.7) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers->-r requirements.txt (line 13)) (2024.7.4) Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 6.2 MB/s eta 0:00:00 Downloading fastparquet-2024.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 47.4 MB/s eta 0:00:00 Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB) Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB) Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB) Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB) Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB) Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB) Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB) Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB) Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB) Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB) Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB) Downloading websockets-13.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (157 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.2/157.2 kB 321.0 kB/s eta 0:00:00 Downloading cramjam-2.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 3.4 MB/s eta 0:00:00 Using cached nvidia_nvjitlink_cu12-12.6.20-py3-none-manylinux2014_x86_64.whl (19.7 MB) Installing collected packages: ninja, websockets, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, cramjam, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12, fastparquet Successfully installed cramjam-2.8.3 fastparquet-2024.5.0 ninja-1.11.1.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.6.20 nvidia-nvtx-cu12-12.1.105 websockets-13.0 Processing /content/exllamav2 Preparing metadata (setup.py) ... done Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (2.1.4) Requirement already satisfied: ninja in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (1.11.1.1) Requirement already satisfied: fastparquet in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (2024.5.0) Requirement already satisfied: torch>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (2.3.1+cu121) Requirement already satisfied: safetensors>=0.3.2 in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (0.4.4) Requirement already satisfied: sentencepiece>=0.1.97 in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (0.1.99) Requirement already satisfied: pygments in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (2.16.1) Requirement already satisfied: websockets in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (13.0) Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (2024.5.15) Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (1.26.4) Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.1.8) (13.7.1) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (3.15.4) Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (4.12.2) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (1.13.2) Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (3.3) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (3.1.4) Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (2024.6.1) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.105) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.105) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.105) Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (8.9.2.26) Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.3.1) Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (11.0.2.54) Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (10.3.2.106) Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (11.4.5.107) Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.0.106) Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (2.20.5) Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (12.1.105) Requirement already satisfied: triton==2.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->exllamav2==0.1.8) (2.3.1) Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=2.2.0->exllamav2==0.1.8) (12.6.20) Requirement already satisfied: cramjam>=2.3 in /usr/local/lib/python3.10/dist-packages (from fastparquet->exllamav2==0.1.8) (2.8.3) Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from fastparquet->exllamav2==0.1.8) (24.1) Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2==0.1.8) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2==0.1.8) (2024.1) Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2==0.1.8) (2024.1) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->exllamav2==0.1.8) (3.0.0) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich->exllamav2==0.1.8) (0.1.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->exllamav2==0.1.8) (1.16.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.2.0->exllamav2==0.1.8) (2.1.5) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.2.0->exllamav2==0.1.8) (1.3.0) Building wheels for collected packages: exllamav2 Building wheel for exllamav2 (setup.py) ... done Created wheel for exllamav2: filename=exllamav2-0.1.8-cp310-cp310-linux_x86_64.whl size=51731505 sha256=d0a886225c9d25596867e0c74a704666e260373813d207e0467a7ee71572b020 Stored in directory: /tmp/pip-ephem-wheel-cache-mgnjedvg/wheels/5a/f8/dc/09b6466fc29d3ca538340226ed551df529f09a59f7650f5ec0 Successfully built exllamav2 Installing collected packages: exllamav2 Successfully installed exllamav2-0.1.8
Once everything installed we can download the model.
[2]
Git LFS initialized. Cloning into 'Mistral-7B-Instruct-v0.3'... remote: Enumerating objects: 93, done. remote: Counting objects: 100% (89/89), done. remote: Compressing objects: 100% (89/89), done. remote: Total 93 (delta 46), reused 0 (delta 0), pack-reused 4 (from 1) Unpacking objects: 100% (93/93), 734.97 KiB | 4.20 MiB/s, done. Filtering content: 100% (5/5), 3.00 GiB | 12.59 MiB/s, done. Encountered 4 file(s) that may not have been copied correctly on Windows: model-00002-of-00003.safetensors model-00003-of-00003.safetensors model-00001-of-00003.safetensors consolidated.safetensors See: `git lfs help smudge` for more details.
Time to quantize! Lets go with a bitrate of 4.0
[ ]
Streaming output truncated to the last 5000 lines.
-- model.layers.2.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.2.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.2.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.2.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.2.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.2.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.2.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.2.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.2.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.2.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.2.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.2.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.2.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.2.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.2.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.2.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.2.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.2.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.2.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.2.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.2.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.2.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.2.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.2.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.2.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.2.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.2.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.2.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.98242324
-- 2.1987 bpw accuracy: 0.98377117
-- 2.2831 bpw accuracy: 0.98568281
-- 2.6768 bpw accuracy: 0.98892056
-- 3.1689 bpw accuracy: 0.99101379
-- 3.1705 bpw accuracy: 0.99098715
-- 4.0439 bpw accuracy: 0.99453021
-- 4.0471 bpw accuracy: 0.99469697
-- 4.0816 bpw accuracy: 0.99504670
-- 4.1381 bpw accuracy: 0.99529099
-- 4.1705 bpw accuracy: 0.99531599
-- 4.1902 bpw accuracy: 0.99563936
-- 4.2737 bpw accuracy: 0.99624683
-- 4.3295 bpw accuracy: 0.99665906
-- 5.2564 bpw accuracy: 0.99801150
-- 5.3295 bpw accuracy: 0.99832702
-- 6.0439 bpw accuracy: 0.99850651
-- 6.3381 bpw accuracy: 0.99915358
-- 8.0439 bpw accuracy: 0.99959020
--------------------------------------------
| Measured: model.layers.2 (Attention) |
| Duration: 12.13 seconds |
| Completed step: 5/67 |
| Avg time / step (rolling): 21.26 seconds |
| Estimated remaining time: 21min 58sec |
| Last checkpoint layer: None |
--------------------------------------------
-- Layer: model.layers.2 (MLP)
-- model.layers.2.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.2.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.2.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.2.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.2.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.2.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.2.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.2.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.2.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.2.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.2.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.2.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.2.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.2.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.2.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.2.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.2.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.2.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.2.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.2.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.2.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.2.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.2.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.2.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.2.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.2.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.2.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.2.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.2.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.2.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.2.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.2.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.97782768
-- 2.3230 bpw accuracy: 0.97843042
-- 2.5958 bpw accuracy: 0.98211724
-- 2.9120 bpw accuracy: 0.98331300
-- 3.2833 bpw accuracy: 0.98874731
-- 3.3655 bpw accuracy: 0.98963368
-- 3.6186 bpw accuracy: 0.99129309
-- 4.1368 bpw accuracy: 0.99411108
-- 4.1977 bpw accuracy: 0.99466143
-- 4.2662 bpw accuracy: 0.99425921
-- 4.3484 bpw accuracy: 0.99497052
-- 5.2491 bpw accuracy: 0.99707504
-- 5.3313 bpw accuracy: 0.99749370
-- 6.0713 bpw accuracy: 0.99841222
-- 6.3032 bpw accuracy: 0.99850878
-- 6.8687 bpw accuracy: 0.99884521
-- 8.0354 bpw accuracy: 0.99957238
--------------------------------------------
| Measured: model.layers.2 (MLP) |
| Duration: 33.99 seconds |
| Completed step: 6/67 |
| Avg time / step (rolling): 23.38 seconds |
| Estimated remaining time: 23min 46sec |
| Last checkpoint layer: None |
--------------------------------------------
-- Layer: model.layers.3 (Attention)
-- model.layers.3.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.3.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.3.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.3.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.3.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.3.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.3.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.3.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.3.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.3.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.3.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.3.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.3.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.3.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.3.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.3.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.3.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.3.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.3.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.3.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.3.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.3.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.3.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.3.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.3.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.3.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.3.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.3.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.3.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.3.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.3.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.3.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.3.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.3.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.3.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.3.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.3.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.3.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.3.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.3.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.3.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.3.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.3.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.3.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.3.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.3.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.3.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.3.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.3.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.3.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.3.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.3.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.3.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.3.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.3.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.3.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.3.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.98109958
-- 2.1987 bpw accuracy: 0.98228964
-- 2.2831 bpw accuracy: 0.98462530
-- 2.6768 bpw accuracy: 0.98846251
-- 3.1689 bpw accuracy: 0.98989627
-- 3.1705 bpw accuracy: 0.99019452
-- 4.0439 bpw accuracy: 0.99267435
-- 4.0471 bpw accuracy: 0.99309814
-- 4.0816 bpw accuracy: 0.99372198
-- 4.1381 bpw accuracy: 0.99405704
-- 4.1705 bpw accuracy: 0.99500043
-- 4.1902 bpw accuracy: 0.99540469
-- 4.2737 bpw accuracy: 0.99572001
-- 4.3295 bpw accuracy: 0.99607138
-- 5.2564 bpw accuracy: 0.99757583
-- 5.3295 bpw accuracy: 0.99799545
-- 6.0439 bpw accuracy: 0.99800412
-- 6.3381 bpw accuracy: 0.99908931
-- 8.0439 bpw accuracy: 0.99947425
--------------------------------------------
| Measured: model.layers.3 (Attention) |
| Duration: 11.98 seconds |
| Completed step: 7/67 |
| Avg time / step (rolling): 21.75 seconds |
| Estimated remaining time: 21min 45sec |
| Last checkpoint layer: None |
--------------------------------------------
-- Layer: model.layers.3 (MLP)
-- model.layers.3.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.3.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.3.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.3.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.3.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.3.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.3.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.3.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.3.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.3.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.3.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.3.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.3.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.3.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.3.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.3.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.3.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.3.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.3.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.3.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.3.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.3.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.3.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.3.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.3.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.3.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.3.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.3.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.3.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.3.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.3.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.3.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.97330996
-- 2.3230 bpw accuracy: 0.97404764
-- 2.5958 bpw accuracy: 0.97811245
-- 2.9120 bpw accuracy: 0.97940267
-- 3.2833 bpw accuracy: 0.98638627
-- 3.3655 bpw accuracy: 0.98746937
-- 3.6186 bpw accuracy: 0.98926819
-- 4.1368 bpw accuracy: 0.99289792
-- 4.1977 bpw accuracy: 0.99355503
-- 4.2662 bpw accuracy: 0.99305358
-- 4.3484 bpw accuracy: 0.99391918
-- 5.2491 bpw accuracy: 0.99645910
-- 5.3313 bpw accuracy: 0.99696968
-- 6.0713 bpw accuracy: 0.99808397
-- 6.3032 bpw accuracy: 0.99819815
-- 6.8687 bpw accuracy: 0.99855809
-- 8.0354 bpw accuracy: 0.99948509
--------------------------------------------
| Measured: model.layers.3 (MLP) |
| Duration: 33.74 seconds |
| Completed step: 8/67 |
| Avg time / step (rolling): 23.25 seconds |
| Estimated remaining time: 22min 51sec |
| Last checkpoint layer: None |
--------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.4 (Attention)
-- model.layers.4.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.4.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.4.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.4.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.4.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.4.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.4.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.4.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.4.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.4.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.4.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.4.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.4.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.4.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.4.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.4.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.4.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.4.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.4.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.4.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.4.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.4.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.4.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.4.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.4.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.4.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.4.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.4.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.4.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.4.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.4.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.4.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.4.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.4.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.4.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.4.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.4.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.4.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.4.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.4.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.4.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.4.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.4.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.4.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.4.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.4.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.4.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.4.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.4.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.4.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.4.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.4.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.4.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.4.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.4.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.4.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.4.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.98252585
-- 2.1987 bpw accuracy: 0.98348221
-- 2.2831 bpw accuracy: 0.98561841
-- 2.6768 bpw accuracy: 0.98871386
-- 3.1689 bpw accuracy: 0.99040757
-- 3.1705 bpw accuracy: 0.99075949
-- 4.0439 bpw accuracy: 0.99289451
-- 4.0471 bpw accuracy: 0.99318537
-- 4.0816 bpw accuracy: 0.99409265
-- 4.1381 bpw accuracy: 0.99428539
-- 4.1705 bpw accuracy: 0.99511318
-- 4.1902 bpw accuracy: 0.99563079
-- 4.2737 bpw accuracy: 0.99580912
-- 4.3295 bpw accuracy: 0.99628595
-- 5.2564 bpw accuracy: 0.99768850
-- 5.3295 bpw accuracy: 0.99816330
-- 6.0439 bpw accuracy: 0.99805899
-- 6.3381 bpw accuracy: 0.99915348
-- 8.0439 bpw accuracy: 0.99948567
-----------------------------------------------
| Measured: model.layers.4 (Attention) |
| Duration: 11.79 seconds |
| Completed step: 9/67 |
| Avg time / step (rolling): 21.98 seconds |
| Estimated remaining time: 21min 14sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.4 (MLP)
-- model.layers.4.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.4.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.4.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.4.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.4.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.4.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.4.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.4.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.4.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.4.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.4.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.4.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.4.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.4.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.4.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.4.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.4.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.4.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.4.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.4.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.4.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.4.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.4.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.4.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.4.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.4.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.4.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.4.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.4.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.4.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.4.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.4.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.96977647
-- 2.3230 bpw accuracy: 0.97067890
-- 2.5958 bpw accuracy: 0.97548738
-- 2.9120 bpw accuracy: 0.97696493
-- 3.2833 bpw accuracy: 0.98466234
-- 3.3655 bpw accuracy: 0.98591853
-- 3.6186 bpw accuracy: 0.98797810
-- 4.1368 bpw accuracy: 0.99197283
-- 4.1977 bpw accuracy: 0.99271050
-- 4.2662 bpw accuracy: 0.99217591
-- 4.3484 bpw accuracy: 0.99316832
-- 5.2491 bpw accuracy: 0.99601106
-- 5.3313 bpw accuracy: 0.99659400
-- 6.0713 bpw accuracy: 0.99782196
-- 6.3032 bpw accuracy: 0.99796597
-- 6.8687 bpw accuracy: 0.99838741
-- 8.0354 bpw accuracy: 0.99940790
-----------------------------------------------
| Measured: model.layers.4 (MLP) |
| Duration: 33.87 seconds |
| Completed step: 10/67 |
| Avg time / step (rolling): 23.17 seconds |
| Estimated remaining time: 22min 0sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.5 (Attention)
-- model.layers.5.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.5.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.5.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.5.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.5.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.5.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.5.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.5.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.5.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.5.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.5.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.5.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.5.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.5.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.5.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.5.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.5.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.5.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.5.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.5.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.5.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.5.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.5.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.5.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.5.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.5.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.5.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.5.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.5.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.5.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.5.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.5.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.5.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.5.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.5.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.5.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.5.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.5.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.5.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.5.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.5.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.5.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.5.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.5.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.5.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.5.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.5.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.5.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.5.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.5.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.5.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.5.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.5.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.5.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.5.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.5.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.5.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.97664799
-- 2.1987 bpw accuracy: 0.97792282
-- 2.2831 bpw accuracy: 0.98103946
-- 2.6768 bpw accuracy: 0.98568327
-- 3.1689 bpw accuracy: 0.98804598
-- 3.1705 bpw accuracy: 0.98837588
-- 4.0439 bpw accuracy: 0.99172002
-- 4.0471 bpw accuracy: 0.99216411
-- 4.0816 bpw accuracy: 0.99320019
-- 4.1381 bpw accuracy: 0.99348132
-- 4.1705 bpw accuracy: 0.99413244
-- 4.1902 bpw accuracy: 0.99458293
-- 4.2737 bpw accuracy: 0.99511615
-- 4.3295 bpw accuracy: 0.99549039
-- 5.2564 bpw accuracy: 0.99733608
-- 5.3295 bpw accuracy: 0.99771802
-- 6.0439 bpw accuracy: 0.99795961
-- 6.3381 bpw accuracy: 0.99883434
-- 8.0439 bpw accuracy: 0.99945571
-----------------------------------------------
| Measured: model.layers.5 (Attention) |
| Duration: 11.86 seconds |
| Completed step: 11/67 |
| Avg time / step (rolling): 22.94 seconds |
| Estimated remaining time: 21min 24sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.5 (MLP)
-- model.layers.5.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.5.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.5.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.5.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.5.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.5.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.5.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.5.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.5.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.5.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.5.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.5.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.5.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.5.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.5.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.5.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.5.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.5.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.5.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.5.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.5.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.5.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.5.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.5.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.5.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.5.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.5.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.5.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.5.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.5.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.5.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.5.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.96435021
-- 2.3230 bpw accuracy: 0.96540036
-- 2.5958 bpw accuracy: 0.97121513
-- 2.9120 bpw accuracy: 0.97297229
-- 3.2833 bpw accuracy: 0.98198438
-- 3.3655 bpw accuracy: 0.98342225
-- 3.6186 bpw accuracy: 0.98589354
-- 4.1368 bpw accuracy: 0.99059182
-- 4.1977 bpw accuracy: 0.99144911
-- 4.2662 bpw accuracy: 0.99083530
-- 4.3484 bpw accuracy: 0.99197662
-- 5.2491 bpw accuracy: 0.99533982
-- 5.3313 bpw accuracy: 0.99600649
-- 6.0713 bpw accuracy: 0.99746123
-- 6.3032 bpw accuracy: 0.99763201
-- 6.8687 bpw accuracy: 0.99813379
-- 8.0354 bpw accuracy: 0.99931030
-----------------------------------------------
| Measured: model.layers.5 (MLP) |
| Duration: 33.92 seconds |
| Completed step: 12/67 |
| Avg time / step (rolling): 22.94 seconds |
| Estimated remaining time: 21min 1sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.6 (Attention)
-- model.layers.6.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.6.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.6.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.6.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.6.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.6.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.6.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.6.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.6.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.6.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.6.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.6.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.6.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.6.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.6.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.6.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.6.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.6.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.6.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.6.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.6.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.6.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.6.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.6.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.6.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.6.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.6.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.6.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.6.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.6.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.6.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.6.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.6.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.6.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.6.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.6.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.6.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.6.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.6.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.6.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.6.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.6.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.6.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.6.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.6.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.6.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.6.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.6.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.6.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.6.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.6.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.6.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.6.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.6.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.6.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.6.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.6.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.97556552
-- 2.1987 bpw accuracy: 0.97681589
-- 2.2831 bpw accuracy: 0.97946853
-- 2.6768 bpw accuracy: 0.98439448
-- 3.1689 bpw accuracy: 0.98764152
-- 3.1705 bpw accuracy: 0.98793082
-- 4.0439 bpw accuracy: 0.99126788
-- 4.0471 bpw accuracy: 0.99168501
-- 4.0816 bpw accuracy: 0.99258404
-- 4.1381 bpw accuracy: 0.99307442
-- 4.1705 bpw accuracy: 0.99386278
-- 4.1902 bpw accuracy: 0.99437859
-- 4.2737 bpw accuracy: 0.99466031
-- 4.3295 bpw accuracy: 0.99514121
-- 5.2564 bpw accuracy: 0.99706046
-- 5.3295 bpw accuracy: 0.99758078
-- 6.0439 bpw accuracy: 0.99770153
-- 6.3381 bpw accuracy: 0.99877531
-- 8.0439 bpw accuracy: 0.99936667
-----------------------------------------------
| Measured: model.layers.6 (Attention) |
| Duration: 11.83 seconds |
| Completed step: 13/67 |
| Avg time / step (rolling): 22.89 seconds |
| Estimated remaining time: 20min 35sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.6 (MLP)
-- model.layers.6.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.6.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.6.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.6.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.6.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.6.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.6.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.6.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.6.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.6.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.6.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.6.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.6.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.6.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.6.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.6.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.6.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.6.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.6.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.6.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.6.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.6.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.6.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.6.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.6.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.6.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.6.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.6.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.6.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.6.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.6.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.6.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.96055157
-- 2.3230 bpw accuracy: 0.96168662
-- 2.5958 bpw accuracy: 0.96828757
-- 2.9120 bpw accuracy: 0.97028112
-- 3.2833 bpw accuracy: 0.98004671
-- 3.3655 bpw accuracy: 0.98168241
-- 3.6186 bpw accuracy: 0.98446003
-- 4.1368 bpw accuracy: 0.98954397
-- 4.1977 bpw accuracy: 0.99051610
-- 4.2662 bpw accuracy: 0.98983606
-- 4.3484 bpw accuracy: 0.99112281
-- 5.2491 bpw accuracy: 0.99482468
-- 5.3313 bpw accuracy: 0.99557457
-- 6.0713 bpw accuracy: 0.99716802
-- 6.3032 bpw accuracy: 0.99736536
-- 6.8687 bpw accuracy: 0.99794299
-- 8.0354 bpw accuracy: 0.99923152
-----------------------------------------------
| Measured: model.layers.6 (MLP) |
| Duration: 33.80 seconds |
| Completed step: 14/67 |
| Avg time / step (rolling): 22.89 seconds |
| Estimated remaining time: 20min 13sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.7 (Attention)
-- model.layers.7.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.7.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.7.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.7.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.7.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.7.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.7.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.7.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.7.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.7.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.7.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.7.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.7.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.7.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.7.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.7.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.7.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.7.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.7.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.7.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.7.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.7.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.7.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.7.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.7.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.7.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.7.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.7.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.7.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.7.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.7.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.7.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.7.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.7.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.7.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.7.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.7.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.7.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.7.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.7.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.7.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.7.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.7.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.7.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.7.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.7.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.7.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.7.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.7.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.7.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.7.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.7.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.7.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.7.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.7.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.7.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.7.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.97214290
-- 2.1987 bpw accuracy: 0.97319920
-- 2.2831 bpw accuracy: 0.97652968
-- 2.6768 bpw accuracy: 0.98208666
-- 3.1689 bpw accuracy: 0.98597534
-- 3.1705 bpw accuracy: 0.98647040
-- 4.0439 bpw accuracy: 0.99031376
-- 4.0471 bpw accuracy: 0.99082901
-- 4.0816 bpw accuracy: 0.99154732
-- 4.1381 bpw accuracy: 0.99214164
-- 4.1705 bpw accuracy: 0.99316714
-- 4.1902 bpw accuracy: 0.99356058
-- 4.2737 bpw accuracy: 0.99389419
-- 4.3295 bpw accuracy: 0.99452353
-- 5.2564 bpw accuracy: 0.99661940
-- 5.3295 bpw accuracy: 0.99726697
-- 6.0439 bpw accuracy: 0.99739566
-- 6.3381 bpw accuracy: 0.99846183
-- 8.0439 bpw accuracy: 0.99932787
-----------------------------------------------
| Measured: model.layers.7 (Attention) |
| Duration: 11.93 seconds |
| Completed step: 15/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 19min 49sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Layer: model.layers.7 (MLP)
-- model.layers.7.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.7.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.7.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.7.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.7.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.7.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.7.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.7.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.7.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.7.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.7.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.7.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.7.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.7.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.7.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.7.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.7.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.7.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.7.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.7.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.7.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.7.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.7.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.7.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.7.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.7.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.7.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.7.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.7.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.7.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.7.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.7.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.95711282
-- 2.3230 bpw accuracy: 0.95840440
-- 2.5958 bpw accuracy: 0.96528786
-- 2.9120 bpw accuracy: 0.96737304
-- 3.2833 bpw accuracy: 0.97833324
-- 3.3655 bpw accuracy: 0.98013736
-- 3.6186 bpw accuracy: 0.98302650
-- 4.1368 bpw accuracy: 0.98867399
-- 4.1977 bpw accuracy: 0.98971790
-- 4.2662 bpw accuracy: 0.98894402
-- 4.3484 bpw accuracy: 0.99036205
-- 5.2491 bpw accuracy: 0.99436124
-- 5.3313 bpw accuracy: 0.99519460
-- 6.0713 bpw accuracy: 0.99691936
-- 6.3032 bpw accuracy: 0.99712549
-- 6.8687 bpw accuracy: 0.99772370
-- 8.0354 bpw accuracy: 0.99917376
-----------------------------------------------
| Measured: model.layers.7 (MLP) |
| Duration: 33.84 seconds |
| Completed step: 16/67 |
| Avg time / step (rolling): 22.86 seconds |
| Estimated remaining time: 19min 25sec |
| Last checkpoint layer: model.layers.3 (MLP) |
-----------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.8 (Attention)
-- model.layers.8.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.8.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.8.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.8.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.8.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.8.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.8.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.8.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.8.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.8.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.8.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.8.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.8.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.8.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.8.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.8.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.8.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.8.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.8.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.8.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.8.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.8.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.8.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.8.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.8.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.8.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.8.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.8.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.8.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.8.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.8.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.8.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.8.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.8.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.8.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.8.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.8.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.8.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.8.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.8.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.8.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.8.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.8.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.8.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.8.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.8.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.8.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.8.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.8.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.8.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.8.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.8.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.8.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.8.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.8.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.8.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.8.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.97076562
-- 2.1987 bpw accuracy: 0.97229849
-- 2.2831 bpw accuracy: 0.97526232
-- 2.6768 bpw accuracy: 0.98203508
-- 3.1689 bpw accuracy: 0.98482448
-- 3.1705 bpw accuracy: 0.98501838
-- 4.0439 bpw accuracy: 0.98959962
-- 4.0471 bpw accuracy: 0.98982176
-- 4.0816 bpw accuracy: 0.99111000
-- 4.1381 bpw accuracy: 0.99150279
-- 4.1705 bpw accuracy: 0.99236452
-- 4.1902 bpw accuracy: 0.99303760
-- 4.2737 bpw accuracy: 0.99337368
-- 4.3295 bpw accuracy: 0.99398298
-- 5.2564 bpw accuracy: 0.99635961
-- 5.3295 bpw accuracy: 0.99702405
-- 6.0439 bpw accuracy: 0.99729517
-- 6.3381 bpw accuracy: 0.99846129
-- 8.0439 bpw accuracy: 0.99928982
-----------------------------------------------
| Measured: model.layers.8 (Attention) |
| Duration: 11.84 seconds |
| Completed step: 17/67 |
| Avg time / step (rolling): 22.84 seconds |
| Estimated remaining time: 19min 2sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.8 (MLP)
-- model.layers.8.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.8.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.8.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.8.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.8.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.8.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.8.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.8.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.8.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.8.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.8.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.8.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.8.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.8.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.8.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.8.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.8.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.8.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.8.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.8.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.8.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.8.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.8.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.8.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.8.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.8.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.8.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.8.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.8.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.8.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.8.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.8.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.95471036
-- 2.3230 bpw accuracy: 0.95603851
-- 2.5958 bpw accuracy: 0.96329175
-- 2.9120 bpw accuracy: 0.96548579
-- 3.2833 bpw accuracy: 0.97712556
-- 3.3655 bpw accuracy: 0.97902408
-- 3.6186 bpw accuracy: 0.98206723
-- 4.1368 bpw accuracy: 0.98805704
-- 4.1977 bpw accuracy: 0.98914909
-- 4.2662 bpw accuracy: 0.98834675
-- 4.3484 bpw accuracy: 0.98983174
-- 5.2491 bpw accuracy: 0.99406063
-- 5.3313 bpw accuracy: 0.99493112
-- 6.0713 bpw accuracy: 0.99675670
-- 6.3032 bpw accuracy: 0.99697408
-- 6.8687 bpw accuracy: 0.99760878
-- 8.0354 bpw accuracy: 0.99911201
-----------------------------------------------
| Measured: model.layers.8 (MLP) |
| Duration: 33.78 seconds |
| Completed step: 18/67 |
| Avg time / step (rolling): 22.85 seconds |
| Estimated remaining time: 18min 39sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.9 (Attention)
-- model.layers.9.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.9.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.9.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.9.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.9.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.9.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.9.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.9.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.9.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.9.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.9.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.9.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.9.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.9.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.9.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.9.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.9.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.9.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.9.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.9.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.9.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.9.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.9.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.9.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.9.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.9.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.9.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.9.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.9.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.9.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.9.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.9.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.9.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.9.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.9.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.9.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.9.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.9.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.9.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.9.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.9.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.9.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.9.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.9.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.9.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.9.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.9.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.9.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.9.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.9.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.9.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.9.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.9.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.9.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.9.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.9.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.9.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96949360
-- 2.1987 bpw accuracy: 0.97062068
-- 2.2831 bpw accuracy: 0.97313075
-- 2.6768 bpw accuracy: 0.97934811
-- 3.1689 bpw accuracy: 0.98462158
-- 3.1705 bpw accuracy: 0.98493923
-- 4.0439 bpw accuracy: 0.98978280
-- 4.0471 bpw accuracy: 0.99026466
-- 4.0816 bpw accuracy: 0.99111835
-- 4.1381 bpw accuracy: 0.99121882
-- 4.1705 bpw accuracy: 0.99238511
-- 4.1902 bpw accuracy: 0.99289908
-- 4.2737 bpw accuracy: 0.99330442
-- 4.3295 bpw accuracy: 0.99388537
-- 5.2564 bpw accuracy: 0.99638738
-- 5.3295 bpw accuracy: 0.99697100
-- 6.0439 bpw accuracy: 0.99746045
-- 6.3381 bpw accuracy: 0.99833393
-- 8.0439 bpw accuracy: 0.99929864
-----------------------------------------------
| Measured: model.layers.9 (Attention) |
| Duration: 11.99 seconds |
| Completed step: 19/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 18min 17sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.9 (MLP)
-- model.layers.9.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.9.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.9.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.9.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.9.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.9.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.9.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.9.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.9.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.9.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.9.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.9.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.9.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.9.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.9.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.9.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.9.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.9.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.9.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.9.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.9.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.9.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.9.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.9.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.9.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.9.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.9.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.9.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.9.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.9.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.9.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.9.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.95229753
-- 2.3230 bpw accuracy: 0.95371278
-- 2.5958 bpw accuracy: 0.96133240
-- 2.9120 bpw accuracy: 0.96366655
-- 3.2833 bpw accuracy: 0.97580603
-- 3.3655 bpw accuracy: 0.97784011
-- 3.6186 bpw accuracy: 0.98109499
-- 4.1368 bpw accuracy: 0.98736480
-- 4.1977 bpw accuracy: 0.98850414
-- 4.2662 bpw accuracy: 0.98766696
-- 4.3484 bpw accuracy: 0.98923480
-- 5.2491 bpw accuracy: 0.99370057
-- 5.3313 bpw accuracy: 0.99464108
-- 6.0713 bpw accuracy: 0.99654098
-- 6.3032 bpw accuracy: 0.99678933
-- 6.8687 bpw accuracy: 0.99744834
-- 8.0354 bpw accuracy: 0.99905349
-----------------------------------------------
| Measured: model.layers.9 (MLP) |
| Duration: 33.90 seconds |
| Completed step: 20/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 17min 54sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.10 (Attention)
-- model.layers.10.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.10.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.10.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.10.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.10.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.10.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.10.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.10.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.10.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.10.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.10.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.10.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.10.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.10.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.10.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.10.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.10.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.10.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.10.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.10.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.10.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.10.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.10.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.10.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.10.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.10.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.10.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.10.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.10.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.10.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.10.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.10.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.10.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.10.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.10.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.10.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.10.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.10.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.10.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.10.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.10.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.10.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.10.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.10.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.10.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.10.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.10.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.10.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.10.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.10.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.10.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.10.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.10.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.10.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.10.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.10.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.10.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96439050
-- 2.1987 bpw accuracy: 0.96588911
-- 2.2831 bpw accuracy: 0.96930288
-- 2.6768 bpw accuracy: 0.97711419
-- 3.1689 bpw accuracy: 0.98197677
-- 3.1705 bpw accuracy: 0.98245664
-- 4.0439 bpw accuracy: 0.98831443
-- 4.0471 bpw accuracy: 0.98894177
-- 4.0816 bpw accuracy: 0.99001351
-- 4.1381 bpw accuracy: 0.99053100
-- 4.1705 bpw accuracy: 0.99116728
-- 4.1902 bpw accuracy: 0.99164836
-- 4.2737 bpw accuracy: 0.99226078
-- 4.3295 bpw accuracy: 0.99289807
-- 5.2564 bpw accuracy: 0.99573026
-- 5.3295 bpw accuracy: 0.99642581
-- 6.0439 bpw accuracy: 0.99697688
-- 6.3381 bpw accuracy: 0.99798415
-- 8.0439 bpw accuracy: 0.99919278
-----------------------------------------------
| Measured: model.layers.10 (Attention) |
| Duration: 11.93 seconds |
| Completed step: 21/67 |
| Avg time / step (rolling): 22.88 seconds |
| Estimated remaining time: 17min 32sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.10 (MLP)
-- model.layers.10.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.10.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.10.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.10.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.10.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.10.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.10.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.10.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.10.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.10.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.10.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.10.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.10.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.10.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.10.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.10.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.10.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.10.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.10.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.10.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.10.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.10.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.10.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.10.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.10.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.10.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.10.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.10.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.10.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.10.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.10.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.10.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.95095831
-- 2.3230 bpw accuracy: 0.95246449
-- 2.5958 bpw accuracy: 0.96071340
-- 2.9120 bpw accuracy: 0.96325856
-- 3.2833 bpw accuracy: 0.97511408
-- 3.3655 bpw accuracy: 0.97720610
-- 3.6186 bpw accuracy: 0.98069898
-- 4.1368 bpw accuracy: 0.98684563
-- 4.1977 bpw accuracy: 0.98811348
-- 4.2662 bpw accuracy: 0.98729146
-- 4.3484 bpw accuracy: 0.98892895
-- 5.2491 bpw accuracy: 0.99352282
-- 5.3313 bpw accuracy: 0.99447654
-- 6.0713 bpw accuracy: 0.99643119
-- 6.3032 bpw accuracy: 0.99670919
-- 6.8687 bpw accuracy: 0.99744479
-- 8.0354 bpw accuracy: 0.99903249
-----------------------------------------------
| Measured: model.layers.10 (MLP) |
| Duration: 33.75 seconds |
| Completed step: 22/67 |
| Avg time / step (rolling): 22.86 seconds |
| Estimated remaining time: 17min 8sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.11 (Attention)
-- model.layers.11.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.11.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.11.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.11.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.11.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.11.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.11.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.11.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.11.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.11.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.11.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.11.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.11.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.11.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.11.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.11.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.11.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.11.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.11.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.11.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.11.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.11.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.11.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.11.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.11.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.11.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.11.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.11.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.11.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.11.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.11.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.11.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.11.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.11.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.11.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.11.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.11.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.11.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.11.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.11.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.11.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.11.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.11.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.11.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.11.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.11.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.11.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.11.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.11.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.11.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.11.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.11.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.11.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.11.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.11.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.11.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.11.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96092784
-- 2.1987 bpw accuracy: 0.96271768
-- 2.2831 bpw accuracy: 0.96625716
-- 2.6768 bpw accuracy: 0.97486000
-- 3.1689 bpw accuracy: 0.97986424
-- 3.1705 bpw accuracy: 0.98046008
-- 4.0439 bpw accuracy: 0.98610930
-- 4.0471 bpw accuracy: 0.98694848
-- 4.0816 bpw accuracy: 0.98803654
-- 4.1381 bpw accuracy: 0.98843495
-- 4.1705 bpw accuracy: 0.98997296
-- 4.1902 bpw accuracy: 0.99068295
-- 4.2737 bpw accuracy: 0.99122523
-- 4.3295 bpw accuracy: 0.99197524
-- 5.2564 bpw accuracy: 0.99494291
-- 5.3295 bpw accuracy: 0.99597608
-- 6.0439 bpw accuracy: 0.99612609
-- 6.3381 bpw accuracy: 0.99765912
-- 8.0439 bpw accuracy: 0.99900815
-----------------------------------------------
| Measured: model.layers.11 (Attention) |
| Duration: 11.98 seconds |
| Completed step: 23/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 16min 46sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Layer: model.layers.11 (MLP)
-- model.layers.11.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.11.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.11.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.11.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.11.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.11.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.11.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.11.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.11.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.11.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.11.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.11.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.11.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.11.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.11.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.11.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.11.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.11.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.11.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.11.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.11.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.11.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.11.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.11.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.11.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.11.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.11.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.11.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.11.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.11.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.11.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.11.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.94947620
-- 2.3230 bpw accuracy: 0.95105945
-- 2.5958 bpw accuracy: 0.95952494
-- 2.9120 bpw accuracy: 0.96215029
-- 3.2833 bpw accuracy: 0.97438584
-- 3.3655 bpw accuracy: 0.97652838
-- 3.6186 bpw accuracy: 0.98013617
-- 4.1368 bpw accuracy: 0.98642455
-- 4.1977 bpw accuracy: 0.98772960
-- 4.2662 bpw accuracy: 0.98692776
-- 4.3484 bpw accuracy: 0.98860136
-- 5.2491 bpw accuracy: 0.99333367
-- 5.3313 bpw accuracy: 0.99430986
-- 6.0713 bpw accuracy: 0.99630646
-- 6.3032 bpw accuracy: 0.99661869
-- 6.8687 bpw accuracy: 0.99737967
-- 8.0354 bpw accuracy: 0.99898830
-----------------------------------------------
| Measured: model.layers.11 (MLP) |
| Duration: 33.77 seconds |
| Completed step: 24/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 16min 23sec |
| Last checkpoint layer: model.layers.7 (MLP) |
-----------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.12 (Attention)
-- model.layers.12.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.12.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.12.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.12.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.12.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.12.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.12.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.12.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.12.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.12.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.12.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.12.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.12.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.12.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.12.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.12.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.12.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.12.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.12.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.12.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.12.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.12.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.12.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.12.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.12.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.12.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.12.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.12.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.12.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.12.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.12.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.12.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.12.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.12.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.12.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.12.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.12.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.12.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.12.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.12.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.12.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.12.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.12.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.12.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.12.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.12.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.12.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.12.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.12.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.12.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.12.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.12.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.12.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.12.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.12.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.12.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.12.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95497149
-- 2.1987 bpw accuracy: 0.95729429
-- 2.2831 bpw accuracy: 0.96096700
-- 2.6768 bpw accuracy: 0.96914752
-- 3.1689 bpw accuracy: 0.97675191
-- 3.1705 bpw accuracy: 0.97728684
-- 4.0439 bpw accuracy: 0.98387393
-- 4.0471 bpw accuracy: 0.98448803
-- 4.0816 bpw accuracy: 0.98542907
-- 4.1381 bpw accuracy: 0.98632431
-- 4.1705 bpw accuracy: 0.98836104
-- 4.1902 bpw accuracy: 0.98919922
-- 4.2737 bpw accuracy: 0.98967935
-- 4.3295 bpw accuracy: 0.99051492
-- 5.2564 bpw accuracy: 0.99421612
-- 5.3295 bpw accuracy: 0.99520169
-- 6.0439 bpw accuracy: 0.99560382
-- 6.3381 bpw accuracy: 0.99738634
-- 8.0439 bpw accuracy: 0.99872250
------------------------------------------------
| Measured: model.layers.12 (Attention) |
| Duration: 11.97 seconds |
| Completed step: 25/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 16min 0sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.12 (MLP)
-- model.layers.12.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.12.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.12.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.12.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.12.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.12.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.12.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.12.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.12.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.12.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.12.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.12.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.12.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.12.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.12.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.12.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.12.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.12.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.12.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.12.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.12.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.12.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.12.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.12.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.12.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.12.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.12.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.12.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.12.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.12.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.12.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.12.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.94482404
-- 2.3230 bpw accuracy: 0.94664729
-- 2.5958 bpw accuracy: 0.95549302
-- 2.9120 bpw accuracy: 0.95822299
-- 3.2833 bpw accuracy: 0.97204762
-- 3.3655 bpw accuracy: 0.97442339
-- 3.6186 bpw accuracy: 0.97817537
-- 4.1368 bpw accuracy: 0.98523562
-- 4.1977 bpw accuracy: 0.98662546
-- 4.2662 bpw accuracy: 0.98571101
-- 4.3484 bpw accuracy: 0.98756677
-- 5.2491 bpw accuracy: 0.99269979
-- 5.3313 bpw accuracy: 0.99377695
-- 6.0713 bpw accuracy: 0.99594282
-- 6.3032 bpw accuracy: 0.99628351
-- 6.8687 bpw accuracy: 0.99705449
-- 8.0354 bpw accuracy: 0.99887945
------------------------------------------------
| Measured: model.layers.12 (MLP) |
| Duration: 33.86 seconds |
| Completed step: 26/67 |
| Avg time / step (rolling): 22.88 seconds |
| Estimated remaining time: 15min 37sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.13 (Attention)
-- model.layers.13.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.13.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.13.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.13.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.13.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.13.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.13.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.13.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.13.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.13.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.13.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.13.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.13.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.13.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.13.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.13.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.13.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.13.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.13.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.13.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.13.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.13.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.13.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.13.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.13.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.13.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.13.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.13.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.13.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.13.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.13.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.13.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.13.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.13.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.13.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.13.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.13.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.13.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.13.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.13.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.13.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.13.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.13.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.13.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.13.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.13.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.13.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.13.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.13.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.13.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.13.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.13.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.13.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.13.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.13.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.13.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.13.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95677763
-- 2.1987 bpw accuracy: 0.95863658
-- 2.2831 bpw accuracy: 0.96279997
-- 2.6768 bpw accuracy: 0.97267952
-- 3.1689 bpw accuracy: 0.97804663
-- 3.1705 bpw accuracy: 0.97853351
-- 4.0439 bpw accuracy: 0.98631115
-- 4.0471 bpw accuracy: 0.98702057
-- 4.0816 bpw accuracy: 0.98794433
-- 4.1381 bpw accuracy: 0.98848109
-- 4.1705 bpw accuracy: 0.98916386
-- 4.1902 bpw accuracy: 0.98979494
-- 4.2737 bpw accuracy: 0.99054718
-- 4.3295 bpw accuracy: 0.99128328
-- 5.2564 bpw accuracy: 0.99471482
-- 5.3295 bpw accuracy: 0.99564732
-- 6.0439 bpw accuracy: 0.99632491
-- 6.3381 bpw accuracy: 0.99759466
-- 8.0439 bpw accuracy: 0.99902323
------------------------------------------------
| Measured: model.layers.13 (Attention) |
| Duration: 11.86 seconds |
| Completed step: 27/67 |
| Avg time / step (rolling): 22.88 seconds |
| Estimated remaining time: 15min 15sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.13 (MLP)
-- model.layers.13.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.13.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.13.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.13.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.13.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.13.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.13.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.13.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.13.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.13.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.13.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.13.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.13.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.13.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.13.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.13.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.13.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.13.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.13.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.13.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.13.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.13.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.13.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.13.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.13.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.13.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.13.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.13.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.13.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.13.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.13.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.13.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.94136187
-- 2.3230 bpw accuracy: 0.94320825
-- 2.5958 bpw accuracy: 0.95275347
-- 2.9120 bpw accuracy: 0.95579984
-- 3.2833 bpw accuracy: 0.97011028
-- 3.3655 bpw accuracy: 0.97265031
-- 3.6186 bpw accuracy: 0.97675317
-- 4.1368 bpw accuracy: 0.98411054
-- 4.1977 bpw accuracy: 0.98563921
-- 4.2662 bpw accuracy: 0.98473041
-- 4.3484 bpw accuracy: 0.98670680
-- 5.2491 bpw accuracy: 0.99220833
-- 5.3313 bpw accuracy: 0.99336434
-- 6.0713 bpw accuracy: 0.99567370
-- 6.3032 bpw accuracy: 0.99603635
-- 6.8687 bpw accuracy: 0.99690129
-- 8.0354 bpw accuracy: 0.99881352
------------------------------------------------
| Measured: model.layers.13 (MLP) |
| Duration: 33.78 seconds |
| Completed step: 28/67 |
| Avg time / step (rolling): 22.88 seconds |
| Estimated remaining time: 14min 52sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.14 (Attention)
-- model.layers.14.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.14.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.14.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.14.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.14.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.14.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.14.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.14.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.14.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.14.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.14.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.14.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.14.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.14.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.14.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.14.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.14.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.14.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.14.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.14.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.14.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.14.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.14.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.14.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.14.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.14.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.14.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.14.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.14.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.14.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.14.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.14.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.14.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.14.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.14.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.14.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.14.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.14.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.14.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.14.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.14.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.14.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.14.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.14.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.14.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.14.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.14.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.14.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.14.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.14.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.14.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.14.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.14.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.14.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.14.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.14.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.14.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95142432
-- 2.1987 bpw accuracy: 0.95376446
-- 2.2831 bpw accuracy: 0.95841915
-- 2.6768 bpw accuracy: 0.96904453
-- 3.1689 bpw accuracy: 0.97473409
-- 3.1705 bpw accuracy: 0.97551306
-- 4.0439 bpw accuracy: 0.98363912
-- 4.0471 bpw accuracy: 0.98478491
-- 4.0816 bpw accuracy: 0.98584172
-- 4.1381 bpw accuracy: 0.98622305
-- 4.1705 bpw accuracy: 0.98736817
-- 4.1902 bpw accuracy: 0.98815438
-- 4.2737 bpw accuracy: 0.98897412
-- 4.3295 bpw accuracy: 0.98997909
-- 5.2564 bpw accuracy: 0.99362657
-- 5.3295 bpw accuracy: 0.99493921
-- 6.0439 bpw accuracy: 0.99524748
-- 6.3381 bpw accuracy: 0.99722645
-- 8.0439 bpw accuracy: 0.99871751
------------------------------------------------
| Measured: model.layers.14 (Attention) |
| Duration: 11.91 seconds |
| Completed step: 29/67 |
| Avg time / step (rolling): 22.87 seconds |
| Estimated remaining time: 14min 29sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.14 (MLP)
-- model.layers.14.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.14.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.14.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.14.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.14.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.14.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.14.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.14.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.14.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.14.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.14.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.14.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.14.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.14.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.14.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.14.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.14.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.14.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.14.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.14.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.14.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.14.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.14.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.14.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.14.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.14.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.14.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.14.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.14.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.14.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.14.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.14.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.93690496
-- 2.3230 bpw accuracy: 0.93886134
-- 2.5958 bpw accuracy: 0.94893391
-- 2.9120 bpw accuracy: 0.95213144
-- 3.2833 bpw accuracy: 0.96797503
-- 3.3655 bpw accuracy: 0.97084171
-- 3.6186 bpw accuracy: 0.97517228
-- 4.1368 bpw accuracy: 0.98287856
-- 4.1977 bpw accuracy: 0.98458308
-- 4.2662 bpw accuracy: 0.98370034
-- 4.3484 bpw accuracy: 0.98578446
-- 5.2491 bpw accuracy: 0.99165915
-- 5.3313 bpw accuracy: 0.99286213
-- 6.0713 bpw accuracy: 0.99531910
-- 6.3032 bpw accuracy: 0.99573994
-- 6.8687 bpw accuracy: 0.99665615
-- 8.0354 bpw accuracy: 0.99864366
------------------------------------------------
| Measured: model.layers.14 (MLP) |
| Duration: 33.73 seconds |
| Completed step: 30/67 |
| Avg time / step (rolling): 22.85 seconds |
| Estimated remaining time: 14min 5sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.15 (Attention)
-- model.layers.15.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.15.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.15.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.15.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.15.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.15.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.15.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.15.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.15.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.15.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.15.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.15.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.15.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.15.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.15.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.15.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.15.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.15.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.15.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.15.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.15.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.15.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.15.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.15.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.15.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.15.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.15.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.15.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.15.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.15.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.15.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.15.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.15.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.15.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.15.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.15.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.15.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.15.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.15.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.15.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.15.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.15.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.15.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.15.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.15.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.15.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.15.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.15.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.15.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.15.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.15.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.15.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.15.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.15.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.15.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.15.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.15.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94746189
-- 2.1987 bpw accuracy: 0.94934057
-- 2.2831 bpw accuracy: 0.95420918
-- 2.6768 bpw accuracy: 0.96364619
-- 3.1689 bpw accuracy: 0.97205734
-- 3.1705 bpw accuracy: 0.97292173
-- 4.0439 bpw accuracy: 0.98082847
-- 4.0471 bpw accuracy: 0.98186975
-- 4.0816 bpw accuracy: 0.98324078
-- 4.1381 bpw accuracy: 0.98430775
-- 4.1705 bpw accuracy: 0.98578498
-- 4.1902 bpw accuracy: 0.98682765
-- 4.2737 bpw accuracy: 0.98746150
-- 4.3295 bpw accuracy: 0.98875030
-- 5.2564 bpw accuracy: 0.99319441
-- 5.3295 bpw accuracy: 0.99443277
-- 6.0439 bpw accuracy: 0.99478138
-- 6.3381 bpw accuracy: 0.99698963
-- 8.0439 bpw accuracy: 0.99858972
------------------------------------------------
| Measured: model.layers.15 (Attention) |
| Duration: 11.88 seconds |
| Completed step: 31/67 |
| Avg time / step (rolling): 22.85 seconds |
| Estimated remaining time: 13min 42sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Layer: model.layers.15 (MLP)
-- model.layers.15.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.15.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.15.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.15.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.15.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.15.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.15.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.15.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.15.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.15.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.15.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.15.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.15.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.15.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.15.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.15.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.15.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.15.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.15.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.15.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.15.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.15.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.15.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.15.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.15.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.15.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.15.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.15.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.15.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.15.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.15.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.15.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.93268816
-- 2.3230 bpw accuracy: 0.93478254
-- 2.5958 bpw accuracy: 0.94569535
-- 2.9120 bpw accuracy: 0.94912148
-- 3.2833 bpw accuracy: 0.96591628
-- 3.3655 bpw accuracy: 0.96870893
-- 3.6186 bpw accuracy: 0.97336368
-- 4.1368 bpw accuracy: 0.98195666
-- 4.1977 bpw accuracy: 0.98360634
-- 4.2662 bpw accuracy: 0.98261731
-- 4.3484 bpw accuracy: 0.98480396
-- 5.2491 bpw accuracy: 0.99113158
-- 5.3313 bpw accuracy: 0.99241681
-- 6.0713 bpw accuracy: 0.99508634
-- 6.3032 bpw accuracy: 0.99549702
-- 6.8687 bpw accuracy: 0.99646459
-- 8.0354 bpw accuracy: 0.99865663
------------------------------------------------
| Measured: model.layers.15 (MLP) |
| Duration: 33.71 seconds |
| Completed step: 32/67 |
| Avg time / step (rolling): 22.84 seconds |
| Estimated remaining time: 13min 19sec |
| Last checkpoint layer: model.layers.11 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.16 (Attention)
-- model.layers.16.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.16.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.16.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.16.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.16.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.16.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.16.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.16.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.16.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.16.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.16.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.16.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.16.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.16.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.16.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.16.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.16.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.16.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.16.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.16.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.16.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.16.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.16.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.16.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.16.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.16.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.16.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.16.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.16.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.16.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.16.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.16.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.16.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.16.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.16.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.16.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.16.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.16.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.16.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.16.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.16.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.16.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.16.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.16.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.16.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.16.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.16.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.16.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.16.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.16.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.16.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.16.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.16.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.16.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.16.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.16.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.16.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95050338
-- 2.1987 bpw accuracy: 0.95214220
-- 2.2831 bpw accuracy: 0.95751995
-- 2.6768 bpw accuracy: 0.96765671
-- 3.1689 bpw accuracy: 0.97444186
-- 3.1705 bpw accuracy: 0.97512672
-- 4.0439 bpw accuracy: 0.98399732
-- 4.0471 bpw accuracy: 0.98493529
-- 4.0816 bpw accuracy: 0.98596075
-- 4.1381 bpw accuracy: 0.98670501
-- 4.1705 bpw accuracy: 0.98724487
-- 4.1902 bpw accuracy: 0.98799390
-- 4.2737 bpw accuracy: 0.98904557
-- 4.3295 bpw accuracy: 0.98993798
-- 5.2564 bpw accuracy: 0.99388331
-- 5.3295 bpw accuracy: 0.99499059
-- 6.0439 bpw accuracy: 0.99562792
-- 6.3381 bpw accuracy: 0.99727731
-- 8.0439 bpw accuracy: 0.99882059
------------------------------------------------
| Measured: model.layers.16 (Attention) |
| Duration: 11.83 seconds |
| Completed step: 33/67 |
| Avg time / step (rolling): 22.83 seconds |
| Estimated remaining time: 12min 56sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.16 (MLP)
-- model.layers.16.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.16.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.16.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.16.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.16.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.16.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.16.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.16.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.16.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.16.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.16.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.16.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.16.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.16.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.16.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.16.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.16.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.16.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.16.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.16.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.16.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.16.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.16.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.16.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.16.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.16.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.16.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.16.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.16.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.16.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.16.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.16.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.92663227
-- 2.3230 bpw accuracy: 0.92892432
-- 2.5958 bpw accuracy: 0.94076666
-- 2.9120 bpw accuracy: 0.94462212
-- 3.2833 bpw accuracy: 0.96285911
-- 3.3655 bpw accuracy: 0.96592391
-- 3.6186 bpw accuracy: 0.97099635
-- 4.1368 bpw accuracy: 0.98016092
-- 4.1977 bpw accuracy: 0.98199537
-- 4.2662 bpw accuracy: 0.98105669
-- 4.3484 bpw accuracy: 0.98345359
-- 5.2491 bpw accuracy: 0.99034267
-- 5.3313 bpw accuracy: 0.99173820
-- 6.0713 bpw accuracy: 0.99458522
-- 6.3032 bpw accuracy: 0.99510760
-- 6.8687 bpw accuracy: 0.99616517
-- 8.0354 bpw accuracy: 0.99849359
------------------------------------------------
| Measured: model.layers.16 (MLP) |
| Duration: 33.73 seconds |
| Completed step: 34/67 |
| Avg time / step (rolling): 22.83 seconds |
| Estimated remaining time: 12min 33sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.17 (Attention)
-- model.layers.17.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.17.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.17.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.17.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.17.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.17.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.17.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.17.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.17.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.17.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.17.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.17.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.17.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.17.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.17.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.17.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.17.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.17.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.17.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.17.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.17.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.17.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.17.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.17.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.17.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.17.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.17.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.17.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.17.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.17.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.17.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.17.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.17.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.17.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.17.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.17.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.17.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.17.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.17.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.17.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.17.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.17.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.17.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.17.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.17.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.17.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.17.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.17.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.17.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.17.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.17.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.17.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.17.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.17.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.17.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.17.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.17.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94687639
-- 2.1987 bpw accuracy: 0.94920247
-- 2.2831 bpw accuracy: 0.95373578
-- 2.6768 bpw accuracy: 0.96568864
-- 3.1689 bpw accuracy: 0.97209875
-- 3.1705 bpw accuracy: 0.97281114
-- 4.0439 bpw accuracy: 0.98125367
-- 4.0471 bpw accuracy: 0.98222374
-- 4.0816 bpw accuracy: 0.98409547
-- 4.1381 bpw accuracy: 0.98476382
-- 4.1705 bpw accuracy: 0.98618766
-- 4.1902 bpw accuracy: 0.98705464
-- 4.2737 bpw accuracy: 0.98775989
-- 4.3295 bpw accuracy: 0.98874353
-- 5.2564 bpw accuracy: 0.99290636
-- 5.3295 bpw accuracy: 0.99435643
-- 6.0439 bpw accuracy: 0.99466827
-- 6.3381 bpw accuracy: 0.99701204
-- 8.0439 bpw accuracy: 0.99865724
------------------------------------------------
| Measured: model.layers.17 (Attention) |
| Duration: 11.95 seconds |
| Completed step: 35/67 |
| Avg time / step (rolling): 22.82 seconds |
| Estimated remaining time: 12min 10sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.17 (MLP)
-- model.layers.17.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.17.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.17.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.17.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.17.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.17.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.17.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.17.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.17.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.17.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.17.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.17.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.17.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.17.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.17.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.17.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.17.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.17.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.17.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.17.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.17.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.17.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.17.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.17.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.17.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.17.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.17.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.17.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.17.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.17.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.17.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.17.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.92048443
-- 2.3230 bpw accuracy: 0.92303085
-- 2.5958 bpw accuracy: 0.93578399
-- 2.9120 bpw accuracy: 0.94030747
-- 3.2833 bpw accuracy: 0.95961078
-- 3.3655 bpw accuracy: 0.96295526
-- 3.6186 bpw accuracy: 0.96854504
-- 4.1368 bpw accuracy: 0.97818065
-- 4.1977 bpw accuracy: 0.98028153
-- 4.2662 bpw accuracy: 0.97935935
-- 4.3484 bpw accuracy: 0.98198899
-- 5.2491 bpw accuracy: 0.98946012
-- 5.3313 bpw accuracy: 0.99100005
-- 6.0713 bpw accuracy: 0.99404954
-- 6.3032 bpw accuracy: 0.99466461
-- 6.8687 bpw accuracy: 0.99586620
-- 8.0354 bpw accuracy: 0.99833643
------------------------------------------------
| Measured: model.layers.17 (MLP) |
| Duration: 33.94 seconds |
| Completed step: 36/67 |
| Avg time / step (rolling): 22.83 seconds |
| Estimated remaining time: 11min 47sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.18 (Attention)
-- model.layers.18.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.18.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.18.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.18.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.18.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.18.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.18.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.18.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.18.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.18.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.18.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.18.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.18.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.18.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.18.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.18.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.18.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.18.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.18.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.18.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.18.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.18.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.18.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.18.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.18.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.18.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.18.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.18.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.18.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.18.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.18.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.18.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.18.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.18.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.18.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.18.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.18.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.18.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.18.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.18.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.18.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.18.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.18.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.18.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.18.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.18.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.18.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.18.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.18.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.18.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.18.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.18.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.18.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.18.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.18.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.18.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.18.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94710512
-- 2.1987 bpw accuracy: 0.94934163
-- 2.2831 bpw accuracy: 0.95427913
-- 2.6768 bpw accuracy: 0.96435621
-- 3.1689 bpw accuracy: 0.97270342
-- 3.1705 bpw accuracy: 0.97349691
-- 4.0439 bpw accuracy: 0.98137824
-- 4.0471 bpw accuracy: 0.98248535
-- 4.0816 bpw accuracy: 0.98321875
-- 4.1381 bpw accuracy: 0.98440606
-- 4.1705 bpw accuracy: 0.98635639
-- 4.1902 bpw accuracy: 0.98716228
-- 4.2737 bpw accuracy: 0.98782826
-- 4.3295 bpw accuracy: 0.98894448
-- 5.2564 bpw accuracy: 0.99218715
-- 5.3295 bpw accuracy: 0.99431925
-- 6.0439 bpw accuracy: 0.99354872
-- 6.3381 bpw accuracy: 0.99704737
-- 8.0439 bpw accuracy: 0.99823556
------------------------------------------------
| Measured: model.layers.18 (Attention) |
| Duration: 11.93 seconds |
| Completed step: 37/67 |
| Avg time / step (rolling): 22.84 seconds |
| Estimated remaining time: 11min 25sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.18 (MLP)
-- model.layers.18.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.18.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.18.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.18.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.18.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.18.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.18.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.18.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.18.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.18.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.18.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.18.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.18.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.18.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.18.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.18.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.18.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.18.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.18.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.18.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.18.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.18.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.18.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.18.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.18.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.18.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.18.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.18.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.18.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.18.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.18.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.18.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91510076
-- 2.3230 bpw accuracy: 0.91783840
-- 2.5958 bpw accuracy: 0.93154725
-- 2.9120 bpw accuracy: 0.93654935
-- 3.2833 bpw accuracy: 0.95695041
-- 3.3655 bpw accuracy: 0.96051874
-- 3.6186 bpw accuracy: 0.96652429
-- 4.1368 bpw accuracy: 0.97655404
-- 4.1977 bpw accuracy: 0.97883857
-- 4.2662 bpw accuracy: 0.97800263
-- 4.3484 bpw accuracy: 0.98082096
-- 5.2491 bpw accuracy: 0.98876678
-- 5.3313 bpw accuracy: 0.99041427
-- 6.0713 bpw accuracy: 0.99360107
-- 6.3032 bpw accuracy: 0.99430635
-- 6.8687 bpw accuracy: 0.99560812
-- 8.0354 bpw accuracy: 0.99816977
------------------------------------------------
| Measured: model.layers.18 (MLP) |
| Duration: 33.68 seconds |
| Completed step: 38/67 |
| Avg time / step (rolling): 22.83 seconds |
| Estimated remaining time: 11min 2sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.19 (Attention)
-- model.layers.19.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.19.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.19.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.19.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.19.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.19.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.19.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.19.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.19.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.19.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.19.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.19.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.19.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.19.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.19.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.19.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.19.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.19.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.19.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.19.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.19.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.19.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.19.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.19.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.19.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.19.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.19.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.19.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.19.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.19.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.19.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.19.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.19.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.19.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.19.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.19.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.19.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.19.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.19.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.19.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.19.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.19.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.19.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.19.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.19.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.19.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.19.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.19.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.19.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.19.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.19.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.19.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.19.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.19.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.19.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.19.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.19.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94883888
-- 2.1987 bpw accuracy: 0.95066694
-- 2.2831 bpw accuracy: 0.95528290
-- 2.6768 bpw accuracy: 0.96576602
-- 3.1689 bpw accuracy: 0.97351192
-- 3.1705 bpw accuracy: 0.97433013
-- 4.0439 bpw accuracy: 0.98241880
-- 4.0471 bpw accuracy: 0.98351122
-- 4.0816 bpw accuracy: 0.98480575
-- 4.1381 bpw accuracy: 0.98549543
-- 4.1705 bpw accuracy: 0.98690823
-- 4.1902 bpw accuracy: 0.98767505
-- 4.2737 bpw accuracy: 0.98855560
-- 4.3295 bpw accuracy: 0.98926044
-- 5.2564 bpw accuracy: 0.99350095
-- 5.3295 bpw accuracy: 0.99476808
-- 6.0439 bpw accuracy: 0.99522363
-- 6.3381 bpw accuracy: 0.99710168
-- 8.0439 bpw accuracy: 0.99872043
------------------------------------------------
| Measured: model.layers.19 (Attention) |
| Duration: 12.09 seconds |
| Completed step: 39/67 |
| Avg time / step (rolling): 22.85 seconds |
| Estimated remaining time: 10min 39sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Layer: model.layers.19 (MLP)
-- model.layers.19.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.19.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.19.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.19.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.19.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.19.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.19.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.19.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.19.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.19.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.19.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.19.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.19.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.19.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.19.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.19.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.19.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.19.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.19.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.19.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.19.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.19.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.19.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.19.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.19.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.19.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.19.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.19.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.19.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.19.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.19.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.19.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91353916
-- 2.3230 bpw accuracy: 0.91625298
-- 2.5958 bpw accuracy: 0.93013112
-- 2.9120 bpw accuracy: 0.93520242
-- 3.2833 bpw accuracy: 0.95600237
-- 3.3655 bpw accuracy: 0.95973639
-- 3.6186 bpw accuracy: 0.96590915
-- 4.1368 bpw accuracy: 0.97611589
-- 4.1977 bpw accuracy: 0.97844007
-- 4.2662 bpw accuracy: 0.97746584
-- 4.3484 bpw accuracy: 0.98043700
-- 5.2491 bpw accuracy: 0.98846898
-- 5.3313 bpw accuracy: 0.99021072
-- 6.0713 bpw accuracy: 0.99345775
-- 6.3032 bpw accuracy: 0.99415462
-- 6.8687 bpw accuracy: 0.99547240
-- 8.0354 bpw accuracy: 0.99813883
------------------------------------------------
| Measured: model.layers.19 (MLP) |
| Duration: 33.84 seconds |
| Completed step: 40/67 |
| Avg time / step (rolling): 22.86 seconds |
| Estimated remaining time: 10min 17sec |
| Last checkpoint layer: model.layers.15 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.20 (Attention)
-- model.layers.20.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.20.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.20.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.20.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.20.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.20.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.20.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.20.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.20.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.20.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.20.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.20.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.20.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.20.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.20.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.20.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.20.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.20.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.20.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.20.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.20.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.20.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.20.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.20.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.20.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.20.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.20.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.20.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.20.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.20.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.20.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.20.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.20.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.20.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.20.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.20.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.20.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.20.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.20.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.20.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.20.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.20.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.20.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.20.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.20.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.20.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.20.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.20.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.20.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.20.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.20.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.20.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.20.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.20.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.20.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.20.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.20.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95325135
-- 2.1987 bpw accuracy: 0.95517738
-- 2.2831 bpw accuracy: 0.95910592
-- 2.6768 bpw accuracy: 0.96907891
-- 3.1689 bpw accuracy: 0.97556290
-- 3.1705 bpw accuracy: 0.97649772
-- 4.0439 bpw accuracy: 0.98323175
-- 4.0471 bpw accuracy: 0.98457339
-- 4.0816 bpw accuracy: 0.98574651
-- 4.1381 bpw accuracy: 0.98617045
-- 4.1705 bpw accuracy: 0.98794535
-- 4.1902 bpw accuracy: 0.98876411
-- 4.2737 bpw accuracy: 0.98934186
-- 4.3295 bpw accuracy: 0.99017358
-- 5.2564 bpw accuracy: 0.99369634
-- 5.3295 bpw accuracy: 0.99502795
-- 6.0439 bpw accuracy: 0.99511157
-- 6.3381 bpw accuracy: 0.99723039
-- 8.0439 bpw accuracy: 0.99869138
------------------------------------------------
| Measured: model.layers.20 (Attention) |
| Duration: 12.05 seconds |
| Completed step: 41/67 |
| Avg time / step (rolling): 22.88 seconds |
| Estimated remaining time: 9min 54sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.20 (MLP)
-- model.layers.20.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.20.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.20.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.20.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.20.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.20.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.20.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.20.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.20.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.20.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.20.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.20.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.20.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.20.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.20.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.20.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.20.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.20.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.20.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.20.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.20.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.20.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.20.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.20.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.20.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.20.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.20.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.20.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.20.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.20.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.20.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.20.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91392138
-- 2.3230 bpw accuracy: 0.91645644
-- 2.5958 bpw accuracy: 0.93007983
-- 2.9120 bpw accuracy: 0.93473345
-- 3.2833 bpw accuracy: 0.95637757
-- 3.3655 bpw accuracy: 0.95989519
-- 3.6186 bpw accuracy: 0.96588621
-- 4.1368 bpw accuracy: 0.97673912
-- 4.1977 bpw accuracy: 0.97888979
-- 4.2662 bpw accuracy: 0.97776568
-- 4.3484 bpw accuracy: 0.98056662
-- 5.2491 bpw accuracy: 0.98866295
-- 5.3313 bpw accuracy: 0.99030392
-- 6.0713 bpw accuracy: 0.99368742
-- 6.3032 bpw accuracy: 0.99425300
-- 6.8687 bpw accuracy: 0.99550527
-- 8.0354 bpw accuracy: 0.99822635
------------------------------------------------
| Measured: model.layers.20 (MLP) |
| Duration: 33.92 seconds |
| Completed step: 42/67 |
| Avg time / step (rolling): 22.90 seconds |
| Estimated remaining time: 9min 32sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.21 (Attention)
-- model.layers.21.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.21.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.21.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.21.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.21.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.21.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.21.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.21.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.21.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.21.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.21.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.21.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.21.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.21.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.21.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.21.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.21.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.21.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.21.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.21.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.21.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.21.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.21.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.21.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.21.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.21.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.21.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.21.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.21.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.21.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.21.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.21.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.21.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.21.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.21.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.21.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.21.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.21.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.21.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.21.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.21.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.21.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.21.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.21.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.21.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.21.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.21.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.21.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.21.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.21.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.21.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.21.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.21.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.21.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.21.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.21.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.21.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95784130
-- 2.1987 bpw accuracy: 0.95949705
-- 2.2831 bpw accuracy: 0.96356448
-- 2.6768 bpw accuracy: 0.97204646
-- 3.1689 bpw accuracy: 0.97850821
-- 3.1705 bpw accuracy: 0.97901424
-- 4.0439 bpw accuracy: 0.98517701
-- 4.0471 bpw accuracy: 0.98583565
-- 4.0816 bpw accuracy: 0.98708409
-- 4.1381 bpw accuracy: 0.98752869
-- 4.1705 bpw accuracy: 0.98882703
-- 4.1902 bpw accuracy: 0.98967209
-- 4.2737 bpw accuracy: 0.99008772
-- 4.3295 bpw accuracy: 0.99093437
-- 5.2564 bpw accuracy: 0.99423382
-- 5.3295 bpw accuracy: 0.99537784
-- 6.0439 bpw accuracy: 0.99536919
-- 6.3381 bpw accuracy: 0.99759657
-- 8.0439 bpw accuracy: 0.99880626
------------------------------------------------
| Measured: model.layers.21 (Attention) |
| Duration: 11.92 seconds |
| Completed step: 43/67 |
| Avg time / step (rolling): 22.91 seconds |
| Estimated remaining time: 9min 9sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.21 (MLP)
-- model.layers.21.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.21.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.21.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.21.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.21.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.21.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.21.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.21.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.21.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.21.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.21.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.21.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.21.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.21.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.21.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.21.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.21.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.21.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.21.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.21.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.21.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.21.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.21.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.21.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.21.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.21.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.21.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.21.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.21.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.21.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.21.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.21.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91480194
-- 2.3230 bpw accuracy: 0.91726028
-- 2.5958 bpw accuracy: 0.93045919
-- 2.9120 bpw accuracy: 0.93470979
-- 3.2833 bpw accuracy: 0.95691981
-- 3.3655 bpw accuracy: 0.96032175
-- 3.6186 bpw accuracy: 0.96605713
-- 4.1368 bpw accuracy: 0.97731551
-- 4.1977 bpw accuracy: 0.97933332
-- 4.2662 bpw accuracy: 0.97808115
-- 4.3484 bpw accuracy: 0.98078477
-- 5.2491 bpw accuracy: 0.98883980
-- 5.3313 bpw accuracy: 0.99042296
-- 6.0713 bpw accuracy: 0.99384326
-- 6.3032 bpw accuracy: 0.99432978
-- 6.8687 bpw accuracy: 0.99547315
-- 8.0354 bpw accuracy: 0.99828143
------------------------------------------------
| Measured: model.layers.21 (MLP) |
| Duration: 34.03 seconds |
| Completed step: 44/67 |
| Avg time / step (rolling): 22.94 seconds |
| Estimated remaining time: 8min 47sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.22 (Attention)
-- model.layers.22.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.22.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.22.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.22.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.22.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.22.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.22.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.22.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.22.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.22.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.22.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.22.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.22.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.22.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.22.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.22.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.22.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.22.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.22.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.22.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.22.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.22.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.22.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.22.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.22.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.22.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.22.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.22.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.22.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.22.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.22.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.22.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.22.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.22.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.22.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.22.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.22.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.22.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.22.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.22.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.22.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.22.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.22.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.22.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.22.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.22.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.22.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.22.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.22.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.22.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.22.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.22.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.22.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.22.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.22.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.22.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.22.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96461019
-- 2.1987 bpw accuracy: 0.96636222
-- 2.2831 bpw accuracy: 0.97005697
-- 2.6768 bpw accuracy: 0.97613398
-- 3.1689 bpw accuracy: 0.98043602
-- 3.1705 bpw accuracy: 0.98131932
-- 4.0439 bpw accuracy: 0.98512823
-- 4.0471 bpw accuracy: 0.98625679
-- 4.0816 bpw accuracy: 0.98792747
-- 4.1381 bpw accuracy: 0.98848560
-- 4.1705 bpw accuracy: 0.99042241
-- 4.1902 bpw accuracy: 0.99103883
-- 4.2737 bpw accuracy: 0.99134844
-- 4.3295 bpw accuracy: 0.99216267
-- 5.2564 bpw accuracy: 0.99475117
-- 5.3295 bpw accuracy: 0.99602106
-- 6.0439 bpw accuracy: 0.99551607
-- 6.3381 bpw accuracy: 0.99799308
-- 8.0439 bpw accuracy: 0.99881047
------------------------------------------------
| Measured: model.layers.22 (Attention) |
| Duration: 11.97 seconds |
| Completed step: 45/67 |
| Avg time / step (rolling): 22.94 seconds |
| Estimated remaining time: 8min 24sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.22 (MLP)
-- model.layers.22.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.22.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.22.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.22.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.22.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.22.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.22.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.22.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.22.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.22.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.22.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.22.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.22.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.22.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.22.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.22.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.22.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.22.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.22.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.22.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.22.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.22.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.22.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.22.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.22.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.22.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.22.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.22.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.22.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.22.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.22.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.22.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91549705
-- 2.3230 bpw accuracy: 0.91785504
-- 2.5958 bpw accuracy: 0.93055295
-- 2.9120 bpw accuracy: 0.93454512
-- 3.2833 bpw accuracy: 0.95715535
-- 3.3655 bpw accuracy: 0.96051844
-- 3.6186 bpw accuracy: 0.96599915
-- 4.1368 bpw accuracy: 0.97749786
-- 4.1977 bpw accuracy: 0.97949812
-- 4.2662 bpw accuracy: 0.97817219
-- 4.3484 bpw accuracy: 0.98086884
-- 5.2491 bpw accuracy: 0.98890260
-- 5.3313 bpw accuracy: 0.99047213
-- 6.0713 bpw accuracy: 0.99392308
-- 6.3032 bpw accuracy: 0.99436556
-- 6.8687 bpw accuracy: 0.99545862
-- 8.0354 bpw accuracy: 0.99831266
------------------------------------------------
| Measured: model.layers.22 (MLP) |
| Duration: 34.10 seconds |
| Completed step: 46/67 |
| Avg time / step (rolling): 22.95 seconds |
| Estimated remaining time: 8min 2sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.23 (Attention)
-- model.layers.23.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.23.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.23.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.23.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.23.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.23.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.23.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.23.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.23.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.23.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.23.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.23.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.23.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.23.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.23.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.23.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.23.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.23.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.23.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.23.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.23.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.23.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.23.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.23.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.23.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.23.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.23.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.23.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.23.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.23.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.23.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.23.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.23.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.23.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.23.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.23.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.23.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.23.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.23.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.23.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.23.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.23.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.23.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.23.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.23.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.23.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.23.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.23.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.23.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.23.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.23.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.23.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.23.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.23.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.23.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.23.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.23.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96144368
-- 2.1987 bpw accuracy: 0.96321644
-- 2.2831 bpw accuracy: 0.96714985
-- 2.6768 bpw accuracy: 0.97383911
-- 3.1689 bpw accuracy: 0.97893453
-- 3.1705 bpw accuracy: 0.98010537
-- 4.0439 bpw accuracy: 0.98434053
-- 4.0471 bpw accuracy: 0.98590600
-- 4.0816 bpw accuracy: 0.98705028
-- 4.1381 bpw accuracy: 0.98767805
-- 4.1705 bpw accuracy: 0.98999631
-- 4.1902 bpw accuracy: 0.99065767
-- 4.2737 bpw accuracy: 0.99120639
-- 4.3295 bpw accuracy: 0.99190866
-- 5.2564 bpw accuracy: 0.99473721
-- 5.3295 bpw accuracy: 0.99599179
-- 6.0439 bpw accuracy: 0.99565781
-- 6.3381 bpw accuracy: 0.99789982
-- 8.0439 bpw accuracy: 0.99887381
------------------------------------------------
| Measured: model.layers.23 (Attention) |
| Duration: 12.06 seconds |
| Completed step: 47/67 |
| Avg time / step (rolling): 22.97 seconds |
| Estimated remaining time: 7min 39sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Layer: model.layers.23 (MLP)
-- model.layers.23.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.23.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.23.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.23.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.23.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.23.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.23.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.23.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.23.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.23.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.23.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.23.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.23.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.23.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.23.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.23.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.23.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.23.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.23.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.23.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.23.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.23.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.23.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.23.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.23.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.23.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.23.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.23.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.23.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.23.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.23.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.23.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91510165
-- 2.3230 bpw accuracy: 0.91747099
-- 2.5958 bpw accuracy: 0.92978290
-- 2.9120 bpw accuracy: 0.93366610
-- 3.2833 bpw accuracy: 0.95690619
-- 3.3655 bpw accuracy: 0.96029718
-- 3.6186 bpw accuracy: 0.96564207
-- 4.1368 bpw accuracy: 0.97737368
-- 4.1977 bpw accuracy: 0.97937206
-- 4.2662 bpw accuracy: 0.97805914
-- 4.3484 bpw accuracy: 0.98075591
-- 5.2491 bpw accuracy: 0.98883342
-- 5.3313 bpw accuracy: 0.99041751
-- 6.0713 bpw accuracy: 0.99387568
-- 6.3032 bpw accuracy: 0.99432397
-- 6.8687 bpw accuracy: 0.99537077
-- 8.0354 bpw accuracy: 0.99829127
------------------------------------------------
| Measured: model.layers.23 (MLP) |
| Duration: 33.97 seconds |
| Completed step: 48/67 |
| Avg time / step (rolling): 23.00 seconds |
| Estimated remaining time: 7min 16sec |
| Last checkpoint layer: model.layers.19 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.24 (Attention)
-- model.layers.24.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.24.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.24.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.24.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.24.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.24.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.24.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.24.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.24.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.24.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.24.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.24.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.24.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.24.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.24.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.24.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.24.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.24.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.24.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.24.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.24.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.24.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.24.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.24.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.24.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.24.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.24.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.24.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.24.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.24.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.24.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.24.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.24.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.24.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.24.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.24.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.24.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.24.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.24.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.24.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.24.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.24.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.24.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.24.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.24.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.24.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.24.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.24.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.24.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.24.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.24.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.24.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.24.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.24.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.24.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.24.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.24.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96257212
-- 2.1987 bpw accuracy: 0.96409544
-- 2.2831 bpw accuracy: 0.96815047
-- 2.6768 bpw accuracy: 0.97486615
-- 3.1689 bpw accuracy: 0.97995368
-- 3.1705 bpw accuracy: 0.98089216
-- 4.0439 bpw accuracy: 0.98570531
-- 4.0471 bpw accuracy: 0.98689639
-- 4.0816 bpw accuracy: 0.98764440
-- 4.1381 bpw accuracy: 0.98846461
-- 4.1705 bpw accuracy: 0.99020582
-- 4.1902 bpw accuracy: 0.99094274
-- 4.2737 bpw accuracy: 0.99156304
-- 4.3295 bpw accuracy: 0.99228689
-- 5.2564 bpw accuracy: 0.99513498
-- 5.3295 bpw accuracy: 0.99612904
-- 6.0439 bpw accuracy: 0.99606761
-- 6.3381 bpw accuracy: 0.99787116
-- 8.0439 bpw accuracy: 0.99891893
------------------------------------------------
| Measured: model.layers.24 (Attention) |
| Duration: 12.01 seconds |
| Completed step: 49/67 |
| Avg time / step (rolling): 22.99 seconds |
| Estimated remaining time: 6min 53sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.24 (MLP)
-- model.layers.24.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.24.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.24.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.24.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.24.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.24.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.24.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.24.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.24.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.24.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.24.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.24.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.24.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.24.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.24.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.24.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.24.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.24.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.24.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.24.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.24.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.24.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.24.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.24.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.24.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.24.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.24.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.24.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.24.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.24.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.24.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.24.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91491852
-- 2.3230 bpw accuracy: 0.91733397
-- 2.5958 bpw accuracy: 0.92939297
-- 2.9120 bpw accuracy: 0.93317702
-- 3.2833 bpw accuracy: 0.95686523
-- 3.3655 bpw accuracy: 0.96027245
-- 3.6186 bpw accuracy: 0.96548226
-- 4.1368 bpw accuracy: 0.97737771
-- 4.1977 bpw accuracy: 0.97933167
-- 4.2662 bpw accuracy: 0.97802880
-- 4.3484 bpw accuracy: 0.98074737
-- 5.2491 bpw accuracy: 0.98882752
-- 5.3313 bpw accuracy: 0.99041133
-- 6.0713 bpw accuracy: 0.99385661
-- 6.3032 bpw accuracy: 0.99432461
-- 6.8687 bpw accuracy: 0.99533995
-- 8.0354 bpw accuracy: 0.99828698
------------------------------------------------
| Measured: model.layers.24 (MLP) |
| Duration: 33.87 seconds |
| Completed step: 50/67 |
| Avg time / step (rolling): 22.99 seconds |
| Estimated remaining time: 6min 30sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.25 (Attention)
-- model.layers.25.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.25.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.25.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.25.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.25.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.25.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.25.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.25.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.25.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.25.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.25.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.25.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.25.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.25.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.25.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.25.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.25.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.25.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.25.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.25.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.25.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.25.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.25.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.25.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.25.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.25.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.25.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.25.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.25.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.25.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.25.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.25.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.25.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.25.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.25.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.25.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.25.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.25.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.25.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.25.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.25.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.25.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.25.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.25.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.25.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.25.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.25.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.25.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.25.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.25.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.25.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.25.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.25.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.25.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.25.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.25.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.25.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96120341
-- 2.1987 bpw accuracy: 0.96268385
-- 2.2831 bpw accuracy: 0.96738063
-- 2.6768 bpw accuracy: 0.97368564
-- 3.1689 bpw accuracy: 0.97990396
-- 3.1705 bpw accuracy: 0.98084363
-- 4.0439 bpw accuracy: 0.98534533
-- 4.0471 bpw accuracy: 0.98675803
-- 4.0816 bpw accuracy: 0.98801213
-- 4.1381 bpw accuracy: 0.98836467
-- 4.1705 bpw accuracy: 0.99026544
-- 4.1902 bpw accuracy: 0.99094024
-- 4.2737 bpw accuracy: 0.99156918
-- 4.3295 bpw accuracy: 0.99229064
-- 5.2564 bpw accuracy: 0.99493717
-- 5.3295 bpw accuracy: 0.99613090
-- 6.0439 bpw accuracy: 0.99582996
-- 6.3381 bpw accuracy: 0.99796351
-- 8.0439 bpw accuracy: 0.99890425
------------------------------------------------
| Measured: model.layers.25 (Attention) |
| Duration: 12.06 seconds |
| Completed step: 51/67 |
| Avg time / step (rolling): 22.99 seconds |
| Estimated remaining time: 6min 7sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.25 (MLP)
-- model.layers.25.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.25.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.25.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.25.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.25.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.25.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.25.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.25.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.25.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.25.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.25.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.25.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.25.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.25.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.25.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.25.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.25.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.25.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.25.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.25.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.25.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.25.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.25.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.25.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.25.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.25.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.25.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.25.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.25.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.25.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.25.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.25.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91408373
-- 2.3230 bpw accuracy: 0.91658161
-- 2.5958 bpw accuracy: 0.92834551
-- 2.9120 bpw accuracy: 0.93212452
-- 3.2833 bpw accuracy: 0.95646317
-- 3.3655 bpw accuracy: 0.95986035
-- 3.6186 bpw accuracy: 0.96498522
-- 4.1368 bpw accuracy: 0.97706547
-- 4.1977 bpw accuracy: 0.97908159
-- 4.2662 bpw accuracy: 0.97785401
-- 4.3484 bpw accuracy: 0.98055461
-- 5.2491 bpw accuracy: 0.98874060
-- 5.3313 bpw accuracy: 0.99031881
-- 6.0713 bpw accuracy: 0.99380535
-- 6.3032 bpw accuracy: 0.99429043
-- 6.8687 bpw accuracy: 0.99529690
-- 8.0354 bpw accuracy: 0.99829267
------------------------------------------------
| Measured: model.layers.25 (MLP) |
| Duration: 34.05 seconds |
| Completed step: 52/67 |
| Avg time / step (rolling): 23.01 seconds |
| Estimated remaining time: 5min 45sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.26 (Attention)
-- model.layers.26.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.26.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.26.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.26.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.26.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.26.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.26.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.26.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.26.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.26.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.26.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.26.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.26.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.26.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.26.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.26.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.26.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.26.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.26.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.26.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.26.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.26.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.26.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.26.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.26.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.26.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.26.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.26.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.26.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.26.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.26.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.26.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.26.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.26.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.26.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.26.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.26.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.26.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.26.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.26.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.26.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.26.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.26.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.26.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.26.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.26.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.26.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.26.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.26.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.26.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.26.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.26.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.26.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.26.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.26.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.26.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.26.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96226496
-- 2.1987 bpw accuracy: 0.96374796
-- 2.2831 bpw accuracy: 0.96728361
-- 2.6768 bpw accuracy: 0.97462485
-- 3.1689 bpw accuracy: 0.97966360
-- 3.1705 bpw accuracy: 0.98046957
-- 4.0439 bpw accuracy: 0.98569630
-- 4.0471 bpw accuracy: 0.98663128
-- 4.0816 bpw accuracy: 0.98772842
-- 4.1381 bpw accuracy: 0.98780873
-- 4.1705 bpw accuracy: 0.99015509
-- 4.1902 bpw accuracy: 0.99067327
-- 4.2737 bpw accuracy: 0.99132762
-- 4.3295 bpw accuracy: 0.99193709
-- 5.2564 bpw accuracy: 0.99476430
-- 5.3295 bpw accuracy: 0.99600561
-- 6.0439 bpw accuracy: 0.99574439
-- 6.3381 bpw accuracy: 0.99789062
-- 8.0439 bpw accuracy: 0.99886594
------------------------------------------------
| Measured: model.layers.26 (Attention) |
| Duration: 11.88 seconds |
| Completed step: 53/67 |
| Avg time / step (rolling): 23.00 seconds |
| Estimated remaining time: 5min 22sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.26 (MLP)
-- model.layers.26.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.26.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.26.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.26.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.26.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.26.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.26.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.26.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.26.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.26.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.26.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.26.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.26.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.26.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.26.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.26.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.26.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.26.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.26.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.26.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.26.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.26.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.26.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.26.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.26.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.26.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.26.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.26.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.26.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.26.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.26.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.26.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91220704
-- 2.3230 bpw accuracy: 0.91475706
-- 2.5958 bpw accuracy: 0.92648144
-- 2.9120 bpw accuracy: 0.93029952
-- 3.2833 bpw accuracy: 0.95540529
-- 3.3655 bpw accuracy: 0.95892851
-- 3.6186 bpw accuracy: 0.96407402
-- 4.1368 bpw accuracy: 0.97650182
-- 4.1977 bpw accuracy: 0.97855930
-- 4.2662 bpw accuracy: 0.97730471
-- 4.3484 bpw accuracy: 0.98009800
-- 5.2491 bpw accuracy: 0.98845722
-- 5.3313 bpw accuracy: 0.99008362
-- 6.0713 bpw accuracy: 0.99363205
-- 6.3032 bpw accuracy: 0.99414638
-- 6.8687 bpw accuracy: 0.99515435
-- 8.0354 bpw accuracy: 0.99821012
------------------------------------------------
| Measured: model.layers.26 (MLP) |
| Duration: 33.86 seconds |
| Completed step: 54/67 |
| Avg time / step (rolling): 22.98 seconds |
| Estimated remaining time: 4min 58sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.27 (Attention)
-- model.layers.27.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.27.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.27.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.27.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.27.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.27.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.27.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.27.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.27.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.27.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.27.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.27.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.27.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.27.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.27.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.27.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.27.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.27.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.27.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.27.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.27.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.27.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.27.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.27.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.27.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.27.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.27.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.27.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.27.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.27.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.27.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.27.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.27.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.27.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.27.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.27.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.27.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.27.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.27.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.27.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.27.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.27.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.27.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.27.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.27.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.27.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.27.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.27.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.27.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.27.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.27.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.27.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.27.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.27.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.27.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.27.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.27.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.96160406
-- 2.1987 bpw accuracy: 0.96318289
-- 2.2831 bpw accuracy: 0.96765929
-- 2.6768 bpw accuracy: 0.97472490
-- 3.1689 bpw accuracy: 0.97987545
-- 3.1705 bpw accuracy: 0.98059758
-- 4.0439 bpw accuracy: 0.98593180
-- 4.0471 bpw accuracy: 0.98691583
-- 4.0816 bpw accuracy: 0.98784235
-- 4.1381 bpw accuracy: 0.98856410
-- 4.1705 bpw accuracy: 0.99007280
-- 4.1902 bpw accuracy: 0.99071385
-- 4.2737 bpw accuracy: 0.99124282
-- 4.3295 bpw accuracy: 0.99189155
-- 5.2564 bpw accuracy: 0.99500938
-- 5.3295 bpw accuracy: 0.99599362
-- 6.0439 bpw accuracy: 0.99611215
-- 6.3381 bpw accuracy: 0.99792379
-- 8.0439 bpw accuracy: 0.99894116
------------------------------------------------
| Measured: model.layers.27 (Attention) |
| Duration: 12.05 seconds |
| Completed step: 55/67 |
| Avg time / step (rolling): 22.99 seconds |
| Estimated remaining time: 4min 35sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Layer: model.layers.27 (MLP)
-- model.layers.27.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.27.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.27.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.27.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.27.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.27.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.27.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.27.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.27.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.27.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.27.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.27.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.27.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.27.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.27.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.27.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.27.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.27.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.27.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.27.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.27.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.27.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.27.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.27.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.27.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.27.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.27.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.27.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.27.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.27.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.27.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.27.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.91032989
-- 2.3230 bpw accuracy: 0.91298755
-- 2.5958 bpw accuracy: 0.92489017
-- 2.9120 bpw accuracy: 0.92879948
-- 3.2833 bpw accuracy: 0.95435773
-- 3.3655 bpw accuracy: 0.95794147
-- 3.6186 bpw accuracy: 0.96321213
-- 4.1368 bpw accuracy: 0.97585048
-- 4.1977 bpw accuracy: 0.97798234
-- 4.2662 bpw accuracy: 0.97677132
-- 4.3484 bpw accuracy: 0.97962346
-- 5.2491 bpw accuracy: 0.98819851
-- 5.3313 bpw accuracy: 0.98984934
-- 6.0713 bpw accuracy: 0.99345932
-- 6.3032 bpw accuracy: 0.99401268
-- 6.8687 bpw accuracy: 0.99504861
-- 8.0354 bpw accuracy: 0.99814431
------------------------------------------------
| Measured: model.layers.27 (MLP) |
| Duration: 33.97 seconds |
| Completed step: 56/67 |
| Avg time / step (rolling): 22.98 seconds |
| Estimated remaining time: 4min 12sec |
| Last checkpoint layer: model.layers.23 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Layer: model.layers.28 (Attention)
-- model.layers.28.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.28.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.28.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.28.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.28.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.28.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.28.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.28.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.28.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.28.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.28.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.28.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.28.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.28.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.28.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.28.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.28.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.28.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.28.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.28.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.28.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.28.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.28.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.28.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.28.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.28.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.28.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.28.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.28.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.28.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.28.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.28.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.28.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.28.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.28.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.28.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.28.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.28.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.28.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.28.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.28.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.28.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.28.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.28.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.28.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.28.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.28.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.28.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.28.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.28.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.28.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.28.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.28.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.28.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.28.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.28.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.28.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.95566610
-- 2.1987 bpw accuracy: 0.95779977
-- 2.2831 bpw accuracy: 0.96239198
-- 2.6768 bpw accuracy: 0.97110887
-- 3.1689 bpw accuracy: 0.97689971
-- 3.1705 bpw accuracy: 0.97764647
-- 4.0439 bpw accuracy: 0.98388138
-- 4.0471 bpw accuracy: 0.98485097
-- 4.0816 bpw accuracy: 0.98628805
-- 4.1381 bpw accuracy: 0.98688308
-- 4.1705 bpw accuracy: 0.98863484
-- 4.1902 bpw accuracy: 0.98942486
-- 4.2737 bpw accuracy: 0.99008797
-- 4.3295 bpw accuracy: 0.99090450
-- 5.2564 bpw accuracy: 0.99428032
-- 5.3295 bpw accuracy: 0.99537329
-- 6.0439 bpw accuracy: 0.99550669
-- 6.3381 bpw accuracy: 0.99745212
-- 8.0439 bpw accuracy: 0.99875961
------------------------------------------------
| Measured: model.layers.28 (Attention) |
| Duration: 11.94 seconds |
| Completed step: 57/67 |
| Avg time / step (rolling): 22.97 seconds |
| Estimated remaining time: 3min 49sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.28 (MLP)
-- model.layers.28.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.28.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.28.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.28.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.28.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.28.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.28.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.28.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.28.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.28.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.28.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.28.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.28.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.28.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.28.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.28.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.28.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.28.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.28.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.28.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.28.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.28.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.28.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.28.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.28.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.28.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.28.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.28.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.28.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.28.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.28.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.28.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.90378511
-- 2.3230 bpw accuracy: 0.90666246
-- 2.5958 bpw accuracy: 0.91989426
-- 2.9120 bpw accuracy: 0.92431042
-- 3.2833 bpw accuracy: 0.95098403
-- 3.3655 bpw accuracy: 0.95486298
-- 3.6186 bpw accuracy: 0.96074617
-- 4.1368 bpw accuracy: 0.97390766
-- 4.1977 bpw accuracy: 0.97622691
-- 4.2662 bpw accuracy: 0.97503045
-- 4.3484 bpw accuracy: 0.97811373
-- 5.2491 bpw accuracy: 0.98730669
-- 5.3313 bpw accuracy: 0.98908933
-- 6.0713 bpw accuracy: 0.99290057
-- 6.3032 bpw accuracy: 0.99357856
-- 6.8687 bpw accuracy: 0.99475442
-- 8.0354 bpw accuracy: 0.99800259
------------------------------------------------
| Measured: model.layers.28 (MLP) |
| Duration: 34.02 seconds |
| Completed step: 58/67 |
| Avg time / step (rolling): 22.97 seconds |
| Estimated remaining time: 3min 26sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.29 (Attention)
-- model.layers.29.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.29.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.29.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.29.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.29.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.29.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.29.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.29.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.29.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.29.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.29.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.29.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.29.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.29.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.29.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.29.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.29.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.29.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.29.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.29.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.29.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.29.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.29.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.29.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.29.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.29.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.29.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.29.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.29.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.29.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.29.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.29.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.29.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.29.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.29.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.29.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.29.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.29.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.29.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.29.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.29.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.29.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.29.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.29.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.29.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.29.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.29.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.29.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.29.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.29.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.29.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.29.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.29.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.29.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.29.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.29.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.29.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94369422
-- 2.1987 bpw accuracy: 0.94609539
-- 2.2831 bpw accuracy: 0.95318407
-- 2.6768 bpw accuracy: 0.96340803
-- 3.1689 bpw accuracy: 0.97089370
-- 3.1705 bpw accuracy: 0.97151723
-- 4.0439 bpw accuracy: 0.98022538
-- 4.0471 bpw accuracy: 0.98100285
-- 4.0816 bpw accuracy: 0.98245847
-- 4.1381 bpw accuracy: 0.98348722
-- 4.1705 bpw accuracy: 0.98524744
-- 4.1902 bpw accuracy: 0.98633939
-- 4.2737 bpw accuracy: 0.98746550
-- 4.3295 bpw accuracy: 0.98837066
-- 5.2564 bpw accuracy: 0.99300754
-- 5.3295 bpw accuracy: 0.99417307
-- 6.0439 bpw accuracy: 0.99456764
-- 6.3381 bpw accuracy: 0.99701945
-- 8.0439 bpw accuracy: 0.99855181
------------------------------------------------
| Measured: model.layers.29 (Attention) |
| Duration: 12.04 seconds |
| Completed step: 59/67 |
| Avg time / step (rolling): 22.98 seconds |
| Estimated remaining time: 3min 3sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.29 (MLP)
-- model.layers.29.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.29.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.29.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.29.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.29.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.29.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.29.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.29.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.29.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.29.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.29.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.29.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.29.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.29.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.29.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.29.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.29.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.29.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.29.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.29.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.29.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.29.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.29.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.29.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.29.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.29.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.29.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.29.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.29.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.29.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.29.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.29.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.90313367
-- 2.3230 bpw accuracy: 0.90606377
-- 2.5958 bpw accuracy: 0.91966039
-- 2.9120 bpw accuracy: 0.92417615
-- 3.2833 bpw accuracy: 0.95051413
-- 3.3655 bpw accuracy: 0.95439962
-- 3.6186 bpw accuracy: 0.96048317
-- 4.1368 bpw accuracy: 0.97343511
-- 4.1977 bpw accuracy: 0.97576757
-- 4.2662 bpw accuracy: 0.97464513
-- 4.3484 bpw accuracy: 0.97765558
-- 5.2491 bpw accuracy: 0.98703892
-- 5.3313 bpw accuracy: 0.98882174
-- 6.0713 bpw accuracy: 0.99276578
-- 6.3032 bpw accuracy: 0.99340849
-- 6.8687 bpw accuracy: 0.99460557
-- 8.0354 bpw accuracy: 0.99800954
------------------------------------------------
| Measured: model.layers.29 (MLP) |
| Duration: 34.06 seconds |
| Completed step: 60/67 |
| Avg time / step (rolling): 23.00 seconds |
| Estimated remaining time: 2min 40sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.30 (Attention)
-- model.layers.30.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.30.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.30.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.30.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.30.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.30.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.30.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.30.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.30.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.30.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.30.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.30.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.30.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.30.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.30.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.30.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.30.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.30.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.30.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.30.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.30.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.30.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.30.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.30.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.30.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.30.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.30.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.30.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.30.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.30.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.30.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.30.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.30.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.30.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.30.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.30.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.30.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.30.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.30.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.30.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.30.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.30.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.30.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.30.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.30.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.30.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.30.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.30.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.30.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.30.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.30.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.30.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.30.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.30.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.30.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.30.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.30.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.94085100
-- 2.1987 bpw accuracy: 0.94362952
-- 2.2831 bpw accuracy: 0.95185889
-- 2.6768 bpw accuracy: 0.96062800
-- 3.1689 bpw accuracy: 0.96788531
-- 3.1705 bpw accuracy: 0.97090238
-- 4.0439 bpw accuracy: 0.97470775
-- 4.0471 bpw accuracy: 0.97797868
-- 4.0816 bpw accuracy: 0.97881716
-- 4.1381 bpw accuracy: 0.97939369
-- 4.1705 bpw accuracy: 0.98599288
-- 4.1902 bpw accuracy: 0.98683970
-- 4.2737 bpw accuracy: 0.98765240
-- 4.3295 bpw accuracy: 0.98852310
-- 5.2564 bpw accuracy: 0.99298498
-- 5.3295 bpw accuracy: 0.99412648
-- 6.0439 bpw accuracy: 0.99433308
-- 6.3381 bpw accuracy: 0.99694584
-- 8.0439 bpw accuracy: 0.99843384
------------------------------------------------
| Measured: model.layers.30 (Attention) |
| Duration: 11.84 seconds |
| Completed step: 61/67 |
| Avg time / step (rolling): 22.97 seconds |
| Estimated remaining time: 2min 17sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.30 (MLP)
-- model.layers.30.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.30.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.30.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.30.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.30.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.30.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.30.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.30.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.30.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.30.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.30.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.30.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.30.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.30.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.30.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.30.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.30.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.30.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.30.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.30.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.30.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.30.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.30.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.30.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.30.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.30.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.30.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.30.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.30.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.30.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.30.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.30.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.89440048
-- 2.3230 bpw accuracy: 0.89717557
-- 2.5958 bpw accuracy: 0.91156459
-- 2.9120 bpw accuracy: 0.91619827
-- 3.2833 bpw accuracy: 0.94636323
-- 3.3655 bpw accuracy: 0.95072527
-- 3.6186 bpw accuracy: 0.95721954
-- 4.1368 bpw accuracy: 0.97073510
-- 4.1977 bpw accuracy: 0.97355560
-- 4.2662 bpw accuracy: 0.97230197
-- 4.3484 bpw accuracy: 0.97564030
-- 5.2491 bpw accuracy: 0.98565681
-- 5.3313 bpw accuracy: 0.98775256
-- 6.0713 bpw accuracy: 0.99186824
-- 6.3032 bpw accuracy: 0.99270174
-- 6.8687 bpw accuracy: 0.99411882
-- 8.0354 bpw accuracy: 0.99773410
------------------------------------------------
| Measured: model.layers.30 (MLP) |
| Duration: 33.91 seconds |
| Completed step: 62/67 |
| Avg time / step (rolling): 22.96 seconds |
| Estimated remaining time: 1min 54sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.31 (Attention)
-- model.layers.31.self_attn.q_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.31.self_attn.q_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.31.self_attn.q_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.31.self_attn.q_proj 1:4b_128g s4 4.04 bpw
-- model.layers.31.self_attn.q_proj 1:4b_64g s4 4.07 bpw
-- model.layers.31.self_attn.q_proj 1:4b_32g s4 4.13 bpw
-- model.layers.31.self_attn.q_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.31.self_attn.q_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.31.self_attn.q_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.31.self_attn.q_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.31.self_attn.q_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.31.self_attn.q_proj 1:6b_128g s4 6.04 bpw
-- model.layers.31.self_attn.q_proj 1:6b_32g s4 6.13 bpw
-- model.layers.31.self_attn.q_proj 1:8b_128g s4 8.04 bpw
-- model.layers.31.self_attn.k_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.31.self_attn.k_proj 0.1:3b_64g/0.9:2b_64g s4 2.20 bpw
-- model.layers.31.self_attn.k_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.31.self_attn.k_proj 1:4b_128g s4 4.06 bpw
-- model.layers.31.self_attn.k_proj 1:4b_64g s4 4.09 bpw
-- model.layers.31.self_attn.k_proj 1:4b_32g s4 4.16 bpw
-- model.layers.31.self_attn.k_proj 0.1:5b_128g/0.9:4b_128g s4 4.19 bpw
-- model.layers.31.self_attn.k_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.31.self_attn.k_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.31.self_attn.k_proj 0.1:6b_128g/0.9:5b_128g s4 5.19 bpw
-- model.layers.31.self_attn.k_proj 0.1:6b_32g/0.9:5b_32g s4 5.26 bpw
-- model.layers.31.self_attn.k_proj 1:6b_128g s4 6.06 bpw
-- model.layers.31.self_attn.k_proj 1:6b_32g s4 6.16 bpw
-- model.layers.31.self_attn.k_proj 1:8b_128g s4 8.06 bpw
-- model.layers.31.self_attn.v_proj 0.05:3b_64g/0.95:2b_64g s4 2.16 bpw
-- model.layers.31.self_attn.v_proj 0.25:3b_64g/0.75:2b_64g s4 2.34 bpw
-- model.layers.31.self_attn.v_proj 0.1:4b_128g/0.9:3b_128g s4 3.19 bpw
-- model.layers.31.self_attn.v_proj 0.1:4b_64g/0.9:3b_64g s4 3.20 bpw
-- model.layers.31.self_attn.v_proj 1:4b_128g s4 4.06 bpw
-- model.layers.31.self_attn.v_proj 1:4b_64g s4 4.09 bpw
-- model.layers.31.self_attn.v_proj 1:4b_32g s4 4.16 bpw
-- model.layers.31.self_attn.v_proj 0.1:5b_64g/0.9:4b_64g s4 4.20 bpw
-- model.layers.31.self_attn.v_proj 0.1:5b_32g/0.9:4b_32g s4 4.26 bpw
-- model.layers.31.self_attn.v_proj 1:5b_64g s4 5.09 bpw
-- model.layers.31.self_attn.v_proj 1:5b_32g s4 5.16 bpw
-- model.layers.31.self_attn.v_proj 1:6b_128g s4 6.06 bpw
-- model.layers.31.self_attn.v_proj 1:6b_32g s4 6.16 bpw
-- model.layers.31.self_attn.v_proj 1:8b_32g s4 8.16 bpw
-- model.layers.31.self_attn.v_proj 1:8b_128g s4 8.06 bpw
-- model.layers.31.self_attn.o_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.31.self_attn.o_proj 0.1:3b_64g/0.9:2b_64g s4 2.18 bpw
-- model.layers.31.self_attn.o_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.31.self_attn.o_proj 1:4b_128g s4 4.04 bpw
-- model.layers.31.self_attn.o_proj 1:4b_64g s4 4.07 bpw
-- model.layers.31.self_attn.o_proj 1:4b_32g s4 4.13 bpw
-- model.layers.31.self_attn.o_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.31.self_attn.o_proj 0.1:5b_64g/0.9:4b_64g s4 4.18 bpw
-- model.layers.31.self_attn.o_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.31.self_attn.o_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.31.self_attn.o_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.31.self_attn.o_proj 1:6b_128g s4 6.04 bpw
-- model.layers.31.self_attn.o_proj 1:6b_32g s4 6.13 bpw
-- model.layers.31.self_attn.o_proj 1:8b_128g s4 8.04 bpw
-- 2.1378 bpw accuracy: 0.93302331
-- 2.1987 bpw accuracy: 0.93534842
-- 2.2831 bpw accuracy: 0.94564187
-- 2.6768 bpw accuracy: 0.95836522
-- 3.1689 bpw accuracy: 0.97124833
-- 3.1705 bpw accuracy: 0.97196335
-- 4.0439 bpw accuracy: 0.98044352
-- 4.0471 bpw accuracy: 0.98173889
-- 4.0816 bpw accuracy: 0.98426918
-- 4.1381 bpw accuracy: 0.98494215
-- 4.1705 bpw accuracy: 0.98609338
-- 4.1902 bpw accuracy: 0.98699982
-- 4.2737 bpw accuracy: 0.98821953
-- 4.3295 bpw accuracy: 0.98907104
-- 5.2564 bpw accuracy: 0.99348937
-- 5.3295 bpw accuracy: 0.99435629
-- 6.0439 bpw accuracy: 0.99523822
-- 6.3381 bpw accuracy: 0.99697390
-- 8.0439 bpw accuracy: 0.99865403
------------------------------------------------
| Measured: model.layers.31 (Attention) |
| Duration: 11.95 seconds |
| Completed step: 63/67 |
| Avg time / step (rolling): 22.97 seconds |
| Estimated remaining time: 1min 31sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Layer: model.layers.31 (MLP)
-- model.layers.31.mlp.gate_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.31.mlp.gate_proj 0.1:3b_64g/0.9:2b_64g s4 2.17 bpw
-- model.layers.31.mlp.gate_proj 0.1:4b_128g/0.9:3b_128g s4 3.16 bpw
-- model.layers.31.mlp.gate_proj 0.1:4b_32g/0.9:3b_32g s4 3.23 bpw
-- model.layers.31.mlp.gate_proj 1:4b_128g s4 4.03 bpw
-- model.layers.31.mlp.gate_proj 1:4b_32g s4 4.13 bpw
-- model.layers.31.mlp.gate_proj 0.1:5b_128g/0.9:4b_128g s4 4.16 bpw
-- model.layers.31.mlp.gate_proj 0.1:5b_32g/0.9:4b_32g s4 4.23 bpw
-- model.layers.31.mlp.gate_proj 0.1:6b_128g/0.9:5b_128g s4 5.16 bpw
-- model.layers.31.mlp.gate_proj 0.1:6b_32g/0.9:5b_32g s4 5.23 bpw
-- model.layers.31.mlp.gate_proj 1:6b_128g s4 6.03 bpw
-- model.layers.31.mlp.gate_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.31.mlp.gate_proj 1:8b_128g s4 8.03 bpw
-- model.layers.31.mlp.up_proj 0.05:3b_64g/0.95:2b_64g s4 2.13 bpw
-- model.layers.31.mlp.up_proj 0.25:3b_64g/0.75:2b_64g s4 2.31 bpw
-- model.layers.31.mlp.up_proj 0.3:3b_64g/0.7:2b_64g s4 2.38 bpw
-- model.layers.31.mlp.up_proj 0.25:4b_128g/0.75:3b_128g s4 3.28 bpw
-- model.layers.31.mlp.up_proj 0.25:4b_32g/0.75:3b_32g s4 3.38 bpw
-- model.layers.31.mlp.up_proj 1:4b_32g s4 4.13 bpw
-- model.layers.31.mlp.up_proj 0.25:5b_128g/0.75:4b_128g s4 4.28 bpw
-- model.layers.31.mlp.up_proj 0.25:5b_32g/0.75:4b_32g s4 4.38 bpw
-- model.layers.31.mlp.up_proj 0.25:6b_128g/0.75:5b_128g s4 5.28 bpw
-- model.layers.31.mlp.up_proj 0.25:6b_32g/0.75:5b_32g s4 5.38 bpw
-- model.layers.31.mlp.up_proj 1:6b_128g s4 6.03 bpw
-- model.layers.31.mlp.up_proj 0.1:8b_128g/0.9:6b_128g s4 6.28 bpw
-- model.layers.31.mlp.up_proj 1:8b_128g s4 8.03 bpw
-- model.layers.31.mlp.down_proj 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4 2.48 bpw
-- model.layers.31.mlp.down_proj 0.05:5b_32g/0.95:3b_32g s4 3.24 bpw
-- model.layers.31.mlp.down_proj 0.05:5b_32g/0.95:4b_32g s4 4.18 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:4b_128g/0.85:3b_128g s4 3.41 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4 3.49 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.95:4b_128g s4 4.25 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.95:4b_32g s4 4.34 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4 4.36 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4 4.44 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:6b_128g/0.85:5b_128g s4 5.31 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.1:6b_32g/0.85:5b_32g s4 5.39 bpw
-- model.layers.31.mlp.down_proj 0.05:8b_32g/0.95:6b_128g s4 6.15 bpw
-- model.layers.31.mlp.down_proj 0.15:8b_128g/0.85:6b_128g s4 6.34 bpw
-- model.layers.31.mlp.down_proj 1:8b_128g s4 8.04 bpw
-- 2.2449 bpw accuracy: 0.89348449
-- 2.3230 bpw accuracy: 0.89637273
-- 2.5958 bpw accuracy: 0.91020142
-- 2.9120 bpw accuracy: 0.91476823
-- 3.2833 bpw accuracy: 0.94536082
-- 3.3655 bpw accuracy: 0.95022537
-- 3.6186 bpw accuracy: 0.95641279
-- 4.1368 bpw accuracy: 0.97063887
-- 4.1977 bpw accuracy: 0.97365729
-- 4.2662 bpw accuracy: 0.97190128
-- 4.3484 bpw accuracy: 0.97561462
-- 5.2491 bpw accuracy: 0.98552038
-- 5.3313 bpw accuracy: 0.98772911
-- 6.0713 bpw accuracy: 0.99174033
-- 6.3032 bpw accuracy: 0.99259526
-- 6.8687 bpw accuracy: 0.99397933
-- 8.0354 bpw accuracy: 0.99742786
------------------------------------------------
| Measured: model.layers.31 (MLP) |
| Duration: 33.79 seconds |
| Completed step: 64/67 |
| Avg time / step (rolling): 22.96 seconds |
| Estimated remaining time: 1min 8sec |
| Last checkpoint layer: model.layers.27 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Layer: model.norm (RMSNorm)
------------------------------------------------
| Measured: model.norm (RMSNorm) |
| Duration: 0.28 seconds |
| Completed step: 65/67 |
| Avg time / step (rolling): 21.78 seconds |
| Estimated remaining time: 0min 43sec |
| Last checkpoint layer: model.layers.31 (MLP) |
------------------------------------------------
-- Layer: lm_head (Linear)
------------------------------------------------
| Measured: lm_head (Linear) |
| Duration: 1.36 seconds |
| Completed step: 66/67 |
| Avg time / step (rolling): 18.52 seconds |
| Estimated remaining time: 0min 18sec |
| Last checkpoint layer: model.layers.31 (MLP) |
------------------------------------------------
-- Saving checkpoint...
-- Optimizing...
-- Optimizing: 1/ 240
-- Optimizing: 8/ 240
-- Optimizing: 15/ 240
-- Optimizing: 22/ 240
-- Optimizing: 29/ 240
-- Optimizing: 36/ 240
-- Optimizing: 43/ 240
-- Optimizing: 50/ 240
-- Optimizing: 57/ 240
-- Optimizing: 64/ 240
-- Optimizing: 71/ 240
-- Optimizing: 78/ 240
-- Optimizing: 80/ 240
-- Optimizing: 87/ 240
-- Optimizing: 94/ 240
-- Optimizing: 101/ 240
-- Optimizing: 108/ 240
-- Optimizing: 115/ 240
-- Optimizing: 122/ 240
-- Optimizing: 129/ 240
-- Optimizing: 136/ 240
-- Optimizing: 143/ 240
-- Optimizing: 150/ 240
-- Optimizing: 157/ 240
-- Optimizing: 164/ 240
-- Optimizing: 171/ 240
-- Optimizing: 178/ 240
-- Optimizing: 185/ 240
-- Optimizing: 192/ 240
-- Optimizing: 199/ 240
-- Optimizing: 206/ 240
-- Optimizing: 213/ 240
-- Optimizing: 220/ 240
-- Optimizing: 227/ 240
-- Optimizing: 234/ 240
-- max(err): 0.028099
-- error_norm: 1.905063
-- Quantization strategy:
-- model.layers.0.self_attn 5.3295 bpw - exp. error: 0.01243892
-- model.layers.0.mlp 4.1977 bpw - exp. error: 0.02316471
-- model.layers.1.self_attn 6.3381 bpw - exp. error: 0.00675082
-- model.layers.1.mlp 3.3655 bpw - exp. error: 0.01386648
-- model.layers.2.self_attn 3.1689 bpw - exp. error: 0.00898621
-- model.layers.2.mlp 2.2449 bpw - exp. error: 0.02217232
-- model.layers.3.self_attn 2.6768 bpw - exp. error: 0.01153749
-- model.layers.3.mlp 2.3230 bpw - exp. error: 0.02595236
-- model.layers.4.self_attn 2.1378 bpw - exp. error: 0.01747415
-- model.layers.4.mlp 2.5958 bpw - exp. error: 0.02451262
-- model.layers.5.self_attn 3.1705 bpw - exp. error: 0.01162412
-- model.layers.5.mlp 3.3655 bpw - exp. error: 0.01657775
-- model.layers.6.self_attn 4.2737 bpw - exp. error: 0.00533969
-- model.layers.6.mlp 3.3655 bpw - exp. error: 0.01831759
-- model.layers.7.self_attn 4.0439 bpw - exp. error: 0.00968624
-- model.layers.7.mlp 3.6186 bpw - exp. error: 0.01697350
-- model.layers.8.self_attn 4.0471 bpw - exp. error: 0.01017824
-- model.layers.8.mlp 3.3655 bpw - exp. error: 0.02097592
-- model.layers.9.self_attn 5.2564 bpw - exp. error: 0.00361262
-- model.layers.9.mlp 4.3484 bpw - exp. error: 0.01076520
-- model.layers.10.self_attn 4.1902 bpw - exp. error: 0.00835164
-- model.layers.10.mlp 4.1977 bpw - exp. error: 0.01188652
-- model.layers.11.self_attn 4.1381 bpw - exp. error: 0.01156505
-- model.layers.11.mlp 3.3655 bpw - exp. error: 0.02347162
-- model.layers.12.self_attn 4.1705 bpw - exp. error: 0.01163896
-- model.layers.12.mlp 4.1368 bpw - exp. error: 0.01476438
-- model.layers.13.self_attn 5.3295 bpw - exp. error: 0.00435268
-- model.layers.13.mlp 4.2662 bpw - exp. error: 0.01526959
-- model.layers.14.self_attn 4.1381 bpw - exp. error: 0.01377695
-- model.layers.14.mlp 4.1977 bpw - exp. error: 0.01541692
-- model.layers.15.self_attn 4.2737 bpw - exp. error: 0.01253850
-- model.layers.15.mlp 4.1368 bpw - exp. error: 0.01804334
-- model.layers.16.self_attn 4.3295 bpw - exp. error: 0.01006202
-- model.layers.16.mlp 4.1977 bpw - exp. error: 0.01800463
-- model.layers.17.self_attn 4.2737 bpw - exp. error: 0.01224011
-- model.layers.17.mlp 4.2662 bpw - exp. error: 0.02064065
-- model.layers.18.self_attn 4.3295 bpw - exp. error: 0.01105552
-- model.layers.18.mlp 4.1977 bpw - exp. error: 0.02116143
-- model.layers.19.self_attn 4.1705 bpw - exp. error: 0.01309177
-- model.layers.19.mlp 4.3484 bpw - exp. error: 0.01956300
-- model.layers.20.self_attn 4.3295 bpw - exp. error: 0.00982642
-- model.layers.20.mlp 4.1368 bpw - exp. error: 0.02326088
-- model.layers.21.self_attn 4.2737 bpw - exp. error: 0.00991228
-- model.layers.21.mlp 4.1977 bpw - exp. error: 0.02066668
-- model.layers.22.self_attn 5.2564 bpw - exp. error: 0.00524883
-- model.layers.22.mlp 4.1977 bpw - exp. error: 0.02050188
-- model.layers.23.self_attn 4.3295 bpw - exp. error: 0.00809134
-- model.layers.23.mlp 4.1977 bpw - exp. error: 0.02062794
-- model.layers.24.self_attn 5.2564 bpw - exp. error: 0.00486502
-- model.layers.24.mlp 4.3484 bpw - exp. error: 0.01925263
-- model.layers.25.self_attn 4.2737 bpw - exp. error: 0.00843082
-- model.layers.25.mlp 4.3484 bpw - exp. error: 0.01944539
-- model.layers.26.self_attn 4.1705 bpw - exp. error: 0.00984491
-- model.layers.26.mlp 4.1977 bpw - exp. error: 0.02144070
-- model.layers.27.self_attn 4.1705 bpw - exp. error: 0.00992720
-- model.layers.27.mlp 4.1977 bpw - exp. error: 0.02201766
-- model.layers.28.self_attn 4.3295 bpw - exp. error: 0.00909550
-- model.layers.28.mlp 4.1977 bpw - exp. error: 0.02377309
-- model.layers.29.self_attn 5.2564 bpw - exp. error: 0.00699246
-- model.layers.29.mlp 4.3484 bpw - exp. error: 0.02234442
-- model.layers.30.self_attn 6.0439 bpw - exp. error: 0.00566692
-- model.layers.30.mlp 4.1977 bpw - exp. error: 0.02644440
-- model.layers.31.self_attn 5.2564 bpw - exp. error: 0.00651063
-- model.layers.31.mlp 4.2662 bpw - exp. error: 0.02809872
-- sum(log(err)): -277.234195
-- max(err): 0.028099
-- Tokenizing samples...
-- Token embeddings again...
-- Quantizing...
-- Layer: model.layers.0 (Attention)
-- Linear: model.layers.0.self_attn.q_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.23 bpw
-- Linear: model.layers.0.self_attn.k_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.26 bpw
-- Linear: model.layers.0.self_attn.v_proj -> 1:6b_32g s4, 6.16 bpw
-- Linear: model.layers.0.self_attn.o_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.23 bpw
-- Module quantized, rfn_error: 0.014221
-- Layer: model.layers.0 (MLP)
-- Linear: model.layers.0.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.0.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.0.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.027821
-- Layer: model.layers.1 (Attention)
-- Linear: model.layers.1.self_attn.q_proj -> 1:6b_32g s4, 6.13 bpw
-- Linear: model.layers.1.self_attn.k_proj -> 1:6b_32g s4, 6.16 bpw
-- Linear: model.layers.1.self_attn.v_proj -> 1:8b_32g s4, 8.16 bpw
-- Linear: model.layers.1.self_attn.o_proj -> 1:6b_32g s4, 6.13 bpw
-- Module quantized, rfn_error: 0.007050
-- Layer: model.layers.1 (MLP)
-- Linear: model.layers.1.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.1.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.1.mlp.down_proj -> 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4, 3.49 bpw
-- Module quantized, rfn_error: 0.007409
-- Layer: model.layers.2 (Attention)
-- Linear: model.layers.2.self_attn.q_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.16 bpw
-- Linear: model.layers.2.self_attn.k_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.19 bpw
-- Linear: model.layers.2.self_attn.v_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.19 bpw
-- Linear: model.layers.2.self_attn.o_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.16 bpw
-- Module quantized, rfn_error: 0.004401
-- Layer: model.layers.2 (MLP)
-- Linear: model.layers.2.mlp.gate_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.13 bpw
-- Linear: model.layers.2.mlp.up_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.13 bpw
-- Linear: model.layers.2.mlp.down_proj -> 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4, 2.48 bpw
-- Module quantized, rfn_error: 0.012257
-- Layer: model.layers.3 (Attention)
-- Linear: model.layers.3.self_attn.q_proj -> 0.1:3b_64g/0.9:2b_64g s4, 2.18 bpw
-- Linear: model.layers.3.self_attn.k_proj -> 0.1:3b_64g/0.9:2b_64g s4, 2.20 bpw
-- Linear: model.layers.3.self_attn.v_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.19 bpw
-- Linear: model.layers.3.self_attn.o_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.16 bpw
-- Module quantized, rfn_error: 0.005541
-- Layer: model.layers.3 (MLP)
-- Linear: model.layers.3.mlp.gate_proj -> 0.1:3b_64g/0.9:2b_64g s4, 2.17 bpw
-- Linear: model.layers.3.mlp.up_proj -> 0.25:3b_64g/0.75:2b_64g s4, 2.31 bpw
-- Linear: model.layers.3.mlp.down_proj -> 0.05:6b_32g/0.2:3b_64g/0.75:2b_64g s4, 2.48 bpw
-- Module quantized, rfn_error: 0.015681
-- Layer: model.layers.4 (Attention)
-- Linear: model.layers.4.self_attn.q_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.13 bpw
-- Linear: model.layers.4.self_attn.k_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.16 bpw
-- Linear: model.layers.4.self_attn.v_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.16 bpw
-- Linear: model.layers.4.self_attn.o_proj -> 0.05:3b_64g/0.95:2b_64g s4, 2.13 bpw
-- Module quantized, rfn_error: 0.010520
-- Layer: model.layers.4 (MLP)
-- Linear: model.layers.4.mlp.gate_proj -> 0.1:3b_64g/0.9:2b_64g s4, 2.17 bpw
-- Linear: model.layers.4.mlp.up_proj -> 0.3:3b_64g/0.7:2b_64g s4, 2.38 bpw
-- Linear: model.layers.4.mlp.down_proj -> 0.05:5b_32g/0.95:3b_32g s4, 3.24 bpw
-- Module quantized, rfn_error: 0.016619
-- Layer: model.layers.5 (Attention)
-- Linear: model.layers.5.self_attn.q_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.16 bpw
-- Linear: model.layers.5.self_attn.k_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.19 bpw
-- Linear: model.layers.5.self_attn.v_proj -> 0.1:4b_64g/0.9:3b_64g s4, 3.20 bpw
-- Linear: model.layers.5.self_attn.o_proj -> 0.1:4b_128g/0.9:3b_128g s4, 3.16 bpw
-- Module quantized, rfn_error: 0.007419
-- Layer: model.layers.5 (MLP)
-- Linear: model.layers.5.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.5.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.5.mlp.down_proj -> 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4, 3.49 bpw
-- Module quantized, rfn_error: 0.012564
-- Layer: model.layers.6 (Attention)
-- Linear: model.layers.6.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.6.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.6.self_attn.v_proj -> 1:5b_64g s4, 5.09 bpw
-- Linear: model.layers.6.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.003608
-- Layer: model.layers.6 (MLP)
-- Linear: model.layers.6.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.6.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.6.mlp.down_proj -> 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4, 3.49 bpw
-- Module quantized, rfn_error: 0.014989
-- Saving checkpoint...
-- Layer: model.layers.7 (Attention)
-- Linear: model.layers.7.self_attn.q_proj -> 1:4b_128g s4, 4.04 bpw
-- Linear: model.layers.7.self_attn.k_proj -> 1:4b_128g s4, 4.06 bpw
-- Linear: model.layers.7.self_attn.v_proj -> 1:4b_128g s4, 4.06 bpw
-- Linear: model.layers.7.self_attn.o_proj -> 1:4b_128g s4, 4.04 bpw
-- Module quantized, rfn_error: 0.007035
-- Layer: model.layers.7 (MLP)
-- Linear: model.layers.7.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.7.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.7.mlp.down_proj -> 0.05:8b_32g/0.95:4b_128g s4, 4.25 bpw
-- Module quantized, rfn_error: 0.014504
-- Layer: model.layers.8 (Attention)
-- Linear: model.layers.8.self_attn.q_proj -> 1:4b_128g s4, 4.04 bpw
-- Linear: model.layers.8.self_attn.k_proj -> 1:4b_128g s4, 4.06 bpw
-- Linear: model.layers.8.self_attn.v_proj -> 1:4b_64g s4, 4.09 bpw
-- Linear: model.layers.8.self_attn.o_proj -> 1:4b_128g s4, 4.04 bpw
-- Module quantized, rfn_error: 0.007372
-- Layer: model.layers.8 (MLP)
-- Linear: model.layers.8.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.8.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.8.mlp.down_proj -> 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4, 3.49 bpw
-- Module quantized, rfn_error: 0.018791
-- Layer: model.layers.9 (Attention)
-- Linear: model.layers.9.self_attn.q_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Linear: model.layers.9.self_attn.k_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.19 bpw
-- Linear: model.layers.9.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.9.self_attn.o_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Module quantized, rfn_error: 0.003062
-- Layer: model.layers.9 (MLP)
-- Linear: model.layers.9.mlp.gate_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.9.mlp.up_proj -> 0.25:5b_32g/0.75:4b_32g s4, 4.38 bpw
-- Linear: model.layers.9.mlp.down_proj -> 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4, 4.44 bpw
-- Module quantized, rfn_error: 0.009912
-- Layer: model.layers.10 (Attention)
-- Linear: model.layers.10.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.10.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.10.self_attn.v_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.10.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.007066
-- Layer: model.layers.10 (MLP)
-- Linear: model.layers.10.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.10.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.10.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.011176
-- Layer: model.layers.11 (Attention)
-- Linear: model.layers.11.self_attn.q_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.11.self_attn.k_proj -> 1:4b_32g s4, 4.16 bpw
-- Linear: model.layers.11.self_attn.v_proj -> 1:4b_32g s4, 4.16 bpw
-- Linear: model.layers.11.self_attn.o_proj -> 1:4b_32g s4, 4.13 bpw
-- Module quantized, rfn_error: 0.010023
-- Layer: model.layers.11 (MLP)
-- Linear: model.layers.11.mlp.gate_proj -> 0.1:4b_32g/0.9:3b_32g s4, 3.23 bpw
-- Linear: model.layers.11.mlp.up_proj -> 0.25:4b_32g/0.75:3b_32g s4, 3.38 bpw
-- Linear: model.layers.11.mlp.down_proj -> 0.05:8b_32g/0.1:4b_32g/0.85:3b_32g s4, 3.49 bpw
-- Module quantized, rfn_error: 0.022409
-- Layer: model.layers.12 (Attention)
-- Linear: model.layers.12.self_attn.q_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.12.self_attn.k_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.19 bpw
-- Linear: model.layers.12.self_attn.v_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.12.self_attn.o_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Module quantized, rfn_error: 0.010207
-- Layer: model.layers.12 (MLP)
-- Linear: model.layers.12.mlp.gate_proj -> 1:4b_128g s4, 4.03 bpw
-- Linear: model.layers.12.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.12.mlp.down_proj -> 0.05:8b_32g/0.95:4b_128g s4, 4.25 bpw
-- Module quantized, rfn_error: 0.014418
-- Layer: model.layers.13 (Attention)
-- Linear: model.layers.13.self_attn.q_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.23 bpw
-- Linear: model.layers.13.self_attn.k_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.26 bpw
-- Linear: model.layers.13.self_attn.v_proj -> 1:6b_32g s4, 6.16 bpw
-- Linear: model.layers.13.self_attn.o_proj -> 0.1:6b_32g/0.9:5b_32g s4, 5.23 bpw
-- Module quantized, rfn_error: 0.004147
-- Layer: model.layers.13 (MLP)
-- Linear: model.layers.13.mlp.gate_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.13.mlp.up_proj -> 0.25:5b_128g/0.75:4b_128g s4, 4.28 bpw
-- Linear: model.layers.13.mlp.down_proj -> 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4, 4.36 bpw
-- Module quantized, rfn_error: 0.015048
-- Saving checkpoint...
-- Layer: model.layers.14 (Attention)
-- Linear: model.layers.14.self_attn.q_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.14.self_attn.k_proj -> 1:4b_32g s4, 4.16 bpw
-- Linear: model.layers.14.self_attn.v_proj -> 1:4b_32g s4, 4.16 bpw
-- Linear: model.layers.14.self_attn.o_proj -> 1:4b_32g s4, 4.13 bpw
-- Module quantized, rfn_error: 0.012923
-- Layer: model.layers.14 (MLP)
-- Linear: model.layers.14.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.14.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.14.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.015563
-- Layer: model.layers.15 (Attention)
-- Linear: model.layers.15.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.15.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.15.self_attn.v_proj -> 1:5b_64g s4, 5.09 bpw
-- Linear: model.layers.15.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.011713
-- Layer: model.layers.15 (MLP)
-- Linear: model.layers.15.mlp.gate_proj -> 1:4b_128g s4, 4.03 bpw
-- Linear: model.layers.15.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.15.mlp.down_proj -> 0.05:8b_32g/0.95:4b_128g s4, 4.25 bpw
-- Module quantized, rfn_error: 0.018786
-- Layer: model.layers.16 (Attention)
-- Linear: model.layers.16.self_attn.q_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.16.self_attn.k_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.16.self_attn.v_proj -> 1:5b_32g s4, 5.16 bpw
-- Linear: model.layers.16.self_attn.o_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Module quantized, rfn_error: 0.010153
-- Layer: model.layers.16 (MLP)
-- Linear: model.layers.16.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.16.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.16.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.018980
-- Layer: model.layers.17 (Attention)
-- Linear: model.layers.17.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.17.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.17.self_attn.v_proj -> 1:5b_64g s4, 5.09 bpw
-- Linear: model.layers.17.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.011752
-- Layer: model.layers.17 (MLP)
-- Linear: model.layers.17.mlp.gate_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.17.mlp.up_proj -> 0.25:5b_128g/0.75:4b_128g s4, 4.28 bpw
-- Linear: model.layers.17.mlp.down_proj -> 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4, 4.36 bpw
-- Module quantized, rfn_error: 0.021872
-- Layer: model.layers.18 (Attention)
-- Linear: model.layers.18.self_attn.q_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.18.self_attn.k_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.18.self_attn.v_proj -> 1:5b_32g s4, 5.16 bpw
-- Linear: model.layers.18.self_attn.o_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Module quantized, rfn_error: 0.010848
-- Layer: model.layers.18 (MLP)
-- Linear: model.layers.18.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.18.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.18.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.022907
-- Layer: model.layers.19 (Attention)
-- Linear: model.layers.19.self_attn.q_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.19.self_attn.k_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.19 bpw
-- Linear: model.layers.19.self_attn.v_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.19.self_attn.o_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Module quantized, rfn_error: 0.013906
-- Layer: model.layers.19 (MLP)
-- Linear: model.layers.19.mlp.gate_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.19.mlp.up_proj -> 0.25:5b_32g/0.75:4b_32g s4, 4.38 bpw
-- Linear: model.layers.19.mlp.down_proj -> 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4, 4.44 bpw
-- Module quantized, rfn_error: 0.021642
-- Layer: model.layers.20 (Attention)
-- Linear: model.layers.20.self_attn.q_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.20.self_attn.k_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.20.self_attn.v_proj -> 1:5b_32g s4, 5.16 bpw
-- Linear: model.layers.20.self_attn.o_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Module quantized, rfn_error: 0.009971
-- Layer: model.layers.20 (MLP)
-- Linear: model.layers.20.mlp.gate_proj -> 1:4b_128g s4, 4.03 bpw
-- Linear: model.layers.20.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.20.mlp.down_proj -> 0.05:8b_32g/0.95:4b_128g s4, 4.25 bpw
-- Module quantized, rfn_error: 0.026400
-- Saving checkpoint...
-- Layer: model.layers.21 (Attention)
-- Linear: model.layers.21.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.21.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.21.self_attn.v_proj -> 1:5b_64g s4, 5.09 bpw
-- Linear: model.layers.21.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.009716
-- Layer: model.layers.21 (MLP)
-- Linear: model.layers.21.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.21.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.21.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.023654
-- Layer: model.layers.22 (Attention)
-- Linear: model.layers.22.self_attn.q_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Linear: model.layers.22.self_attn.k_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.19 bpw
-- Linear: model.layers.22.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.22.self_attn.o_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Module quantized, rfn_error: 0.005309
-- Layer: model.layers.22 (MLP)
-- Linear: model.layers.22.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.22.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.22.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.023554
-- Layer: model.layers.23 (Attention)
-- Linear: model.layers.23.self_attn.q_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.23.self_attn.k_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.23.self_attn.v_proj -> 1:5b_32g s4, 5.16 bpw
-- Linear: model.layers.23.self_attn.o_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Module quantized, rfn_error: 0.008197
-- Layer: model.layers.23 (MLP)
-- Linear: model.layers.23.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.23.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.23.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.023779
-- Layer: model.layers.24 (Attention)
-- Linear: model.layers.24.self_attn.q_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Linear: model.layers.24.self_attn.k_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.19 bpw
-- Linear: model.layers.24.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.24.self_attn.o_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Module quantized, rfn_error: 0.004919
-- Layer: model.layers.24 (MLP)
-- Linear: model.layers.24.mlp.gate_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.24.mlp.up_proj -> 0.25:5b_32g/0.75:4b_32g s4, 4.38 bpw
-- Linear: model.layers.24.mlp.down_proj -> 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4, 4.44 bpw
-- Module quantized, rfn_error: 0.022033
-- Layer: model.layers.25 (Attention)
-- Linear: model.layers.25.self_attn.q_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Linear: model.layers.25.self_attn.k_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.25.self_attn.v_proj -> 1:5b_64g s4, 5.09 bpw
-- Linear: model.layers.25.self_attn.o_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.18 bpw
-- Module quantized, rfn_error: 0.008494
-- Layer: model.layers.25 (MLP)
-- Linear: model.layers.25.mlp.gate_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.25.mlp.up_proj -> 0.25:5b_32g/0.75:4b_32g s4, 4.38 bpw
-- Linear: model.layers.25.mlp.down_proj -> 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4, 4.44 bpw
-- Module quantized, rfn_error: 0.022276
-- Layer: model.layers.26 (Attention)
-- Linear: model.layers.26.self_attn.q_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.26.self_attn.k_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.19 bpw
-- Linear: model.layers.26.self_attn.v_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.26.self_attn.o_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Module quantized, rfn_error: 0.009997
-- Layer: model.layers.26 (MLP)
-- Linear: model.layers.26.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.26.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.26.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.024846
-- Layer: model.layers.27 (Attention)
-- Linear: model.layers.27.self_attn.q_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.27.self_attn.k_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.19 bpw
-- Linear: model.layers.27.self_attn.v_proj -> 0.1:5b_64g/0.9:4b_64g s4, 4.20 bpw
-- Linear: model.layers.27.self_attn.o_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Module quantized, rfn_error: 0.010132
-- Layer: model.layers.27 (MLP)
-- Linear: model.layers.27.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.27.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.27.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.025698
-- Saving checkpoint...
-- Layer: model.layers.28 (Attention)
-- Linear: model.layers.28.self_attn.q_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.28.self_attn.k_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.26 bpw
-- Linear: model.layers.28.self_attn.v_proj -> 1:5b_32g s4, 5.16 bpw
-- Linear: model.layers.28.self_attn.o_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Module quantized, rfn_error: 0.009447
-- Layer: model.layers.28 (MLP)
-- Linear: model.layers.28.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.28.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.28.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.027911
-- Layer: model.layers.29 (Attention)
-- Linear: model.layers.29.self_attn.q_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Linear: model.layers.29.self_attn.k_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.19 bpw
-- Linear: model.layers.29.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.29.self_attn.o_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Module quantized, rfn_error: 0.007283
-- Layer: model.layers.29 (MLP)
-- Linear: model.layers.29.mlp.gate_proj -> 0.1:5b_32g/0.9:4b_32g s4, 4.23 bpw
-- Linear: model.layers.29.mlp.up_proj -> 0.25:5b_32g/0.75:4b_32g s4, 4.38 bpw
-- Linear: model.layers.29.mlp.down_proj -> 0.05:8b_32g/0.1:5b_32g/0.85:4b_32g s4, 4.44 bpw
-- Module quantized, rfn_error: 0.026197
-- Layer: model.layers.30 (Attention)
-- Linear: model.layers.30.self_attn.q_proj -> 1:6b_128g s4, 6.04 bpw
-- Linear: model.layers.30.self_attn.k_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.30.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.30.self_attn.o_proj -> 1:6b_128g s4, 6.04 bpw
-- Module quantized, rfn_error: 0.005726
-- Layer: model.layers.30 (MLP)
-- Linear: model.layers.30.mlp.gate_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.30.mlp.up_proj -> 1:4b_32g s4, 4.13 bpw
-- Linear: model.layers.30.mlp.down_proj -> 0.05:8b_32g/0.95:4b_32g s4, 4.34 bpw
-- Module quantized, rfn_error: 0.031432
-- Layer: model.layers.31 (Attention)
-- Linear: model.layers.31.self_attn.q_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Linear: model.layers.31.self_attn.k_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.19 bpw
-- Linear: model.layers.31.self_attn.v_proj -> 1:6b_128g s4, 6.06 bpw
-- Linear: model.layers.31.self_attn.o_proj -> 0.1:6b_128g/0.9:5b_128g s4, 5.16 bpw
-- Module quantized, rfn_error: 0.007069
-- Layer: model.layers.31 (MLP)
-- Linear: model.layers.31.mlp.gate_proj -> 0.1:5b_128g/0.9:4b_128g s4, 4.16 bpw
-- Linear: model.layers.31.mlp.up_proj -> 0.25:5b_128g/0.75:4b_128g s4, 4.28 bpw
-- Linear: model.layers.31.mlp.down_proj -> 0.05:8b_32g/0.1:5b_128g/0.85:4b_128g s4, 4.36 bpw
-- Module quantized, rfn_error: 0.032691
-- Layer: model.norm (RMSNorm)
-- Module quantized, rfn_error: 0.000000
-- Layer: lm_head (Linear)
-- Linear: lm_head -> 0.15:8b_128g/0.85:6b_128g s4, 6.34 bpw
-- Module quantized, calibration perplexity (quant): 8.1155
-- Saving checkpoint...
-- Compiling output file...
-- Writing shard 1...
-- Creating directory Mistral-7B-Instruct-v0.3-exl2/4.0bpw/
Model quantized and saved! You can test it with the following:
[13]
-- Model: Mistral-7B-Instruct-v0.3-exl2/4.0bpw -- Options: [] -- Loading model... -- Loaded model in 2.0031 seconds -- Loading tokenizer... -- Warmup... -- Generating... Once upon a time, in a land far, far away, children had to work hard to get their three meals a day. The chores were backbreaking and tedious. However, they were grateful for their food because they knew how difficult it was to produce, process, and prepare it. Fast forward to the present time: a century of technological advancement has made food readily available at our fingertips. We no longer have to toil in the fields to gather food, or spend hours in the kitchen to prepare it. The ease with which we can now access delicious food has led to the proliferation of fast food chains -- Response generated in 2.27 seconds, 128 tokens, 56.51 tokens/second (includes prompt eval.)