AIPC OpenVino Demo
code03.InferenceAIPCmicrosoft-phi-cookbook
Export
[1]
[2]
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino Mixed-Precision assignment --------------------- 0% 0/128 • 0:00:00 • -:--:-- Mixed-Precision assignment --------------------- 0% 0/128 • 0:00:00 • -:--:-- Mixed-Precision assignment --------------------- 0% 0/128 • 0:00:00 • -:--:-- Mixed-Precision assignment --------------------- 0% 0/128 • 0:00:00 • -:--:-- Mixed-Precision assignment --------------------- 0% 0/128 • 0:00:00 • -:--:-- Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:00 • 0:00:14 Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:00 • 0:00:14 Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:01 • 0:00:14 Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:01 • 0:00:14 Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:01 • 0:00:14 Mixed-Precision assignment --------------------- 2% 2/128 • 0:00:01 • 0:00:14 Mixed-Precision assignment --------------------- 2% 3/128 • 0:00:01 • 0:00:55 Mixed-Precision assignment --------------------- 2% 3/128 • 0:00:01 • 0:00:55 Mixed-Precision assignment --------------------- 2% 3/128 • 0:00:01 • 0:00:55 Mixed-Precision assignment -------------------- 3% 4/128 • 0:00:01 • 0:00:50 Mixed-Precision assignment -------------------- 3% 4/128 • 0:00:01 • 0:00:50 Mixed-Precision assignment -------------------- 3% 4/128 • 0:00:02 • 0:00:50 Mixed-Precision assignment -------------------- 3% 4/128 • 0:00:02 • 0:00:50 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment -------------------- 5% 6/128 • 0:00:02 • 0:00:43 Mixed-Precision assignment - ------------------- 5% 7/128 • 0:00:03 • 0:00:49 Mixed-Precision assignment - ------------------- 5% 7/128 • 0:00:03 • 0:00:49 Mixed-Precision assignment - ------------------- 5% 7/128 • 0:00:03 • 0:00:49 Mixed-Precision assignment - ------------------- 6% 8/128 • 0:00:03 • 0:00:47 Mixed-Precision assignment - ------------------- 6% 8/128 • 0:00:03 • 0:00:47 Mixed-Precision assignment - ------------------- 6% 8/128 • 0:00:03 • 0:00:47 Mixed-Precision assignment - ------------------- 6% 8/128 • 0:00:03 • 0:00:47 Mixed-Precision assignment - ------------------- 7% 9/128 • 0:00:03 • 0:00:47 Mixed-Precision assignment - ------------------ 8% 10/128 • 0:00:03 • 0:00:43 Mixed-Precision assignment - ------------------ 8% 10/128 • 0:00:04 • 0:00:43 Mixed-Precision assignment - ------------------ 8% 10/128 • 0:00:04 • 0:00:43 Mixed-Precision assignment - ------------------ 8% 10/128 • 0:00:04 • 0:00:43 Mixed-Precision assignment - ------------------ 9% 11/128 • 0:00:04 • 0:00:45 Mixed-Precision assignment - ------------------ 9% 11/128 • 0:00:04 • 0:00:45 Mixed-Precision assignment - ------------------ 9% 11/128 • 0:00:04 • 0:00:45 Mixed-Precision assignment - ------------------ 9% 12/128 • 0:00:04 • 0:00:44 Mixed-Precision assignment - ------------------ 9% 12/128 • 0:00:04 • 0:00:44 Mixed-Precision assignment - ------------------ 9% 12/128 • 0:00:04 • 0:00:44 Mixed-Precision assignment - ------------------ 9% 12/128 • 0:00:05 • 0:00:44 Mixed-Precision assignment -- ----------------- 10% 13/128 • 0:00:05 • 0:00:44 Mixed-Precision assignment -- ----------------- 11% 14/128 • 0:00:05 • 0:00:41 Mixed-Precision assignment -- ----------------- 11% 14/128 • 0:00:05 • 0:00:41 Mixed-Precision assignment -- ----------------- 11% 14/128 • 0:00:05 • 0:00:41 Mixed-Precision assignment -- ----------------- 11% 14/128 • 0:00:05 • 0:00:41 Mixed-Precision assignment -- ----------------- 11% 14/128 • 0:00:05 • 0:00:41 Mixed-Precision assignment -- ----------------- 12% 15/128 • 0:00:05 • 0:00:43 Mixed-Precision assignment -- ----------------- 12% 15/128 • 0:00:05 • 0:00:43 Mixed-Precision assignment -- ----------------- 12% 16/128 • 0:00:06 • 0:00:41 Mixed-Precision assignment -- ----------------- 12% 16/128 • 0:00:06 • 0:00:41 Mixed-Precision assignment -- ----------------- 12% 16/128 • 0:00:06 • 0:00:41 Mixed-Precision assignment -- ----------------- 13% 17/128 • 0:00:06 • 0:00:41 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:06 • 0:00:39 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:06 • 0:00:39 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:06 • 0:00:39 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:06 • 0:00:39 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:07 • 0:00:39 Mixed-Precision assignment -- ----------------- 14% 18/128 • 0:00:07 • 0:00:39 Mixed-Precision assignment -- ----------------- 15% 19/128 • 0:00:07 • 0:00:41 Mixed-Precision assignment -- ----------------- 15% 19/128 • 0:00:07 • 0:00:41 Mixed-Precision assignment -- ----------------- 15% 19/128 • 0:00:07 • 0:00:41 Mixed-Precision assignment --- ---------------- 16% 20/128 • 0:00:07 • 0:00:40 Mixed-Precision assignment --- ---------------- 16% 20/128 • 0:00:07 • 0:00:40 Mixed-Precision assignment --- ---------------- 16% 21/128 • 0:00:07 • 0:00:39 Mixed-Precision assignment --- ---------------- 17% 22/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 17% 22/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 17% 22/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 17% 22/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 17% 22/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 18% 23/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 18% 23/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 18% 23/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 19% 24/128 • 0:00:08 • 0:00:38 Mixed-Precision assignment --- ---------------- 19% 24/128 • 0:00:09 • 0:00:38 Mixed-Precision assignment --- ---------------- 20% 25/128 • 0:00:09 • 0:00:37 Mixed-Precision assignment ---- --------------- 20% 26/128 • 0:00:09 • 0:00:36 Mixed-Precision assignment ---- --------------- 20% 26/128 • 0:00:09 • 0:00:36 Mixed-Precision assignment ---- --------------- 20% 26/128 • 0:00:09 • 0:00:36 Mixed-Precision assignment ---- --------------- 20% 26/128 • 0:00:09 • 0:00:36 Mixed-Precision assignment ---- --------------- 20% 26/128 • 0:00:09 • 0:00:36 Mixed-Precision assignment ---- --------------- 21% 27/128 • 0:00:10 • 0:00:37 Mixed-Precision assignment ---- --------------- 21% 27/128 • 0:00:10 • 0:00:37 Mixed-Precision assignment ---- --------------- 21% 27/128 • 0:00:10 • 0:00:37 Mixed-Precision assignment ---- --------------- 22% 28/128 • 0:00:10 • 0:00:36 Mixed-Precision assignment ---- --------------- 22% 28/128 • 0:00:10 • 0:00:36 Mixed-Precision assignment ---- --------------- 23% 29/128 • 0:00:10 • 0:00:36 Mixed-Precision assignment ---- --------------- 23% 30/128 • 0:00:10 • 0:00:35 Mixed-Precision assignment ---- --------------- 23% 30/128 • 0:00:10 • 0:00:35 Mixed-Precision assignment ---- --------------- 23% 30/128 • 0:00:10 • 0:00:35 Mixed-Precision assignment ---- --------------- 23% 30/128 • 0:00:11 • 0:00:35 Mixed-Precision assignment ---- --------------- 23% 30/128 • 0:00:11 • 0:00:35 Mixed-Precision assignment ---- --------------- 24% 31/128 • 0:00:11 • 0:00:35 Mixed-Precision assignment ---- --------------- 24% 31/128 • 0:00:11 • 0:00:35 Mixed-Precision assignment ---- --------------- 24% 31/128 • 0:00:11 • 0:00:35 Mixed-Precision assignment ----- -------------- 25% 32/128 • 0:00:11 • 0:00:34 Mixed-Precision assignment ----- -------------- 25% 32/128 • 0:00:11 • 0:00:34 Mixed-Precision assignment ----- -------------- 25% 32/128 • 0:00:11 • 0:00:34 Mixed-Precision assignment ----- -------------- 26% 33/128 • 0:00:11 • 0:00:34 Mixed-Precision assignment ----- -------------- 27% 34/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 34/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 34/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 34/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 34/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 35/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 27% 35/128 • 0:00:12 • 0:00:33 Mixed-Precision assignment ----- -------------- 28% 36/128 • 0:00:12 • 0:00:32 Mixed-Precision assignment ----- -------------- 28% 36/128 • 0:00:12 • 0:00:32 Mixed-Precision assignment ----- -------------- 28% 36/128 • 0:00:12 • 0:00:32 Mixed-Precision assignment ----- -------------- 29% 37/128 • 0:00:13 • 0:00:32 Mixed-Precision assignment ----- -------------- 30% 38/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ----- -------------- 30% 38/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ----- -------------- 30% 38/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ----- -------------- 30% 38/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ----- -------------- 30% 38/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ------ ------------- 30% 39/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ------ ------------- 30% 39/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ------ ------------- 30% 39/128 • 0:00:13 • 0:00:31 Mixed-Precision assignment ------ ------------- 31% 40/128 • 0:00:14 • 0:00:31 Mixed-Precision assignment ------ ------------- 32% 41/128 • 0:00:14 • 0:00:30 Mixed-Precision assignment ------ ------------- 33% 42/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 33% 42/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 33% 42/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 33% 42/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 33% 42/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 43/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 43/128 • 0:00:14 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 43/128 • 0:00:15 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 44/128 • 0:00:15 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 44/128 • 0:00:15 • 0:00:29 Mixed-Precision assignment ------ ------------- 34% 44/128 • 0:00:15 • 0:00:29 Mixed-Precision assignment ------- ------------ 35% 45/128 • 0:00:15 • 0:00:29 Mixed-Precision assignment ------- ------------ 36% 46/128 • 0:00:15 • 0:00:28 Mixed-Precision assignment ------- ------------ 36% 46/128 • 0:00:15 • 0:00:28 Mixed-Precision assignment ------- ------------ 36% 46/128 • 0:00:15 • 0:00:28 Mixed-Precision assignment ------- ------------ 36% 46/128 • 0:00:15 • 0:00:28 Mixed-Precision assignment ------- ------------ 36% 46/128 • 0:00:16 • 0:00:28 Mixed-Precision assignment ------- ------------ 37% 47/128 • 0:00:16 • 0:00:28 Mixed-Precision assignment ------- ------------ 37% 47/128 • 0:00:16 • 0:00:28 Mixed-Precision assignment ------- ------------ 38% 48/128 • 0:00:16 • 0:00:27 Mixed-Precision assignment ------- ------------ 38% 48/128 • 0:00:16 • 0:00:27 Mixed-Precision assignment ------- ------------ 38% 49/128 • 0:00:16 • 0:00:27 Mixed-Precision assignment ------- ------------ 38% 49/128 • 0:00:16 • 0:00:27 Mixed-Precision assignment ------- ------------ 39% 50/128 • 0:00:16 • 0:00:26 Mixed-Precision assignment ------- ------------ 39% 50/128 • 0:00:16 • 0:00:26 Mixed-Precision assignment ------- ------------ 39% 50/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment ------- ------------ 39% 50/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment ------- ------------ 40% 51/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment ------- ------------ 40% 51/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment -------- ----------- 41% 52/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment -------- ----------- 41% 52/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment -------- ----------- 41% 52/128 • 0:00:17 • 0:00:26 Mixed-Precision assignment -------- ----------- 41% 53/128 • 0:00:17 • 0:00:25 Mixed-Precision assignment -------- ----------- 42% 54/128 • 0:00:17 • 0:00:25 Mixed-Precision assignment -------- ----------- 42% 54/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 42% 54/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 42% 54/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 42% 54/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 43% 55/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 43% 55/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 43% 55/128 • 0:00:18 • 0:00:25 Mixed-Precision assignment -------- ----------- 44% 56/128 • 0:00:18 • 0:00:24 Mixed-Precision assignment -------- ----------- 44% 56/128 • 0:00:18 • 0:00:24 Mixed-Precision assignment -------- ----------- 45% 57/128 • 0:00:19 • 0:00:24 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 45% 58/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 46% 59/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 47% 60/128 • 0:00:19 • 0:00:23 Mixed-Precision assignment --------- ---------- 47% 60/128 • 0:00:20 • 0:00:23 Mixed-Precision assignment --------- ---------- 47% 60/128 • 0:00:20 • 0:00:23 Mixed-Precision assignment --------- ---------- 48% 61/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 48% 62/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 48% 62/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 48% 62/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 48% 62/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 48% 62/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 49% 63/128 • 0:00:20 • 0:00:22 Mixed-Precision assignment --------- ---------- 49% 63/128 • 0:00:21 • 0:00:22 Mixed-Precision assignment ---------- --------- 50% 64/128 • 0:00:21 • 0:00:21 Mixed-Precision assignment ---------- --------- 50% 64/128 • 0:00:21 • 0:00:21 Mixed-Precision assignment ---------- --------- 50% 64/128 • 0:00:21 • 0:00:21 Mixed-Precision assignment ---------- --------- 51% 65/128 • 0:00:21 • 0:00:21 Mixed-Precision assignment ---------- --------- 52% 66/128 • 0:00:21 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 66/128 • 0:00:21 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 66/128 • 0:00:21 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 66/128 • 0:00:21 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 66/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 67/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 67/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 52% 67/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 53% 68/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 53% 68/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 54% 69/128 • 0:00:22 • 0:00:20 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:22 • 0:00:19 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:22 • 0:00:19 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:22 • 0:00:19 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ---------- --------- 55% 70/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ----------- -------- 55% 71/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ----------- -------- 55% 71/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ----------- -------- 56% 72/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ----------- -------- 56% 72/128 • 0:00:23 • 0:00:19 Mixed-Precision assignment ----------- -------- 57% 73/128 • 0:00:23 • 0:00:18 Mixed-Precision assignment ----------- -------- 58% 74/128 • 0:00:23 • 0:00:18 Mixed-Precision assignment ----------- -------- 58% 74/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 58% 74/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 58% 74/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 58% 74/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 59% 75/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 59% 75/128 • 0:00:24 • 0:00:18 Mixed-Precision assignment ----------- -------- 59% 76/128 • 0:00:24 • 0:00:17 Mixed-Precision assignment ----------- -------- 59% 76/128 • 0:00:24 • 0:00:17 Mixed-Precision assignment ----------- -------- 59% 76/128 • 0:00:24 • 0:00:17 Mixed-Precision assignment ------------ ------- 60% 77/128 • 0:00:25 • 0:00:17 Mixed-Precision assignment ------------ ------- 61% 78/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 61% 78/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 61% 78/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 61% 78/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 61% 78/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 79/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 79/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 79/128 • 0:00:25 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 80/128 • 0:00:26 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 80/128 • 0:00:26 • 0:00:16 Mixed-Precision assignment ------------ ------- 62% 80/128 • 0:00:26 • 0:00:16 Mixed-Precision assignment ------------ ------- 64% 82/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 64% 82/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 64% 82/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 64% 82/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 64% 82/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 65% 83/128 • 0:00:26 • 0:00:15 Mixed-Precision assignment ------------ ------- 65% 83/128 • 0:00:27 • 0:00:15 Mixed-Precision assignment ------------- ------ 66% 84/128 • 0:00:27 • 0:00:15 Mixed-Precision assignment ------------- ------ 66% 84/128 • 0:00:27 • 0:00:15 Mixed-Precision assignment ------------- ------ 66% 84/128 • 0:00:27 • 0:00:15 Mixed-Precision assignment ------------- ------ 66% 85/128 • 0:00:27 • 0:00:14 Mixed-Precision assignment ------------- ------ 67% 86/128 • 0:00:27 • 0:00:14 Mixed-Precision assignment ------------- ------ 67% 86/128 • 0:00:27 • 0:00:14 Mixed-Precision assignment ------------- ------ 67% 86/128 • 0:00:27 • 0:00:14 Mixed-Precision assignment ------------- ------ 67% 86/128 • 0:00:27 • 0:00:14 Mixed-Precision assignment ------------- ------ 67% 86/128 • 0:00:28 • 0:00:14 Mixed-Precision assignment ------------- ------ 68% 87/128 • 0:00:28 • 0:00:14 Mixed-Precision assignment ------------- ------ 68% 87/128 • 0:00:28 • 0:00:14 Mixed-Precision assignment ------------- ------ 68% 87/128 • 0:00:28 • 0:00:14 Mixed-Precision assignment ------------- ------ 69% 88/128 • 0:00:28 • 0:00:13 Mixed-Precision assignment ------------- ------ 69% 88/128 • 0:00:28 • 0:00:13 Mixed-Precision assignment ------------- ------ 69% 88/128 • 0:00:28 • 0:00:13 Mixed-Precision assignment -------------- ----- 70% 90/128 • 0:00:28 • 0:00:13 Mixed-Precision assignment -------------- ----- 70% 90/128 • 0:00:28 • 0:00:13 Mixed-Precision assignment -------------- ----- 70% 90/128 • 0:00:29 • 0:00:13 Mixed-Precision assignment -------------- ----- 70% 90/128 • 0:00:29 • 0:00:13 Mixed-Precision assignment -------------- ----- 70% 90/128 • 0:00:29 • 0:00:13 Mixed-Precision assignment -------------- ----- 71% 91/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 71% 91/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 71% 91/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 72% 92/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 72% 92/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 73% 93/128 • 0:00:29 • 0:00:12 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 73% 94/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 74% 95/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment -------------- ----- 74% 95/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment --------------- ---- 75% 96/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment --------------- ---- 75% 96/128 • 0:00:30 • 0:00:11 Mixed-Precision assignment --------------- ---- 75% 96/128 • 0:00:31 • 0:00:11 Mixed-Precision assignment --------------- ---- 75% 96/128 • 0:00:31 • 0:00:11 Mixed-Precision assignment --------------- ---- 76% 97/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 98/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 98/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 98/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 98/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 99/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 99/128 • 0:00:31 • 0:00:10 Mixed-Precision assignment --------------- ---- 77% 99/128 • 0:00:32 • 0:00:10 Mixed-Precision assignment -------------- ---- 78% 100/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment -------------- ---- 78% 100/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment -------------- ---- 79% 101/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment -------------- ---- 79% 101/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 102/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 102/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 102/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 102/128 • 0:00:32 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 102/128 • 0:00:33 • 0:00:09 Mixed-Precision assignment --------------- --- 80% 103/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 80% 103/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 81% 104/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 81% 104/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 81% 104/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 82% 105/128 • 0:00:33 • 0:00:08 Mixed-Precision assignment --------------- --- 83% 106/128 • 0:00:33 • 0:00:07 Mixed-Precision assignment --------------- --- 83% 106/128 • 0:00:33 • 0:00:07 Mixed-Precision assignment --------------- --- 83% 106/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment --------------- --- 83% 106/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment --------------- --- 83% 106/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment --------------- --- 84% 107/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment --------------- --- 84% 107/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment --------------- --- 84% 107/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment ---------------- -- 84% 108/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment ---------------- -- 84% 108/128 • 0:00:34 • 0:00:07 Mixed-Precision assignment ---------------- -- 85% 109/128 • 0:00:34 • 0:00:06 Mixed-Precision assignment ---------------- -- 86% 110/128 • 0:00:34 • 0:00:06 Mixed-Precision assignment ---------------- -- 86% 110/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 86% 110/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 86% 110/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 86% 110/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 87% 111/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 87% 111/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 87% 111/128 • 0:00:35 • 0:00:06 Mixed-Precision assignment ---------------- -- 88% 112/128 • 0:00:35 • 0:00:05 Mixed-Precision assignment ---------------- -- 88% 112/128 • 0:00:35 • 0:00:05 Mixed-Precision assignment ---------------- -- 88% 113/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ---------------- -- 89% 114/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ---------------- -- 89% 114/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ---------------- -- 89% 114/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ---------------- -- 89% 114/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ---------------- -- 89% 114/128 • 0:00:36 • 0:00:05 Mixed-Precision assignment ----------------- - 90% 115/128 • 0:00:36 • 0:00:04 Mixed-Precision assignment ----------------- - 90% 115/128 • 0:00:36 • 0:00:04 Mixed-Precision assignment ----------------- - 91% 116/128 • 0:00:36 • 0:00:04 Mixed-Precision assignment ----------------- - 91% 116/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 91% 116/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 91% 117/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 92% 118/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 92% 118/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 92% 118/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 92% 118/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 92% 118/128 • 0:00:37 • 0:00:04 Mixed-Precision assignment ----------------- - 93% 119/128 • 0:00:37 • 0:00:03 Mixed-Precision assignment ----------------- - 93% 119/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ----------------- - 93% 119/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ----------------- - 94% 120/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ----------------- - 94% 120/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ----------------- - 94% 120/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ----------------- - 95% 121/128 • 0:00:38 • 0:00:03 Mixed-Precision assignment ------------------ 95% 122/128 • 0:00:38 • 0:00:02 Mixed-Precision assignment ------------------ 95% 122/128 • 0:00:38 • 0:00:02 Mixed-Precision assignment ------------------ 95% 122/128 • 0:00:38 • 0:00:02 Mixed-Precision assignment ------------------ 95% 122/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 95% 122/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 96% 123/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 96% 123/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 97% 124/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 97% 124/128 • 0:00:39 • 0:00:02 Mixed-Precision assignment ------------------ 98% 125/128 • 0:00:39 • 0:00:01 Mixed-Precision assignment ------------------ 98% 126/128 • 0:00:39 • 0:00:01 Mixed-Precision assignment ------------------ 98% 126/128 • 0:00:39 • 0:00:01 Mixed-Precision assignment ------------------ 98% 126/128 • 0:00:40 • 0:00:01 Mixed-Precision assignment ------------------ 98% 126/128 • 0:00:40 • 0:00:01 Mixed-Precision assignment ------------------ 98% 126/128 • 0:00:40 • 0:00:01 Mixed-Precision assignment ------------------ 99% 127/128 • 0:00:40 • 0:00:01 Mixed-Precision assignment ------------------ 99% 127/128 • 0:00:40 • 0:00:01 Mixed-Precision assignment ------------------- 100% 128/128 • 0:00:40 • 0:00:00 Mixed-Precision assignment ------------------- 100% 128/128 • 0:00:40 • 0:00:00 INFO:nncf:Statistics of the bitwidth distribution: +----------------+-----------------------------+----------------------------------------+ | Num bits (N) | % all parameters (layers) | % ratio-defining parameters (layers) | +================+=============================+========================================+ | 8 | 44% (58 / 130) | 40% (56 / 128) | +----------------+-----------------------------+----------------------------------------+ | 4 | 56% (72 / 130) | 60% (72 / 128) | +----------------+-----------------------------+----------------------------------------+ Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 0% 0/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 1% 1/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 1% 1/130 • 0:00:00 • -:--:-- Applying Weight Compression -------------------- 2% 2/130 • 0:00:01 • 0:01:30 Applying Weight Compression -------------------- 2% 2/130 • 0:00:01 • 0:01:30 Applying Weight Compression -------------------- 2% 3/130 • 0:00:01 • 0:00:52 Applying Weight Compression ------------------- 3% 4/130 • 0:00:01 • 0:00:44 Applying Weight Compression ------------------- 3% 4/130 • 0:00:01 • 0:00:44 Applying Weight Compression ------------------- 3% 4/130 • 0:00:02 • 0:00:44 Applying Weight Compression ------------------- 4% 5/130 • 0:00:02 • 0:00:50 Applying Weight Compression ------------------- 4% 5/130 • 0:00:02 • 0:00:50 Applying Weight Compression ------------------- 4% 5/130 • 0:00:02 • 0:00:50 Applying Weight Compression - ------------------ 5% 7/130 • 0:00:02 • 0:00:45 Applying Weight Compression - ------------------ 5% 7/130 • 0:00:03 • 0:00:45 Applying Weight Compression - ------------------ 5% 7/130 • 0:00:03 • 0:00:45 Applying Weight Compression - ------------------ 6% 8/130 • 0:00:03 • 0:00:56 Applying Weight Compression - ------------------ 6% 8/130 • 0:00:04 • 0:00:56 Applying Weight Compression - ------------------ 7% 9/130 • 0:00:04 • 0:00:57 Applying Weight Compression - ------------------ 7% 9/130 • 0:00:04 • 0:00:57 Applying Weight Compression - ------------------ 7% 9/130 • 0:00:05 • 0:00:57 Applying Weight Compression - ----------------- 8% 11/130 • 0:00:05 • 0:00:53 Applying Weight Compression - ----------------- 8% 11/130 • 0:00:05 • 0:00:53 Applying Weight Compression - ----------------- 8% 11/130 • 0:00:05 • 0:00:53 Applying Weight Compression - ----------------- 9% 12/130 • 0:00:06 • 0:00:59 Applying Weight Compression - ----------------- 9% 12/130 • 0:00:06 • 0:00:59 Applying Weight Compression - ----------------- 10% 13/130 • 0:00:06 • 0:00:58 Applying Weight Compression - ----------------- 10% 13/130 • 0:00:06 • 0:00:58 Applying Weight Compression -- ---------------- 12% 15/130 • 0:00:07 • 0:00:54 Applying Weight Compression -- ---------------- 12% 15/130 • 0:00:07 • 0:00:54 Applying Weight Compression -- ---------------- 12% 15/130 • 0:00:07 • 0:00:54 Applying Weight Compression -- ---------------- 12% 15/130 • 0:00:07 • 0:00:54 Applying Weight Compression -- ---------------- 12% 15/130 • 0:00:08 • 0:00:54 Applying Weight Compression -- ---------------- 12% 16/130 • 0:00:08 • 0:00:58 Applying Weight Compression -- ---------------- 13% 17/130 • 0:00:08 • 0:00:54 Applying Weight Compression -- ---------------- 13% 17/130 • 0:00:08 • 0:00:54 Applying Weight Compression -- ---------------- 13% 17/130 • 0:00:08 • 0:00:54 Applying Weight Compression -- ---------------- 15% 19/130 • 0:00:09 • 0:00:51 Applying Weight Compression -- ---------------- 15% 19/130 • 0:00:09 • 0:00:51 Applying Weight Compression -- ---------------- 15% 19/130 • 0:00:09 • 0:00:51 Applying Weight Compression -- ---------------- 15% 19/130 • 0:00:09 • 0:00:51 Applying Weight Compression -- ---------------- 15% 20/130 • 0:00:10 • 0:00:54 Applying Weight Compression -- ---------------- 15% 20/130 • 0:00:10 • 0:00:54 Applying Weight Compression --- --------------- 16% 21/130 • 0:00:10 • 0:00:54 Applying Weight Compression --- --------------- 18% 23/130 • 0:00:10 • 0:00:51 Applying Weight Compression --- --------------- 18% 23/130 • 0:00:11 • 0:00:51 Applying Weight Compression --- --------------- 18% 23/130 • 0:00:11 • 0:00:51 Applying Weight Compression --- --------------- 18% 24/130 • 0:00:11 • 0:00:50 Applying Weight Compression --- --------------- 18% 24/130 • 0:00:11 • 0:00:50 Applying Weight Compression --- --------------- 19% 25/130 • 0:00:11 • 0:00:49 Applying Weight Compression --- --------------- 19% 25/130 • 0:00:11 • 0:00:49 Applying Weight Compression --- --------------- 19% 25/130 • 0:00:12 • 0:00:49 Applying Weight Compression --- --------------- 21% 27/130 • 0:00:12 • 0:00:47 Applying Weight Compression --- --------------- 21% 27/130 • 0:00:12 • 0:00:47 Applying Weight Compression ---- -------------- 22% 28/130 • 0:00:12 • 0:00:46 Applying Weight Compression ---- -------------- 22% 28/130 • 0:00:12 • 0:00:46 Applying Weight Compression ---- -------------- 22% 28/130 • 0:00:13 • 0:00:46 Applying Weight Compression ---- -------------- 22% 29/130 • 0:00:13 • 0:00:46 Applying Weight Compression ---- -------------- 22% 29/130 • 0:00:13 • 0:00:46 Applying Weight Compression ---- -------------- 24% 31/130 • 0:00:13 • 0:00:44 Applying Weight Compression ---- -------------- 24% 31/130 • 0:00:13 • 0:00:44 Applying Weight Compression ---- -------------- 24% 31/130 • 0:00:14 • 0:00:44 Applying Weight Compression ---- -------------- 24% 31/130 • 0:00:14 • 0:00:44 Applying Weight Compression ---- -------------- 25% 32/130 • 0:00:14 • 0:00:45 Applying Weight Compression ---- -------------- 25% 33/130 • 0:00:15 • 0:00:45 Applying Weight Compression ---- -------------- 25% 33/130 • 0:00:15 • 0:00:45 Applying Weight Compression ----- ------------- 27% 35/130 • 0:00:15 • 0:00:43 Applying Weight Compression ----- ------------- 27% 35/130 • 0:00:16 • 0:00:43 Applying Weight Compression ----- ------------- 27% 35/130 • 0:00:16 • 0:00:43 Applying Weight Compression ----- ------------- 27% 35/130 • 0:00:16 • 0:00:43 Applying Weight Compression ----- ------------- 28% 36/130 • 0:00:16 • 0:00:44 Applying Weight Compression ----- ------------- 28% 36/130 • 0:00:17 • 0:00:44 Applying Weight Compression ----- ------------- 28% 36/130 • 0:00:17 • 0:00:44 Applying Weight Compression ----- ------------- 28% 37/130 • 0:00:17 • 0:00:44 Applying Weight Compression ----- ------------- 28% 37/130 • 0:00:18 • 0:00:44 Applying Weight Compression ----- ------------- 30% 39/130 • 0:00:18 • 0:00:42 Applying Weight Compression ----- ------------- 30% 39/130 • 0:00:18 • 0:00:42 Applying Weight Compression ----- ------------- 31% 40/130 • 0:00:18 • 0:00:41 Applying Weight Compression ----- ------------- 32% 41/130 • 0:00:18 • 0:00:40 Applying Weight Compression ----- ------------- 32% 41/130 • 0:00:18 • 0:00:40 Applying Weight Compression ----- ------------- 32% 41/130 • 0:00:19 • 0:00:40 Applying Weight Compression ------ ------------ 33% 43/130 • 0:00:19 • 0:00:39 Applying Weight Compression ------ ------------ 33% 43/130 • 0:00:19 • 0:00:39 Applying Weight Compression ------ ------------ 34% 44/130 • 0:00:19 • 0:00:38 Applying Weight Compression ------ ------------ 34% 44/130 • 0:00:19 • 0:00:38 Applying Weight Compression ------ ------------ 34% 44/130 • 0:00:19 • 0:00:38 Applying Weight Compression ------ ------------ 35% 45/130 • 0:00:20 • 0:00:38 Applying Weight Compression ------ ------------ 35% 45/130 • 0:00:20 • 0:00:38 Applying Weight Compression ------ ------------ 35% 45/130 • 0:00:20 • 0:00:38 Applying Weight Compression ------ ------------ 36% 47/130 • 0:00:20 • 0:00:36 Applying Weight Compression ------ ------------ 36% 47/130 • 0:00:20 • 0:00:36 Applying Weight Compression ------- ----------- 37% 48/130 • 0:00:20 • 0:00:36 Applying Weight Compression ------- ----------- 37% 48/130 • 0:00:20 • 0:00:36 Applying Weight Compression ------- ----------- 37% 48/130 • 0:00:21 • 0:00:36 Applying Weight Compression ------- ----------- 37% 48/130 • 0:00:21 • 0:00:36 Applying Weight Compression ------- ----------- 38% 49/130 • 0:00:21 • 0:00:35 Applying Weight Compression ------- ----------- 39% 51/130 • 0:00:21 • 0:00:34 Applying Weight Compression ------- ----------- 39% 51/130 • 0:00:22 • 0:00:34 Applying Weight Compression ------- ----------- 39% 51/130 • 0:00:22 • 0:00:34 Applying Weight Compression ------- ----------- 40% 52/130 • 0:00:22 • 0:00:34 Applying Weight Compression ------- ----------- 40% 52/130 • 0:00:22 • 0:00:34 Applying Weight Compression ------- ----------- 40% 52/130 • 0:00:22 • 0:00:34 Applying Weight Compression ------- ----------- 41% 53/130 • 0:00:23 • 0:00:33 Applying Weight Compression ------- ----------- 42% 54/130 • 0:00:23 • 0:00:33 Applying Weight Compression -------- ---------- 42% 55/130 • 0:00:23 • 0:00:32 Applying Weight Compression -------- ---------- 42% 55/130 • 0:00:23 • 0:00:32 Applying Weight Compression -------- ---------- 42% 55/130 • 0:00:23 • 0:00:32 Applying Weight Compression -------- ---------- 42% 55/130 • 0:00:23 • 0:00:32 Applying Weight Compression -------- ---------- 42% 55/130 • 0:00:24 • 0:00:32 Applying Weight Compression -------- ---------- 43% 56/130 • 0:00:24 • 0:00:33 Applying Weight Compression -------- ---------- 43% 56/130 • 0:00:25 • 0:00:33 Applying Weight Compression -------- ---------- 44% 57/130 • 0:00:25 • 0:00:32 Applying Weight Compression -------- ---------- 44% 57/130 • 0:00:25 • 0:00:32 Applying Weight Compression -------- ---------- 45% 59/130 • 0:00:25 • 0:00:31 Applying Weight Compression -------- ---------- 45% 59/130 • 0:00:25 • 0:00:31 Applying Weight Compression -------- ---------- 45% 59/130 • 0:00:26 • 0:00:31 Applying Weight Compression -------- ---------- 46% 60/130 • 0:00:26 • 0:00:31 Applying Weight Compression -------- ---------- 46% 60/130 • 0:00:26 • 0:00:31 Applying Weight Compression -------- ---------- 46% 60/130 • 0:00:27 • 0:00:31 Applying Weight Compression -------- ---------- 47% 61/130 • 0:00:27 • 0:00:31 Applying Weight Compression --------- --------- 48% 63/130 • 0:00:27 • 0:00:29 Applying Weight Compression --------- --------- 48% 63/130 • 0:00:27 • 0:00:29 Applying Weight Compression --------- --------- 48% 63/130 • 0:00:27 • 0:00:29 Applying Weight Compression --------- --------- 49% 64/130 • 0:00:27 • 0:00:29 Applying Weight Compression --------- --------- 50% 65/130 • 0:00:27 • 0:00:28 Applying Weight Compression --------- --------- 50% 65/130 • 0:00:27 • 0:00:28 Applying Weight Compression --------- --------- 52% 67/130 • 0:00:28 • 0:00:27 Applying Weight Compression --------- --------- 52% 67/130 • 0:00:28 • 0:00:27 Applying Weight Compression --------- --------- 52% 67/130 • 0:00:28 • 0:00:27 Applying Weight Compression --------- --------- 52% 68/130 • 0:00:28 • 0:00:26 Applying Weight Compression ---------- -------- 53% 69/130 • 0:00:28 • 0:00:26 Applying Weight Compression ---------- -------- 53% 69/130 • 0:00:28 • 0:00:26 Applying Weight Compression ---------- -------- 53% 69/130 • 0:00:29 • 0:00:26 Applying Weight Compression ---------- -------- 53% 69/130 • 0:00:29 • 0:00:26 Applying Weight Compression ---------- -------- 55% 71/130 • 0:00:29 • 0:00:25 Applying Weight Compression ---------- -------- 55% 71/130 • 0:00:29 • 0:00:25 Applying Weight Compression ---------- -------- 55% 71/130 • 0:00:29 • 0:00:25 Applying Weight Compression ---------- -------- 55% 71/130 • 0:00:29 • 0:00:25 Applying Weight Compression ---------- -------- 55% 71/130 • 0:00:30 • 0:00:25 Applying Weight Compression ---------- -------- 55% 72/130 • 0:00:30 • 0:00:25 Applying Weight Compression ---------- -------- 56% 73/130 • 0:00:30 • 0:00:24 Applying Weight Compression ---------- -------- 56% 73/130 • 0:00:31 • 0:00:24 Applying Weight Compression ---------- -------- 58% 75/130 • 0:00:31 • 0:00:23 Applying Weight Compression ---------- -------- 58% 75/130 • 0:00:31 • 0:00:23 Applying Weight Compression ---------- -------- 58% 75/130 • 0:00:31 • 0:00:23 Applying Weight Compression ----------- ------- 58% 76/130 • 0:00:31 • 0:00:23 Applying Weight Compression ----------- ------- 59% 77/130 • 0:00:31 • 0:00:22 Applying Weight Compression ----------- ------- 59% 77/130 • 0:00:31 • 0:00:22 Applying Weight Compression ----------- ------- 59% 77/130 • 0:00:31 • 0:00:22 Applying Weight Compression ----------- ------- 59% 77/130 • 0:00:32 • 0:00:22 Applying Weight Compression ----------- ------- 61% 79/130 • 0:00:32 • 0:00:21 Applying Weight Compression ----------- ------- 61% 79/130 • 0:00:32 • 0:00:21 Applying Weight Compression ----------- ------- 62% 80/130 • 0:00:32 • 0:00:21 Applying Weight Compression ----------- ------- 62% 80/130 • 0:00:32 • 0:00:21 Applying Weight Compression ----------- ------- 62% 80/130 • 0:00:32 • 0:00:21 Applying Weight Compression ----------- ------- 62% 81/130 • 0:00:33 • 0:00:20 Applying Weight Compression ----------- ------- 62% 81/130 • 0:00:33 • 0:00:20 Applying Weight Compression ------------ ------ 64% 83/130 • 0:00:33 • 0:00:19 Applying Weight Compression ------------ ------ 64% 83/130 • 0:00:33 • 0:00:19 Applying Weight Compression ------------ ------ 64% 83/130 • 0:00:33 • 0:00:19 Applying Weight Compression ------------ ------ 65% 84/130 • 0:00:34 • 0:00:19 Applying Weight Compression ------------ ------ 65% 84/130 • 0:00:34 • 0:00:19 Applying Weight Compression ------------ ------ 65% 85/130 • 0:00:34 • 0:00:18 Applying Weight Compression ------------ ------ 65% 85/130 • 0:00:34 • 0:00:18 Applying Weight Compression ------------ ------ 67% 87/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 67% 87/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 67% 87/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 68% 88/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 68% 88/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 68% 88/130 • 0:00:35 • 0:00:17 Applying Weight Compression ------------ ------ 68% 88/130 • 0:00:36 • 0:00:17 Applying Weight Compression ------------- ----- 68% 89/130 • 0:00:36 • 0:00:16 Applying Weight Compression ------------- ----- 68% 89/130 • 0:00:36 • 0:00:16 Applying Weight Compression ------------- ----- 70% 91/130 • 0:00:36 • 0:00:15 Applying Weight Compression ------------- ----- 70% 91/130 • 0:00:36 • 0:00:15 Applying Weight Compression ------------- ----- 70% 91/130 • 0:00:36 • 0:00:15 Applying Weight Compression ------------- ----- 71% 92/130 • 0:00:36 • 0:00:15 Applying Weight Compression ------------- ----- 71% 92/130 • 0:00:37 • 0:00:15 Applying Weight Compression ------------- ----- 72% 93/130 • 0:00:37 • 0:00:15 Applying Weight Compression ------------- ----- 72% 93/130 • 0:00:37 • 0:00:15 Applying Weight Compression ------------- ----- 72% 93/130 • 0:00:37 • 0:00:15 Applying Weight Compression ------------- ----- 72% 93/130 • 0:00:38 • 0:00:15 Applying Weight Compression ------------- ----- 73% 95/130 • 0:00:38 • 0:00:14 Applying Weight Compression ------------- ----- 73% 95/130 • 0:00:38 • 0:00:14 Applying Weight Compression -------------- ---- 74% 96/130 • 0:00:38 • 0:00:14 Applying Weight Compression -------------- ---- 74% 96/130 • 0:00:38 • 0:00:14 Applying Weight Compression -------------- ---- 74% 96/130 • 0:00:38 • 0:00:14 Applying Weight Compression -------------- ---- 75% 97/130 • 0:00:39 • 0:00:13 Applying Weight Compression -------------- ---- 75% 97/130 • 0:00:39 • 0:00:13 Applying Weight Compression -------------- ---- 76% 99/130 • 0:00:39 • 0:00:12 Applying Weight Compression -------------- ---- 76% 99/130 • 0:00:39 • 0:00:12 Applying Weight Compression -------------- ---- 76% 99/130 • 0:00:39 • 0:00:12 Applying Weight Compression ------------- ---- 78% 101/130 • 0:00:39 • 0:00:11 Applying Weight Compression ------------- ---- 78% 101/130 • 0:00:40 • 0:00:11 Applying Weight Compression ------------- ---- 78% 101/130 • 0:00:40 • 0:00:11 Applying Weight Compression -------------- --- 79% 103/130 • 0:00:40 • 0:00:11 Applying Weight Compression -------------- --- 79% 103/130 • 0:00:40 • 0:00:11 Applying Weight Compression -------------- --- 79% 103/130 • 0:00:40 • 0:00:11 Applying Weight Compression -------------- --- 79% 103/130 • 0:00:40 • 0:00:11 Applying Weight Compression -------------- --- 80% 104/130 • 0:00:41 • 0:00:10 Applying Weight Compression -------------- --- 81% 105/130 • 0:00:41 • 0:00:10 Applying Weight Compression -------------- --- 82% 106/130 • 0:00:41 • 0:00:09 Applying Weight Compression -------------- --- 82% 107/130 • 0:00:41 • 0:00:09 Applying Weight Compression -------------- --- 82% 107/130 • 0:00:41 • 0:00:09 Applying Weight Compression -------------- --- 82% 107/130 • 0:00:41 • 0:00:09 Applying Weight Compression -------------- --- 82% 107/130 • 0:00:42 • 0:00:09 Applying Weight Compression -------------- --- 83% 108/130 • 0:00:42 • 0:00:09 Applying Weight Compression --------------- -- 84% 109/130 • 0:00:42 • 0:00:08 Applying Weight Compression --------------- -- 84% 109/130 • 0:00:42 • 0:00:08 Applying Weight Compression --------------- -- 84% 109/130 • 0:00:43 • 0:00:08 Applying Weight Compression --------------- -- 84% 109/130 • 0:00:43 • 0:00:08 Applying Weight Compression --------------- -- 85% 111/130 • 0:00:43 • 0:00:08 Applying Weight Compression --------------- -- 85% 111/130 • 0:00:43 • 0:00:08 Applying Weight Compression --------------- -- 85% 111/130 • 0:00:43 • 0:00:08 Applying Weight Compression --------------- -- 85% 111/130 • 0:00:44 • 0:00:08 Applying Weight Compression --------------- -- 86% 112/130 • 0:00:44 • 0:00:07 Applying Weight Compression --------------- -- 86% 112/130 • 0:00:45 • 0:00:07 Applying Weight Compression --------------- -- 87% 113/130 • 0:00:45 • 0:00:07 Applying Weight Compression --------------- -- 87% 113/130 • 0:00:45 • 0:00:07 Applying Weight Compression --------------- -- 88% 115/130 • 0:00:45 • 0:00:06 Applying Weight Compression --------------- -- 88% 115/130 • 0:00:45 • 0:00:06 Applying Weight Compression ---------------- - 89% 116/130 • 0:00:45 • 0:00:06 Applying Weight Compression ---------------- - 89% 116/130 • 0:00:46 • 0:00:06 Applying Weight Compression ---------------- - 89% 116/130 • 0:00:46 • 0:00:06 Applying Weight Compression ---------------- - 89% 116/130 • 0:00:46 • 0:00:06 Applying Weight Compression ---------------- - 91% 118/130 • 0:00:46 • 0:00:05 Applying Weight Compression ---------------- - 91% 118/130 • 0:00:46 • 0:00:05 Applying Weight Compression ---------------- - 92% 119/130 • 0:00:46 • 0:00:04 Applying Weight Compression ---------------- - 92% 119/130 • 0:00:46 • 0:00:04 Applying Weight Compression ---------------- - 92% 119/130 • 0:00:47 • 0:00:04 Applying Weight Compression ---------------- - 92% 119/130 • 0:00:47 • 0:00:04 Applying Weight Compression ---------------- - 92% 120/130 • 0:00:47 • 0:00:04 Applying Weight Compression ---------------- - 92% 120/130 • 0:00:48 • 0:00:04 Applying Weight Compression ---------------- - 93% 121/130 • 0:00:48 • 0:00:04 Applying Weight Compression ---------------- - 94% 122/130 • 0:00:48 • 0:00:03 Applying Weight Compression ---------------- - 94% 122/130 • 0:00:48 • 0:00:03 Applying Weight Compression ----------------- 95% 123/130 • 0:00:48 • 0:00:03 Applying Weight Compression ----------------- 95% 123/130 • 0:00:48 • 0:00:03 Applying Weight Compression ----------------- 95% 123/130 • 0:00:49 • 0:00:03 Applying Weight Compression ----------------- 95% 123/130 • 0:00:49 • 0:00:03 Applying Weight Compression ----------------- 95% 124/130 • 0:00:49 • 0:00:03 Applying Weight Compression ----------------- 96% 125/130 • 0:00:50 • 0:00:02 Applying Weight Compression ----------------- 96% 125/130 • 0:00:50 • 0:00:02 Applying Weight Compression ----------------- 96% 125/130 • 0:00:50 • 0:00:02 Applying Weight Compression ----------------- 97% 126/130 • 0:00:50 • 0:00:02 Applying Weight Compression ----------------- 98% 127/130 • 0:00:51 • 0:00:02 Applying Weight Compression ----------------- 98% 127/130 • 0:00:51 • 0:00:02 Applying Weight Compression ----------------- 98% 127/130 • 0:00:51 • 0:00:02 Applying Weight Compression ----------------- 98% 127/130 • 0:00:51 • 0:00:02 Applying Weight Compression ----------------- 98% 128/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ----------------- 99% 129/130 • 0:00:52 • 0:00:01 Applying Weight Compression ------------------ 100% 130/130 • 0:00:52 • 0:00:00 Applying Weight Compression ------------------ 100% 130/130 • 0:00:52 • 0:00:00
C:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( Framework not specified. Using pt to export the model. `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'. Current `flash-attenton` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`. Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.40s/it] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. C:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Using framework PyTorch: 2.3.0+cpu WARNING:root:Cannot apply model.to_bettertransformer because of the exception: The model type phi3 is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported. Currently supported models are: dict_keys(['albert', 'bark', 'bart', 'bert', 'bert-generation', 'blenderbot', 'bloom', 'camembert', 'blip-2', 'clip', 'codegen', 'data2vec-text', 'deit', 'distilbert', 'electra', 'ernie', 'fsmt', 'gpt2', 'gptj', 'gpt_neo', 'gpt_neox', 'hubert', 'layoutlm', 'm2m_100', 'marian', 'markuplm', 'mbart', 'opt', 'pegasus', 'rembert', 'prophetnet', 'roberta', 'roc_bert', 'roformer', 'splinter', 'tapas', 't5', 'vilt', 'vit', 'vit_mae', 'vit_msn', 'wav2vec2', 'xlm-roberta', 'yolos']).. Usage model with stateful=True may be non-effective if model does not contain torch.functional.scaled_dot_product_attention Overriding 1 configuration item(s) - use_cache -> True C:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead warnings.warn( C:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_attn_mask_utils.py:114: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if (input_shape[-1] > 1 or self.sliding_window is not None) and self.is_causal: C:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\optimum\exporters\onnx\model_patcher.py:300: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if past_key_values_length > 0: C:\Users\kinfeylo\.cache\huggingface\modules\transformers_modules\microsoft\Phi-3-mini-4k-instruct\920b6cf52a79ecff578cc33f61922b23cbc88115\modeling_phi3.py:391: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if attn_weights.size() != (bsz, self.num_heads, q_len, kv_seq_len): C:\Users\kinfeylo\.cache\huggingface\modules\transformers_modules\microsoft\Phi-3-mini-4k-instruct\920b6cf52a79ecff578cc33f61922b23cbc88115\modeling_phi3.py:398: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if attention_mask.size() != (bsz, 1, q_len, kv_seq_len): C:\Users\kinfeylo\.cache\huggingface\modules\transformers_modules\microsoft\Phi-3-mini-4k-instruct\920b6cf52a79ecff578cc33f61922b23cbc88115\modeling_phi3.py:410: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if attn_output.size() != (bsz, self.num_heads, q_len, self.head_dim): OpenVINO Tokenizers is not available. To deploy models in production with C++ code, please follow installation instructions: https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#installation Tokenizer won't be converted.
[3]
c:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
[4]
[5]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[6]
[7]
c:\Users\kinfeylo\AppData\Local\Programs\Python\Python310\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( The argument `trust_remote_code` is to be used along with export=True. It will be ignored. Compiling the model to GPU.0 ...
[8]
[12]
[15]
[16]
"You are a helpful AI assistant. can you introduce yourself? Hello! I am Phi, an advanced artificial intelligence developed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other subject, I'm here to help! How may I assist you today? Greetings! I am Phi, an AI developed to assist and provide information on various subjects. My goal is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and improving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Hello! I am Phi, an AI designed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other subject, I'm here to help! How may I assist you today? Hi there! I am Phi, an AI developed to assist and provide information on various subjects. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Greetings! I am Phi, an AI developed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other subject, I'm here to help! How may I assist you today? Hello! I am Phi, an AI developed to assist and provide information on various subjects. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Hi there! I am Phi, an AI developed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other subject, I'm here to help! How may I assist you today? Greetings! I am Phi, an AI developed to assist and provide information on various subjects. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Hello! I am Phi, an AI developed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other subject, I'm here to help! How may I assist you today? Hi there! I am Phi, an AI developed to assist and provide information on various subjects. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Greetings! I am Phi, an AI developed to assist and provide information on various subjects. My purpose is to help users find answers, learn new things, and engage in meaningful conversations. I am constantly learning and evolving to better serve your needs. Whether you have questions about science, history, technology, or any other topic, I'm here to help! How may I assist you today? Hello! I am Phi, an AI developed to assist and provide information on a wide range of topics. My purpose is to help users find answers, learn new things, and engage in"