Exporting Qwen2.5-VL AWQ checkpoints fails with KeyError: 'num_heads'

Describe the bug

Exporting the officially supported Qwen2.5-VL AWQ checkpoints fails during the export step with:

File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/models/qwen2_5_vl/modeling_qwen2_5_vl_visual.py", line 288, in __init__
    self.num_heads: int = config["num_heads"]
                        ~~~~~~^^^^^^^^^^^^^
KeyError: 'num_heads'

For comparison, Qwen 2.5 an Qwen3 AWQ LLM checkpoints export and build successfully.

I reproduced this with all Qwen2.5 AWQ models listed in the TensorRT Edge-LLM 0.9.1 Supported Models documentation:

LLM
- Qwen2.5-0.5B-Instruct-AWQ -> ✅ export + ✅ build
- Qwen2.5-1.5B-Instruct-AWQ -> ✅ export + ✅ build
- Qwen2.5-3B-Instruct-AWQ -> ✅ export + ✅ build (+ ❌ garbage output)
- Qwen2.5-7B-Instruct-AWQ -> ✅ export + ✅ build
- Qwen2.5-14B-Instruct-AWQ -> ✅ export + ✅ build
- Qwen3-4B-AWQ -> ✅ export + ✅ build
- Qwen3-8B-AWQ -> ✅ export + ✅ build
- Qwen3-14B-AWQ -> ✅ export + ✅ build
VLM
- Qwen2.5-VL-3B-Instruct-AWQ -> ❌ export
- Qwen2.5-VL-7B-Instruct-AWQ -> ❌ export

Steps/Code to reproduce bug

Installation method:
As described in TensorRT Edge-LLM on Jetson, I used an NVIDIA Jetson Thor as both the host and target device. I installed TensorRT Edge-LLM v0.9.1 by following the Installation Guide. The installation completed successfully without any issues.

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jul_16_07:31:19_PM_PDT_2025
Cuda compilation tools, release 13.0, V13.0.48
Build cuda_13.0.r13.0/compiler.36260728_0

$ nvidia-smi
Mon Jul 27 08:52:49 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00                 Driver Version: 580.00         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA Thor                    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   36C  N/A               3W /  N/A  | Not Supported          |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A           15507      G   /usr/lib/xorg/Xorg                       92MiB |
|    0   N/A  N/A           15707      G   /usr/bin/gnome-shell                     55MiB |
+-----------------------------------------------------------------------------------------+

$ python3 --version
Python 3.12.3

$ cat /etc/nv_tegra_release
# R38 (release), REVISION: 4.0, GCID: 43443517, BOARD: generic, EABI: aarch64, DATE: Wed Dec 31 00:15:19 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
INSTALL_TYPE=

Export command used:
I use the following command to export models:

tensorrt-edgellm-export path/to/model_name model_name/onnx

For instance:

tensorrt-edgellm-export Qwen/Qwen2.5-VL-3B-Instruct-AWQ Qwen2.5-VL-3B-Instruct-AWQ/onnx

Details

08:07:29  INFO      tensorrt_edgellm.scripts.export: Downloading Qwen/Qwen2.5-VL-3B-Instruct-AWQ from Hugging Face Hub ...
08:07:29  INFO      httpx: HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen2.5-VL-3B-Instruct-AWQ/revision/main "HTTP/1.1 200 OK"

Fetching 14 files:   0%|          | 0/14 [00:00<?, ?it/s]
Fetching 14 files: 100%|██████████| 14/14 [00:00<00:00, 11100.24it/s]
08:07:29  INFO      tensorrt_edgellm.scripts.export: ============================================================
08:07:29  INFO      tensorrt_edgellm.scripts.export: Model type    : qwen2_5_vl
08:07:29  INFO      tensorrt_edgellm.scripts.export: Checkpoint    : /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct-AWQ/snapshots/e7b623934290c5a4da0ee3c6e1e57bfb6b5abbf2
08:07:29  INFO      tensorrt_edgellm.scripts.export: Output dir    : Qwen2.5-VL-3B-Instruct-AWQ/onnx
08:07:29  INFO      tensorrt_edgellm.scripts.export:   thinker        : yes
08:07:29  INFO      tensorrt_edgellm.scripts.export:   mtp_draft      : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   mtp_draft      : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   dflash_draft   : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   talker         : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   code_predictor : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   visual         : yes
08:07:29  INFO      tensorrt_edgellm.scripts.export:   audio          : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   code2wav       : no
08:07:29  INFO      tensorrt_edgellm.scripts.export:   action         : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: FP8 embedding : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: MTP capable   : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: MTP export    : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: Gemma4 MTP    : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: DFlash base   : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: DFlash draft  : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: Reduced vocab : no
08:07:29  INFO      tensorrt_edgellm.scripts.export: External weights: no
08:07:29  INFO      tensorrt_edgellm.scripts.export: TP size       : 1
08:07:29  INFO      tensorrt_edgellm.scripts.export: ============================================================
08:07:29  INFO      tensorrt_edgellm.scripts.export: [LLM] Loading checkpoint from /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct-AWQ/snapshots/e7b623934290c5a4da0ee3c6e1e57bfb6b5abbf2
08:07:33  INFO      tensorrt_edgellm.checkpoint.loader: Loaded 938 tensors, skipped 390 from /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct-AWQ/snapshots/e7b623934290c5a4da0ee3c6e1e57bfb6b5abbf2
08:08:00  INFO      tensorrt_edgellm.checkpoint.repacking: Repacked GPTQ weights
08:08:00  INFO      tensorrt_edgellm.checkpoint.loader: Tied lm_head.weight to embed_tokens.weight
08:08:00  INFO      tensorrt_edgellm.scripts.export: [LLM] Exporting to Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx
08:08:00  INFO      tensorrt_edgellm.onnx.export: Exporting ONNX to Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx (opset 24, dynamo) ...
/usr/lib/python3.12/contextlib.py:144: UserWarning: The tensor attribute self._model.model.last_pre_norm_hidden_states was assigned during export. Such attributes must be registered as buffers using the `register_buffer` API (https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.register_buffer).
  next(self.gen)
/usr/lib/python3.12/copyreg.py:99: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
08:08:15  INFO      onnx_ir.passes.common.unused_removal: No unused functions to remove
08:08:15  INFO      onnx_ir.passes.common.unused_removal: Removed 253 unused nodes
08:08:15  INFO      onnx_ir.passes.common.unused_removal: No unused functions to remove
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_Shape_0' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_Shape_1' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node__to_copy' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_4' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_add_79' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_12' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_20' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_28' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_36' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_44' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_52' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_60' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_68' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_76' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_84' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_92' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_100' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_108' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_116' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_124' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_132' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_140' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_148' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_156' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_164' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_172' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_180' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_188' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_196' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_204' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_212' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_220' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_228' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_236' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_244' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_252' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_260' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_268' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_276' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_284' because it is graph input to preserve graph signature
08:08:15  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n1' because it is graph input to preserve graph signature
08:08:16  INFO      onnx_ir.passes.common.unused_removal: Removed 184 unused nodes
08:08:16  INFO      onnxscript.rewriter: Applied 72 of general pattern rewrite rules.
08:08:16  INFO      onnx_ir.passes.common.unused_removal: No unused functions to remove
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_Shape_0' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_Shape_1' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node__to_copy' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_4' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'node_add_79' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_12' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_20' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_28' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_36' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_44' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_52' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_60' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_68' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_76' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_84' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_92' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_100' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_108' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_116' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_124' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_132' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_140' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_148' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_156' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_164' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_172' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_180' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_188' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_196' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_204' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_212' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_220' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_228' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_236' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_244' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_252' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_260' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_268' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_276' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n0_284' because it is graph input to preserve graph signature
08:08:16  INFO      onnxscript.optimizer._constant_folding: Skipping constant folding for node 'n1' because it is graph input to preserve graph signature
08:08:17  INFO      onnx_ir.passes.common.unused_removal: Removed 72 unused nodes
08:08:17  INFO      onnx_ir.passes.common.unused_removal: No unused functions to remove
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_15' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_20' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_29' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_34' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_43' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_48' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_57' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_62' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_71' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_76' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_85' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_90' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_99' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_104' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_113' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_118' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_127' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_132' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_141' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_146' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_155' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_160' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_169' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_174' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_183' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_188' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_197' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_202' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_211' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_216' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_225' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_230' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_239' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_244' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_253' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_258' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_267' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_272' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_281' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_286' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_295' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_300' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_309' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_314' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_323' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_328' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_337' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_342' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_351' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_356' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_365' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_370' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_379' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_384' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_393' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_398' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_407' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_412' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_421' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_426' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_435' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_440' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_449' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_454' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_463' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_468' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_477' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_482' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_491' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_496' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_505' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_510' with existing initializer 'val_5'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_13' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_18' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_25' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_27' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_32' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_39' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_41' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_46' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_53' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_55' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_60' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_67' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_69' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_74' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_81' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_83' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_88' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_95' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_97' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_102' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_109' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_111' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_116' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_123' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_125' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_130' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_137' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_139' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_144' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_151' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_153' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_158' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_165' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_167' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_172' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_179' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_181' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_186' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_193' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_195' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_200' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_207' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_209' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_214' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_221' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_223' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_228' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_235' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_237' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_242' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_249' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_251' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_256' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_263' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_265' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_270' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_277' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_279' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_284' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_291' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_293' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_298' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_305' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_307' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_312' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_319' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_321' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_326' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_333' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_335' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_340' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_347' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_349' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_354' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_361' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_363' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_368' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_375' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_377' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_382' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_389' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_391' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_396' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_403' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_405' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_410' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_417' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_419' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_424' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_431' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_433' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_438' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_445' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_447' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_452' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_459' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_461' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_466' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_473' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_475' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_480' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_487' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_489' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_494' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_501' with existing initializer 'val_11'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_503' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'val_508' with existing initializer 'val_2'
08:08:17  INFO      onnx_ir.passes.common.initializer_deduplication: Replaced initializer 'axes' with existing initializer 'val_5'
/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:486: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
  rename_mapping = _dynamic_shapes.create_rename_mapping(
/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:486: UserWarning: # The axis name: past_len will not be used, since it shares the same shape constraints with another axis: past_len.
  rename_mapping = _dynamic_shapes.create_rename_mapping(
08:08:27  INFO      tensorrt_edgellm.onnx.export: Export complete: Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx
08:08:27  INFO      tensorrt_edgellm.checkpoint.checkpoint_utils: Wrote config.json to /workspace/tensorrt-edgellm-workspace/VLM/Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm
08:08:28  INFO      tensorrt_edgellm.checkpoint.checkpoint_utils: Wrote embedding.safetensors ([151936, 2048])
08:08:28  INFO      tensorrt_edgellm.checkpoint.checkpoint_utils: Copied tokenizer.json
08:08:28  INFO      tensorrt_edgellm.checkpoint.checkpoint_utils: Copied tokenizer_config.json
08:08:28  INFO      tensorrt_edgellm.checkpoint.checkpoint_utils: Copied special_tokens_map.json
[transformers] Kwargs passed to `processor.__call__` have to be in `processor_kwargs` dict, not in `**kwargs`
08:08:28  INFO      tensorrt_edgellm.chat_template: Chat template saved to /workspace/tensorrt-edgellm-workspace/VLM/Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/processed_chat_template.json
08:08:28  INFO      tensorrt_edgellm.scripts.export: [LLM] Patched multimodal token IDs: image_token_id=151655, video_token_id=151656, vision_start_token_id=151652, vision_end_token_id=151653
08:08:28  INFO      tensorrt_edgellm.scripts.export: [LLM] Done: Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx
08:08:28  INFO      tensorrt_edgellm.scripts.export: Loading safetensors weights ...
08:08:28  INFO      tensorrt_edgellm.scripts.export:   Loading shard: model.safetensors
08:08:28  INFO      tensorrt_edgellm.scripts.export: [Visual] Exporting qwen2_5_vl visual encoder to Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx
08:08:28  INFO      tensorrt_edgellm.onnx.export_encoder: Building qwen2_5_vl visual model ...
[torch.onnx] Obtain model graph for `_Wrapper([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `_Wrapper([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decompositions...
[torch.onnx] Run decompositions... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
[torch.onnx] Optimize the ONNX graph...
[torch.onnx] Optimize the ONNX graph... ✅
Traceback (most recent call last):
  File "/workspace/TensorRT-Edge-LLM/host_venv/bin/tensorrt-edgellm-export", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/export.py", line 2812, in main
    fn(
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/export.py", line 2696, in _export_visual_component
    _export_visual(model_dir,
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/scripts/export.py", line 1094, in _export_visual
    export_visual_onnx(
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/onnx/export_encoder.py", line 299, in export_visual_onnx
    visual_model: nn.Module = build_fn(vcfg,
                              ^^^^^^^^^^^^^^
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/models/qwen2_5_vl/modeling_qwen2_5_vl_visual.py", line 529, in build_qwen25_vl_visual
    model = Qwen2_5VLVisualModel(config, model_config=model_config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/TensorRT-Edge-LLM/host_venv/lib/python3.12/site-packages/tensorrt_edgellm/models/qwen2_5_vl/modeling_qwen2_5_vl_visual.py", line 288, in __init__
    self.num_heads: int = config["num_heads"]
                          ~~~~~~^^^^^^^^^^^^^
KeyError: 'num_heads'

System information (Nvidia Jetson Thor)

Container used: nvcr.io/nvidia/pytorch:25.12-py3
OS: Ubuntu 24.04.3 LTS
CPU architecture: aarch64
GPU name: Nvidia Jetson T5000
GPU memory size: 128 GB
Number of GPUs: 1
Library versions:
- Python: 3.12.3
- TensorRT Edge-LLM version: 0.9.1
- CUDA: 13.1
- PyTorch: 2.12.0+cu130
- Transformers: 5.9.0
- ModelOpt: 0.44.0
- ONNX: 1.19.0

I've been trying to narrow down this issue, and it seems to be related to the Hugging Face checkpoint's config.json. The Qwen2.5-VL-3B-Instruct-AWQ checkpoint configuration lacks some of the attributes present in Qwen2.5-VL-3B-Instruct.

Compared to Qwen2.5-VL-3B-Instruct, the Qwen2.5-VL-3B-Instruct-AWQ checkpoint is missing several fields in vision_config, including num_heads, patch_size, depth, and intermediate_size.

Adding these missing fields is sufficient for the model to export and both the visual encoder and LLM engines to build successfully. However, inference still fails with:

# Export
06:30:05  INFO      tensorrt_edgellm.onnx.export_encoder: Export complete: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx
06:30:05  INFO      tensorrt_edgellm.scripts.export: [Visual] Done: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx
06:30:05  INFO      tensorrt_edgellm.scripts.export: [Visual] Copied preprocessor_config.json to Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual
06:30:05  INFO      tensorrt_edgellm.scripts.export: [Visual] Wrote config.json: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/config.json

============================================================
Export complete
output dir: Qwen2.5-VL-3B-Instruct-AWQ/onnx
thinker        : Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx  (3.3 MB)
                + embedding.safetensors  (622.3 MB)
visual         : Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx  (2.9 MB)
============================================================

# Visual encoder build
[08:32:06.858] [INFO] [visual_build.cpp:202:main] Visual engine built successfully.

# Decoder build
[08:32:37.917] [INFO] [llm_build.cpp:247:main] LLM engine built successfully.

# Inference
[08:32:46.174] [ERROR] [llm_inference.cpp:592:main] Failed to initialize runtime: /home/fabien/aide/TensorRT-Edge-LLM/target/TensorRT-Edge-LLM/cpp/runtime/llmInferenceRuntime.cpp:414: No valid multimodal engine found in /home/fabien/aide/TensorRT-Edge-LLM/target/tensorrt-edgellm-workspace/VLM/Qwen2.5-VL-3B-Instruct-AWQ/engines/visual

I also tried replacing the entire vision_config in Qwen2.5-VL-3B-Instruct-AWQ with the one from Qwen2.5-VL-3B-Instruct. In that case:

Export succeeds
Both engines build successfully
Inference no longer reports an error, but the model returns an empty output ("output_text": "")

# Export
06:41:38  INFO      tensorrt_edgellm.onnx.export_encoder: Export complete: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx
06:41:38  INFO      tensorrt_edgellm.scripts.export: [Visual] Done: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx
06:41:38  INFO      tensorrt_edgellm.scripts.export: [Visual] Copied preprocessor_config.json to Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual
06:41:38  INFO      tensorrt_edgellm.scripts.export: [Visual] Wrote config.json: Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/config.json

============================================================
Export complete
output dir: Qwen2.5-VL-3B-Instruct-AWQ/onnx
thinker        : Qwen2.5-VL-3B-Instruct-AWQ/onnx/llm/model.onnx  (3.3 MB)
                + embedding.safetensors  (622.3 MB)
visual         : Qwen2.5-VL-3B-Instruct-AWQ/onnx/visual/model.onnx  (2.9 MB)
============================================================

# Visual encoder build
[08:43:55.672] [INFO] [visual_build.cpp:202:main] Visual engine built successfully.

# Decoder build
[08:44:23.031] [INFO] [llm_build.cpp:247:main] LLM engine built successfully.

# Inference
# No error but "output_text": ""

This suggests that the missing vision_config fields are at least part of the problem, but simply copying the configuration from the non-AWQ checkpoint is not sufficient to produce a working model.

Hope this helps narrow down the issue!

Navigation Menu

Repository navigation

Exporting Qwen2.5-VL AWQ checkpoints fails with KeyError: 'num_heads' #163

Describe the bug

Steps/Code to reproduce bug

System information (Nvidia Jetson Thor)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Exporting Qwen2.5-VL AWQ checkpoints fails with KeyError: 'num_heads' #163

Description

Describe the bug

Steps/Code to reproduce bug

System information (Nvidia Jetson Thor)

Activity

FABallemand commented on Aug 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions