Qwen/Qwen3.5-9B

Overview

Qwen3.5-9B is a dense multimodal model from the Qwen3.5 family — same gated delta networks hybrid attention, vision encoder, 262K context, and MTP support as its larger siblings, but sized to fit comfortably on a single 24 GB GPU.

Prerequisites

• vLLM version: >= 0.17.0
• Hardware: single 24 GB GPU (RTX 4090 / L4 / A10G / H100) or Intel Arc Pro B60/B70

Install vLLM

uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend=auto

Docker

docker pull vllm/vllm-openai-xpu:latest  # Intel XPU (B60 / B70)

Launching the Server

Single-GPU BF16

vllm serve Qwen/Qwen3.5-9B \
  --max-model-len 262144 \
  --reasoning-parser qwen3

MTP speculative decoding

vllm serve Qwen/Qwen3.5-9B \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}' \
  --reasoning-parser qwen3

Docker (Intel XPU B60 / B70)

Validated on 1× Intel Arc Pro B60 / B70 (B60 24 GB, B70 32 GB per card) with the official vLLM XPU image vllm/vllm-openai-xpu:latest.

docker run --device /dev/dri \
  -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint bash vllm/vllm-openai-xpu:latest \
  -c "source /opt/intel/oneapi/setvars.sh && exec vllm serve Qwen/Qwen3.5-9B \
  --reasoning-parser qwen3 \
  --max-model-len 8192 \
  --enforce-eager"

Client Usage

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-9B",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=128,
)
print(resp.choices[0].message.content)

Troubleshooting

• CUDA graph / Mamba cache size error: reduce --max-cudagraph-capture-size (default 512). See vLLM PR #34571.
• Disable reasoning: add --default-chat-template-kwargs '{"enable_thinking": false}'.

References

• Model card
• Base checkpoint