Qwen3.5 / Qwen3.6 / Qwen3.8

Qwen3.5 / Qwen3.6 / Qwen3.8#

The Qwen3.5 architectures pair a vision encoder with a decoder that combines Gated DeltaNet linear attention and full attention. Qwen3.5-0.8B and Qwen3.6-27B use the dense architecture (Qwen3_5ForConditionalGeneration), while Qwen3.6-35B-A3B uses the Mixture-of-Experts (MoE) architecture (Qwen3_5MoeForConditionalGeneration) with 35B total parameters and 3B activated per token. All three models support reasoning, tool (function) calling, and visual understanding. Thinking is enabled by default on the Qwen3.6 models and disabled by default on Qwen3.5-0.8B; either default can be switched per request.

Qwen3.8-27B-FP8 uses the same architecture, kernel shapes, and FP8 quantization configuration as Qwen3.6-27B-FP8, so it can use the same Furiosa-LLM compilation path. The usage examples below use Qwen3.6-27B-FP8.

FuriosaAI packages the upstream weights (BF16 for Qwen3.5-0.8B, FP8 for the Qwen3.6 models) together with a Furiosa Executable Bundle (FXB) for running them on FuriosaAI RNGD with Furiosa-LLM under the furiosa-ai organization on the Hugging Face Hub. The same upstream weights also run on other frameworks (such as vLLM, SGLang, and Transformers); for usage with those, see the upstream model cards linked below.

For the Qwen3 dense chat models see Qwen3 (dense), and for the Qwen3-VL vision-language models see Qwen3-VL.

Available Models#

Model

Quantization

RNGD cards

Notes

furiosa-ai/Qwen3.5-0.8B

None (BF16)

0.5 (4 PEs)

0.8B dense; hybrid thinking / non-thinking (non-thinking by default); performance experimental

furiosa-ai/Qwen3.6-27B-FP8

FP8

2

27B dense; hybrid thinking / non-thinking; experimental

furiosa-ai/Qwen3.6-35B-A3B-FP8

FP8

2

35B total / 3B active MoE; hybrid thinking / non-thinking; correctness and performance experimental

  • Architecture: Qwen3.5 (dense), Qwen3_5ForConditionalGeneration; Qwen3.5-MoE, Qwen3_5MoeForConditionalGeneration

  • Input / Output: Image + Text / Text

  • Quantization: Qwen3.5-0.8B is not quantized; its weights and activations stay in BF16, as in the upstream checkpoint. In the Qwen3.6 models, quantized projections use static FP8 weights (E4M3, 128 × 128 blocks) and dynamic FP8 activation quantization at runtime (per-token / per-block), without offline activation calibration. Modules excluded by the upstream quantization configuration, including the vision encoder and LM head, retain higher precision. The KV cache stays in 16-bit precision.

Usage#

To run these models with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.

Launch the server#

Pass the model’s furiosa-ai/<repo> identifier. Qwen3.6-27B-FP8 runs on two RNGD cards per replica (tensor-parallel size 16 PEs, pipeline-parallel size 1). Qwen3.6-35B-A3B-FP8 also runs on two RNGD cards (tensor-parallel size 16 PEs), and Qwen3.5-0.8B runs on half of one RNGD card (tensor-parallel size 4 PEs); use their identifiers in place of the model ID in the examples below. The Qwen3.6 models reason by default, while Qwen3.5-0.8B answers without reasoning by default; all of them can switch thinking on and off (see Advanced Usage).

Serve it with the qwen3 reasoning parser so the reasoning is returned in a separate field:

furiosa-llm serve furiosa-ai/Qwen3.6-27B-FP8 \
  --reasoning-parser qwen3

To also enable tool (function) calling, add the qwen3_coder tool-call parser (also available as qwen3_xml) for this model’s XML tool-call format:

furiosa-llm serve furiosa-ai/Qwen3.6-27B-FP8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Without --devices, the server uses every available PE, so Qwen3.5-0.8B runs one replica on each group of four PEs. To run a single replica on specific PEs, pass --devices; for example, --devices "npu:0:0-3" uses PEs 0–3 of NPU 0.

When the server is ready, you will see:

INFO:     Started server process [27507]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Basic Usage#

The server exposes an OpenAI-compatible API. You can send a text-only request with curl:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen3.6-27B-FP8",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
    }' \
    | python -m json.tool

To ask about an image, pass an image_url content part in the message:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen3.6-27B-FP8",
    "messages": [{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"}},
            {"type": "text", "text": "Describe this image."}
        ]
    }]
    }' \
    | python -m json.tool

The image_url.url field accepts a remote http:///https:// URL, an inline base64 data: URL, or a local file:// path (the last requires the --allowed-local-media-path flag described under Advanced Usage). Furiosa-LLM currently serves text and image inputs for these models.

With thinking enabled, the model returns its reasoning separately from the final answer:

  • response.choices[].message.reasoning (non-streaming)

  • response.choices[].delta.reasoning (streaming)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="furiosa-ai/Qwen3.6-27B-FP8",
    messages=[{"role": "user", "content": "How many r's are in 'strawberry'?"}],
)

print("Reasoning:", response.choices[0].message.reasoning)
print("Answer:", response.choices[0].message.content)

Note: The reasoning field is not part of the OpenAI API specification but is a widely followed convention (the OpenAI Agents SDK, vLLM, and others). It appears only in responses that contain reasoning content; accessing it otherwise raises an AttributeError.

Advanced Usage#

Toggling thinking. The Qwen3.6 models reason by default. To turn thinking off for a single request, pass enable_thinking through chat_template_kwargs; the response then carries no reasoning content, so read only message.content:

# Disable thinking for a single request
response = client.chat.completions.create(
    model="furiosa-ai/Qwen3.6-27B-FP8",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

To default every request to non-thinking, launch the server with --default-chat-template-kwargs (a request can still re-enable thinking with its own chat_template_kwargs):

furiosa-llm serve furiosa-ai/Qwen3.6-27B-FP8 \
  --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Qwen3.5-0.8B works the other way around: it answers without reasoning by default, so pass "enable_thinking": True per request, or '{"enable_thinking": true}' as the server default, to turn thinking on. The upstream model card notes that, in thinking mode, Qwen3.5-0.8B is more prone to thinking loops than larger Qwen3.5 models, and recommends tuning the sampling parameters and streaming the response so that such generations can be stopped.

Multimodal serving options. This example allows local images under /srv/media, restricts remote image fetches to cdn.example.com, and limits each request to four images:

furiosa-llm serve furiosa-ai/Qwen3.6-27B-FP8 \
  --reasoning-parser qwen3 \
  --allowed-local-media-path /srv/media \
  --allowed-media-domains cdn.example.com \
  --image-limit-per-prompt 4

See Server Options in the Vision-Language Models guide for the full option reference and defaults.

Tool calling. With the server launched using --enable-auto-tool-choice --tool-call-parser qwen3_coder (see Launch the server), pass tools in the request and let the model decide when to call them. See the Tool Calling guide for a complete client example and details on tool-choice options.

Learn more#