Qwen3-MoE

Qwen3-MoE#

The Qwen3 Mixture-of-Experts (MoE) models are auto-regressive MoE transformers with 30.5B total parameters, of which about 3.3B are activated per token. They offer strong instruction following, reasoning over text, multilingual coverage, and tool usage. They come in the original hybrid thinking / non-thinking release as well as updated instruct, thinking, and coding-specialized editions.

Furiosa-LLM runs the Qwen3-MoE models in FP8 (static FP8 weights with dynamic FP8 activation quantization; the KV cache stays in 16-bit precision). FuriosaAI publishes pre-compiled FP8 builds under the furiosa-ai organization on the Hugging Face Hub, each shipping a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same upstream weights also run on other frameworks (such as vLLM, SGLang, and Transformers); for usage with those, see the upstream model cards linked below.

For the dense Qwen3 chat models see Qwen3 (dense).

Available Models#

Model

Quantization

RNGD cards

Notes

furiosa-ai/Qwen3-30B-A3B-FP8

FP8

4

Original release; hybrid thinking / non-thinking

furiosa-ai/Qwen3-30B-A3B-Instruct-2507-FP8

FP8

4

Updated (2507) instruct; non-thinking only

furiosa-ai/Qwen3-30B-A3B-Thinking-2507-FP8

FP8

4

Updated (2507) thinking; always reasons

furiosa-ai/Qwen3-Coder-30B-A3B-Instruct-FP8

FP8

4

Agentic coding; non-thinking only

  • Architecture: Qwen3-MoE (Mixture-of-Experts), Qwen3MoeForCausalLM

  • Input / Output: Text / Text

  • Quantization: Weights are quantized to FP8 (static), following the upstream FP8 release, and activations use dynamic FP8 quantization at runtime (per-token / per-block). The KV cache stays in 16-bit precision.

Usage#

To run these models with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.

Launch the server#

Pass the model’s furiosa-ai/<repo> identifier; each variant runs on four RNGD cards (tensor-parallel size 32 PEs). The examples below use the hybrid Qwen3-30B-A3B-FP8, which reasons by default and can switch thinking on and off (see Advanced Usage).

Serve it with the qwen3 reasoning parser so the chain of thought is returned in a separate field:

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3

To also enable tool (function) calling, add the hermes tool-call parser (the parser used by the Qwen3 series):

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

The other variants launch the same way, with one difference: the Qwen3-30B-A3B-Thinking-2507-FP8 variant always reasons (keep --reasoning-parser qwen3), while the non-thinking Qwen3-30B-A3B-Instruct-2507-FP8 and Qwen3-Coder-30B-A3B-Instruct-FP8 variants omit the reasoning parser:

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-Instruct-2507-FP8

When the server is ready, you will see:

INFO:     Started server process [27507]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Basic Usage#

The server exposes an OpenAI-compatible API. You can send a request with curl (replace the model id with the variant you launched):

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen3-30B-A3B-FP8",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
    }' \
    | python -m json.tool

The reasoning variants (Qwen3-30B-A3B-FP8 and Qwen3-30B-A3B-Thinking-2507-FP8) return their reasoning separately from the final answer:

  • response.choices[].message.reasoning (non-streaming)

  • response.choices[].delta.reasoning (streaming)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="furiosa-ai/Qwen3-30B-A3B-FP8",
    messages=[{"role": "user", "content": "How many r's are in 'strawberry'?"}],
)

print("Reasoning:", response.choices[0].message.reasoning)
print("Answer:", response.choices[0].message.content)

Note: The reasoning field is not part of the OpenAI API specification but is a widely followed convention (the OpenAI Agents SDK, vLLM, and others). It appears only in responses that contain reasoning content; accessing it otherwise raises an AttributeError.

Advanced Usage#

Toggling thinking. Only the hybrid Qwen3-30B-A3B-FP8 supports switching thinking on and off; it reasons by default. To turn thinking off for a single request, pass enable_thinking through chat_template_kwargs; the response then carries no reasoning content, so read only message.content:

# Disable thinking for a single request
response = client.chat.completions.create(
    model="furiosa-ai/Qwen3-30B-A3B-FP8",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

To default every request to non-thinking, launch the server with --default-chat-template-kwargs (a request can still re-enable thinking with its own chat_template_kwargs):

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

The other variants do not read enable_thinking — the Instruct and Coder variants are non-thinking, and Qwen3-30B-A3B-Thinking-2507-FP8 always reasons — so the switch has no effect on them.

Tool calling. With the server launched using --enable-auto-tool-choice --tool-call-parser hermes (see Launch the server), pass tools in the request and let the model decide when to call them. See the Tool Calling guide for a complete client example and details on tool-choice options.

Learn more#