Supported Models

FuriosaAI Software Stack supports a variety of Transformer-based models in HuggingFace Hub. The following is the list of model architectures that are currently supported by Furiosa SDK. If your model is based on the following architectures, you can use Furiosa SDK to compile, quantize, and run the model on FuriosaAI RNGD.

Decoder-only Models

Decoder-only Models

Model Name	Architecture	Example HuggingFace Models
Llama 2, Llama 3.1	LlamaForCausalLM	meta-llama/Llama-2-70b-hf, meta-llama/Llama-3.1-70B, meta-llama/Llama-3.1-70B-Instruct, meta-llama/Llama-3.1-8B, meta-llama/Llama-3.1-8B-Instruct, ..
GPT-J	GPTJForCausalLM	EleutherAI/gpt-j-6b
Solar	LlamaForCausalLM	upstage/SOLAR-10.7B-v1.0, upstage/SOLAR-10.7B-Instruct-v1.0
EXAONE	LlamaForCausalLM	LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct
CodeLlama	LlamaForCausalLM	meta-llama/CodeLlama-7b-Instruct-hf

Encoder-only Models

Encoder-only Models

Model Name	Architecture	Example HuggingFace Models
Bert	BertForQuestionAnswering	google-bert/bert-large-uncased, google-bert/bert-base-uncased, ..

Planned Models for Future Releases

• Falcon

• Mistral-7B-v0.1, Mistral-7B-Instruct-v0.1

• Mixtral-8x7B-v0.1, Mixtral-8x7B-Instruct-v0.1

• Phi-3, Phi-3.5,

• Phi-3.5-MoE

• Qwen2

• gemma-2

• DeepSeek-Coder-V2-Instruct

• Stable Diffusion XL

• Llama 3.2 multi-model models

• Llama 3.3