Skip to main content

Ctrl+K

📣 SDK 2025.1.0 has been released on Feb 24, 2025. Please checkout SDK Release Announcement of 2025.1.0.

Overview

FuriosaAI RNGD
FuriosaAI’s Software Stack
Supported Models
What’s New
Roadmap

Getting Started

Installing Prerequisites
Quick Start with Furiosa LLM
Running MLPerf™ Inference Benchmark
Upgrading FuriosaAI’s Software

Furiosa LLM

Furiosa LLM
Model Preparation Workflow
OpenAI-Compatible Server
Model Parallelism
API Reference
Examples
- Chat
- Chat with tools

Cloud Native Toolkit

Cloud Native Toolkit
Container Support
Kubernetes Plugins

Device Management

Furiosa SMI
- Furiosa SMI CLI
- Furiosa SMI Library

Customer Support

Forums
Customer Support

Other Links

FuriosaAI Homepage
Furiosa Gen 1 NPU SDK Doc

© Copyright 2025 FuriosaAI Inc.

Examples

Examples#

Chat
Chat with tools

previous

AsyncLLMEngine class

next

Chat

By FuriosaAI, Inc.

© Copyright 2025 FuriosaAI Inc.