Skip to main content
Ctrl+K
📣 SDK 2025.1.0 has been released on Feb 24, 2025. Please checkout SDK Release Announcement of 2025.1.0.

Furiosa Docs

Overview

  • FuriosaAI RNGD
  • FuriosaAI’s Software Stack
  • Supported Models
  • What’s New
  • Roadmap

Getting Started

  • Installing Prerequisites
  • Quick Start with Furiosa LLM
  • Running MLPerf™ Inference Benchmark
  • Upgrading FuriosaAI’s Software

Furiosa LLM

  • Furiosa LLM
  • Model Preparation Workflow
  • OpenAI-Compatible Server
  • Model Parallelism
  • API Reference
    • LLM class
    • SamplingParams class
    • ArtifactBuilder
    • LLMEngine class
    • AsyncLLMEngine class
  • Examples
    • Chat
    • Chat with tools

Cloud Native Toolkit

  • Cloud Native Toolkit
  • Container Support
  • Kubernetes Plugins
    • Installing Furiosa Feature Discovery
    • Installing Furiosa Device Plugin
    • Installing Furiosa Metrics Exporter

Device Management

  • Furiosa SMI
    • Furiosa SMI CLI
    • Furiosa SMI Library

Customer Support

  • Forums
  • Customer Support

Other Links

  • FuriosaAI Homepage
  • Furiosa Gen 1 NPU SDK Doc

© Copyright 2025 FuriosaAI Inc.

Index

A | B | C | F | G | H | L | M | S

A

  • abort() (furiosa_llm.AsyncLLMEngine method)
  • abort_request() (furiosa_llm.LLMEngine method)
  • add_request() (furiosa_llm.LLMEngine method)
  • Artifact (class in furiosa_llm.artifact)
  • ArtifactBuilder (class in furiosa_llm.artifact)
  • ArtifactMetadata (class in furiosa_llm.artifact)
  • AsyncLLMEngine (class in furiosa_llm)

B

  • build() (furiosa_llm.artifact.ArtifactBuilder method)

C

  • chat() (furiosa_llm.LLM method)

F

  • from_artifacts() (furiosa_llm.LLM class method)
  • from_engine_args() (furiosa_llm.AsyncLLMEngine class method)
    • (furiosa_llm.LLMEngine class method)

G

  • generate() (furiosa_llm.AsyncLLMEngine method)
    • (furiosa_llm.LLM method)

H

  • has_unfinished_requests() (furiosa_llm.LLMEngine method)

L

  • LLM (class in furiosa_llm)
  • LLMEngine (class in furiosa_llm)
  • load_artifact() (furiosa_llm.LLM class method)
  • load_artifacts() (furiosa_llm.LLM class method)

M

  • model_computed_fields (furiosa_llm.artifact.Artifact attribute)
    • (furiosa_llm.artifact.ArtifactMetadata attribute)
  • model_config (furiosa_llm.artifact.Artifact attribute)
    • (furiosa_llm.artifact.ArtifactMetadata attribute)
  • model_fields (furiosa_llm.artifact.Artifact attribute)
    • (furiosa_llm.artifact.ArtifactMetadata attribute)

S

  • SamplingParams (class in furiosa_llm)
  • step() (furiosa_llm.LLMEngine method)
  • stream_generate() (furiosa_llm.LLM method)

By FuriosaAI, Inc.

© Copyright 2025 FuriosaAI Inc.