Commit Adapter

The Commit Adapter applies element-wise transformations to the packet stream before the Commit Engine writes it to DM. It mirrors the Fetch Adapter on the output side of the Tensor Unit.

The adapter’s stages chain as dedicated .commit_xxx(...) methods on the upstream tensor, and the chain always ends in .commit(...) for the actual DM write. Trimming is the mandatory first stage: .commit() / .commit_view() are reachable only after .commit_trim(...), so every commit is trimmed first (it is how flit padding is dropped). The other stages are rare and chain after trimming. Main and sub contexts then diverge. A separate operation, Generate Mode, is conceptually part of the Commit Adapter but stands alone (it does not chain off a TuTensor).

Main pipeline: Trimming → Type Casting (optionally fusing ReLU) → .commit().
Sub pipeline: Trimming → Valid Count Packing → .commit().
Sub bypass: Generate Mode writes a single 32-bit constant to DM directly via a standalone API, skipping the Tensor Unit pipeline entirely.

Operation	Main	Sub
Trimming	✅	✅
Type Casting (optional fused ReLU)	✅	❌
Valid Count Packing	❌	✅
Generate Mode	❌	✅ (UC, see §Generate Mode)

Trimming

Stream packets in the Tensor Unit pipeline are always 32-byte flits (see Collect Engine), but a flit may carry fewer valid elements than its capacity, with trailing elements filled by padding. Writing the full flit verbatim would clobber DM bytes beyond the valid region with the flit’s padding values.

Trimming solves this by writing only the leading valid_size elements of each flit to DM, discarding the trailing padding. The compiler derives valid_size from the output tensor mapping. Users do not set it directly. D[valid_size] must be 8, 16, 24, or 32 bytes (where 32 means no trim). Trimming adds nearly zero latency.

Trimming is the mandatory first stage of the Commit Adapter, even though not every commit has padding to drop: when valid_size is already 32 bytes the flit is fully valid and the trim is a no-op. It is mandatory because .commit() is reachable only after .commit_trim(...), so it anchors the chain and runs ahead of Type Casting (main) and Valid Count Packing (sub).

// `D: MaterializableScalar` here (trim is the commit path's mandatory first stage) keeps i5/i9 uncommittable.
impl<
    'l,
    const T: Tu,
    P: CanApplyCommitTrim,
    D: MaterializableScalar,
    Chip: M,
    Cluster: M,
    Slice: M,
    Time: M,
    Packet: M,
    B: Backend,
> TuTensor<'l, T, P, D, Chip, Cluster, Slice, Time, Packet, B>
{
    /// Runs the Commit Adapter's trimming stage.
    ///
    /// Drops the trailing padding from each flit so DM stores only valid
    /// elements. `OutPacket` is the post-trim layout the kernel
    /// promises; the compiler derives the trim count from the input and
    /// output mappings.
    #[primitive(TuTensor::commit_trim)]
    pub fn commit_trim<OutPacket: M>(self) -> CommitTrimTensor<'l, T, D, Chip, Cluster, Slice, Time, OutPacket, B> {
        verify_commit_trim::<D, Packet, OutPacket>();
        // `transpose(false)` is type-system filler; real trim lowering lands with the backend wiring.
        CommitTrimTensor::new(self.ctx, self.inner.transpose(false))
    }
}

.commit_trim::<OutPacket>() declares the post-trim packet, and the chained .commit(...) then performs the DM write on the trimmed stream. The two are fully separate.

#![feature(adt_const_params)]
extern crate furiosa_opt_std;
use furiosa_opt_std::prelude::*;
axes![M = 4, K = 2, W = 8, N = 16, J = 64];

fn commit_trim_i8_padding<'l, const T: Tu>(
    input: CastTensor<'l, T, i8, m![1], m![1], m![1], m![M, K], m![W # 32]>,
) -> CommitTrimTensor<'l, T, i8, m![1], m![1], m![1], m![M, K], m![W]> {
    // 8 valid i8 out of 32 padded; OutPacket drops the `# 32` padding.
    input.commit_trim::<m![W]>()
}

fn commit_trim_f32_non_padding<'l, const T: Tu>(
    input: ContractTensor<'l, T, f32, m![1], m![1], m![1], m![M, K], m![W]>,
) -> CommitTrimTensor<'l, T, f32, m![1], m![1], m![1], m![M, K], m![W = 4]> {
    // 4 valid f32 out of 8; OutPacket resizes `W` to 4.
    input.commit_trim::<m![W = 4]>()
}

fn commit_trim_bf16_with_transpose<'l, const T: Tu>(
    input: CastTensor<'l, T, bf16, m![1], m![1], m![1], m![M, K], m![N]>,
) -> CommitTrimTensor<'l, T, bf16, m![1], m![1], m![1], m![M, K], m![N = 8]> {
    // 8 valid bf16 out of 16; OutPacket resizes `N` to 8.
    input.commit_trim::<m![N = 8]>()
}

fn commit_trim_i4_no_trim<'l, const T: Tu>(
    input: CastTensor<'l, T, i4, m![1], m![1], m![1], m![M, K], m![J]>,
) -> CommitTrimTensor<'l, T, i4, m![1], m![1], m![1], m![M, K], m![J]> {
    // No trimming; `OutPacket == Packet`.
    input.commit_trim::<m![J]>()
}

let mut ctx = Context::acquire();
let a: CastTensor<'_, _, i8, m![1], m![1], m![1], m![M, K], m![W # 32]> = CastTensor::new(&mut ctx.main, Tensor::zero());
let _o = commit_trim_i8_padding(a);
let b: ContractTensor<'_, _, f32, m![1], m![1], m![1], m![M, K], m![W]> = ContractTensor::new(&mut ctx.main, Tensor::zero());
let _o = commit_trim_f32_non_padding(b);
let c: CastTensor<'_, _, bf16, m![1], m![1], m![1], m![M, K], m![N]> = CastTensor::new(&mut ctx.main, Tensor::zero());
let _o = commit_trim_bf16_with_transpose(c);
let d: CastTensor<'_, _, i4, m![1], m![1], m![1], m![M, K], m![J]> = CastTensor::new(&mut ctx.main, Tensor::zero());
let _o = commit_trim_i4_no_trim(d);

Type Casting

Type casting converts f32 data to bf16 format on the commit path, optionally fusing a ReLU activation into the same pass. The Cast Engine handles most type conversions in the Tensor Unit pipeline. Commit Adapter type casting exists for one specific case, running main-context contraction in parallel with sub-context Vector Engine work. The Cast Engine sits on top of the Vector Engine and so occupies it during a conversion. If the main-context performed its f32 → bf16 conversion through the Cast Engine, the Vector Engine would be busy and the sub-context could not run in parallel. Routing the conversion through the Commit Adapter instead leaves the Vector Engine free for the sub-context. Sub-context itself does not support type casting (consistent with the support matrix above).

commit_cast takes an Activation. Activation::None is a plain cast. Activation::Relu clamps negative values to zero as part of the same cast. ReLU has no standalone hardware stage, and exists only fused with a narrowing cast (f32 → bf16 + ReLU).

impl<
    'l,
    const T: Tu,
    P: CanApplyCommitCast,
    D: MaterializableScalar,
    Chip: M,
    Cluster: M,
    Slice: M,
    Time: M,
    Packet: M,
    B: Backend,
> TuTensor<'l, T, P, D, Chip, Cluster, Slice, Time, Packet, B>
{
    /// Runs the Commit Adapter's type-casting stage, optionally fusing a
    /// ReLU.
    ///
    /// Folds an `f32` → `bf16` (or other narrowing) cast into the commit
    /// path, leaving the [Cast Engine](crate::engine::cast) free for
    /// sub-context Vector Engine work. `activation` selects the optional
    /// fused ReLU; ReLU has no standalone hardware stage.
    #[primitive(TuTensor::commit_cast)]
    pub fn commit_cast<OutD: Scalar>(
        self,
        _activation: Activation,
    ) -> CommitCastTensor<'l, T, OutD, Chip, Cluster, Slice, Time, Packet, B>
    where
        D: Cast<OutD>,
    {
        verify_commit_cast::<D, OutD>();
        CommitCastTensor::new(self.ctx, self.inner.map(|v| v.cast()))
    }
}

#![feature(adt_const_params)]
extern crate furiosa_opt_std;
use furiosa_opt_std::prelude::*;
axes![N = 4, C = 3, H = 4, W = 8];

fn commit_cast_example<'l, const T: Tu>(
    input: ContractTensor<'l, T, f32, m![1], m![1], m![1], m![N, C, H], m![W]>,
) -> CommitCastTensor<'l, T, bf16, m![1], m![1], m![1], m![N, C, H], m![W]> {
    // Cast f32 to bf16 (values preserved), no activation. A real main
    // commit runs `.commit_trim()` first, then `.commit(...)` after.
    // W = 8 f32 elements (32 bytes) stays 8 bf16 elements (16 bytes).
    input.commit_cast::<bf16>(Activation::None)
}

fn commit_cast_relu_example<'l, const T: Tu>(
    input: ContractTensor<'l, T, f32, m![1], m![1], m![1], m![N, C, H], m![W]>,
) -> CommitCastTensor<'l, T, bf16, m![1], m![1], m![1], m![N, C, H], m![W]> {
    // f32 -> bf16 with a fused ReLU: negative values clamped to zero.
    // e.g. [-5.0, -0.1, 0.0, 3.7] -> [0.0, 0.0, 0.0, 3.7]
    input.commit_cast::<bf16>(Activation::Relu)
}

Valid Count Packing

Valid Count Packing is a sub-context-only stage that commits a variable number of valid elements per packet, excluding padding from the output.

impl<
    'l,
    const T: Tu,
    P: CanApplyCommitValidCountPack,
    D: Scalar,
    Chip: M,
    Cluster: M,
    Slice: M,
    Time: M,
    Packet: M,
    B: Backend,
> TuTensor<'l, T, P, D, Chip, Cluster, Slice, Time, Packet, B>
{
    /// Runs the Commit Adapter's valid-count-packing stage (sub-context
    /// only). The count comes from a per-call argument; the trailing
    /// elements are discarded. The packed stream keeps the input
    /// `Time` / `Packet` shape at this skeleton stage.
    // TODO: `_valid_count` is currently discarded. The backend
    // `TuOperationCommitValidCountPack` record does not store it yet.
    #[primitive(TuTensor::commit_valid_count_pack)]
    pub fn commit_valid_count_pack(
        self,
        _valid_count: usize,
    ) -> CommitValidCountPackTensor<'l, T, D, Chip, Cluster, Slice, Time, Packet, B> {
        verify_commit_valid_count_pack::<D, Time, Packet>();
        CommitValidCountPackTensor::new(self.ctx, self.inner.transpose(false))
    }
}

Generate Mode

Generate Mode is used for sub-context-only ITOS (immediate-to-SRAM) writes. The sequencer hands the hardware a constant u32 value and a sub-context-derived DM address, and the runtime writes the constant directly to that address.

It does not fetch from DM and does not consume any upstream Tensor Unit stream. The constant value and the destination are the only inputs; the rest of the Tensor Unit pipeline (Fetch / Switch / Collect / Contraction / Vector / Cast / Transpose / Commit Adapter stages) is bypassed entirely.

Keyboard shortcuts

Programming Tensor Contraction Processors

Commit Adapter

Trimming

Type Casting

Valid Count Packing

Generate Mode