stdlib/inferView source ↗
infer
9 items · generated by scribe fromstdlib/infer.fd· synced 2026-11-15
Contents
- infer_matvec (function)
- infer_rmsnorm (function)
- infer_rope (function)
- infer_softmax (function)
- infer_argmax (function)
- infer_attn (function)
- infer_top_k (function)
- infer_sample (function)
- infer_sample_top_p (function)
infer_matvec
y = W x for W in row-major [rows][cols] f64.
pub fn infer_matvec(w: Vec<f64>, rows: i64, cols: i64, x: Vec<f64>) -> Vec<f64> {Parameters:
w:Vec<f64>rows:i64cols:i64x:Vec<f64>
Returns: Vec<f64>
Defined at line 7
infer_rmsnorm
RMSNorm: x / sqrt(mean(x^2) + eps) * weight.
pub fn infer_rmsnorm(x: Vec<f64>, w: Vec<f64>, eps: f64) -> Vec<f64> {Parameters:
x:Vec<f64>w:Vec<f64>eps:f64
Returns: Vec<f64>
Defined at line 25
infer_rope
Apply rotary position embedding to a q/k head: pairs (x[2i], x[2i+1]) rotate by pos * theta^(-2i/head_dim). Returns a new Vec so it works on every backend.
pub fn infer_rope(v: Vec<f64>, head_dim: i64, pos: i64, theta: f64) -> Vec<f64> {Parameters:
v:Vec<f64>head_dim:i64pos:i64theta:f64
Returns: Vec<f64>
Defined at line 45
infer_softmax
Numerically stable softmax (max subtracted).
Example (native only): use vec; var x: Vec
pub fn infer_softmax(x: Vec<f64>) -> Vec<f64> {Parameters:
x:Vec<f64>
Returns: Vec<f64>
Defined at line 73
infer_argmax
Index of the largest element (ties -> lowest index), -1 when empty.
pub fn infer_argmax(x: Vec<f64>) -> i64 {Parameters:
x:Vec<f64>
Returns: i64
Defined at line 104
infer_attn
Single-head attention: q (dim), keys/values flat [seq][dim]. Returns sum_i softmax(q.k_i / sqrt(dim)) * v_i — a Vec of dim f64.
pub fn infer_attn(q: Vec<f64>, k: Vec<f64>, v: Vec<f64>, seq: i64, dim: i64) -> Vec<f64> {Parameters:
q:Vec<f64>k:Vec<f64>v:Vec<f64>seq:i64dim:i64
Returns: Vec<f64>
Defined at line 121
infer_top_k
Top-k indices of logits, largest-first (selection sort over k).
pub fn infer_top_k(x: Vec<f64>, k: i64) -> Vec<i64> {Parameters:
x:Vec<f64>k:i64
Returns: Vec<i64>
Defined at line 154
infer_sample
Sample from a probability Vec using u in [0,1). Returns an index. Caller normalizes/softmaxes first (or passes probs from top-k).
pub fn infer_sample(probs: Vec<f64>, u: f64) -> i64 {Parameters:
probs:Vec<f64>u:f64
Returns: i64
Defined at line 188
infer_sample_top_p
Top-p (nucleus): keep the smallest prefix of sorted probs whose mass reaches p, then sample among them. order from infer_top_k over the softmax’d logits, probs the same softmax values. Returns the chosen LOGIT index.
pub fn infer_sample_top_p(probs: Vec<f64>, order: Vec<i64>, p: f64, u: f64) -> i64 {Parameters:
probs:Vec<f64>order:Vec<i64>p:f64u:f64
Returns: i64
Defined at line 208
Generated by scribe, the Fides documentation generator