session
Update 17: AI & LLM serving
The AI story deepened from tensors-in-the-stdlib to language-level ML semantics.
layer/modelkeywords: desugar to plain structs and impls, so a network definition reads like the math:x |> self.l1.forward |> relu |> self.l2.forward.- Pipeline operator
|>:x |> f |> gisg(f(x)), resolved at parse time. - Compile-time shape safety:
Tensor<f32, [784, 128]>with symbolic batch dimensions;matmul,conv2d,concat, and friends are shape-checked before the program runs. - Tape-based autodiff:
grad(w) { ... backward(loss); }records and replays a gradient tape (interpreter runtime; IR support across backends). - GPU kernels:
@kernelfunctions withthread_id()/ workgroup builtins compile to SPIR-V, with a CPU fallback so kernels are testable in the interpreter. - Quantization: the
~> q8cast operator, symmetric INT8 parameters,qmatmul, and dequantization. - LLM serving primitives: KV-cache regions with mandatory memory budgets and sliding-window eviction, gradient checkpointing, differential-privacy budget tracking, simulated tensor parallelism, and LoRA adapter hot-swap that survives hot reload via
persist.
Honesty note: checkpointing, DP, and tensor parallelism are partial (interpreter-level or single-rank simulation today); see Feature Status.