#inference

lyogavin/airllm

AirLLM 70B inference with single 4GB GPU

antirez/ds4

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

Predictive Speculative KV Replication for Bursty LLM Inference

ChatGPT Optimizes Its Agent Loop: Harness, API, and Inference

Kimi K3 Now Available via Telnyx Inference API

Ask HN: HotPin – lossless 120B MoE inference on 24GB RAM (CPU, 50 loc)

Hetzner is working on LLM Inference

AMD and Cerebras Launch AI Inference Solution

Show HN: Flow Matching model inference in C

kvcache-ai/ktransformers

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

handy-computer/transcribe.cpp

ggml speech-to-text inference for 16+ model families

Homomorphically encrypted CIFAR-10 inference in 200ms

Zro

Private inference for coding agents

Accelerating Block Low-Rank Foundation Model Inference on MemoryConstrained GPUs

Performant C/CUDA inference engine for Qwen 3.6 35B on RTX 5090 / Blackwell

Litert.js, Google's High Performance Web AI Inference

Show HN: Reame – a CPU inference server that gets faster as it runs

Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

AI inference is obviously profitable