llm-d 1.0 Advances Open, Cost-Efficient AI Inference for Production Scale
IBM-backed open-source project llm-d released version 1.0, a hardened core for distributed AI inference now governed by the CNCF. IBM co-founded the project with Red Hat, Google Cloud, CoreWeave and NVIDIA, and IBM Research demonstrations claim up to 11.5x lower inference cost and 5-10x savings versus commercial API pricing on agentic workloads.
Key figures
- Contributors
- 500
- Throughput claim
- 3x greater output throughput
- Glm 5 2 parameters
- approximately 753 billion
- Cost reduction claim
- 5 to 10 times vs commercial API pricing
- Prefix cache hit rate
- approximately 98%
- Concurrent coding agents
- 3000
- Participating organizations
- 25
- Peak output tokens per minute
- 6600000
- Blended serving cost per million tokens
- 0.11
- Quantum error correction inference cost reduction
- 11.5 times
AI analysis
The rest of the AI analysis, red flags and sentiment are part of Signal8 Pro.
AI-generated analysis of a public disclosure. Not investment advice; verify against the original document.