BLOG
The latest news and updates from the Inference.net team.
Michael Ryaboy
Jul 31, 2025
GPU-Rich Labs Have Won: What's Left for the Rest of Us is Distillation
massive training runs and powerful but expensive models means another technique is starting to dominate: distillation
Amar Singh
Jul 29, 2025
On the Economics of Hosting Open Source Models
The open source community is buzzing around the new Wan release, but what are the economics of the businesses hosting it right now? Or hosting open source models in general?
Michael Ryaboy
Jul 24, 2025
Batch vs Real-Time LLM APIs: When to Use Each
Not every LLM request needs an immediate response. Chat interfaces need real-time. But data extraction, enrichment, and background jobs can wait hours.
Michael Ryaboy
Jul 22, 2025
Do You Need Model Distillation? The Complete Guide
Model distillation is particularly valuable in scenarios where large models are impractical due to resource constraints or performance requirements.
Michael Ryaboy
Jul 21, 2025
The Cheapest LLM Call Is the One You Don’t Await
Asynchronous requests – fire‑and‑forget calls that finish whenever idle GPUs are free.
Michael Ryaboy
May 31, 2025
Osmosis-Structure-0.6B: The Tiny Model That Fixes Structured Outputs
We're excited to announce that Osmosis-Structure-0.6B is now available on the Inference.net platform alongside our comprehensive DeepSeek R1 family.
Michael Ryaboy
May 29, 2025
How Smart Routing Saved Exa 90% on LLM Costs During Their Viral Moment
They thought of a clever solution that saved them 90% on tokens: route people with the most followers to Claude, and everyone else to dirt cheap open-source models
Sean
May 1, 2025
Migrating our Website and Dashboard to TanStack Start
We evaluated a few frontend frameworks and eventually settled on TanStack Start as the tool of choice to re-implement are dashboard and website. In particular, we wanted a flexible solution that would allow us to server-render static content while also powering a rich, JS-heavy client side application.
Sam Hogan
Feb 19, 2025
Introducing Inference.net
Inference.net is a global network of compute providers delivering affordable, serverless inference for the top open source AI models. We built a distributed infrastructure that allows developers to access state-of-the-art language models with the reliability of major cloud providers—but at a fraction of the cost.
'%20stroke='url(%23paint0_linear_1744_28313)'%20stroke-width='1.5'/%3e%3cg%20clip-path='url(%23clip0_1744_28313)'%3e%3crect%20x='-0.75'%20y='-0.75'%20width='30.4397'%20height='30.4397'%20rx='15.2199'%20transform='matrix(-4.37114e-08%20-1%20-1%204.37114e-08%2075.4219%2030.4399)'%20stroke='url(%23paint1_linear_1744_28313)'%20stroke-width='1.5'/%3e%3c/g%3e%3crect%20x='-0.75'%20y='-0.75'%20width='30.4397'%20height='86.8666'%20rx='15.2199'%20transform='matrix(-4.37114e-08%20-1%20-1%204.37114e-08%20131.848%2030.4399)'%20stroke='url(%23paint2_linear_1744_28313)'%20stroke-width='1.5'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_1744_28313'%20x1='31.4122'%20y1='-0.255341'%20x2='30.4937'%20y2='39.1623'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20offset='0.04'%20stop-color='%23FFD100'/%3e%3cstop%20offset='1'%20stop-color='%23F03000'/%3e%3c/linearGradient%3e%3clinearGradient%20id='paint1_linear_1744_28313'%20x1='31.9397'%20y1='40.5636'%20x2='2.79196'%20y2='-13.3446'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2341C0F9'/%3e%3cstop%20offset='0.98'%20stop-color='%231024C2'/%3e%3c/linearGradient%3e%3clinearGradient%20id='paint2_linear_1744_28313'%20x1='39.4078'%20y1='98.4798'%20x2='-21.6781'%20y2='12.168'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2341C0F9'/%3e%3cstop%20offset='0.98'%20stop-color='%231024C2'/%3e%3c/linearGradient%3e%3cclipPath%20id='clip0_1744_28313'%3e%3crect%20width='31.9397'%20height='88.3666'%20rx='15.9699'%20transform='matrix(-4.37114e-08%20-1%20-1%204.37114e-08%20133.348%2031.9399)'%20fill='white'/%3e%3c/clipPath%3e%3c/defs%3e%3c/svg%3e){kind=small}