← Back
News · September 14, 2026

Google and Inferact Announce Engineering Partnership to Make TPU a First-Class Target for Open Inference

Google Cloud and Inferact align on one engineering roadmap to bring production-grade vLLM inference to TPU, with all resulting work upstreamed to open source

San Francisco, September 14, 2026 — Google Cloud, the cloud platform behind the Tensor Processing Units (TPUs) that power Gemini and frontier AI workloads at scale, and Inferact, the company founded by the creators of vLLM to build the production inference layer for AI, today announced a joint engineering and go-to-market partnership to make TPU a first-class citizen in vLLM. The partnership puts both teams on one roadmap across serving features, model coverage, and market adoption, and ensures that everything the collaboration produces is open sourced.

Engineers from both teams are already working to that shared roadmap. The results will reach anyone deploying on TPU, from faster inference as kernels and frameworks improve to broader model and TPU generation support at launch and a shorter path to production.

From Frontier Training to Open Inference

vLLM's job is to run models efficiently on the best hardware available for the workload. TPU is proven at frontier scale: Google trains and serves Gemini on it, Anthropic has trained and served its models on it for years and committed last October to expand to as many as one million chips, and Ironwood, the latest generation, was built specifically for inference.

Inferact is the company behind vLLM, the most widely adopted open source inference engine, which supports more than 5,000 model architectures, counts a community of over 3,000 contributors, and runs a large share of the world's inference traffic. Inferact and Google are investing in vLLM as the path to open TPU to that broader model and enterprise ecosystem for production inference.

Bill Jia, Vice President of Engineering, Core ML/AI, Google: “Inferact’s inference expertise, combined with the team’s depth on vLLM and PyTorch, ensures that AI models achieve optimal inference performance on TPU. We are excited to partner with Inferact to build out the TPU ecosystem, enabling ML developers and engineers to leverage TPUs for superior production performance and TCO.”

Where vLLM on TPU Stands Today

vLLM runs on TPU today, across current and earlier generations. The current TPU backend is tpu-inference, which runs a broad set of production serving features on Ironwood, Trillium, and earlier hardware and supports a wide range of leading models, including Gemma, Kimi, and Qwen. Support status is published openly, with correctness and performance testing across available models, and the current support matrix is at github.com/vllm-project/tpu-inference. While tpu-inference has served users well, community feedback has shaped our roadmap toward a native PyTorch approach via TorchTPU for an even more streamlined and performant experience.

What the Partnership Is Building

The partnership's goal is to optimize vLLM on TPU for production serving. The work spans agentic workloads, production serving features, kernel performance, and more.

Agentic workloads. Agents now account for a large share of production traffic, and they stress a serving engine differently than chat does. An agent replays a long conversation on every turn, carries tool output and screenshots forward, and pushes contexts into the hundreds of thousands of tokens. Serving these workloads efficiently requires unique optimizations, such as prefix caching, and the partnership is building them as core serving infrastructure on TPU.

Production serving features. vLLM's production features include prefill/decode disaggregation, speculative decoding, KV cache offload, and structured output and tool calling. The goal is for these features to compose interchangeably on TPU, optimized and validated both separately and in combination.

Woosuk Kwon, Co-Founder & CTO, Inferact | Creator and Lead Maintainer, vLLM: “Supporting a new accelerator on vLLM is not one project, it extends across every project: attention kernels, MoE routing, collectives, quantization, cache management. Inferact engineers and Google’s TPU team are working through that combinatorial matrix together, and each model we bring up makes the next one cheaper. That compounding is what turns ‘it runs’ into ‘it is optimized on day zero.’”

Optimized open-source kernels. Kernel performance sets the latency floor for every request. The joint engineering work covers attention, MoE, quantization, and collectives, and all of it goes upstream into open source for anyone to use and build on.

Native PyTorch. Google introduced TorchTPU in April, a stack that makes TPU a native PyTorch device, and named vLLM among its roadmap integrations. vLLM is a PyTorch codebase, so a native PyTorch backend means that supporting a new model on TPU stops being a TPU-specific project. Inferact and Google are building a first-class TorchTPU path on vLLM, available through an updated repo in the coming weeks, while the current tpu-inference backend remains available and supported today.

Qi Zhou, Senior Staff Software Engineer, Google TPU: “The fastest way to put Ironwood in developers’ hands is through the framework they already use. Our TPU team is working alongside Inferact and the vLLM maintainers to land TPU-native attention and MoE kernels directly into open-source vLLM, where anyone can read them, improve them, and build on them. There is a lot of headroom left, but we are confident that we will get there.”

Day-0 support. The partnership's goal is frontier inference on TPU from day zero for each major model release. That capability gets built by partnering with model vendors ahead of release and bringing frontier models up one at a time, until the path from announcement to production serving is routine.

Kimi K3 is a headline example. Inferact worked with Moonshot AI ahead of the K3 release, co-designing the cache handling its hybrid attention stack needs, and vLLM served the model on day zero. Inferact and Google are now bringing K3 to TPU, including its hybrid attention, optimized throughput, and low-latency decode path. MiniMax M3 followed the same pattern: Inferact and MiniMax collaborated to serve the model on day zero, and Inferact and Google are now hardening it on TPU.

A New Community Program

Hardware alone does not serve AI. The software ecosystem around it makes the hardware reachable, and building that ecosystem for TPU is the point of this work. Inferact and Google are launching a community program with two commitments at its center.

Chips for contributors. A shared pool of TPU capacity for open-source contributors to develop and test work on the vLLM TPU backend.

Direct support on pull requests. Dedicated review and design help for TPU contributors from the Core vLLM Maintainers and Contributors at Inferact.

Progress will be public. The release support matrices show where help is needed, and TPU results will be published continuously on ci.vllm.ai. Inferact and Google are aligned on accelerating inference wherever TPU runs, and this program is how the developer ecosystem around that work gets built. A healthy, open ecosystem for TPU inference software is what unlocks the next wave of innovation.

Get Involved

Simon Mo, Co-Founder & CEO, Inferact: “vLLM’s job is to run well on the best hardware for the job, and that only holds when every inference implementation has a joint team behind it. Inferact and Google are aligned on making that true for TPU in vLLM, working in the same repositories and upstreaming to open source. Our goal is to make vLLM on TPU optimized and production ready for the whole community.”

About Google Cloud

Google Cloud offers a powerful, optimized AI stack — including AI infrastructure, leading models like Gemini, data management capabilities, multicloud security solutions, developer tools and platform, as well as agents and applications — that enables organizations to transform their business for the Agentic Era. Customers in more than 200 countries and territories turn to Google Cloud as their trusted technology partner.

About Inferact

Inferact is building the production inference infrastructure for AI, enabling companies to serve optimized inference on their infrastructure. Founded by the creators and core maintainers of vLLM, the world's most widely adopted open-source LLM inference engine, Inferact makes it faster, cheaper, and easier to serve frontier AI models at scale. vLLM supports more than 5,000 model architectures and 1,000 accelerator types and is used by frontier labs, hyperscalers, and startups around the world. Learn more at inferact.ai.