Skip to content
NewsResearch

Google open-sources HEIR, a compiler that runs ML inference on fully encrypted inputs

· by Pondero Newsdesk

The short version

Google released HEIR on August 14, 2026, an MLIR-based open-source compiler that converts pre-trained AI models to run inference on fully homomorphic encrypted data, so the server never decrypts what it processes.

Google open-sources HEIR, a compiler that runs ML inference on fully encrypted inputs

Until now, running a machine learning model in the cloud required decrypting user data before the model could touch it. Google's HEIR compiler, released as open source on August 14, 2026, removes that requirement: it converts pre-trained models to operate on fully homomorphic encrypted inputs, so the server processes ciphertexts and returns encrypted results without the underlying data ever being exposed.

What

HEIR (Homomorphic Encryption Intermediate Representation) is an MLIR-based open-source compiler toolchain that takes pre-trained AI models and transforms them to run on fully homomorphic encrypted (FHE) data. With standard cloud inference, the service provider must decrypt user data to compute on it. HEIR-compiled models skip that step entirely. Per the Google Security Blog post by Jeremy Kun, published August 14, 2026, the project is part of Google's Private Computing Toolkit and the code is available at github.com/google/heir.

Google shared four live FHE inference demonstrations compiled with HEIR: a deep learning recommendation model (with Belfort Labs, LG, and NYU), a credit card fraud detector (with Niobium and hardshell.ai), a network intrusion detector using the Kitsune anomaly-detection model, and a hotword detector that processes audio without exposing the recording to the server. Latency figures are reported for single-threaded CPU. Per the Google blog post, four peer-reviewed publications have been built on the HEIR platform, and the project has academic collaborations with Carnegie Mellon, Georgia Tech, UC Santa Barbara, and four other universities.

The underlying compiler design is documented in arXiv:2508.11095, a paper by ten Google researchers submitted in August 2025. Hardware acceleration partners include Belfort, Niobium, Cornami, and Optalysys.

Why it matters

The FHE model solves a specific compliance exposure that blocks cloud AI adoption in healthcare, finance, and any sector with strict data-residency rules: the service provider's server has always had to see plaintext to run the model. HEIR's approach removes that window. The tradeoff Google acknowledges is cost: per the blog post, FHE inference carries "nontrivial cost overhead" compared to plaintext inference, and the published benchmark figures are single-threaded CPU results with no comparison against GPU or TPU throughput. The current version is better framed as a research-grade foundation than a production swap-in for existing pipelines.

For teams evaluating privacy-preserving inference options, the signal is that Google is now offering FHE as a supported path inside its Private Computing Toolkit rather than a research curiosity. Google's stated goal, per the blog, is to make HEIR a "one-click solution to enable non-experts to incorporate encrypted inference into production applications." The four demo use cases show the capability exists at meaningful model sizes, but real-world latency comparisons against plaintext pipelines remain unpublished.

What to watch next

Two milestones will indicate whether HEIR moves from toolkit component to production path. First, whether Google integrates it into Vertex AI or Cloud TPU inference with published latency-versus-plaintext figures. Second, whether the hardware acceleration partners (Belfort, Niobium, Cornami, Optalysys) publish throughput numbers that close the cost gap enough for latency-sensitive workloads outside batch processing.

Sources