Back to feed

Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations

Engineering at Meta

Oct 12, 2026

10/12/2026

Meta Uses Two Elastic Execution Modes To Balance CPU And Accelerator Resources For Inference

Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations · Engineering at Meta

Science, Technology & Innovation · Oct 12, 2026

Meta uses two complementary elasticity controls: Rank More Ads consumes both CPU and accelerator capacity by expanding candidate retrieval, while Serve Better Models primarily consumes spare accelerator capacity by routing selected requests to stronger models; capacity is periodically shifted between them based on impact and resource availability, so elastic systems should support multiple workload modes rather than one generic burst.


10/12/2026

Fast Local Admission Control Is A Mandatory Backstop To Fleet-Level Controllers For Latency-Sensitive AI Systems

Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations · Engineering at Meta

Science, Technology & Innovation · Oct 12, 2026

Meta’s safety design combines physical or logical isolation, slow fleet-level optimization, and millisecond-scale local shedding to protect baseline latency-sensitive service during sudden capacity loss while still exploiting unreliable opportunistic capacity.


10/12/2026

Idle Inference Capacity Is Treated As A Preemptible Elastic Resource Managed By Workload Scheduling To Protect Baseline Reliability

Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations · Engineering at Meta

Science, Technology & Innovation · Oct 12, 2026

Meta uses fragmented idle inference capacity as preemptible, opportunistic compute for optional workloads that can rapidly scale down without affecting baseline reliability. This approach depends on workload elasticity, graceful fallback, and measurable quality gains across varying compute levels—not merely high hardware utilization.