KEY POINTS
- Meta's agents ran 1,097 experiments in 31 days and had 325 accepted, a 29.6% acceptance rate.
- The campaign used about 360,000 GPU-hours and raised search-session volume by 0.23% in live traffic.
- The final model rendered Meta's search interface more slowly, and the size of the slowdown was not disclosed.
Meta researchers used AI agents to run 1,097 machine-learning experiments over 31 days, consuming roughly 360,000 GPU-hours to lift search activity by 0.23%, CryptoSlate reported. The work is described in a research paper dated Nov. 18.
The team built a system in which agents propose changes to machine-learning models, edit the code, train new versions and evaluate the results. It was applied to Meta's search-suggestion system, which decides what users see when they open the search bar before typing anything.
Normally, engineers pick candidate changes, implement them and judge them through repeated training and testing. In Meta's setup, much of that loop was automated and many experiments ran at once, while engineers kept control over research priorities and final deployment decisions.
According to the report, the system relied on Anthropic's Claude Opus 4.8 to explore architectural improvements and generate candidate changes. Of the 1,097 experiments, 325 were accepted, a rate of 29.6%. An earlier version that evaluated changes one after another had an acceptance rate of 15.6%. The researchers also said the agents found several architectural improvements that Meta's engineers had not explored before.
The compute bill was large. Meta estimated the total at about 360,000 GPU-hours, a figure that includes completed experiments, extra tests to measure training variability, and failed or retried jobs. Because single training and evaluation runs took several hours, running experiments in parallel was necessary to finish within a month.
After the accepted changes were merged into one model, Meta compared it with its existing search-ranking system on live traffic. Search-session volume rose by 0.23%, with a 95% confidence interval of plus or minus 0.15 percentage points. Other engagement measures, such as participation and retention, also moved in a positive direction. CryptoSlate noted that on a platform handling more than a billion search-ranking requests a day, a small gain could still matter operationally.
There was a drawback, though. The new model made the search interface slower to render, and the researchers did not say by how much. The agents had optimized against offline measures of search engagement, while interface rendering speed was tracked separately. The researchers acknowledged the system could not identify every trade-off between its assigned targets and other parts of the user experience.
Meta did not disclose the financial cost of the campaign or compare its resource use with that of a conventional engineering team. The researchers argued the results show that autonomous agents can do meaningful machine-learning engineering on production-scale systems. The open question for Meta, as the report frames it, is whether the extra search activity and engineering productivity justify the computing use and slower interface before wider adoption.