Harvard Weighs $25 Million Investment to Double University-Wide AI Computing Capacity

Harvard is considering putting roughly $25 million from a new centrally administered fund toward computing infrastructure to significantly boost the University’s computing capacity, according to two people familiar with the planning.

The money is penciled in to purchase a cluster of GPUs roughly the same size as the existing cluster at the Kempner Institute for the Study of Natural and Artificial Intelligence, according to Bernardo L. Sabatini ’91, who co-directs the institute. Sabatini estimated the investment could roughly double the University’s AI computing capacity.

While the Kempner cluster — roughly 1,100 GPUs — prioritizes access for its own affiliates through a tiered system, the new GPUs would be “a University resource for all,” Sabatini wrote. Decisions about which GPUs to purchase and how they are used would be made centrally rather than by Kempner, he added.

A second person, granted anonymity to discuss plans that are not public, confirmed a similar figure had been discussed. Both Sabatini and the person said the number was not finalized but had circulated in recent talks about how the University plans to allocate a recently announced $50 million investment in cutting-edge scientific research.

A University spokesperson declined to comment on the size or scope of the computing investment. The spokesperson referred The Crimson to a Sept. 28 announcement in the Harvard Gazette that said the money would support servers, computational capacity, cloud computing, and leading AI models.

The $50 million fund is the centrally administered portion of a $150 million research initiative by Harvard President Alan M. Garber ’76 announced last week, with the remaining $100 million to be distributed by Harvard’s schools. The $50 million fund will support research in areas including neuroscience, immunology, energy and climate, and industry-oriented research, as well as computing infrastructure. The University has not publicly detailed how that money will be divided.

“Half of it is compute — is going to go to computing,” Sabatini said of the $50 million pot. “I don’t know that that’s a firm number, but that’s what I’ve heard bandying about.”

If Harvard moves forward with the purchase, it would echo a bet Kempner made several years ago: that it was better off building out its own infrastructure than renting computing power from cloud providers.

Sabatini called the proposed investment of roughly $25 million in GPUs “tremendously important” and a “significant investment for academia.”

Kempner has poured money into building one of the largest academic clusters in the country since its launch in fall 2022, propelled by a $500 million gift from Meta CEO Mark E. Zuckerberg and Priscilla Chan ’07.

A 2024 Yale University task force report, citing a senior Harvard faculty member, said $150 million of the gift, which is paid out over 15 years, was allocated to computing. That figure includes salaries for engineers over the 15-year time frame, Sabatini wrote. A University spokesperson declined to comment on the gift’s parameters.

The Kempner cluster runs on 1,144 Nvidia GPUs — including 424 H200s and 384 H100s, the high-end chips used to train large AI models. The institute more than doubled the cluster’s size in an expansion announced in March.

“The University certainly realizes at the highest levels that they need to invest in machine learning and AI capacity for faculty to be able to do what they need to do,” Sabatini said. “That’s happening, and this is the first example of that.”

The proposed investment follows months of signals from University leadership that Harvard plans to lean hard into AI. Garber said in July that scientific researchers could not stay at the forefront of their fields without incorporating artificial intelligence, and he argued in a September address that AI would enable research to progress “more rapidly than at any other time in human existence.”

Max Shad, Kempner’s senior director of research engineering, said the institute had not received a plan for how it would “operationalize” the investment, but that there had been conversations for years about the need for more centrally coordinated research computing infrastructure.

“I think they are working on that to see how they can team up and benefit from central computing resources. And of course, Kempner plays an important role because we have the largest AI cluster in University,” Shad said.

Shad said there were certain advantages to the “decentralized” computing model Harvard had pursued, in which schools across the University sign agreements with Kempner to use computing capacity and Harvard Medical School maintains its own cluster with higher security for sensitive medical data.

Harvard has already begun developing a Central GPU Cluster to serve researchers across the University. But Shad argued greater central coordination was long overdue, allowing researchers to share data, work on joint projects, and avoid duplicating work across research computing centers.

“The most important concern to cover is this disaggregated research computing infrastructure,” Shad said. “How University is going to create a central resource, or act, or coordinate with other centers to provide a seamless and smooth access for people to do collaboration across campus.”

Sabatini said demand for GPUs around the University was “essentially infinite” and predicted that a larger, more broadly accessible cluster would let researchers harness, if needed, a significant number of GPUs in monthslong projects. Kempner’s standard allocation policies currently limit each user to 16 GPUs at once and each account, which could span multiple users, to 96 GPUs.

“That’s what’s really big about these,” Sabatini said. “What’s important about these big clusters is that it suddenly allows you to tackle that kind of problem that now has largely been absent from academia, definitely absent from Harvard.”

—Staff writer Elise A. Spenner can be reached at [email protected] or on Signal at elisespenner.82. Follow her on X @EliseSpenner.