Harvard IT Launches Pilot of AI Sandbox to Enable ‘Walled-Off’ Use of Large Language Models

Harvard University Information Technology began a limited rollout of the pilot version of its artificial intelligence “sandbox” tool on Sept. 4, with the aim of providing Harvard affiliates with secure access to large language models.

The AI sandbox provides a “walled-off” environment where prompts and data entered into the interface are seen by the user only — the data is not shared with LLM vendors and cannot be used as training data for these models, according to a press release on HUIT’s website.

HUIT designed the tool in collaboration with the Office of the Vice Provost for Advances in Learning, the Faculty of Arts and Sciences Division of Science, and other colleagues across the University. HUIT spokesperson Tim J. Bailey wrote the pilot aims to encourage safe experimentation with LLMs, inform how Harvard can offer increased accessibility of the tools, and explore applications of AI in the classroom and workplace.

Pilot access is being rolled out in phases, with access expanded to instructors within the FAS two weeks ago. Still, affiliates must request access to use the AI sandbox for each specific use case.

Harvard Business School professor Mitchell B. Weiss ’99 was one of the inaugural participants in the pilot AI sandbox.

Weiss said the AI sandbox was a crucial educational resource in his course HBSMBA 1623: “Public Entrepreneurship.” He praised the AI sandbox for offering easy access to a range of generative AI models.

Weiss, by incorporating AI into his course, said he hoped to provide insight into the broader question of, “How can generative AI be useful in helping solve public problems?”

“The interest is spreading as examples of uses for teaching and learning spread,” Weiss said.

Feedback from the pilot, obtained through surveys and discussions with participants, will be shared among faculty and University leadership to advise Harvard’s strategy toward AI, according to Bailey.

The University has continued to negotiate with vendors on enterprise agreements that can help expand the variety of consumer AI tools available on the platform, per Bailey.

Weiss said he looks forward to future versions that include the ability to upload files — like data and PDFs — and the “advanced data analysis tool in GPT-plus.”

As generative AI has gained traction on campus and in the world, the Office of Undergraduate Education has rolled out guidance for faculty approaches regarding generative AI use in FAS courses. These guidelines range from “maximally restrictive” to “fully-encouraging,” though the FAS has not imposed a blanket policy on AI use.

Weiss said the pilot version of the AI sandbox has been received positively by his students.

“Oh, this changes my whole job,” Weiss said, quoting a conversation between two students in his class.

“They really saw the magnitude of these tools in a way they hadn’t. I think using them is a very important way to understanding them,” Weiss added.

—Staff writer Camilla J. Martinez can be reached at camilla.martinez@thecrimson.com. Follow her on X @camillajinm.

– Staff writer Tiffani A. Mezitis can be reached at tiffani.mezitis@thecrimson.com.