Laboratory researchers converge to test and train scientific AI models

Scientists from New Mexico’s two national laboratories converged to test new artificial intelligence tools that may drive science and technology in the future. Nearly 250 researchers and professionals from Los Alamos and Sandia National Laboratories set conference rooms buzzing with AI-fueled investigation and collaboration as New Mexico’s contribution to the 1,000 Scientist AI Jam Session, the first such event in a nationwide Department of Energy initiative, to gather data on AI model performance across a diverse range of tasks in scientific research and development.

“This is the first time more than a thousand scientists from across national labs worked to understand the power of frontier AI for our most important science and security challenges,” said Jason Pruet, director of Los Alamos’ National Security AI Office. “Based on the energy in the room, the answer seems clear: With these technologies we can change the slope of progress in the country. 

“Partnership must be at the forefront of our approach to AI,” said Kevin Dixon, director of Applied Information Sciences at Sandia. “The pace of change is rapid, and these developments are uncharted territory for all of us. Given the swift evolution of AI, no single team or laboratory possesses all the answers. As national laboratories, we are uniquely positioned to embrace a co-design philosophy that enhances both our hardware and software capabilities.”

Researchers came from across a wide variety of disciplines at both laboratories in everything from physics, to computational science and statistics, to documentation and support services. Participants were given access to AI models from Anthropic and OpenAI, including each organization’s newest reasoning models — Anthropic’s Claude 3.7 and OpenAI’s o3-mini — AI applications intended to work by reasoning through complex information.  Individuals and collaborative teams brought problems from their scientific domain and spent the day using the AI models to sift through and solve their problems, gaining or renewing familiarity with the AI tools while also helping analyze the performance of the models.

Solving complex scientific problems quickly

Kipton Barros works on several specialized problems involving mathematical manipulations for theoretical physics. He uses AI tools already in some of this work, seeing it as particularly useful for pulling together different areas of knowledge.

“For me, large language models like this are valuable as a personal tool that complement the creativity and expertise a researcher brings,” Barros said. “We’re experts in our fields, but none of us can know everything about all of science. If I want to expand my research into new areas, these tools are a great way to learn quickly. Reliability of model output remains a concern, but with recent reasoning abilities, the situation is improving.”

Russell Bent, applied mathematician at Los Alamos, works on the challenges of optimization in systems, including the reliability of power grids. He focused his time during the AI Jam on the problem of how to formulate the modeling of natural gas systems from an optimization standpoint. He hoped to build useful code for models that show and assess different ways natural gas moves through pipelines, comparing his results on the AI models with other implementations he’s done.

“One of the tasks I gave the AI model was to summarize natural gas system optimization formulations from the academic literature, and the model correctly identified many of the formulations in the literature,” Bent said. “Once I asked it to actually generate code, it did a good job of implementing a code that could be integrated directly into the codebase. Based on my experience, I view AI as a very important tool for improving the efficiency of many of the tasks we do on a daily basis, whether it’s implementing particular pieces of code, modeling different aspects of the energy system, or doing extensive literature review.”

Los Alamos was joined by nearly 50 researchers and staff members from Sandia National Laboratories, a fellow National Nuclear Security Administration laboratory.

Ben Newton joined the AI Jam from Sandia. A geospatial data scientist, he studies trajectories, relying heavily on machine learning to track objects across the sea and sky. That kind of work involves looking for patterns and anomalies.

“I don’t use a lot of large language models in general, so what I’m trying to do today is to see how well it performs in some tasks related to aircraft trajectory data or vessel data,” Newton said. “The performance of the models, in many respects, is amazing.”

Research scientist Jonas Actor joined Newton from Sandia. As a member of the computational mathematics department in the Center for Computing Research, Actor works extensively with machine learning models.

“I know I now have a much better grasp of the abilities and limitations of large language models at the current moment,” Actor said. “Part of the goal here was to learn where the models are at, what the frontiers of their abilities are, and where science functions fall at the frontier. I have a much better sense of where that is now, and it should be encouraging for science.”

Similar AI jam sessions were held concurrently at other national labs throughout the U.S. — including participants from Argonne, Berkeley, Brookhaven, Idaho, Livermore, Los Alamos, Oak Ridge, Pacific Northwest, and Princeton Plasma Physics laboratories — bringing together hundreds of scientific minds on the same day whose research evaluated the latest AI models. The research will serve as an evaluation of the potential impact of the latest AI tools for the scientific community.

LA-UR-25-23634