This past spring, the world experienced a rare moment of collective joy and awe when NASA’s Artemis II mission, the first crewed flight to the Moon since 1972, completed its historic lunar flyby. On April 6, astronauts Reid Wiseman, Christina Koch, Victor Glover and Jeremy Hansen flew farther from Earth than any humans before them. Now, as NASA prepares for the next Artemis mission and beyond, the space agency is working with IBM to give scientists new tools to study the Moon.
On Thursday, the two organizations released the NASA-IBM Lunar Foundation Model. It’s an open-source AI system that’s available to download from Hugging Face. As a foundation model, it can do a few different things, says Dr. Juan Bernabé-Moreno, the director of IBM Research Europe, UK and Ireland. To start, NASA and IBM found it was particularly adept at identifying areas on the lunar surface where there might be ice.
The two organizations ran a test where they compared the model’s predictions against a map they made using a published scientific workflow that incorporates terrain, thermal and other environmental data. They then pitted the model against SwinV2-B, a Microsoft-trained vision system for processing high-resolution images. The NASA-IBM model reduced errors by 23 percent. NASA and IBM compared the model against SwinV2-B because the tool is often used as the baseline for many image analysis tasks.
NASA and IBM also benchmarked their model’s ability to identify and classify craters. There, the new model outperformed SwinV2-B by 19 percent while using half the training data. NASA and IBM got a chance to verify the model’s capabilities recently when a SpaceX Falcon 9 rocket crashed into the Moon on August 5. When IBM fed an image of the impact to the model, it correctly identified the crash site as a new crater, despite it closely overlapping with an existing crater. “It worked fantastically,” says Bernabé-Moreno, with the model nailing the identification on its first attempt.
IBM
Bernabé-Moreno says training the model was a challenge. “In Earth observation, you have pristine images,” he explains. Our planet’s atmosphere scatters sunlight, filling shadows with ambient light that softens their edges. Shadows on the Moon instead appear knife-edged and pitch black, meaning any time they’re photographed, the shadowed pixels don’t carry any information. “On the Moon, what you have is the sun at different moments during the day, and it illuminates and it cuts shadows, and you need to deal with those shadows,” he says. The result is that a crater can look one way in one image and completely different in the next image.
Another challenge came as a result of how computer vision models are typically trained and tested. Most often, that process involves removing some parts of an image and then tasking the model with reconstructing what’s missing based on the patterns it has learned in training. “You might remove 90 percent of a crater, and with [the remaining] 10 percent, you ask the model to reconstruct the other 90 percent,” explains Bernabé-Moreno. That doesn’t work for the Moon. As you might imagine, the reason for that is that a lot of craters look very similar to one another when photographed from orbit.
In fact, IBM’s attempts to train the model in the traditional way were a “complete disaster,” says Bernabé-Moreno, but the research team eventually devised a creative solution. “We took the Moon and divided it into wedges, like an orange, and we completely separated the training wedges from the testing wedges. This way, we could give the model a lot of consistency,” he says.
One of the reasons NASA and IBM are excited about this release is that it includes a first-of-its-kind open-source dataset that other researchers can use to build future models. It incorporates tens of thousands of images and instrument data from NASA’s Lunar Reconnaissance Orbiter (LRO) and Gravity Recovery and Interior Laboratory (GRAIL) missions, as well as Japan’s Selenological and Engineering Explorer (SELENE).
“One of the reasons we’ve never seen a comprehensive, powerful lunar model before is that we didn’t have the data [organized] in the right way,” says Bernabé-Moreno. Organization in this case meant creating a grid where every pixel of imaging data corresponds to other modalities of data a model can use to inform its processing. “That alone is a massive scientific contribution, because put the model aside: the community now has a co-registered dataset with more than two million [data] points,” says Bernabé-Moreno. “And we know that AI models get superseded by other models. The data is what really creates the industry of AI models.”
