RePAIR projet: Robots use AI to restore Pompeii's shattered Frescoes

Pompeii's ancient Roman frescoes, long shattered and buried, may soon receive a second life thanks to an innovative robotic system aimed at assisting archaeologists with the intricate task of reassembling fragmented artifacts. This technology is part of the EU-funded project called RePAIR, which combines advanced image recognition, AI-driven puzzle-solving, and ultra-precise robotic hands to enhance restoration efforts that are typically slow and laborious.

Launched in 2021 and coordinated by Ca' Foscari University in Venice, the project was showcased in Pompeii, bringing together international research teams to test the technology at this historic site. According to Gabriel Zuchtriegel, the director of Pompeii, the initiative arose from a pressing need to recombine fragments of frescoes that had been damaged during World War II.

The robotic system employs twin arms fitted with flexible hands of varying sizes and vision sensors to identify, grip, and assemble fragments without harming their delicate surfaces. Researchers are optimistic that this technology could revolutionize restoration practices around the globe.

Pompeii, located near Naples, was buried under volcanic ash when Mount Vesuvius erupted in AD 79. The current focus is on frescoes stored in Pompeii's storerooms, particularly two large ceiling paintings that were damaged during the eruption and later shattered by bombings, as well as frescoes from the House of the Gladiators, which collapsed in 2010. During the testing phase, replicas of the original pieces were created to ensure their preservation.

While robotics teams engineered the system, experts in artificial intelligence and machine learning developed algorithms that help reconstruct the frescoes by matching colors and patterns that might be invisible to the human eye. The challenge of reassembly is akin to solving a massive jigsaw puzzle, complicated further by missing pieces and the absence of a reference image for the end result.

Marcello Pelillo, the project coordinator from Venice University, likens the task to mixing multiple jigsaw puzzles together, discarding their original boxes, and attempting to solve them simultaneously without clear guidance on what the final images should look like. The project's success could pave the way for more efficient restoration techniques in archaeology and beyond.