Codebreakers Hope To Use AI to Unearth History's Well-Kept Secrets

More than 400 years ago, Mary, Queen of Scots, was imprisoned in an English castle on charges that she'd conspired against Queen Elizabeth I, her cousin. As she tried to save her head, Mary wrote dozens of secret letters to friends and allies. In several of them, she discussed with France's ambassador how rivals had abducted her son; in others she complained about conditions of her captivity and said she supported a controversial marriage that would've aided a political alliance between France and England.

Mary used a "spiral locking" technique to fold the letters, so the recipients would be able to tell if anyone had opened them before they were received. She and her associates also corresponded in a complex code, which protected the content for 436 years. Then in February, a group of codebreakers detailed in a paper how they'd designed a computer system to break it. In the paper, they showed how Mary had used a type of encryption known as a complex homophonic cipher, substituting various symbols for individual letters of the alphabet.

The researchers, part of a group of computer-savvy historians who call themselves the Decrypt Project, have spent the past decade poring over manuscripts from European archives, trying to solve hidden messages. The inspiration for the idea came in part from the 2011 cracking of the Copiale cipher, a 105-page collection of handwritten characters that German occultists wrote in the 1700s. Among the things depicted in those decoded manuscripts was an initiation ritual for a secret society that involved repeatedly asking recruits to read a blank piece of paper before plucking a single one of their eyebrows.

The Decrypt Project uses a combination of image analysis, computer algorithms designed to identify patterns and the researchers' own expertise in centuries-old languages to translate forgotten texts into readable formats. As technology improves, the people behind the project, along with other historians doing similar work, anticipate that artificial intelligence will help unlock the secrets of an increasingly wide range of centuries-old texts.

The Decrypt Project's long-term aspiration is to design a kind of Google Translate for the ages - a tool that could scan historical documents and translate them into modern English, regardless of their age, the language they were written in or the encryption they used. The prospects of building such a tool remain unclear, but historians are gaining confidence in their code-cracking abilities. "For historical texts we don't have a lot of uniform data because people wrote in different ways with different writing systems and different handwriting," says Beata Megyesi, leader of the Decrypt Project and a professor of computational linguistics at Sweden's Uppsala University. "We developed AI models to transcribe these systems, and that can make things more efficient."

History is full of coded mysteries, many of which can't be solved through decryption alone. One study is examining how the sophistication of the encryption the Roman Catholic Church employed when communicating with world leaders diminished between the 16th and 18th centuries. While the regression has been documented, the reasons behind it are unclear. One theory is that there was a group within the Vatican that specialised in coded messages, then failed to pass their techniques down to their successors.

In 2020, a group of mathematicians said they'd cracked the cipher that the Zodiac serial killer used when communicating with police officers in 1969. Like many coded messages, the notes used a complex substitution methodology, in which one or multiple characters were used in the place of the alphabet. The Zodiac killer used multiple symbols for each letter, varying which symbols he used in a way that made the code harder to break. Using custom-built software, the mathematicians found that, in one note, the killer had cycled through the symbols in a regular pattern, allowing them to break the code, says Kevin Knight, a professor of computer science at the University of Southern California, who was involved in the Copiale cipher decryption and analyzed some Zodiac correspondence.

The decoded message, which the FBI confirmed, said, "I hope you are having fun in trying to catch me." Other Zodiac letters remain encrypted, and the killer's identity is still a mystery. But Knight says the evolution of code-breaking techniques is giving historians a leg up in revealing long-kept secrets. "The historical significance is big," he says. "It's been an arms race since ancient times between the codemakers and the breakers."