Generative AI ‘fools’ scientists by bringing in artificial data

The same AI technologies used to mimic human art can now synthesize artificial scientific data, advancing efforts towards fully automated data analysis.

Researchers at the University of Illinois Urbana-Champaign have developed an AI that generates artificial data from microscopy experiments commonly used to characterize the atomic-level structure of materials. Drawing from the underlying technology of state-of-the-art generators, AI allows researchers to incorporate background noise and experimental imperfections into the resulting data, enabling material features to be detected faster and more efficiently than ever before.

“Generative AI takes information and generates new things like never before in the world, and now we are leveraging it for automated data analysis purposes,” said Pinshane Huang, professor of materials science and engineering and co-lead of the project. “What used to get Monet’s llama paintings on the internet can now make scientific data so good it fools me and my colleagues.”

Other forms of AI and machine learning are routinely used in materials science to aid data analysis, but require frequent and time-consuming human intervention. Making these analysis routines more efficient requires large sets of labeled data to indicate what programs to look for. In addition, data sets need to account for various background noise and experimental imperfections to be effective, an effect that is difficult to model.

Because collecting and labeling such large data sets using a real microscope was impossible, Huang worked with U. of I physics professor Bryan Clark to develop a generative AI that could create large sets of artificial training data from relatively small real data sets. , labeled data. To achieve this, the researchers used a generative cycle adversarial network, or CycleGAN.

“You can think of CycleGAN as a competition between two entities,” says Clark. “There are ‘generators’ whose job is to imitate the provided data set, and there are ‘discriminators’ whose job is to find differences between the generator and the actual data. They take turns trying to fail each other, improving themselves based on what the other side is capable of. In the end, generators can generate artificial data that is nearly indistinguishable from real data.”

By providing CycleGAN with small samples of real microscope images, the AI ​​learns to generate images that are used to train analysis routines. Now able to recognize various structural features despite background noise and systematic imperfections.

“The great part of this is we never have to tell the AI ​​about things like background noise and imperfections like aberrations in the microscope,” said Clark. “This means that even if there is something we haven’t thought about, CycleGAN can learn it and run with it.”

Huang’s research group has incorporated CycleGAN into their experiments to detect defects in two-dimensional semiconductors, a class of materials that hold promise for applications in

electronics and optics but difficult to characterize without AI assistance. However, he observed that this method had a much wider reach.

“His dream is to one day have a ‘self-propelled’ microscope, and the biggest obstacle is understanding how to process the data,” he says. “Our work fills this gap. We show how you can teach a microscope how to find interesting things without having to know what you’re looking for.”

***

The study, “Utilizing generative adversarial networks to create realistic scanning transmission electron microscopy images,” was published in the journal NPJ Computing Materials.

Huang is a Racheff Faculty Scholar and an affiliate of the Materials Research Laboratory at the U. of I. Clark is an affiliate of the Center for Artificial Intelligence Innovation at the National Center for Supercomputing Applications and the Illinois Center for Quantum Science and Information Technology at U. of I.

This work was supported by the US Department of Energy.