Many restaurant industry experts believe a food-making robot revolution is inevitable—and researchers at the Massachusetts Institute of Technology (MIT) and the Quatar Computing Research Institute (QCRI) hope to lead the way through advances in artificial intelligence (AI) and machine learning.

In September 2018, PMQ reported on an AI program, called Strono, developed by MIT scientist Pinar Yanardag, who leads the How to Generate (Almost) Anything Project, and her team of AI geniuses. After learning hundreds of existing pizza recipes, Strono was trained to develop its own original artisan recipes. Weirdly, some of the ingredients Strono proposed don’t actually exist in the real world, such as “snipped caramel cheese,” but most featured legit topping combos, including shrimp, jam and Italian sausage.

Try out these pizza recipes dreamed up by a computer!

Adding to Yanardag’s research, the MIT/QCRI researchers have been working on a system called PizzaGAN (Generative Adversarial Network), a neural network that learns how to make pizza using photos of real pies downloaded from Instagram as well as pizza clip art. The AI system has been trained to identify individual toppings, distinguish layers of ingredients and determine the right order and distribution for placing them on the pizza.

The AI system doesn’t actually prep and bake a pizza, but it’s a good step forward in helping mechanical robots carry out those tasks based on recipes or photos.

The MIT/QCRI team has published a paper titled, “How to Make a Pizza: Learning a Compositional Layer-Based GAN Model.” In the abstract for the paper, the researchers summarized the project: “A food recipe is an ordered set of instructions for preparing a particular dish. From a visual perspective, every instruction step can be seen as a way to change the visual appearance of the dish by adding extra objects (e.g., adding an ingredient) or changing the appearance of the existing ones (e.g., cooking the dish). In this paper, we aim to teach a machine how to make a pizza by building a generative model that mirrors this step-by-step procedure.”

Each GAN is called an “operator” in the paper. “Given only weak image-level supervision, the operators are trained to generate a visual layer that needs to be added to or removed from the existing image,” the researchers wrote. The GAN model has been shown to be able to segment pizza toppings, remove them by revealing the layer underneath, and infer the correct ordering of the toppings.

A website for PizzaGAN offers a visual example of how the AI system can remove or add toppings in pizza photos. Click here to view it.

The researchers note the system isn’t perfect, but for pizzas with two toppings, PizzaGAN could determine the right order of toppings 88 percent of the time.

These results could fuel further advances in pizza-making robot technology. The system would also be useful in other segments of the restaurant industry, according to the researchers.

“Though we have evaluated our model only in the context of pizza, we believe that a similar approach is promising for other types of foods that are naturally layered, such as burgers, sandwiches and salads,” they wrote. “Beyond food, it will be interesting to see how our model performs on domains such as digital fashion shopping assistants, where a key operation is the virtual combination of different layers of clothes.”