Instituto de Óptica “Daza de Valdés”

On the synthesis of visual illusions using deep generative models

12 Aug, 2022 | Ciencias de la imagen-en

Do artificial neural networks see optical illusions?
A scientific study plays with neural networks that see visual illusions and are also capable of inventing new examples

A visual illusion is an image that induces a visual perception that does not correspond to the actual description of the scene, which can be correctly measured with sensors such as spectroradiometers, rulers, protractors, etc.
They are images that the human eye misinterprets due to its otherwise optimal image coding strategy.
Two squares, one black and one white, both with a small gray square in the center

In this image one gray box appears darker than the other when they are the same

Últimas noticias

Visual illusion as a way of studying vision

Visual illusions serve as strategies to broaden our understanding of the human visual system because they help us to check first if the artificial vision models also suffer from them and also as a second strategy we can ask the artificial model to generate new images with convincing visual illusions for humans to check if you have “understood” its operation.

Visual illusions in artificial neural networks

Since 2018, it has been observed that artificial neural networks (ANNs) trained on natural images can also be “fooled” by visual illusions, in the sense that their response to an illusion-inducing input in humans is (qualitatively) the same. .
In this work we focus on the second part of the strategy (less explored): we propose a framework for the neural network to generate new visual illusions and check on their own if they are good or not.

With these studies, it is possible to study the differences between human perception and the artificial vision model in the particular case of visual illusions and thus adjust the model so that it behaves as closely as possible to human vision.
The direct study of visual perception is an extremely challenging problem, and for this reason most psychophysical research is done on the study of particular cases such as these that reduce the complexity of the system.

In this work the scientific team has proposed a framework that creates novel visual illusions by using generative adversarial networks (GANs) which are optimized to produce illusions that will be displayed to a given vision model. These synthesized stimuli will also be displayed to human observers, for verification.

The contributions of this work are the following:

• A new method to generate visual illusions in humans in which the images will be generated without human input.
• A new way of studying the differences between a given vision model and human perception through visual illusions generated to deceive the vision model.
• An extensive collection of synthetic visual illusions produced by different frame choices.

Scheme of different configurations of the proposed framework

Different configurations of the proposed framework, in the top one the best possible image is sought and in the bottom one the best generator of visual illusions is sought

Illusion Meter

The illusion meter is the most important element of the proposed framework because it is in charge of measuring the effect produced by the generated visual illusion and therefore it needs to “see” the image as a human would. It is made up of two well-differentiated parts, the first is a vision module that processes the image (thus acting as a replicator of human vision) and a second module that measures how much the vision module has made a mistake as a result of the visual illusion. In this way the image generator has been able to search for the generated images that cause a greater visual illusion.

This is a collaborative project between the Computer Vision Center of the Autonomous University of Barcelona, ​​the Department of Information and Communication Technologies of the Pompeu Fabra University, the Group of Image and Vision Sciences< /a> of the Institute of Optics and the Image Processing Laboratory of the University of Valencia

Related News