On the synthesis of visual illusions using deep generative models
Do artificial neural networks see optical illusions?
A scientific study plays with neural networks that see visual illusions and are also capable of inventing new examples
They are images that the human eye misinterprets due to its otherwise optimal image coding strategy.
In this image one gray box appears darker than the other when they are the same
Visual illusion as a way of studying vision
Visual illusions serve as strategies to broaden our understanding of the human visual system because they help us to check first if the artificial vision models also suffer from them and also as a second strategy we can ask the artificial model to generate new images with convincing visual illusions for humans to check if you have “understood” its operation.
Visual illusions in artificial neural networks
Since 2018, it has been observed that artificial neural networks (ANNs) trained on natural images can also be “fooled” by visual illusions, in the sense that their response to an illusion-inducing input in humans is (qualitatively) the same. .
In this work we focus on the second part of the strategy (less explored): we propose a framework for the neural network to generate new visual illusions and check on their own if they are good or not.
With these studies, it is possible to study the differences between human perception and the artificial vision model in the particular case of visual illusions and thus adjust the model so that it behaves as closely as possible to human vision.
The direct study of visual perception is an extremely challenging problem, and for this reason most psychophysical research is done on the study of particular cases such as these that reduce the complexity of the system.
In this work the scientific team has proposed a framework that creates novel visual illusions by using generative adversarial networks (GANs) which are optimized to produce illusions that will be displayed to a given vision model. These synthesized stimuli will also be displayed to human observers, for verification.
The contributions of this work are the following:
• A new method to generate visual illusions in humans in which the images will be generated without human input.
• A new way of studying the differences between a given vision model and human perception through visual illusions generated to deceive the vision model.
• An extensive collection of synthetic visual illusions produced by different frame choices.
Different configurations of the proposed framework, in the top one the best possible image is sought and in the bottom one the best generator of visual illusions is sought
Illusion Meter
The illusion meter is the most important element of the proposed framework because it is in charge of measuring the effect produced by the generated visual illusion and therefore it needs to “see” the image as a human would. It is made up of two well-differentiated parts, the first is a vision module that processes the image (thus acting as a replicator of human vision) and a second module that measures how much the vision module has made a mistake as a result of the visual illusion. In this way the image generator has been able to search for the generated images that cause a greater visual illusion.
This is a collaborative project between the Computer Vision Center of the Autonomous University of Barcelona, the Department of Information and Communication Technologies of the Pompeu Fabra University, the Group of Image and Vision Sciences< /a> of the Institute of Optics and the Image Processing Laboratory of the University of Valencia
Related News
Using Decoupled Features for Photorealistic Style Transfer
Madrid / October 10, 2023A team of researchers from York University of Toronto and the Institute of CSIC Optics has developed a new automatic...
Two photographs of María Egües Ortiz (1917-2008): a periscopic gaze in times of misty silence
Madrid / July 26, 2023This essay published by our colleague Sergio Barbero from the IVIS group focuses on the figure of María Egües Ortiz (...
Presentation of the Exhibition: “The beginnings of the Institute of Optics ‘Daza de Valdés’
Madrid / November 18, 2022On the occasion of the 75th anniversary of the creation of the “Daza de Valdés” Institute of Optics we inaugurate an...