Github
This website is fully interactive playground! In this project, we generate optical illusions and perform stenography using off-the-shelf diffusion models - all of which work in the physical world too!
Diffusion Illusions won the best demo award
in
CVPR
2023!
We also exhibited at OpenSauce 2023. See
JacksFilms' reaction
here!
More Details: Check the bottom of the page to learn more about this project!
Our Paper: diffusionillusions.pdf - please give it a read!
Source code: github.com/RyannDaGreat/Diffusion-Illusions. If you like it, please give it a star ★!
If you wait a few seconds, the below images will flip upside-down - to reveal a new image!
You can
Try it on Colab!In this section, we have examples with two images. As the one on the top is rotated, the image changes! The one on the top is multiplied by the one on the bottom (pixelwise), simulating how light would be filtered through two transparent sheets with images printed on them. Effectively, we've made 4 images out of 2!
This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.
You can
Click here for a printable PDF
full of these
images,
so you can try
it yourself!
Try it on Colab!In this section, we have sets of 4 images. You can
This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.
I won't spoil it for you in this paragraph; try it out!
Try it on Colab!
Auxetic mechanical linkages of connected squares.
Here are some videos of the above illusion, in case your browser is too slow to render them nicely.
Click the this link to learn much more about Diffusion Illusions!
The project is comprised of four demonstrations:
Another project with a similar goal was released in February 2023: Illusion-Diffusion, authored by Matthew Tancik. That project only achieves the first part of this demo, aka image flipping.
Our other paper, "Peekaboo: Text to Image Diffusion Models are Zero Shot Segmentors", has an extremely similar formulation to the illusions on this website - it optimizes images using a variant of score distillation loss to perform zero shot segmentation.
Inspired by our Diffuion Illusions project, Daniel Geng et al wrote Visual Anagrams. It has beautiful results and is definitely worth checking out!
To help explain how these were calculated, I've included some easy-to-read pseudocode that covers how each of these illusions were made. This code has been oversimplified for the sake of simplicity - for all the details please read our paper. The full source code can be found at github.com/RyannDaGreat/Diffusion-Illusions, or at any of the Google Colab links throughout this webpage.
Diffusion illusions are generated through an iterative process: we optimize the output images with gradient descent, starting from pure noise. Here are some timelapses showing how they evolve over time.
For reference, here are the end results of the above timelapse video:
Arxiv
Paper
