Restoring Images by Adding Noise

Wait, what? Restoring images by adding noise?

Yes, you read that right. It's possible to beautifully restore, upscale, and enhanced images and videos by adding noise to the picture. That may seem counterintuitive at first, but cutting-edge AI image generation tools like Midjourney, Dall-E, and Stable Diffusion do exactly that. This can be done either as Text-to-Image (T2I) from a text prompt, or as Image-to-Image (I2I) with an input source image instead of a prompt. They deliberately add random noise, similar to old-fashioned analog TV static, to empty latent images or existing but damaged images and then transform them into clear, vibrant images. This innovative technique, known as "noise diffusion," is revolutionizing how we preserve and revitalize old, damaged photos, and it's only a matter of time until it becomes applicable to video, as well. Let’s take a look at this intriguing process and discover how it works to bring cherished memories and artworks back to life.

What Is Noise Diffusion?

Noise diffusion is a technique that artificial intelligence (AI) uses to create or enhance images by introducing random noise and then refining the image through a series of noise reduction steps, known as sampling. In simple terms, it’s a process that helps the AI figure out what an image should look like, even if parts of it are damaged, faded, or missing. This technique helps the AI focus on the essential details and reconstruct the image accurately.

How Does Noise Diffusion Work?

Adding Noise: The process starts with the AI adding random noise to the image. This noise looks like tiny specks, similar to what you might see on a TV when it’s not tuned to any channel. This might seem odd, but adding noise helps the AI handle uncertainty and enhances its ability to generate a clear image.

Refining the Image: After adding noise, the AI gradually removes it through several sampling steps, which can be done using many different algorithms and techniques. In each step, the AI takes a closer look at the image and tries to remove more of the unwanted noise while keeping the important details intact. This process is repeated many times, with the AI refining the image a little more with each step. A typical image might use 20-40 sampling steps, although some models may use as few as 6 or as many as 100.

Recovering the Image: As the AI refines the image, it gradually removes the noise and brings out the desired picture. The AI has been trained on millions of images, so it has a good idea of what various objects, faces, and scenes should look like. By the end of the process, the AI has removed most of the noise and restored the image, often making it look as clear and vibrant as possible.

Here’s an illustration showing how noise diffusion works for image restoration: