Jason Antic's DeOldify is a Self-Attention Generative Adversarial Network-based machine learning system that colorizes and restores old images. It's only in the early stages but it's already producing really impressive results, and the pipeline includes a "defade" model that is "just training the same model to reconstruct images that augmented with ridiculous contrast/brightness adjustments, as a simulation of fading photos and photos taken with old/bad equipment."
Read the restThis is a deep learning based model. More specifically, what I've done is combined the following approaches:
* Self-Attention Generative Adversarial Network (https://arxiv.org/abs/1805.08318). Except the generator is a pretrained Unet, and I've just modified it to have the spectral normalization and self-attention. It's a pretty straightforward translation. I'll tell you what though – it made all the difference when I switched to this after trying desperately to get a Wasserstein GAN version to work. I liked the theory of Wasserstein GANs but it just didn't pan out in practice. But I'm in love with Self-Attention GANs.
* Training structure inspired by (but not the same as) Progressive Growing of GANs (https://arxiv.org/abs/1710.10196). The difference here is the number of layers remains constant – I just changed the size of the input progressively and adjusted learning rates to make sure that the transitions between sizes happened successfully. It seems to have the same basic end result – training is faster, more stable, and generalizes better.
* Two Time-Scale Update Rule (https://arxiv.org/abs/1706.08500). This is also very straightforward – it's just one to one generator/critic iterations and higher critic learning rate.
