Hate to wake up early enough to photograph the sunrise? Pierre Yves-Laffont has a solution: sleep in, take a photo later, and let a computer do the work of turning back time. He and other computer scientists at Brown University have come up with a program that can edit outdoor photographs using keywords or phrases like “more snow”, “less gloomy”, or “more daylight”.
To build it, the researchers funneled 8500 images from over 100 outdoor webcams into a massive database, and used crowdsourcing and machine learning to teach the tool to recognize more than 40 attributes of outdoor scenes. Is it sunny? Raining? Dry? Gloomy? An algorithm identifies the differences between the same scene when viewed under various conditions by separating out the elements, such as the sky, buildings, and the ground. Each element is affected differently by an attribute change. In real life, snow is more than a simple white layer. It might make the ground white, a building darker, and the sky greyer. So the program must apply multiple changes to the user's image on a pixel level to make it look realistic.
To use it, you simply upload a photo and enter a keyword. The algorithm can recognize the 40 attributes in any photo—not just those in the database—so it can edit most outdoor scenes.
The photo editor can automatically change a photo to a new weather scheme, or you can use an interactive version for more control. The interactive option presents effect choices via a selection of photos that have the desired attribute in diverse ways. Then the user can specify whether they want “more snow” as in a clear winter day or "more snow" as in a blizzard.
Researchers can now gain access to the whole dataset. According to Laffont, this is the first time an annotated database of this size has been compiled—and as more photos and variations are added, the system will become more flexible, more powerful, and better able to make dramatic changes.
