Training a Computer to Find Shapes in the Clouds

I spent my afternoons on a small wooden pier, relaxing and staring up at clouds, looking for shapes. That pursuit is an exploration of pareidolia, the psychological phenomenon in which the mind responds to a stimulus, usually an image or a sound, by perceiving a familiar pattern where none exists. It was in that leisure time, trying to make sense of the clouds, that I found the comfort that I’d been seeking.

Nine years later, I found myself back in America, interning at IDEO Chicago, where I had far less opportunity for leisurely cloud gazing. Without the spare time to hang out on a pier, I wondered if I could engineer the same kind of comfort. Could I design an experience that brought back those long dreamy pareidolia days with the help of an app?

After all, people in my age bracket, between 25-34 years old, only spend an average of 13 minutes per day relaxing, according to the American Use of Time Survey, and experts say that we need to acknowledge and accept that we’re leisure-limited. Maybe this could be a shortcut to that leisure we’re missing.

So I created “Pareidolia,” an app that searches the sky on my behalf. It’s my way of trying to keep a part of my mind exploring, even when I need to be inside, working. If the app thinks it sees something in the clouds, it pings me to take a look.

As I started to share Pareidolia with colleagues and friends, my creation was met with mixed reviews. Ross Weijer, a data scientist here at IDEO Chicago, thought that it created efficiency while offering a different perspective. “Technology is something that speeds up your life and, lets you do things quicker,” he said. “So I think it’s subversive and helpful to have the aim of this project be to spare your imagination. It’s saying, ‘Stop, stop, take a moment to consider this.’”

Others, though, felt that the payoff wasn’t seeing images in the clouds, but actually, the act of searching for them. “I look at the sky so little, that when I do it’s because I am looking for an escape,” said Kevin Matuszeski, regional director of business operations “There are very few places left without the presence of screens or where you have to read something. So the pure joy of staring at something just for its beauty should be respected."

In other words, for some people, the joy lays in the process, and Pareidolia seemed to eliminate that.

Pareidolia didn’t turn out to be quite the leisure fix that I hoped it would. And it didn’t quite prove to be an antidote to homesickness, either. But it did encourage me to get outside, take my eyes off a screen for a few minutes, and enjoy nature as much as I could from the concrete of my downtown Chicago office. And even if it wasn’t a total fix, it does brighten up my days.

To create Pareidolia, I set up a Raspberry Pi and a camera on the terrace of our Chicago studio, and programmed it to look continuously at the sky, searching for interesting shapes. The project uses computer vision, tensorflow, and keras to perceive images in clouds, then snaps an image, generates a drawing of what it imagines it might look like, and shares them via Slack. (The code can be found and downloaded from this Github repo.) It tells us what it sees in the sky in real time and invites us to go out and look for it.