When I last spoke with former autonomous vehicle researcher Alex Teichman, he was busy making a company out of a little computer vision system he and a fellow autonomous car researcher had hacked together to watch out for burglars when he was out of town. At the time, in 2015, he called the product Snitch, and had just unveiled a demo that distinguished between your dog roaming your house and a person that probably shouldn’t be there.
Now that little demo is all grown up, and Teichman’s vision of using autonomous vehicle technologies as the eyes of the smart home is close to fruition.
His company, now called Lighthouse, has $17 million in funding and its first product has started to roll off manufacturing lines, with shipments to begin in September. The company has 30 employees housed inside the cavernous startup warehouse that is Andy Rubin’s Playground Global, where it was able to tap the incubator’s hardware team for initial product design before hiring its own hardware engineers.
Here’s what Lighthouse does. The gadget (or two or three) sits on a shelf or table—somewhere with a good view of a key area of your house, such as the front door or perhaps a child’s room. Its cameras record everything within its 95-degree field of view, storing that recording in the cloud for 30 days. While it is recording, it’s sending the images to cloud servers for analysis, using its algorithms to separate objects from background, figure out which moving objects are pets or people, and track those pets’ and people as they move around. The user, via a smart phone app, can see what the camera sees, ask about particular activity (Did you see anybody with the dog?), or set up alerts (“Tell me if you don’t see anybody come home between 11 and 1.”) The system also reacts to certain behaviors—if someone in the home waves at the camera, it will push an alert to the app user that somebody says hello, with a photo of the hand-waver. If it spots a person when nobody is supposed to be home, it can set off a siren.
“What we are building,” Teichman says, “is the inverse of Alexa. Alexa is a way to connect to the outside world when you are home; Lighthouse is a way to connect to your home when you are in the outside world.”
Says Lighthouse general manager Jessica Gilmartin, “I am always wondering what the dog and the kids are doing. I can say ‘What did the kids do today while I was gone,’ and it shows me my husband taking my kids out. I have a dog walker, and am always nervous that they’ll forget, so I ask to be pinged, ‘Tell me if you see anyone with a dog between 12 and 2 pm on weekdays.’”
Right now, Lighthouse doesn’t distinguish between individuals, but that will be operational by the time the product ships in September, Teichman says. Initially, the product will come with basic face recognition. But in the not so distant future, says Teichman, the system will often be able to identify people even if it can’t see their faces, because it will track recognized people and learn about their body shapes and sizes as seen from different perspectives. He says the company is also working on object and spatial relationships, for example, “Tell me if the dog jumps on the couch.”
So, how does it work? The Lighthouse system sends out modulated light from four “illuminators,” as Teichman calls the solid state devices he declined to identify more precisely. It then looks at the phase shifts of this light as it arrives at a camera sensor; these help it determine the precise 3D location of every pixel in the scene. Teichman says Lighthouse designed custom optics for this purpose. “This is a direct translation of our work on self-driving cars,” he notes. “It is similar to the time-of-flight measurements done by laser rangefinders, but done in a more cost-effective way.”
The system also has a standard color camera sensor, and combines this image data with the data from the 3D sensor. It includes a microphone and a speaker for two-way communication, as well as a siren—a reminder that it started out as a home security device. And, Teichman assured me, security is designed into the device itself and into every connection between the device and the outside world; the only user data shared with the company is done so on an opt-in basis by users who agree to provide training data for the deep learning algorithms in exchange for early access to new features.
Pricing, Gilmartin said, is still a bit in flux, but right now the company is taking preorders at $299 for the hardware; users also pay a monthly subscription fee (right now, $10 or less, depending on how many years are prepaid).
Of course, Teichman isn’t the only one trying to put eyes inside the smart home. Smart home gadget maker Nest just announced a $299 camera that can recognize faces, and, with a $10 a month subscription, allow users to check back over ten days to determine who was in a room and when. Teichman, says that Nest IQ is “great validation that consumers are looking for more intelligence from their home cameras,” though, of course, he thinks Lighthouse’s 3D sensor gives it an advantage because it’s better able to decode the activities of pets and people in a home.
Tekla S. Perry is a senior editor at IEEE Spectrum. Based in Palo Alto, Calif., she's been covering the people, companies, and technology that make Silicon Valley a special place for more than 40 years. An IEEE member, she holds a bachelor's degree in journalism from Michigan State University.