Review: Neato BotVac Connected

Six years ago, I drove from my crummy little apartment in the part of Berkeley that’s too close to Oakland to somewhere in the south bay that I don’t really remember to pick up, in person, what I’m pretty sure was a development prototype of the Neato XV-11 robotic vacuum. I was instructed to return it in 24 hours, or they’d send a robotic hit squad after me. I wrote a blazing fast review of the XV-11, taped a butter knife to it and let it duel my iRobot Roomba 560, and then brought it back to Neato, having inflicted a bare minimum of physical (and emotional) scarring. 

Since then, Neato Robotics has established itself as a solid and capable competitor to iRobot’s Roombas in the autonomous vacuum space. The XV-11 series has been incrementally upgraded, with a much more significant redesign in 2014 in the form of the BotVac series. Late last year, Neato announced the BotVac Connected, which adds WiFi connectivity and an app that lets you control your robot from anywhere in the world. This is Neato’s top of the line model and currently sells for US $700. We took a look at it at CES, and then Neato promised to send us one to check out at home.

Neato’s robots, starting with the XV-11 and continuing with the BotVac Connected, are notable because of their ability to rapidly generate accurate maps of the spaces that they’re in, and then localize and navigate to efficiently clean those spaces in nice straight lines. For a long time, this was a capability that was almost entirely unique to Neato, but over the last few years, other robot vacuum manufacturers have added mapmaking to their robots, most recently iRobot with its Roomba 980, which uses a camera for helping with localization and mapping. So the question is, now that Neato isn’t the only robot vacuum on the block with this technology, how does the latest version look? Does its six-year-old navigation system still hold up, and can it survive in an increasingly competitive market without the feature that made it unequivocally unique?

We’ve got a full review for you after the break, complete with some long-exposure cleaning pics and an interview with the Neato robotics team.

img

Hardware

You’ll notice two things about the BotVac Connected right away. The first is that it’s not round or square, but “D”-shaped, and the back of the robot is the rounded bit, which takes some getting used to. Neato says that the square front helps the robot fit itself into corners and be generally more effective throughout the square-ish nature of your home, which is generally true, although the sweeping brush is set back far enough from the front of the robot that even if the bot is nose-to-nose with a wall, the effective distance is still a few inches. More on the brush later.

img

The other thing you’ll notice is that it’s got a bump at the back. This houses the laser turret, which is literally a laser turret, housing a spinning infrared laser that the robot does its navigation with. More on that later, too. Otherwise, the top of the robot is spartan but functional, with a color screen and four button interface, plus two other dedicated buttons: one for a spot clean, and one for a full house clean. Most of the top of the robot lifts off to allow access to the dustbin, which is one of my favorite features. Overall, it’s a very polished, slick-looking little robot.

img

Flip it over, and you’ll see some wheels and casters, the main brush, and a side brush. There are two drop sensors right behind the bumper at the front of the robot, but none at the sides or back. The side brush is pretty small, and looks like it exists primarily to sweep dirt from close to a wall out in front of the robot. In a thoughtful little touch, the side brush held onto its axle with a magnet, so it pops off and on again quite easily if you need to clean or replace it.

img

It’s not immediately obvious how to remove the main brush for cleaning, but Neato’s website has some handy how-to videos to help you out. I didn’t watch those videos because I prefer haphazard flailing, but I discovered that a.) you stick your fingers in behind the wheels and pop the whole bottom plate off, and b.) if you do this without removing the side brush first, it will fly off into your face. Once the bottom plate is removed, the brush lifts right out for you to clean, check the bearings (they require some extra force to remove), and all that stuff to keep your Neato in top shape. 

img

On the back of the robot (the round part that looks like it should be the front) are two contacts that line up with the charging dock. The Neato docks itself in reverse (using an adorable little butt-wiggle motion) and can safely spend most of its time on the dock, keeping itself topped up and at your beck and call.

Sensing and Navigation

img

The thing that makes the Neato cooler than any other robot vacuum out there is the fact that it’s got an actual laser navigation system inside of it. That turret on the top of the robot houses a spinning laser cannon and detection system called the Revo LDS (for revolving laser distance sensor). It fires up to 4,000 infrared laser pulses per second while rotating at up to 10 Hz, watches for the reflection of each pulse, and then does some math to figure out the distance and angle of whatever the pulse reflected off of. A range of 5-ish meters with centimeter accuracy is plenty for building up a map of the rooms in your home to navigate with.

There are all kinds of reasons why laser distance sensors are the sensors of choice for robotic map-making applications. They’re fast and accurate, and have proven to be very robust, which is why you see the same fundamental technology spinning away on the roofs of just about every single autonomous car. Those things (usually beefy sensors from Velodyne) cost a ton, but Neato managed to get the price of the Revo LDS down to under $30 by developing it from scratch, and for indoor applications, it’s easily as effective as it needs to be. You can read all of the technical details on Neato’s laser sensor (the first gen, anyway) in this paper from ICRA 2008 by Kurt Konolige, Joseph Augenbraun, Nick Donaldson, Charles Fiebig, and Pankaj Shah.

In the context of a robotic vacuum, Neato’s navigation system is, in my opinion, still the best out there. Other vacuums that do vision-based simultaneous localization and mapping (SLAM) can slowly build up maps of their environments with relative positioning and then navigate around them, but with the Neato, you turn the thing on and BAM, in just a few seconds the laser has located walls, furniture, your legs, its dock, and anything else close to floor level. As it moves, that map is continuously being refined and updated to make sure that the robot never misses a spot or gets lost. Just about every single mobile base that you see doing anything in a commercial or industrial environment has a LDS on it, and it’s awesome that you can buy a robot vacuum with that technology built in, too.

img

As impressive at the Neato’s LDS is, it does have plenty of other sensors that it uses to refine its navigation and obstacle avoidance. Having a big front bumper is especially important since the LDS operates in one horizontal plane, meaning that it can’t detect obstacles shorter than it is, or any hovering furniture you may own. It also has a wall-following sensor, along with two infrared drop sensors at the front, right behind the bumper.

img

The biggest problem that I have with robot vacuums in general is with these drop sensors, used to keep the robots from hurling themselves down stairs and off of balconies. They (generally) work by shining infrared lights onto your floor and then watching for the reflection: if they can’t see the reflected light, it means that the light is shining down into nothingness, and they’re probably about to drive off of a ledge. However, these sensors also don’t see reflected light if the light is absorbed by the surface that it’s shining on, and robots can have problems distinguishing black carpets made of certain materials from a void.

img

As a frequent reviewer of robot vacuums, I made it my business to acquire such a carpet (and my girlfriend liked the pattern, so win-win). Roombas will not function on this carpet at all. If you start them from off the carpet, they won’t vacuum the black areas, and if you start them on one of the black areas directly, they will completely freak out and refuse to move while making panicked beeping noises. It’s possible to solve this problem by taping tinfoil over the Roomba’s cliff sensors, fooling them into thinking that they can always see the floor. Doing so renders them inoperable, and consequently, the Roomba will make a beeline for the nearest stairway and attempt to throw itself down it. This happened to me first with my 880 (the virtual wall I was using to protect the stairway ran out of batteries), and then more recently with my new 980 (I forgot to turn the virtual wall on). Both Roombas bounced down the entire flight of stairs and landed on concrete and were somehow fine, but this is not something that you want to happen frequently. And to be frank, it shouldn’t happen: I shouldn’t have to modify my robot in a way that is dangerous to it in order for it to work on a color of carpet that is not uncommon for people to have in their homes. If the Neato BotVac Connected can handle my black carpet, that’ll be a major victory for it over iRobot’s Roombas. But before we get into that, let’s look at the robot’s control interface.

Interface and Connectivity

The big new feature with the BotVac Connected is the Connected part, since the BotVac part has been out for more than a year. Connected means that the robot has WiFi it in and can talk to your home network. Through that, it can talk to Neato’s servers and then to an app on your phone, allowing you to monitor the status of the robot, get alerts, and do a limited amount of remote scheduling and control.

img

Getting the app to work was not a particularly smooth process. Initially, I couldn’t get it to work at all on my Android phone (a not very old Nexus 5x): the app would connect to the robot and then spend 5 minutes failing to connect to my home network, which resulted in a frustratingly useless “something has gone wrong” error message before giving up completely and then offering a manual connection mode that didn’t work either. At that point, I just ignored the app and used the controls on the robot itself, which mostly worked fine, and forgot about the whole "connected" part of the Neato Botvac.

A few weeks later, after the app was updated on the Google Play store, I gave it another try, and everything worked fine. It’s possible that this was just a temporary Android issue, and that it works flawlessly on iOS, but the mixed reviews on Google Play suggest that at least on Android, there have been some systematic problems. Hopefully they’ve been fixed now, but it kind of seems like Neato may have let the Android app out of the box before it was completely ready to go.

If for whatever reason you can’t get the app to work, or don’t want to use it, it doesn’t actually matter all that much. You can use the app tell your robot to do a full cleaning cycle or a spot cleaning cycle. You can switch from Turbo mode to Eco mode. You can set the cleaning schedule. You can check the battery level. That’s just about about it. Yeah, it’s kind of cool that you can tell your robot to start cleaning the house from anywhere in the world, but since it’s got a scheduling function anyway, I’m not sure how much utility it actually offers. Really, the only fundamentally exciting thing that the app does that you can’t do through the robot’s own control panel is a “Manual clean” option where you can drive the robot around yourself. It’s a neat-o little feature (I’ll only do that once, I promise), but hardly worth getting excited about. At this point, the app seems like more of a proof-of-concept for having a connected robot vacuum than something designed to add a significant amount of value for most users, but we’re expecting a lot more with future updates.

img

Generally, I got the robot to do my bidding more directly, through the little color screen on it and the two big buttons on the front. Weirdly, the screen offers all kinds of options that the app doesn’t, including options for lights and sounds, scheduling, usage info, and even log file downloads and software updates. The most option-y option is deciding whether you want to use Turbo mode or Eco mode; Turbo spins the brush faster and cranks the vacuum up, while Eco is quieter and more energy efficient but in some situations may not clean as well.

Cleaning Behavior

img

I stuck my Neato BotVac Connected in a corner of my living room, hemmed in by the end of my couch, the end of a bookcase, and a lamp. It’s just the right size for a robot like this, but I never housed any of my Roombas there for fear that they’d have trouble getting out and would never be able to find their way home again. The Neato’s LIDAR gave me confidence that the robot would be able to detect the way out, so I figured I’d give it a try.

When you start the BotVac from its dock, it plays a happy little song, drives forward a bit, and then sits for a second as the vacuum powers up and the LIDAR gets up to speed. Generally, it follows a counterclockwise edge-to-center pattern, at least at the beginning, but this can vary immensely (and immediately) depending on what obstacles (furniture) the robot encounters and where. 

Here’s a long-exposure picture that I took of the Neato BotVac Connected covering my living room over the course of about 20 minutes. I put some red LEDs on it to leave a red light trail, but the robot’s screen was on the whole time as well, leaving a parallel blue light trail at the same time:

img

The dock is in the upper left corner. The BotVac left that corner and immediately went under the couch, probably in an effort to favor its left side (with the side brush and wall sensor). It finished under the couch, and then spent a big chunk of time carefully going around each leg of the coffee table before covering the rest of the room in more or less straight lines.

You’d expect that since the BotVac is using its LIDAR to map the room and plot the most efficient route, running it in the same room would result in a similar pattern, and that’s exactly what you see. Note that for this second run, I manually stopped the robot before it finished cleaning and returned to its dock, which is why you don’t see the return path in the image:

img

And here’s the navigation path of a Roomba 980 in the same space to compare:

img

And one more, of a Roomba 880, which navigates by zigzagging randomly across a room:

img

All of these pictures were between 20 and 22 minute exposures; the BotVac is perhaps best described as “methodical.” The Roomba 980 covered the same area over multiple passes in almost exactly the same amount of time (within a minute or two). I’d characterize it as “brisk.”

You’ll also notice that the transitions onto the black carpet patches don’t even cause a wiggle: I’m delighted to be able to report that the Neato BotVac was not the least bit intimidated by the black areas on my rug that no Roomba has been able to handle without modification. If you have a carpet in your house that’s a deep dark color and a little shaggy and don’t want to have to mess with your robot vacuum to get it to behave itself, Neato is hands-down the way to go. 

The Neato seems to be a bit louder than the Roomba, at least when the Neato is in Turbo mode. This is partly because of the vacuum and the laser, but also because of how the robot is designed underneath. On hardwood, it tends to make thunk thunk thunk noises with its casters, and the plastic scraper makes, well, scraping noises that aren’t particularly pleasant. I might consider hanging out in the living room while the Roomba was running, but not while the BotVac is, if it’s on hardwood. On carpet, it’s not so bad, and in Eco mode, it’s much better, although I think the Roomba still wins because of its ability to toggle its own (louder) carpet boost mode on and off when necessary. I do, however, much prefer the lower-pitched hum of the Neato’s vacuum. It’s actually kind of soothing, especially when the bot is moving slowly.

img

During normal cleaning, the Neato tends to be a bit aggressive at times, and it’s not always clear to me why. In some situations, it’ll pass right by the leg of a chair with just a few millimeters of clearance, whereas it sometimes instead seems to deliberately use its bumper to feel it was around corners or along walls, with repeated gentle nudges to make sure that it gets as close as possible. I haven’t noticed any damage to the robot or my furniture, but I can’t help but feel like the poor thing must be giving itself a headache.

After the Neato decides that it’s done, it immediately shuts its vacuum off and drives back to its dock. I was expecting it to make a beeline every time, but often the robot would go back towards the center of the room and then rotate back and forth a bit as of trying to decide where to go. It never failed to find its dock, though I’m not sure what the state of its internal map is such that it can’t plot a direct path every time. When the robot does make it back, it’ll notify you that it’s done through the app, which is nice.

Cleaning Performance

img

As we’ve come to expect from Neato, the BotVac Connected cleans like a champ. Like most robot vacuums, it’s especially good on hard surfaces, but it also manages to give carpets that “I just vacuumed” look. It won’t replace your upright vacuum (no robot vacuums can realistically do this), but it will keep your floors much cleaner, while you yourself vacuum much less often. The long exposure pictures above give a good sense of how the robot performs in an average living room: coverage is complete, with special attention paid to walls, edges of furniture, and furniture legs.

Over a few solid months of testing, I had the BotVac running once or twice a week. Running it while I was at home was only a minimal hassle, since I could be doing something else while the robot got to work. Using the scheduler was even better, since I could just come home and the robot would be on its dock charging (right where I’d left it) and the floor would be obviously cleaner, like magic. A single charge should be sufficient for the BotVac to cover your floor, even in Turbo mode, but if you have lots of rooms you want it to do, it’s smart enough to return home for a charge if necessary and then continue where it left off.

img

Using my Roombas as a baseline comparison (the 980 and the 880), I can unscientifically say that it’s unlikely that you’ll notice much of a difference between them in normal day to day cleaning. Both the Neato and the Roombas somehow manage to fill their dustbins on almost every cleaning pass, which for the life of me I don’t understand, because that’s a lot of dirt that somehow continually makes it into my house. The Neato is particularly good at dealing with hair (pet and otherwise), and seems to do a better job with carpets, especially carpets with deep pile. And dark carpets, of course.

No robot vacuum is flawless, and the BotVac is no exception. For whatever reason it will occasionally miss spots, and it had intermittent trouble in my kitchen, where it would leave some dirt close to the base of the fridge and stove, possibly because the side brush was too short to properly reach. The Neato will also have trouble if you have crap strewn all over your floors, which bothers some people, but I view it as a feature: for the robot to keep your floors clean, you have to first keep your floors tidy.

In general, the BotVac seemed to be very durable, but I pride myself on my robot care. The more simple maintenance you give it, the less complex maintenance it will need. Simple maintenance includes emptying out its dirt bin after every run or two, wiping the filter clean, and making sure that the brush and bearings are in good shape. It’s not a big deal, and if you do take care of your robot, it’ll take care of your floors for years.

Q&A with Neato Robotics

For more details, we spoke with Nancy Nunziati, VP of Marketing, and Matt Tenuta, director of hardware engineering at Neato Robotics.

IEEE Spectrum: What’s new with the Neato since my first review of the XV-11?

Matt Tenuta: There have been a lot of changes to the platform. Going from the XV series to the BotVac is a pretty major change in terms of the system architecture. With Connected, obviously the connectivity is the big piece, but so far we haven’t shown off too much about exactly what that’s going to do, for strategic reasons.

As far as functionality, there have been improvements in the navigation and behavior. Going from XV to BotVac, you should notice that the robot requires less help and is more autonomous: we do a better job of escaping from situations where would have given up in the XV days. We reacted to user feedback on the XV, and added quite a bit: new escape maneuvers, new detection on stuck conditions, and figuring out the right strategy to get out of those situations. 

Can you talk about some of those challenging situations, and how the robot handles them?

Matt Tenuta: The Neato algorithms have always prioritized coverage. The first goal of the robot is to mark every cell in its map as ’clean.’ Historically, the robot has been very persistent in trying to find its way into the middle of things like chair legs, and in the XV days, there were situations where we could get into those chair legs, and we’d be reacting to a number of bumper hits and tilt events and things where the robot is just bouncing around, and eventually, we’d give up and ask the user for help. In the BotVac, we added some ability to detect those conditions, and we have some behavior in the robot that once it’s able to detect them, it’s able to identify a maneuver that’s likely to get it out. It may take a few tries, but the data that we have from our testing indicate that the BotVac is much better in terms of autonomy than the XV was. 

Drops are something as well that we very extensively test. We come up with these platforms that are designed to create very challenging environments for the robot to navigate around. We’ve basically got a table with a number of different cutouts, and the robot has to be very careful in terms of how it’s reacting and how far it’s willing to back up. So, we don’t just test against simple ledges, we test against these platforms that have holes and juts and all kinds of different geometries that challenge the robot.

We have well over 2,000 square feet of testing environments, with different environments set up to represent kitchens, home offices, a number of different drop platforms with different surfaces and different features. For example, there’s one particular type of carpet made of this particular material that can cause additional friction with the plastic that we use in the squeegee that funnels the dirt up into the intake path. If we didn’t have this specific type of carpet in our lab, a few percent of users in Europe might have a problem that we would never have noticed. Neato’s had products on the market for nearly six years now, and we’ve learned a lot over that period. 

What was a challenge of operating a robot vacuum in a home environment that surprised you?

Matt Tenuta: Drapes are the one that surprised me. We’ve had issues with sheer drapes. Because we have a number of different sensors that we use for wall tracking, we’ve had drapes that allow our laser light to pass through and return back, but don’t allow the IR light from the LED in the wall follower [on the right side of the robot]. So, you’ve got an area where the LIDAR says, "sure, you can get there," and then as soon as you get there, the wall follower says, "you can’t get there." That’s one that been challenging and that we’re working through for our next generation.

I’ve been impressed by the robot’s ability to deal with black carpet. How’d you get that to work?

Matt Tenuta: We realized there was an issue [with robot vacuums and black carpeting]; we spent a lot of time reading reviews, both of our products and our competitor’s products, and we wanted to make sure that we did better. It’s one of our qualification items: we do quite a bit of testing on carpets that we know to be problematic, usually black long-pile carpet, and we’ve tuned our drop sensors to be able to continue to range successfully on those surfaces. In the future, we’ll be doing more with different types of sensor technologies to better distinguish surface reflectivity from distance.

Neato’s LIDAR system was initially designed eight years ago. How’s it been holding up?

Matt Tenuta: For what we’re doing indoors, [Neato’s LIDAR system] works very, very well. It reacts to the features that we care about; things like walls, doorways, chair legs, table legs, we get very high fidelity data back from those. We get a ton of returns that are very linear and very consistent, and that allows us to write very simple algorithms to detect features and also to reference our position relative to those objects.

So why isn’t everyone using LIDAR for these applications?

Matt Tenuta: Neato’s had the LDS technology for many years. We get people quite frequently asking about the technology and licensing and things, and I think that goes to show that it’s very, very difficult to copy. The amount of understanding that Neato has in terms of how to align the optics, how to test and calibrate, how to align the firmware, there’s just a ton of internal research that’s been done. I think we do it better than anybody else, and I think everybody else who’s tried to come up with something that works just as well has gotten frustrated.

But for us, it’s more about the product. It’s more about what the LIDAR enables. It’s a fantastic technology, but it can do much more than SLAM. While we can compare and contrast SLAM and VSLAM, we’re thinking beyond SLAM. We’re thinking about what LIDAR is going to be doing in the future.  

Cool, so what will LIDAR be doing in the future?

Matt Tenuta: You can imagine a number of things. If the robot is able to wake up, and within a few seconds figure out exactly what room it’s in, maybe that would change its cleaning behavior. Rather than doing an entire SLAM-based cleaning run, it could do a more dedicated cleaning run. And it could also do more in terms of working with other devices: if the Neato knows exactly where it is in the environment, then maybe it could collaborate with other kinds of connected devices, share information, and obtain information to help it learn more about the environment to make a more useful product. There’s quite a bit there, and I’m sure you can let your imagination run a little.

Something that we’re working on for the future is trying to make it a little more flexible— giving the user the option to set behavior depending on their environment, their furniture, and the level of autonomy that they feel is right.

Nancy Nunziati: From a purely speculative viewpoint, I think that you’ll see us putting increasing importance on connectivity, and really being the leader in that area. From the beginning, Neato has had a fast development cycle, and you’ll see that in frequent app releases and a more rapid product cycle for release.

How about opening up the Neato to people who want to use it as a mobile base?

Matt Tenuta: We have a ton of people on the team who are really passionate about the ROS community, and these robotic base platforms that are available for personal projects. We’d love to support it, but obviously, we’re a really small team, so in terms of developer outreach and SDKs and things, we can’t do too much because we have far too many projects as it is. But we’ll be doing more over-the-air type potential things, making the system accessible to hobbyists and developers and students. I don’t think we’ll have anything as robust as [the iRobot Create] anytime soon because it takes a big team to support those efforts, but maybe down the road when we have a little breathing room.

Conclusions

img

If you’re in the market for a new robotic vacuum, we have no trouble recommending the Neato BotVac Connected: it’s a solid robot that will do you proud. In our experience, the quality of its cleaning is easily comparable to the iRobot Roomba, which (like it or not) has over the years become the standard of comparison in this space. The Neato is noticeably better at getting dirt and hair out of carpet, and a bit less capable in some specific kitchen situations. Realistically, for most people, the biggest difference between the top of the line Neato and the top of the line Roomba is as simple as the $200 more that the Roomba 980 costs: the BotVac Connected is $700, while the Roomba 980 is $900.

However, this is the reason why we usually don’t recommend top of the line robots to most people: you’re paying a premium for new features that you may not get hundreds of dollars worth of value out of. So, the real question is whether the premium you’ll pay for the BotVac Connected’s connectedness is really worth it: the non-connected BotVac D series starts at $450, while the Connected is $700.

Fortunately, the WiFi connectivity and app isn’t the only upgrade that the Neato BotVac Connected has over its siblings. You’ll also get some additional cleaning settings (including Turbo and Eco mode), a more powerful internal vacuum, and a lithium battery (instead of a NiMH) that will add up to 2,000 square feet to the area that the robot can clean on one charge. Is all of this worth an extra $200? It depends: if you’re likely to run the vacuum when you’re home most of the time, then the app won’t do a whole lot for you, and if if you’re willing to commit to a fixed schedule, you can just use the scheduling function on the vacuum itself anyway. On the other hand, if you want to be able to initiate a cleaning cycle from halfway around the world, the app might be a good idea.

The other reason to invest in the Connected is to (potentially) benefit from the future of a house full of connected devices. This is the future, mind you, not the present, and while it sounds like Neato is actively working on a bunch of stuff, we don’t have a list or a timeline or anything tangible that you could use to talk yourself into buying a robot with a lot of expensive potential baked into it.

As much as I’ve tried to remain impartial by comparing the performance and features and whatnot of these robot vacuums, I have to mention one more thing: the technology that Neato’s robots have inside them is just awesome. The LIDAR is a small miracle of engineering, and it works amazingly well. It seems so much more futuristic than using VSLAM like every other navigating robot vacuum does, and having a LIDAR in your household robot has the potential to enable much, much more. 

The Neato BotVac Connected is available on Amazon, Best Buy, and other retailers.

[ Neato Robotics ]

Advertisement

Automaton

IEEE Spectrum’s award-winning robotics blog, featuring news, articles, and videos on robots, humanoids, drones, automation, artificial intelligence, and more.
Contact us:  e.guizzo@ieee.org

Editor
Erico Guizzo
New York City
Senior Writer
Evan Ackerman
Washington, D.C.
 

Newsletter Sign Up

Sign up for the Automaton newsletter and get biweekly updates about robotics, automation, and AI, all delivered directly to your inbox.

Advertisement