left
right
left
right
left
right
left
right
left
right
left
right
left
right
left
right
left
right
left
right
left
right

DEPARTMENTS

DIY Street-View Camera

Create Google Street View-like panoramas with cheap webcams and open-source software


PAGE 12 // VIEW ALL

Photo: Roy D. Ragsdale

OFF THE GRID: A do-it-yourself camera array lets you create your own street views of places where Google's cameras don't go.

BY Roy D. Ragsdale // October 2009

If you use Google Maps, you're probably familiar with its Street View feature, which shows actual ground-level photos of many cities around the world. Google creates the images by mounting special cameras on vehicles and driving them around.

Now wouldn't it be great if you could have your own Street View–like camera? You could hike a trail and later share the photos with friends. The photos would carry GPS tags, so you could display them on Google Earth and include annotations—good water here, poison ivy there. Realtors could display whole neighborhoods to potential clients. A country club could offer a virtual tour of its golf course. Architects could monitor progress at a construction site.

Last year, as part of a "disruptive technologies" course at the United States Military Academy, in West Point, N.Y., I set out to develop a prototype. I thought such a system would have many applications in the battlefield, for example, helping soldiers patrol dangerous routes. My system—I call it PhotoTrail—uses off-the-shelf components and open-source software. It consists of webcams, a GPS receiver, a notebook computer, and imaging software.

For the camera system, I chose the Microsoft LifeCam NX-6000, which is small and has UVC (USB video class) compatibility. It was also cheap (although it lists for US $79.95, I got it for $25 new). It has a megapixel video resolution and shoots 8-megapixel still images.

The NX-6000 has a lens with a 71-degree field of view. In order to stitch images together for 360-degree panoramas, I bought eight units, for a total of 568 degrees of coverage, allowing a healthy image overlap. To connect all the cameras to the notebook, I used two D-Link USB hubs ($25 each), which ran unpowered.

For the GPS receiver, I chose the GlobalSat BU-353, a self-contained waterproof device with good signal reception and accuracy, which costs a mere $37. If you attach it to a USB port, the GPS coordinates will appear in a log file, using a standard GPS encoding scheme.

Construction was straightforward. On a flat octagonal heavy-cardboard base, I glued small posts for the cameras' clips to latch onto. I aligned each unit and then placed the USB hubs and the GPS receiver in the middle. I secured the cables with Velcro and sandwiched everything with another piece of cardboard. The whole thing's the size of a small pizza box, weighing less than 1 kilogram. Excluding the notebook (a 2-gigahertz machine with 512 megabytes of RAM running Ubuntu Linux), the hardware cost about $300.

To start capturing images, I installed a UVC driver and a device driver compatible with the camera array. For the capture itself, I used luvcview, a small open-source webcam program by Logitech. (Uvccapture, also by Logitech, lets you take still shots, but it was incompatible with this camera.)

I had set the camera array on video capture, so I needed to tweak luvcview's source code to get still images from the video feed. The tweaks call for the array to capture a few frames and then stabilize itself so that the images are in focus and have good light contrast. I wrote a Python script to capture the eight 1280-by-1024 JPEG files. That capture takes about 8 seconds. Images captured within that time frame can be considered a single cluster to be stitched together.

Digital cameras normally add data about the photograph, but because luvcview operates at the file level, these images have no such metadata. So I wrote a Python script to read the date and time the file was created. I then used Exiftool, a command-line image metadata editor, to put the date and time into the file.

The images also need to be GPS-tagged. Gpicsync, an open-source tool, can automatically get the latitude and longitude data from a GPS receiver's log and add the coordinates to the image's metadata field. Gpicsync also lets you transform this image set into a single file that you can view using Google Earth.

I used two tools to generate panoramas. The first, autopano-sift, identifies common features in different images and aligns them along a horizon line. Another tool, hugin, uses those common elements to effectively stitch the images into a single panorama. I again used gpicsync to GPS-tag the panorama and generate a Google Earth file. To see the panorama as a 360-degree image and zoom in and pan about the scene, you can use PTViewer.

On my underpowered computer, it took 15 minutes to stitch each panorama. It's a long time. But you can do the capture first and the stitching later, or transmit the images to a more powerful server for remote processing.

With all this development work done, it was time to test the prototype. During a trip to the Boston area, I walked around the MIT campus holding the system above my head. Passersby didn't seem bothered. I guess students attached to weird contraptions are a common sight there. On Google Earth, I can retrace my route and see the surroundings with great detail [see photo, previous page].

JEEP CAM
Photo: Roy D. Ragsdale

JEEP CAM: Mounted on a moving vehicle, the camera array [detail] can capture images of streets and their surroundings—just as Google does to produce its Street View panoramas.

I also mounted the array on a Jeep [see photos above] and drove around West Point, capturing images while driving up to 100 kilometers per hour. I programmed it to take one set of images every 20 seconds. In an hour I had 300 MB of data from 180 sets of images. When the jeep isn't stationary, the images can't be clustered into panoramas. (Recall that it takes 8 seconds to grab a single set.) Still, the individual images are perfectly clear and on a par with those available on Google Street View.

I'm now working on some improvements. One idea is to replace the notebook with a smaller computer, such as one based on the Pico-ITX board, and shrink the camera system (the actual CCD, or charge-coupled device, and lens elements are no bigger than a fingernail). Eventually, you could build a camera system small enough to be integrated into a headband or hat.

The software could use some tweaks as well. I'm planning to write an Adobe Flash application to allow the user to see the panoramas as 360-degree images and be able to navigate from one panorama to another, just as in Google Street View.

The U.S. Army is currently evaluating my prototype. Eventually, a contractor could produce a field version for tests. Meanwhile, as this article goes to print, I'm preparing to travel far and wide. If I have space in my backpack, I'll have the camera capturing my journey, step by step.

TOOLBOX FOR BUILDING PHOTOTRAIL

HARDWARE
Microsoft LifeCam NX-6000 camera (8)
D-Link USB hub (2)
GlobalSat BU-353 GPS receiver
Laptop running Ubuntu Linux

SOFTWARE
UVC driver: http://linux-uvc.berlios.de
Webcam device driver: http://linuxtv.org
luvcview: http://www.quickcamteam.net
Exiftool: http://www.sno.phy.queensu.ca/~phil/exiftool
gpicsync: http://code.google.com/p/gpicsync
autopano-sift: http://user.cs.tu-berlin.de/~nowozin/autopano-sift
hugin: http://hugin.sourceforge.net
PTViewer: http://www.fsoft.it/panorama/ptviewer.htm

About the Author

Roy D. Ragsdale studied computer science at the United States Military Academy, in West Point, N.Y. Ragsdale will report to his first unit, in Germany, next year.


PAGE 12 // VIEW ALL





1

1
1

Comments will appear after moderation 

payam 10.30.2009
Hi I interested about this article i want give me more information about source code , how did you change source code ? and Which section must be change in source code? .
joe ki7wv 10.28.2009
LT Ragsdale, Nice Work! One question- how do you orient the images to N/S/E/W from the datastream? That part got past me. Does your GPS receiver also have a magnetic compass heading included? Thanks.
Bogdan Cristea 10.19.2009
Excellent article. I was looking for some time for a software allowing to create panoramas from overlapping pictures. Can I put a link on my personal page to this article ?.
Rod Gaffar 10.16.2009
Hi Roy, Wow thats great what you have done, is there a chance you could sell your protype or newer version as iam interested in street view my town in africa. Please do email me or send a contact so we can chat.# Thanks .
Streetview 10.10.2009
That's certainly a very impressive piece of work! You managed something great and that's not given to anyone: from conceptualization to realization, that's been some work ;) Congratulations!.
EKRAMUL HAQUE 10.02.2009
PARBATIPUR.DINAJPUR.BANGLADESH .
Jeffrey Johnson 10.01.2009
Get in touch! We just deployed a very similar rig at the burning man festival ... http://www.flickr.com/photos/nym/3948247009/ ... .
Yuval Levy 10.01.2009
Great Project. For the Flash application to show 360°, Panosalado <http://panosalado.com/panosalado> is the place to go. Autopano-sift is encumbered by a patent in the US. The SIFT algorithm is protected. It is free for educational purposes but if the US Army is serious about using your prototype, they better negotiate a license. Also autopano-sift is slow. In the Hugin repository we have a faster version, autopano-SIFT-C. The last version (2.5.0) had severe memory leaks. 2.5.1 is due out soon and fixed them. You may want to consider Panomatic instead, although the algorythm (SURF) is also patented. We (the Hugin/Panotools team) had two Google Summer of Code projects to build patent-free versions of the two elements (feature detector and feature matcher) that make a control points generator. So far the results are not to par with the patent-encumbered ones. You should also mention Enblend as part of your toolbox - http://enblend.sf.net - this is the workhorse that blends Hugin's stitches into a single seamless panorama. And you may want to look at the Panotools - http://panotools.sf.net as a way to further script / automate the Hugin process. Hugin is a GUI to all these functionalities, and if you would like to contribute your Python scripts, maybe one of the next Hugin version will ship with this functionality as well?.
craig rasch cpo ret 09.30.2009
would like to contact you on future advancements of your street view. i would like to do a street view type presentation of the museums here in San Diego. please contact me at crasch48@gmail,com.
Steve 09.30.2009
This sounds like something that would be useful for the OpenStreetMap project for recording POI's and tagging road names after recording the road data..
Gene Ressler 09.30.2009
"About the Author" could also include that Roy is going to Germany as an active duty Army Second Lieutenant. He did this project as the project portion of a 3-credit course while he was taking five others. Pretty impressive stuff..
John R 09.30.2009
Way cool!!! One note about your PC though - the problem is you don't have enough RAM - for $30 you can upgrade your memory from 512MB to 2GB and it will be much faster..
Phil Harvey 09.30.2009
Exiftool will also do geotagging from GPS logs, so if you are lucky you could save the Gpicsync step..
John McKerrell 09.30.2009
I've recently launched a project called "OpenStreetView" which would be a great home for your images, head over to http://openstreetview.org/ to take a look. The idea is that there's lots of people trying out this sort of thing and this gives them a single central place to host all the photos. This also means that if someone is interested in writing a stitching tool or visualisation software, they can concentrate on that without having to work on the hardware or take photos themselves. Currently the site is quite basic but if you can upload images with lat/lon in the EXIF headers then they should show up on the site. Ping photos AT openstreetmap DOT org if you want to discuss it..
Clyde 09.29.2009
It should be noted that "shoots 8-megapixel still images" is a marketing gimmick. Yes I'm sure you can get a picture with 8 million pixels, but it is done through interpolation from a smaller (roughly 2 million pixel from available specs) image. .