Feeding the World With Big Data

Buckets of Data: Agriculture experts say that data-sharing will make Africa more productive.Photo: ISSOUF SANOGO/AFP/GettyImages

Farmers today produce three times as much food as they did 50 years ago using just 12 percent more land, thanks to new technologies and better farming practices. But the global playing field isn’t level. In Africa, farmers produce a fraction of what they could, according to the Forum for Agricultural Research in Africa, and most barely get by, struggling against infertile soil, drought, and diseases.

Helping farmers—in Africa and elsewhere—produce more will be key to lifting millions out of poverty and sustainably feeding a world population of 9 billion in 2050. Food-policy experts believe that a crucial step toward that goal is to give farmers, scientists, and entrepreneurs unhindered access to agricultural data which is generated at research centers worldwide. At a two-day international conference on open data in agriculture last week, leaders of the G8—the world’s eight wealthiest countries—brainstormed the best ways to make data available without restrictions, in formats easy for humans and machines to parse.

“Agricultural data is interesting because it comes in several flavors,” says James Hendler, computer science professor at Rensselaer Polytechnic Institute, in Troy, N.Y., who guides the U.S. government’s Data.gov website. There are deeply detailed data sets on things like plant genomics and local weather conditions. Then there are broad data sets on such topics as the best crops for certain soils, changes in rainfall levels, signs of pests and diseases, and anticipated prices at local markets.

If these data sets are made freely available, the possibilities for their use are endless, says Piers Bocock of the CGIAR Consortium of International Agricultural Research Centers, in Montpellier, France. At the conference, experts from universities and research institutions presented apps they’ve developed using data that’s already publicly available. These included MyFarm, an Android-based country-specific multilingual app that helps train farmers to give agricultural advice to other small farmers, and Aqueduct, an interactive tool that provides high-resolution maps of water-related risks.

In Africa, where even the poorest farmer carries a cellphone, open-data evangelists envision an incredible—and not completely improbable—scenario. “Imagine this,” Bocock says. “A woman standing in a field in Malawi has just borrowed money to start her own farm. What if an app on her mobile phone geo-locates her and then, from this ever-growing data ecosystem of knowledge, is able to identify the soil type and needs of that specific field, and then tell her where, locally, she can buy the seeds she needs and when to plant, harvest, et cetera?”

Making such “what if” scenarios a reality will require increasing amounts of free, accessible agricultural research data that’s easy to use—not just by humans but also by machines. Much of the data has been collected by scientists at universities and research centers and made purposefully inaccessible for security or privacy reasons. “There’s the culture of ‘I don’t want to share it, it’s mine,’ or ‘It’s government property,’ ” says Bocock.

Even if some data is free, it’s not necessarily easy to find and use. For instance, Hendler says there is a growing trend for governments and institutions to make data available in raw form or through application-programming interfaces (APIs), which can be used in data-manipulating software or to create mobile apps. However, such data sets may be difficult to locate on the Internet. They may also be poorly documented with missing units and annotation or be otherwise hard to read and apply.

Hendler, of Semantic Web fame, likens the current situation to the pre-Web hypertext world, when there were no standards for sharing information nor any search engines to find it. There’s a need for new approaches and technology that make it easier to find and interact with data, he says.

The solutions are, of course, not limited to agriculture. One idea being discussed is simpler and more URL-based metadata, the descriptive data about the data sets. This would make data sets more accessible to search engines and allow linking to databases from websites and from other databases. There is also a need for better visualization and analysis tools, says Hendler, as well as standards for sharing, archiving, and interacting with databases.

The G8 open-data conference is a sign of change. At the conference, several countries released action plans to make agricultural data streams available. Canada, India, and the United States, among others, are pushing for an open data-sharing platform. The U.S. Department of Agriculture, for instance, announced the launch of a portal on the Data.gov website, which links to 348 agriculture data sets.

Still, truly open, accessible data on a large scale will require years of effort among G8 and partner countries, international organizations, and the private sector. “At the heart of all this is a change in culture,” says Bocock. “The old-school model of research is that the data collected is the gold mine: ‘Why would I give that to someone else?’ Open data is turning that model on its head to change the world.”

Look for our special report, “The Age of Plenty” in the June 2013 issue.