Imagine you're a farmer in the northern United States. It's early spring, and nighttime temperatures are just starting to rise above freezing. You need to fertilize your newly-planted crops, but you also know that at freezing temperatures, the fertilizer will kill your crops. The weather forecast out of the closest town, which is 50 miles away, predicts that temperatures will stay above freezing for the next few days, and so you decide to go ahead and fertilize. But that night, temperatures in some parts of your fields dip below freezing. Over a quarter of your crops die.
Unfortunately, this is a common situation, especially when weather data comes from farther away. Now, researchers at Microsoft have developed a framework called DeepMC that can very accurately predict local weather, and could be used by farmers, renewable energy producers, and others. Microsoft researchers presented a study on the framework and its application at the Association of Computing Machinery's Conference on Knowledge Discovery and Data Mining in August.
DeepMC uses machine learning and artificial intelligence to localize predications related to weather and climate. It combines two different sources of data: One from on-site sensors, and the other from standard local weather forecasting data. DeepMC gets that data directly from application programming interfaces (APIs) that come from sources like the National Oceanic and Atmospheric Administration, Dark Sky, and the National Weather Service.
"We have a fusion mechanism by which we combine these two signals," says Peeyush Kumar, a senior research scientist at Microsoft Research and the study's lead author.
DeepMC works by training an AI to precisely find the error between the local weather forecast and the micro-climate weather conditions. The system uses historical data on both weather forecasts and local sensor data for training, and predicts each weather parameter, like temperature and wind speed, individually. The system also uses a method called decomposition to find both short-term and long-term trends and patterns in weather data, which Kumar says makes it even more accurate.
Applying machine learning and AI to weather prediction in this way isn't new, says Andrew Blum, journalist and author of The Weather Machine, a book exploring the science, history, and future of weather prediction. "There has definitely been a trend recently, of machine learning scientists kind of poking out a little bit," he says. IBM, who owns The Weather Company, already uses machine learning to make predictions more accurate, as do other companies like Tomorrow.io. Dark Sky, a weather app recently bought by Apple which DeepMC has used as a data source, already uses machine learning to localize weather predictions.
Kumar says that DeepMC is more accurate than any other comparable model. In the study he worked on, researcher examined four real-world uses of DeepMC—predictions of wind speed, solar radiation, soil moisture content, and temperature. In all cases, the prediction outperformed comparable models. In the case of temperature prediction, the researchers followed an actual farmer in Eastern Washington during early spring using DeepMC to time when he fertilized his crops. The researchers found that temperature forecasts made by DeepMC were over 90% accurate. Kumar says that accuracy comes from the way the model decomposes signals into trends that can be extremely specific to an area.
Microsoft researchers say the DeepMC framework could be useful for applications beyond local weather prediction. This is because the framework is generic—it works for many types of data. In the study, the researchers examined how radiation predictions made with DeepMC can be used to predict energy generation on a commercial solar farm, but Kumar says he can also imagine the model being used more generally by energy grids to predict how much energy people will use. Some farms are starting to develop their own microgrids, in some cases including generating their own renewable energy, for which Kumar says that DeepMC would also be useful.
Of course, weather affects everyone. As the Earth's climate changes, some weather is becoming more extreme, causing serious flooding, stronger hurricanes, and intense heat waves. If weather forecasts become available only as a private good, not a public one, it might be that only richer countries and individuals would have accurate forecasts. That would effectively mean "the entire system of global weather observation exchange will fall apart," says Blum.
Kumar says that DeepMC is not intended to predict extreme weather events. He also says Microsoft Research wants to make the framework as accessible as possible, adding that just under 1,000 people and businesses around the world have already used it. In another study presented to the United Nations Food and Agriculture Organization and co-authored with farmer and DeepMC user Andrew Nelson, the researchers presented the framework as sustainable and affordable.
At the same time, Kumar acknowledges that it takes time, money, and resources to implement the system, though he couldn't provide the cost. But he says that the framework is for anyone who wants to use it, from small farms to corporations.
"The vision that we started with is that we want to bring, what we want to do is we want to democratize decision making," he says.
Rebecca Sohn is a freelance science journalist. Her work has appeared in Live Science, Slate, and Popular Science, among others. She has been an intern at STAT and at CalMatters, as well as a science fellow at Mashable.