Restoring Microgrids After Power Loss Requires Smarts

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

It’s well known that microgrids can offer resiliency against power outages, but what happens when microgrids themselves run out of moving electrons? Researchers have tested this question, using machine learning as a tool for optimizing power restoration on microgrids after power outages.

Microgrids are, as the name suggests, like regular power grids but on a smaller scale. They contain interconnected systems of energy sources and energy loads that can operate alongside or separate from the larger power grid. Microgrids often include renewable energy sources, and they can be as small as a group of residential buildings with solar panels on the roofs. Or they can be citywide—with solar, wind, fuel-powered generators as well as various energy storage devices.

However, if there’s a power outage, load restoration on a microgrid isn’t as simple as flipping all the switches back on.

Researchers at the University of California, Santa Cruz, in fact tried using deep reinforcement learning to manage the load restoration process in bringing a microgrid back online after a power loss. Reinforcement learning is a type of machine learning where a model learns actions that achieve desirable results through interactions with its environment. Yu Zhang, a coauthor on the study and an assistant professor of electrical and computer engineering at Santa Cruz, was inspired to apply reinforcement learning to the load restoration problem due to his firsthand experience with frequent power outages at his university campus. The campus is remote, and a single downed tree on a transmission line during high wind can cut power to much of the campus, ruining ongoing laboratory experiments and damaging expensive computing equipment.

Practical constraints like the cost of running fuel-powered generators and the fluctuating availability of renewable energy make it a complex optimization problem. At any given point, loads can draw from any of the energy sources on the microgrid, and the energy storage systems can also store or output energy.

Which is part of the reason why it’s not so easy to turn the system back on.

Today, microgrid operators often lean on an optimization technique called model predictive control is currently used to decide which actions to take during a microgrid’s load restoration process. But one of MPC’s drawbacks is a reliance on high quality forecasting, which isn’t always available. Load restoration uses weather forecasting to predict the future availability of renewable energy sources and demand-side forecasting to predict when and how much power the loads on the microgrid will need to consume.

Zhang and colleagues, by contrast, used a style of reinforcement learning that can take constraints into account during the training process. The model stays within the given constraints—such as not allowing energy storage devices to both store and output energy at the same time—while optimizing for the maximum amount of loads restored and minimum amount of fuel used by generators.

Zhang says the training process begins with the model taking random actions. After each action, it reassesses the environment and determines how the results compare to the goal and to previous results. For load restoration, the training model calculates the amount of load restored and the cost of the restoration, then adjusts future actions to optimize for better results. Training the model took several hours, Zhang says, while using the model to determine proper actions for load distribution required only seconds.

When given low-quality forecasting data, Zhang’s team found, reinforcement learning models outperform MPC-trained models. Zhang says this may be due to reinforcement learning’s ability to learn the signals that accompany inaccurate forecasts. If the model judges a forecast to be inaccurate, it can choose to rely on it less.

“During the learning process, the model can know if your forecast is good or bad—if it underestimates or overestimates the reality,” Zhang says. “Then the [model] will take the forecast error into account when it helps the microgrid controller make decisions.”

While reinforcement learning models outperform MPC models for load restoration problems overall, the difference can be slight. Zhang says it isn’t enough to make an overwhelming case for abandoning the MPC technique in favor of reinforcement learning, and further exploration of each technique’s strengths and weaknesses are needed. Different circumstances can tip the scale toward one or the other technique, including reinforcement learning’s ability to train more complex models given more time and computing power, and the variable of forecast quality.

Zhang says he’d like to apply the new technique to the Santa Cruz campus’s power outage problem and see how the system performs in practice. He says his research team is also working on other issues that affect the power system, including techniques to better forecast the availability of renewable energy.