Sensor fusion and integrated MEMS are essential tools for today’s athletes
A golfer stands in the dreaded sand trap, carefully considering how to balance his weight as he eyes the ball. He takes a few practice swings. If he swings too deeply, he’ll hit the ground and lose another stroke. It’s a tough shot, but he swings without hesitation. Embedded in his club are microelectromechanical system (MEMS) devices—tiny machines with elements about the thickness of a human hair. These devices aren’t going to swing the club for him, but he’s been using them to analyze his swing and practice this shot. Maybe this time he’ll make it.
The wild popularity of smartphones and wearables has been driving down the cost of MEMS devices, including accelerometers, gyroscopes, magnetometers, and pressure sensors. These minuscule chips help to count steps, track calories burned, and monitor heart rate. Such data are useful, sure, but while these devices may nudge users to be more active, they don’t actually improve a swing, a punch, or a kick. To do so means moving sensors off the wrist and into sports gear—and that’s quickly happening. Indeed, you can now buy sensor-based equipment that can boost your performance, not only for golf but also for tennis, baseball, boxing, and soccer.
Equipment: Babolat Pure Drive Play racket
Sensors on board: InvenSense motion-sensing system embedded in its handle
What it tracks: Type of shot made (forehand, backhand, serve, and smash), location where the tennis ball connects with the racket’s strings, overall playing time versus active playing time, and ball speed during play and when serving
How it works: Players can connect the racket via Bluetooth to a mobile device to share data with a coach for postgame analysis or over social media.
Commercial status: Announced at French Open 2013; available now at US $200
Measuring the often-complex motion of an athlete takes many more metrics than just tracking steps. Consider the sport of sculling: To gauge the efficiency of a rower as his oar moves through the phases of catch, drive, release, and recovery, you need to track the movement of his legs, back, and arms. If you want to analyze a baseball player as he whips the bat around, you need to consider rotational angles and swing speed.
The current boom in sensor-laden sports equipment is an outgrowth of the dramatic drop in sensor prices and the development of technology that makes it easy to integrate data from multiple sensors. In the past, synthesizing the data outputs from multiple sensors in real time was a nightmare. Figuring out how to make sense of streams of data flying into a general-purpose processor—so-called sensor fusion—presented a real hurdle to developing sports-training products that used such data, even though the need was clear.
In the past couple of years, though, a number of companies have come out with off-the-shelf technology for sensor fusion. Now, rather than develop an algorithm from scratch that measures linear acceleration, for example—which is useful in measuring the speed of a kicked or hit ball—a design engineer can use a third-party linear-acceleration algorithm to measure motion across the x, y, and z axes.
Sensor on board: Tri-axis accelerometer
What it tracks: Punch types (jab, cross, hook, uppercut), speed, power, hit/miss ratio, blocks, combinations, punch rate, pacing
How it works: A small device clips onto the boxer’s hand wraps and fits under the boxing gloves. An onboard microprocessor analyzes the accelerometer data to detect punches and extract key features from each punch’s waveform. The device then sends this data via Bluetooth to a smart device for display and storage to build a picture of the boxer’s performance in each round, session, week, and month of training.
Commercial status: Announced in January 2016; will begin shipping in August 2016, with a pari of devices sold for US $70
Equipment: miCoach Smart Ball
Sensors on board: Tri-axis accelerometer/tri-axis gyroscope combo
What it tracks: How hard the ball has been struck, speed, spin, and flight path
How it works: Soccer ball with integrated sensors tracks the dead-ball kicking used for penalties, shooting, corners, long passes, and goal kicks; transmits data via Bluetooth to an iPhone or Android device. Coaches can use this data to train players.
Commercial status: Announced May 2014; available now for $200
Sensor-fusion hardware and software have also taken on another tricky task: managing power consumption, which is key if you want to avoid having to frequently charge your smart balls, bats, boxing gloves, tennis rackets, or golf clubs. These systems may, for example, turn off a gyroscope, which is notoriously power hungry, when not in use. They may allocate work to the most efficient sensors, like an accelerometer or an accelerometer-magnetometer combination instead of a gyro, if it can do the job.
While both sensor manufacturers and the companies that make tools to process sensor data agree that sensor fusion is a good idea, they have tackled the implementation problem in different ways, creating systems that range from very proprietary to essentially open.
On the proprietary end of that spectrum, the makers of sensors and MEMS—such as Analog Devices, Bosch Sensortec, Freescale, InvenSense, Kionix, and STMicroelectronics—are running their own sensor-fusion algorithms on the chips they build that include multiple sensors and a microcontroller, for example Bosch Sensortec’s BN0055. Some of these manufacturers also provide application programming interfaces that can run on other hardware but work only with one company’s brand of sensors. The combo sensor-microcontroller chips allow for data batching, which means that they’ll store sensor readings in a buffer until the application processor is awake. This power-saving mechanism is a big benefit of sensor fusion.
In a more flexible approach, Hillcrest Labs, PNI Sensor, and QuickLogic have developed stand-alone sensor-fusion coprocessors that support chips from a variety of sensor makers. These hardware-software systems poll the sensors for data, push it to their own low-power coprocessors, and manage the data using proprietary algorithms.
Photo: Sony Corp.
Equipment: Sony Smart Tennis Sensor
Sensors on board: Three-axis motion-tracking sensor and vibration sensor
What it tracks: Shot type (serve, smash, volley forehand, volley backhand, topspin forehand, topspin backhand, slice forehand, and slice backhand), ball speed, swing speed, ball spin, and ball impact spot; will match shot data to video recorded on smart device
How it works: Attaches to the grip end of a compatible tennis racket.
Commercial status: Released in Japan May 2014; in United States 2015; US $200
Manufacturer: Sony Corp.
Photo: Diamond Kinetics
Sensors on board: Tri-axis accelerometer and tri-axis gyroscope
What it tracks: Fifteen different swing metrics, including power, speed, efficiency, and distance the bat travels in the hitting zone
How it works: Mounts to the bottom of a bat, capturing 11,000 data points per second to analyze swing data in real time; shares that information with coaches via Bluetooth to a mobile device.
Commercial status: Announced 2014; available now for US $150
Manufacturer: Diamond Kinetics
Finally, my organization, MEMS & Sensors Industry Group, has rolled out an open-source option that can help designers jump-start development of systems that require the use of sensor fusion. Our Accelerated Innovation Community, which began in late 2014, makes some of the most popular sensor-fusion algorithms available for free to design engineers.
With cheap sensors and simple sensor fusion on the market, the current wave of smart sports gear is not likely to crest anytime soon.
Sensors have already been inserted into just about every type of sports equipment that can be swung, punched, or worn, and more are in development. Noel C. Perkins of the University of Michigan and William Clark of the University of Pittsburgh developed one recent example, a smart baseball bat add-on. The prototype is a small, sensor-laden circuit board affixed to the knob end of a bat. It contains MEMS chipsets that measure the three components of the bat’s acceleration as well as rotation rates around three orthogonal axes. A smartphone collects the data and analyzes the swing time, speed, control, and other factors that relate to a successful hit.
The University of Michigan has licensed the technology to Clark’s new company, Diamond Kinetics, which in 2014 rolled out its first commercial product, SwingTracker, a lightweight sensing accessory that attaches temporarily to the knob of the bat. The technology, which Perkins originally developed to improve his fly fishing, can and likely will be applied to other sports.
Sensors in athletic clothing, already in limited use by elite athletes, will soon become accessible to ordinary consumers. These can be used to analyze body motion for multiple sports. Such wearable gear is still in the nascent stages of evolution. But more than one startup has already begun to sell sensor-equipped workout clothes. California-based MAD Apparel, for example, claims that its US $400 Athos Upper Body Package analyzes in real time how hard an athlete’s muscles are working by using a shirt with 14 embedded biosensors and a hardware “core” positioned over the heart.
Equipment: Lumo Run running shorts and capris
Sensors on board: Nine-axis inertial measurement unit (accelerometer, gyroscope, magnetometer), pressure sensor
What it tracks: Measures 14 different running metrics, including pelvic rotation, tilt and drop, braking, bounce, ground contact time, cadence, and stride length
How it works: A removable sensor in the waistband provides real-time audio cues via Bluetooth to headphones and data to a smart device for detailed postrun analysis.
Commercial status: Announced October 2015; shipping in March 2016; US $150 (shorts), $170 (capris)
Photo: Wilson Sporting Goods Co.
Equipment: Wilson X Connected Basketball
Sensor on board: Tri-axis accelerometer
What it tracks: Shots made and shots missed
How it works: An algorithm developed by a machine learning system detects shots made and missed based on accelerometer data. Requires standard-height hoop with a net, communicates to a smart device via Bluetooth technology; nonreplaceable battery tracks at least 100,000 shots.
Commercial status: Available in United States now, Canada by end of March 2016; US $200
Manufacturer: Wilson Sporting Goods Co.
Coming soon using the same technology: Wilson X Connected Football
What it tracks: Throw distance, speed, spiral efficiency, and whether a football has been caught or dropped
Another maker of smart clothing is the Dutch company Xsens Technologies, which Fairchild Semiconductor purchased in April 2014. Xsens is adapting its full-body motion-tracking suits (used to make animated movies and video games) to sports applications. Independent researchers are using Xsens’s bodysuits for such sports as ski jumping, shot put, archery, baseball pitching, rowing, and even wingsuit flight.
And, in another sign that smart clothing is moving toward the mass market, Under Armour chief digital officer Robin Thurston has said his company is developing smart clothing and expects to have commercial products within the next two to three years.
For many sports, the remaining challenge lies not in collecting data but in making that information accessible and meaningful. Both professional athletes and casual sports enthusiasts want to ensure that they are maximizing the effects of their time at the gym. That’s coming.
Within a few years, our MEMS-enabled golfer should have plenty of help for his game: His smart eyewear will give him a hole-by-hole analysis of the course and will record and instantly play back each shot while MEMS motion sensors in his shoes will help him position his feet. His clothes will have sensors sewn into the fabric so he can adjust his hips and legs appropriately. He’ll swing his MEMS-based golf club to strike a MEMS-enabled golf ball, connecting just right. The ball will land on the green, transmitting its short distance to the hole immediately to his smartphone. At long last, he’ll really enjoy the sport of golf.
The HITS of Concussion Detection
Hit Hard: David Bruton Jr. of the Denver Broncos lies on the ground after a hit in 2014; he left the game with a reported concussion. Photo: Doug Pensinger/Getty Images
I can’t describe the use of sensors in athletics without mentioning where they will have the biggest impact—literally. I’m talking, of course, about concussion detection. The numbers are staggering—and growing, particularly for children who play sports. According to the Centers for Disease Control and Prevention, diagnoses of concussion or traumatic brain injury rose a whopping 57 percent in children 19 years old and younger between 2001 and 2009. There were more than 248,000 sports-related concussions in children in 2009 alone.
Today’s push for more extensive concussion analysis stems from a Simbex Study funded by the National Institutes of Health. More than eight years of detailed data from football practices and games at all levels, from Pop Warner Little Scholars to the National Football League, showed that concussions occur far more often and have longer-lasting effects than previously believed.
The Head Impact Telemetry System, or HITS, developed for that study is now the property of the football helmet manufacturer Riddell, which acquired HITS in 2004. HITS uses multiple single-axis MEMS accelerometers to record the location, magnitude, duration, and direction of impacts. Players and coaches can upload and evaluate the data either right there on the playing field or after the game.
Football, Ice Hockey*
Equipment: Reebok Checklight skullcap for head-impact detection
Sensors on board: MEMS accelerometers embedded in MC10’s conformal electronics
What it tracks: Potential severity of head impacts
How it works: MEMS accelerometers track direct accelerations to the head; red and yellow indicator lights display the severity of impact.
Commercial status: Announced July 2013; available now
*and other helmeted sports
“The HITS system showed that there are actually two events in each impact—a high g-force event on initial contact and then a low g-force event due to the percussive reaction of the head,” says Rob O’Reilly, a senior member of the technical staff at Analog Devices. “At first, we were adjusting the same single-axis technology used in automotive airbag deployment sensing, and a lot of people thought we just couldn’t do it.”
While that technology worked reasonably well, Analog Devices introduced the ADXL377, a three-axis MEMS accelerometer able to measure up to 200 g’s of force instantly and accurately while withstanding peak impacts of up to 10,000 g’s. Intended specifically for contact sports of all kinds, that accelerometer is small enough to fit easily inside an earpiece. The military is already using the company’s ADXL375, a 200-g digital-output accelerometer, in a body-worn blast gauge for sensing the likelihood of traumatic brain injury from explosions. Data from the use of this sensor will improve the technology and likely drive a broader adoption in youth sports.
A handful of companies are building accelerometers and gyroscopes to gauge the possibility of brain injury into products worn by nonhelmeted athletes playing such sports as soccer and rugby. X2 Biosystems in Seattle makes technology-laden sports mouth guards, and Triax Technologies of Norwalk, Conn., is selling sensor-studded headbands and skullcaps. Reebok markets the US $100 Checklight skullcap, which uses accelerometers incorporated into a mesh of stretchable electronics to measure the severity of an impact. Even U.S. Women’s National Team soccer star Abby Wambach is in the game. Having sustained some serious concussions because of her affinity for heading the ball, Wambach has partnered with Triax on the launch of its head-impact monitoring products.
What’s still debatable among athletes, coaches, and parents with regard to concussion-detection equipment is its accuracy. According to recent Washington State University research, some of the concussion-detection products (such as smart mouth guards) used for nonhelmeted sports aren’t fast enough to measure hard hits and don’t accurately measure angular hits, which may be more dangerous than straight-on impacts.
This doesn’t invalidate the merit of concussion-detection equipment for nonhelmeted (or helmeted) athletes. It just means that engineers must work more closely with sports-equipment manufacturers to produce products with greater accuracy. —K.L.
About the Author
Karen Lightman is executive director of MEMS & Sensors Industry Group. She began exploring microelectromechanical systems in sports a few years ago when her daughter sustained a serious concussion while skiing. Lightman realized that a more advanced ski helmet, with built-in MEMS-based concussion-detection technology, could have provided potentially valuable information to her daughter’s medical team. She also suggests that sensors in ski-lift passes might help detect dangerous skiers and prevent accidents.
This article appears in the March 2016 print issue as “Silicon Gets Sporty.”