Someone who is a data scientist today is said by Harvard Business Review to have the sexiest job alive . And if sexy isn’t enough, how about being a savior of the economy? According to a 2011 report by consulting company McKinsey & Company, “Big Data” is “ the next frontier for innovation, competition and productivity .” That is, of course, if enough of those sexy data scientists can be found.
For also according to McKinsey’s report, “the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions,” by 2018.
However, Peter Sondergaard, senior vice president at Gartner and global head of research asserts that the shortage situation is even more frightening than what McKinsey implies. Sondergaard stated in October 2012 that, “By 2015, 4.4 million IT jobs globally will be created to support Big Data, generating 1.9 million IT jobs in the United States. In addition, every big data‐related role in the U.S. will create employment for three people outside of IT, so over the next four years a total of 6 million jobs in the U.S. will be generated by the information economy.”
Wow. Not only will Big Data make a significant dent in the U.S. unemployment rate, but the U.S. IT technical workforce of 3.9 million or so needs to increase by almost 50 percent within the next two years.
But wait, there’s more.
Photo: Baran Ozdemir/Getty Images
According to a 2012 IDC study [PDF] commissioned by Microsoft , cloud computing ( which made storage sexy ) is going to require 2.7 million cloud-related IT specialists in the U.S. and Canada by 2015 as well.
And as icing on the IT worker short(age) cake, the U.S. government’s $30 billion electronic health record (EHR) initiative has apparently created an even greater shortage of health information technology workers over the next few years, above the 50 0000 the U.S. government estimated, according to a report earlier this year by the consulting company Price-Water House Coopers. Unfortunately, I can’t find anyone who says these HIT jobs are sexy, so they may go wanting.
Given the intense competition for IT workers these thee shortage projections imply (and this doesn’t include the recent claims that corporate IT spending will take off from 2014 to 2016 which should increase IT worker demand at least some), one would expect a huge, surging corporate and IT industry demand for IT professionals across the board, with job position listings, salaries and perks soaring as they did during the late 1990s Dot.com boom . I would also expect government to publish worrisome studies about the issue, like the U.S. Department of Commerce did in 1997, under the title of America’s New Deficit: The Shortage of Information Technology Workers [PDF].
I would also expect those 100 high tech executives wanting an increase in the number of H-1B visas issued would be baying for the limit to be raised to 500 000 or more instead of a measly 115 000. And why aren’t these executives saying they have tens of thousands of open big data, cloud computing and HIT positions, instead of a paltry 10 000 being used as justification for more H-1B visas?
But strangely enough, recent salary and anecdotal data doesn’t show this happening. Nor are tech companies like Cisco, IBM or HP (some of the ones which are the ones begging for more H-1B visas) rescinding their decisions to lay off thousands of IT workers. And those laid off IT workers don’t seem to be sucked into some huge Big Data-Cloud-HIT worker shortage whirlpool that should be growing by the day out there somewhere.
Perhaps Cisco, IBM and HP among others have discovered that perhaps these predictions are more fiction than reality. Even Gartner has recently admitted that Big Data may be a wee bit overhyped. Or maybe, in the words of a recent New York Times article , Big Data is really just a Big (unsexy) Dud, and no one wants to admit it.
We’ve been here before, some 15 years ago, during the great computer programmer shortage scare . Projections at the time said that by 2008, the U.S. would need 839 000 computer programmers [PDF] by 2008. The actual number computer programmers actually employed was in 2008 closer to 427 000 , due in part to outsourcing.
And speaking of outsourcing, an article in CIO magazine last week reports that a study by The Hackett Group says that since 2002, offshoring, new technology productivity improvements, and low business growth will have “killed” 1.5 million IT jobs across North America and Europe by 2017 after new job creation is factored in. The losses were roughly the same for both continents. Perhaps The Hackett Group hasn’t heard about the Big Data-Cloud-HIT worker shortage either?
Even out in Silicon Valley, which is begging for more H-1B guestworkers because of a shortage of engineering and IT talent, all is not what it seems. As described in a recent New York Times article, the San Francisco talent search company Bright found that after examining three million of its résumés of job seekers in the United States, there are “ plenty of potential candidates [who] exist for thousands of positions for which companies want to import guest workers. ”
Steve Goodman, Bright’s CEO, was quoted by the Times as saying, “I didn’t expect this result.”
The Bright report [PDF] shows only a couple of jobs such as application developers or specialized computer occupations where hiring H-1B guestworkers might possibly be appropriate. However, in the other job categories where shortages are being claimed by high tech companies, qualified U.S. job seekers who could do the job already exist.
Goodman, whose company, the Times says, earns its revenue in part by placing qualified candidates with recruiters went on to say, “We’re Silicon Valley people, we just assumed the shortage was true. It turns out there is a little Silicon Valley groupthink going on about this, though it’s not comfortable to say that.”
Without trying to be too cynical, the claims of an impending multimillion IT worker shortage in the next 30 months almost make Bill Gates look like Nostradamus with his 2004 prediction that, “Two years from now, spam will be solved.”
Robert N. Charette is a Contributing Editor to IEEE Spectrum and an acknowledged international authority on information technology and systems risk management. A self-described “risk ecologist,” he is interested in the intersections of business, political, technological, and societal risks. Charette is an award-winning author of multiple books and numerous articles on the subjects of risk management, project and program management, innovation, and entrepreneurship. A Life Senior Member of the IEEE, Charette was a recipient of the IEEE Computer Society’s Golden Core Award in 2008.