U.S. government commits big R&D money to 'Big Data'

Summary: The U.S. government is investing $200 million in big data projects to help the U.S. jump ahead in the next frontier of computing.

Photo credit: iStockphoto/Nikada

Calling it one of the most important public investments in technology since the rise of supercomputing and the Internet, six U.S. federal agencies teamed up on Thursday to announce a new $200 million investment "to greatly improve the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data" said the White House Office of Science and Technology Policy (OSTP).

This is a pure research and development initiative that will manifest itself as public/private partnerships and new projects that will drive big data investments in government, education, and business. The catalyst for this move came from the President's Council of Advisors on Science and Technology, which recommended investing more in big data in order to deal with some the biggest challenges in the U.S., including issues in health care, energy, and defense.

The agencies involved in the announcement were:

• National Institues of Health
• Department of Energy
• National Science Foundation
• Department of Defense
• DARPA
• U.S. Geological Survey

The most obvious agencies missing were NASA and the National Oceanic and Atmospheric Administration, which both gather massive amounts of data that could benefit from big data tools and serve the new cause. White House officials said that both agencies will eventually be involved.

The OSTP's stated goals for the $200 million will be:

• Advance state-of-the-art core technologies needed to collect, store, preserve, manage, analyze, and share huge quantities of data.
• Harness these technologies to accelerate the pace of discovery in science and engineering, strengthen our national security, and transform teaching and learning; and
• Expand the workforce needed to develop and use Big Data technologies.

Confusingly, the Department of Defense also stated that it is "placing a big bet on big data." It stated that it will invest $250 million annually in big data projects across various departments of the military. The DoD said that it wants to "harness and utilize massive data in new ways and bring together sensing, perception and decision support to make truly autonomous systems that can maneuver and make decisions on their own." That's the kind of thing that will scare some people because it sounds like robots and drones that are going to become smart enough to make their own decisions. It's the stuff science fiction writers have been anticipating, and in some cases fearing, for over half a century. It's unclear whether the DoD's $250 annual million investment is separate from the overall $200 million R&D that the OSTP announced.

John Holdren, Director of the White House Office of Science and Technology Policy, said, "In the same way that past federal investments in information technology R&D led to dramatic advances in supercomputing and the creation of the Internet, the initiative we are launching today promises to transform our ability to use Big Data for scientific discovery, environmental and biomedical research, education, and national security."

In the joint press conference, the agency chiefs not only threw out flowery hyperbole about the potential impact of big data in helping solve some of the most important problems that the U.S. is facing in the years ahead, but there were also bureaucrats and technologists who clearly have a deep understanding of the engineering and computer science behind big data and were very enthusiastic about this investment. They were confident that it's going to enable the U.S. to take a big step forward in one of the next frontiers of computing.