Combined, Artificial Intelligence (AI) and big data offer researchers in every field opportunities that didn’t exist a few short years ago.
But as more data has become available to researchers, the ability to analyze this data in a meaningful way has become more and more problematic. In the “hard” sciences and especially in the social and other “soft” sciences, analyzing the raw data can be a monumental task.
In general, machines are capable of analyzing large datasets for quantitative information, but detecting trends or overarching themes — in other words, qualitative information — still requires the human touch. Human minds, not machines, are needed to do qualitative analyses.
That might be changing thanks to advances in artificial intelligence. A recent study by a team from the University of Bristol,“Content Analysis of 150 years of British Periodicals,” confirms that analyzing historical data using AI techniques can reveal qualitative outcomes as well as quantitative outcomes.
In the study, researchers at Bristol used artificial intelligence tools to analyze 35 million articles from a hundred British newspapers spanning the years 1800–1950 (a total of 30 billion words). As they expected, analyzing the digitized text disclosed coverage of significant historical events such as the Napoleonic Wars and World War I.
But the analysis also uncovered trends and events the researchers didn’t expect to uncover. For example, the researchers could clearly see how gender bias and cultural tastes changed over time. Their artificial intelligence tools traced the decline of the steam engine and the corresponding rise in the use of electricity. The researchers were able to pinpoint when trains overtook horses as the favorite means of transportation (1902).
Among other surprising discoveries, the Bristol researchers found that politicians and writers have the best chance of becoming celebrities in their lifetimes; scientists and mathematicians are less likely to become famous, but if they do become famous their notoriety lasts longer.
These broad discoveries hold promise for future researchers not just in the hard sciences (such as physics), but in the soft sciences (such as sociology) as well.
Said Dr. Tom Lansdall-Welfare, a computer scientist at the University of Bristol who led the computational part of the study, “What cannot be automated is the understanding of the implications of these findings for people. That will always be the realm of the humanities and social sciences, and never that of machines.”
Understanding the implications of a study will always be a task for humans, but the Bristol research demonstrates that AI can nevertheless be a very valuable partner in the qualitative analysis of information. AI technology allows humans to find deeper meaning in the subjects they explore. AI can now make associations and find inferences — tasks that not so long ago were thought to be strictly in the realm of human thinking.
Said Ryan Welsh, CEO of Kyndi, “The goal is not to replace knowledge workers, but to make them one hundred times smarter, a hundred times faster.”