8 surprising things data science has revealed about us over the past decade

Big data analysis has long supported major feats in physics and astronomy. But more recently we’ve seen it underpin breakthroughs in the social sciences and humanities.

Since the landmark paper Computational Social Science was published in 2009, a new generation of data analytics tools has given researchers insight into fundamental questions about how we communicate, who we are and what we value.

For instance, by analysing the relative frequency of certain words in historical texts, researchers can identify important changes in our use of language over time.

In some cases these shifts will be obvious, such as the use of archaic words being replaced by more contemporary words. But in other cases, they may reflect more subtle but widespread social and cultural changes. Below are some of the most influential data-centric discoveries from the past 10 years.

How we communicate

Over the past decade, a growing number of global open data sources have helped researchers reveal patterns in what we read, write and pay attention to. Google Books, Worldcat and Project Gutenberg are just some examples.

The release of the Google Books n-gram viewer in the early 2010s was a game changer on this front. Using the entire Google Books database, this tool shows you the relative frequency of a specific term or phrase as it has been used over hundreds of years. Researchers have used this data to explore the systematic suppression of the mention of Jewish painters, such as Marc Chagall, in German books during World War II.

Data analysis can also reveal patterns in the expression of human emotions over time. CSIRO’s We Feel tracks emotions in communities around the world. It does this by analysing the language people are using on social media in real time and mapping it out.

The tool can be used to determine the general mood over time (hour by hour, day by day) within particular cities and countries. Patterns in these data can then be explored in association with other information, such as weather, holidays and economic fluctuations.

Some research findings even claim to represent fundamental changes in humans’ social values, community sentiment and how we think (for example, the rise and fall of words associated with rationality such as “method”, “analysis” and “determine”).

Here are some key findings in this space:

• Cultural turnover is accelerating

  A Harvard University-led analysis of more than a century of data from millions of books provides evidence that society’s attention span for historical events is declining, as appetite for new material grows.

  In other words, we are forgetting the past faster. You can see this in the graph below, which tracks how often three specific years are mentioned across a vast range of literature through time. As time passes, the “half-life” of each year (the point at which it receives just half the attention it had at its peak) comes quicker.

  
  Our collective attention for historical events has shrunk over the past century. Michel et al., Science 2010
• Human language diversity and biodiversity are correlated

  By mapping linguistic diversity and the diversity of animal species, researchers have shown these two worlds are correlated geographically – both increasing with temperature and proximity to the equator. So the closer to the equator you get, the more variation there is in spoken language and the greater the variety of species there is.

  The authors propose this is due to heat near the equator producing greater productivity and variety in plant life, which in turn provides more complex and interactive environments for both animals and humans alike – feeding into a cycle whereby “diversity begets more diversity”.