Digital technology and social change: the digital transformation of society from a historical perspective


Abstract

Digital technology, including its omnipresent connectedness and its powerful artificial intelligence, is the most recent long wave of humanity’s socioeconomic evolution. The first technological revolutions go all the way back to the Stone, Bronze, and Iron Ages, when the transformation of material was the driving force in the Schumpeterian process of creative destruction. A second metaparadigm of societal modernization was dedicated to the transformation of energy (aka the “industrial revolutions”), including water, steam, electric, and combustion power. The current metaparadigm focuses on the transformation of information. Less than 1% of the world's technologically stored information was in digital format in the late 1980s, surpassing more than 99% by 2012. Every 2.5 to 3 years, humanity is able to store more information than since the beginning of civilization. The current age focuses on algorithms that automate the conversion of data into actionable knowledge. This article reviews the underlying theoretical framework and some accompanying data from the perspective of innovation theory.


New digital wine into the old wineskins ofinnovation theory


The discussion of digital technology and social change is part of the broader literature of innovation theory.¹ Innovation theory is most commonly based on Schumpeter’s notion of socioeconomic evolution through technological change.^2,3 The reputed “prophet of innovation” himself gave it an illustrative name: “creative destruction.”⁴ Creative destruction works on different levels, reaching from product cycles, over fashion and investment- lifecycles (including so-called Kitchin and Juglar cycles), to so-called business cycles. The result is “an indefinite number of wavelike fluctuations which will roll on simultaneously and interfere with one another in the process… of different span and intensity… superimposed on each other.”² High-level business cycles (also known as great surges or long waves) are emergent phenomena linked to technological paradigms^5,6 that modernize the modus operandi of society as a whole, including its economic, social, cultural, and political organization.^7,8 


Schumpeter extended, theorized, and generalized⁹ the work of the Soviet economist Nikolai Kondratieff, who had already identified two cycles of expansion, stagnation, and recession.¹⁰ He identified the key carrier technology of his first industrial revolution (1770-1850) as water-powered mechanization (including mills and irrigation systems, see Figure 1 ). The following long wave (1850-1900) was enabled by steam-powered technology (including trains and industrial machinery). Kondratieff speculated that around 1900 a new cycle had started, which Schumpeter later called the “Third Kondratieff.”² It was characterized by the electrification of social and productive organization, including manufacturing (1900-1940). Schumpeterian economists later added the long wave of motorization (1940-1970s), and the age of information and telecommunications thereafter.^8,11 


Figure 1. Schematic presentation of Schumpeterian long waves. GDP, gross domestic product.

Note that this specific scheme of historical classification promoted foremost by industrial economists could be complemented with other perspectives, including historical advances in medicine,¹² military technology,¹³ institution or cultural evolution,^14,15 or the very nature of communication itself.¹⁶ Independently of the detail of what technology transforms society exactly when, it is common practice in innovation theory to name long-term paradigms of human history after the dominating technological toolset. This practice is borrowed from historians, who commonly subdivide archaeological periodization of early civilizations into the descending sequence of the Stone Age, Bronze Age, and Iron Age ( Figure 1 ). The general notion is that “civilization advances by extending the number of important operations which we can perform without thinking about them.”¹⁷ In order to trigger a great surge in form of a long wave, the automation needs to be driven by a so-called general purpose technology.¹⁸ Those fulfill “the following conditions: (i) clearly perceived low-and descending- relative cost; (ii) unlimited supply for all practical purposes; (iii) potential all-pervasiveness; (iv) a capacity to reduce the costs of capital, labor and products as well as to change them qualitatively.”¹⁹ 


The fact that the consecutive long waves have tended to become shorter over the course of history (note that the Stone Age lasted 2 000 000, and the Bronze Age 2000 years) is due to the combinatorial logic of technological innovation²⁰ (Schumpeter defined innovation as “carrying out New Combinations”).² An accumulatively larger repertoire of possibilities leads to exponential progress.^21-23 


The creative process of societal modernization is at the same time also destructive, and inseparably intertwined with financial bubbles, recession, and social crisis. 


Each technological revolution, originally received as a bright new set of opportunities, is soon recognized as a threat to the established way of doing things in firms, institutions, and society at large. The new techno-economic paradigm gradually takes shape as a different “common sense” for effective action in any area of endeavor. But while competitive forces, profit seeking, and survival pressures help diffuse the changes in the economy, the wider social and institutional spheres — where change is also needed — are held back by strong inertia stemming from routine, ideology, and vested interests. It is this difference in rhythm of change, between the techno-economic and the socio-institutional spheres, that would explain the turbulent period.¹¹ 


In short, the initial euphoria about the (often economic) opportunities is in every cycle followed by a subsequent sobering discovery of the (often societal) downsides. It is well known that the industrial revolutions have contributed much wealth, but also much inequality and many economic problems. The same is true for the current period of digital technology and social change.


The diffusion of the digital paradigm


The most recent period of this ancient and incessant logic of societal transformation was given many names between the 1970s and the year 2000, among them (in chronological order) post-industrial society,²⁴ information economy,²⁵ information society,²⁶ fifth Kondratieff,¹⁹ information technology revolution,²⁷ digital age,²⁸ and information age.²⁹ While only time will provide the required empirical evidence to set any categorization of this current period on a solid footing, recent developments have suggested that we are living through different long waves within the continuously evolving information age. Starting with Shannon’s conceptualization of “digital” in 1948 in the area of telecommunication (aka the “bit”),³⁰ the Kuhnian process of scientific puzzle solving³¹ started by focusing on the problem of communication. The search for Shannon’s limit of utmost communication capacity kept engineers busy for almost half a century, but was eventually solved in the early 1990s (for all practical purposes).³² Since then, broadband communication has been sending entropic information through radio waves and fiberoptic cables at the speed of light, which seems to be a fundamental limit to the speed of information transmission in our universe.


Figure 2. The world’s technological capacity to store and telecommunicate information. Non-high-income telecom refers to the ratio of installed bandwidth capacity between non-high-income countries and high-income countries. EB, exabytes; ZB, zetabytes; Tb/s, terabits per second.

As always in technological paradigms, the process of successful technological innovation was closely followed by a process of technological diffusion.^33,34 The world was swamped with internet connections and mobile phones in record time.³⁵ The result was the resolution of space-time constraints in global communication²⁹ and the accumulation of vast amounts of stored data, which has more recently been termed “big data.”³⁶ We estimate the beginning of the “digital age” to be in 2002, when the world was first able to store more digital than analog information in its technological tools ( Figure 2 ).³⁷ In the late 1980s, still less than 1% was in digital format, whereas in 2012, 99% of the world’s stored information was digital.³⁸ During these decades, the world’s technological capacity to communicate and store information has grown 25% to 35% per year (doubling every 2.5-3 years — see logarithmic left-hand side axis in Figure 2 ).^38-40 


As always, the diffusion of a new paradigm is never instantaneous, but takes place over social networks over time, which inevitably creates a divide between the haves and have nots.³⁴ Figure 2 also shows that the resulting digital divide has increasingly been closed internationally. Non-high-income countries provided 16% of the installed bandwidth capacity in the late 1980s but hosted more bandwidth than high-income countries after 2015 (led by China). It is good news that the divide among countries has become smaller. At the same time, within countries and among people worldwide, independent from their nationality, bandwidth capacity continues to correlate strongly with income.^41,42 Since income inequality is notoriously persistent, it is expected that the digital bandwidth divide has become a systematic and permanent characteristic of modern societies, especially as its focus migrates from minimum connectivity to bandwidth. 


The digital growth of information and communication led to the often-lamented information overload for humans, whose mental capacities get crunched in the ambitions of the information economy.^43,44 At the same time, it led to the much-celebrated “unreasonable effectiveness of data”⁴⁵ in discovering actionable knowledge through artificially intelligent machines. The world’s computational capacity has grown three times faster than our information storage and communication capacity (some 80% per year^37,39 ), which enabled us to analyze the provided data in an automated fashion. For many practitioners, artificial intelligence (AI) has become synonymous with data-driven machine learning, including the neural networks of deep learning architectures.⁴⁶ 


Advancements in the field of AI have been dazzling. AI has not only superseded humans in many intellectual tasks, like several kinds of cancer diagnosis⁴⁷ and speech recognition (reducing AI’s word-error rate from 26% to 4% just between 2012 and 2016),⁴⁸ but has also become an indispensable pillar of the most crucial building blocks of society. By now, most humans not only trust AI blindly with their lives on a daily basis through anti-lock braking systems in cars (ABS) and autopilots in planes, but also with the filtering of their cultural, economic, social, and political opinions.⁴⁹ The electric grid is in the hands of AI⁵⁰ ; three out of four transactions on the US stock markets are executed by it⁵¹ ; and one in three marriages in America begins online.⁵² If we were to study any other species that has outsourced almost all of its energy distribution decisions, three-quarters of its resource distribution decisions, and an average one-third of its procreation decision to some kind of intelligent and proactive system, it is unlikely that we would treat them as two distinct and independent systems. We would look at it as one inseparable and organically interwoven socio-technological system. From a historical perspective of social change, the merger between biological and AI has already crossed beyond any point of return, at least from the social science perspective of society as a whole. Currently, the downsides of this merger are starting to become obvious, including the loss of privacy, political polarization, psychological manipulation, addictive use, social anxiety and distraction, misinformation, and mass narcissism.^53,54 


Amid the third metaparadigm


Summing up, we can distinguish three different long-term metaparadigms, each with different long waves ( Figure 1 ). The first focused on the transformation of material, including stone, bronze, and iron. The second, often referred to as industrial revolutions, was dedicated to the transformation of energy, including water, steam, electric, and combustion power. Finally, the most recent metaparadigm aims at transforming information. It started out with the proliferation of communication and stored data and has now entered the age of algorithms, which aims at creating automated processes to convert the existing information into actionable knowledge.