Digital Transformation: Exploring big data Governance in Public Administration

Abstract

As economies become increasingly data-driven, big data technologies and software products are turning into key tools for managing technological processes in real time for more efficient delivery of public services to citizens. The change in the interaction between the state and society implies the creation of a unified state digital ecosystem, centered around big data. Such paradigm calls for rethinking of public administration principles ensuring the transition from an electronic state to a digital one. As a result, the likelihood of creating values ​​that meet the shifting expectations of citizens in relation to public services increases.

Introduction

Nowadays big data technologies and software products are turning into key tools for managing digital transformation. The development of the state and society interaction implies the creation of a unified state digital ecosystem with dig data placed in the center. The global big data market is projected to grow to US$ 103 billion by 2027, more than double the market size in 2018. At the same time software will become the largest big data market segment by 2027 with a share of 45% (Mlitz, 2021).

Investments in big data, artificial intelligence and machine learning technologies (Yukhno, 2019) are essential for the development of this area. Its volume depends on the industry and market segment. For example, the 2021 Global Data survey shows that for the pharmaceutical industry, big data (51%) and artificial intelligence (51%), along with APIs and digital platforms (39%), will be the most important investments in the next two years (Big data and AI are likely to lead emerging technology investments: Survey, 2021). Surveys show that most of these investments spent on the budget for ICT solutions (55%), training employees (48%), hiring new employees (30%) and paying for external technical consultants (26%). Financing of such investments mainly comes from the IT budget (46%), implying that big data is still considered a technical issue (Big data is a job engine, n.d).

The COVID-19 pandemic has contributed to the implementation into practice the tools and processes for working with big data, which reflects the key technological trends of state and corporate development and have the greatest potential impact on the competitive advantages of states and organizations in the coming years.

Methodology

The current study presents a comprehensive analysis of big data governance in public administration.

The author applies empirical and comparative analysis, expert assessments, synthesis, deduction, and induction to draw the conclusions.

Big data is becoming an increasingly important intangible asset for states and organizations that affects the implementation of their strategies and competitive advantages in the market. Thus, the key technological trends in a state and corporate development concerning data-driven public administration in the post-Covid period were examined.In this context the following aspects of big data application were studied: the main features of big data and reasons for introducing it to public administration, the critical challenges to its efficient application and a data-driven public sector. Based on the results obtained, the need to create a unified state digital ecosystem with data and algorithms in the center is logically justified.

The article begins with a brief overview section. The next part presents the analysis and findings about efficient big data application. It is followed by discussion on a path to efficient data-driven public administration and its implications for states and organizations. The article is concluded with final remarks and discissions for further research.

Big Data As a New Effective Tool For States and Organizations

Altough big data existed in the past, the widespread adoption of the term is credited to C.Lynch, the editor of the Journal Nature, who used it in 2008 in an issue dedicated to the explosive growth of the world’s information (Lynch, 2008). Gartner understands big data as high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making (Sicular, 2013). ISO/IEC 20546:2019 (Information technology — Big data — Overview and vocabulary) defines big data as extensive datasets — primarily in the data characteristics of volume, variety, velocity, and/or variability — that require a scalable technology for efficient storage, manipulation, management, and analysis (GOST R ISO/IEC 20546 – 2019. Information technology. Big data. Overview and vocabulary, 2020). In essence, the term describes the exponentially increasing volume and variety of data belonging to organizations and states that cannot be processed using conventional tools (Bennett, 2017, p. 462).

The main sources of big data include mobile and social platforms, streaming data that comes from the Internet of Things (IoT), public data from open sources, cloud data, databases (Oracle, SQL, Amazon Simple, etc.), ERP systems, data warehouses, etc. (Big data: what is it and why it matters, n.d; Joshi, 2017). In most cases, data is generated by an extremely heterogeneous user base and social and organizational mechanisms that differ significantly from the technical models that have traditionally been used in economic life. The situation is further complicated by the fact that users of the Internet and social networks often do not belong to the organizations or networks to which they transmit data (Constatiou and Kallinikos, 2015, pp.4–5). In this regard, the issue of ensuring its confidentiality becomes extremely relevant.

In recent years, researchers have developed various features of big data that characterize this term and reveal its content1. The author proposes to highlight the key ones among them:

1. Volume - the amount of big data created, collected, copied, and consumed around the world is growing exponentially and will reach 180 zettabytes by 2025 (Holst, 2021).

2. Velocity - big data is created at a tremendous speed and requires real time processing, storage, and analysis.

3. Variety - big data is formed from various types of data in a wide variety of formats (structured, semi-structured and unstructured data, text data, speech data, video data, etc.).

4. Veracity - big data must be characterized by accuracy, truthfulness, and reliability, otherwise they can lead to incorrect decisions.

5. Variability - understanding the meaning of big data requires consideration of the chosen context.

6. Visualization - for greater accessibility of perception of big data, its visualization is required.

7. Value - the analysis of big data allows you to derive value to solve the problems of specific users (Ishwarappa, 2015; Sicular, 2013; Rialti et al., 2016).

Nowadays big data is used in almost all sectors of the economy and is finding new areas of application, such as identifying consumer buying habits, personalized marketing, health monitoring using data from wearable devices, road mapping for autonomous vehicles, customized health plans for cancer patients, traffic analysis to reduce congestion in cities, production optimization, real-time data monitoring and cybersecurity protocols, etc.

T.H. Davenport and J. Dyché highlight the following among the main reasons for using big data technologies: cost and task execution time reduction, development of new products and offerings, and support of internal business decisions (Davenport and Dyché, 2013, pp. 3–8). This position could be complemented by such benefits as identifying potential risks that can improve the quality and effectiveness of the risk management system, the ability to create targeted products and services for a specific market, retaining and attracting new customers (Mills, 2019).

The COVID-19 has only accelerated the indicated trends, demonstrating the key role of big data in promptly responding to emerging challenges and making decisions that affect all or most people in states and organizations2. It has rapidly influenced many aspects of governments and organizations activities, significantly increasing the role of artificial intelligence and machine learning in big data analytics in the public and private sectors. Due to the need to collect, segment and promptly interpret huge amounts of data related to the spread of the virus, the important role of artificial intelligence algorithms in this process was revealed (Marr, 2020). In addition, due to the use of artificial intelligence, states and organizations received a functional tool which can predict, detect, diagnose and treate various viruses for the coming years in real-time mode (Ibid). At the same time, states and organizations are facing problems in the field of data protection (issues of confidentiality, security, data leakage, etc.), which have underlined the need for new approaches to regulating this area of our life.

A Path to Efficient Big Data Application

Big data usability depends on two key factors, which are the main goals of the applied business processes: the ability to create value from it and the quality of such data (Hilb, 2019, p. 60). V.D. Milovidov notes that the presence of a large amount of data and technologies for its processing does not have “any pronounced correlation with the economic results of economic activity.“ As a confirmation of this thesis, the author points out that “the total volume of accumulated data in American companies practically does not correlate with the indicators of their market capitalization” (Milovidov, 2019, p. 288). In reality, the presence of a large amount of generated data does not always allow to determine at first glance potential options for extracting value from it and using it for solving practical problems. On average, from 60 to 73% of all data in an organization is not used for analytics, which considering the specifics of data-driven economy may lead to a decrease in competitiveness (Gualtieri, 2016). Moreover, the intended effects of data analysis cannot be verified retrospectively, and the transparency of the data can cause controversy among the participants in the digital ecosystem. The Organization for Economic Cooperation and Development (OECD), while noting the inexhaustibility of data, emphasizes that it can be reused to open up significant growth opportunities or create benefits for the whole society in ways that could not have been foreseen when they were first created (Enhancing Access to and Sharing of Data: Reconciling Risks and Benefits for Data Re-use across Societies, 2019). With more data being produced today than ever before, the ability to use it becomes the key competitive advantage for states and organizations. Understanding and interpretation of the data can provide them with valuable information needed to improve its performance (forecasting demand, potential problems, etc.).

The issue of data quality is one of the key areas of activity of states and organizations, not just their ICT departments (Yukhno, 2021). During the coronavirus pandemic, the low quality of data did not allow the use of various artificial intelligence models for diagnosing the disease and predicting risks for patients, as they were not suitable for clinical use. Thus, it becomes clear that poor data quality jeopardizes the achievement of state goals and largely depends on the quality of the process for data obtaining, as well as on the quality of the processes for its storing, managing, transferring, and presenting (GOST R 54524 – 2011/ISO/TS 8000 – 100:2009. Data quality. Part 100. Master data. Exchange of characteristic data. Overview, 2012). At the same time, according to GOST R 56214 – 2014 / ISO / TS 8000-1: 2011, data quality affects only those data that are involved in making any decision, while preventing the repetition of data defects and reducing unnecessary costs (GOST R 56214 – 2014/ISO/TS 8000-1:2011. Data quality. Part 1. Overview, 2015). In this regard, interaction in the field of data management should include stakeholders with a clear division of areas of responsibility and be directly linked to the development priorities of states and organizations (Goasduff, 2020).

A report by Precisely and Corinium Global Intelligence shares that data reliability implies three characteristics: accuracy, consistency, and context. The integration of cloud technologies helps improve the quality of collection and analysis of big data. However, 82% of data leaders find it difficult to provide a constant stream of reliable data that is suitable for making informed business decisions. In addition, 9 out of 10 managers face a shortage of workers with the right skills (Data Integrity Trends: Chief Data Officer Perspectives in 2021, 2021; Precisely: 82% of data executives cite data quality as a barrier, 2021).

In the digital era, data quality needs to be directly linked to the KPIs of governments and organizations (Moore, 2018b). Data collection requires certain costs and clear regulations for assessing the use and profitability of acquiring various types of data for specific tasks. However, according to Gartner, about 60% of organizations do not even measure the annual financial costs (lost revenue opportunities, reduced organizational efficiency and productivity, etc. (Barrett, 2022) on low quality data (Moore, 2018b).

Governments and organizations need to clearly understand the relationship between additional data, improving forecasting accuracy and creating added value (Heath, 2019, p. 46). It also means quantifying the financial benefits of big data, as well as the costs (and consequent savings) of investing in risk management in this area (Bennett, 2017). That said, according to a Gartner survey, 42% of data analytics leaders do not measure or track metrics in this area. Those who do carry out such activities are mainly focused on achieving compliance goals (Goasduff, 2020). In this regard, it is recommended that states and organizations:

• establish a unified approach to data;

• measure the values of big data indicators;

• identify key data workers;

• optimize the cost of data quality management tools;

• set realistic time frames for deploying data quality tools (Moore, 2018b);

• implement a quality control system for big data;

• create a unified repository of big data and provide access to it for all employees in accordance with internal regulations;

• develop a corporate culture of working with big data;

• implement employee-friendly big data analytics tools.

Big data is becoming an increasingly important intangible asset (The world’s most valuable resource is no longer oil, but data: Regulating the Internet giants, 2017) for states and organizations, contributing to the creation of added value in various sectors of the economy. Such approach contributes to digital transformation (Dremel et al., 2017, p. 97) and gaining competitive advantages in the market (Ibid) by predicting needs and responding to changes in the economy (Strengthening Digital Government, 2019, p. 3). In other words, big data needs to be viewed as an asset and both its value and the value it stores should be measured. In this regard, states and organizations need to implement measurable indicators that will allow to trace the relationship between data analytics and ongoing initiatives and projects to create value and achieve their goals and objectives. Specifically, they are invited to:

1. determine the key priorities of activities;

2. approve KPIs (financial, operational, reputation, etc.) in achieving goals and objectives using big data analytics;

3. determine the current level of quality of existing big data and its impact on their activities;

4. approve the target state of data quality to achieve the set goals (Moore, 2018a).

Currently the importance of solutions that allow monitoring of real time data quality metrics is growing. States and organizations are now using artificial intelligence and machine learning to support more complex data management tasks, including data cataloging, metadata managing, and displaying and anomalies detecting (Hunt, 2021). In addition, the use of artificial intelligence and machine learning contributes to significant improvements in the quality of the data itself as an intangible asset. At the same time, high-quality and compatible data is required, for the efficient operation of artificial intelligence (Artificial Intilligence. Towards a choice of strategy, 2019, p. 56). In this regard, states and organizations, first of all, need to systematically consider the issue of introducing strategies and tools for managing such data. It is noted that aligning strategies for big data, cloud computing and mobile technologies leads to a 53% economic growth in comparison to competitors that do not use these technologies (Columbus, 2015).

High data quality provides the ground for an effective data management structure, supports business initiatives, minimizes emerging risks, and ensures the reliability, completeness, uniqueness, and relevance of data itself. In practice, the main tools for improving the quality of data are as follows: adopting a data management strategy, data profiling, data checking as it is entered, data cleaning, data quality monitoring and controlling, building interaction between organizational units, making changes to the organizational structure (for example, the introduction of data manager position or the creation of a data quality control center) with the setting of appropriate business processes, roles, functionality, stakeholder responsibilities and realistic deadlines for the implementation of these tools in practice. At the same time, the widespread introduction of tools to improve data quality, as a rule, is undermined by their high cost (Moore, 2018b).

With the exponential scaling of the volume of big data moving between nations and organizations, value chains will become more complex and analytical tools more and more diversified. Thus, states and organizations need to carry out continuous and comprehensive work to systematize and control big data quality used to create in order to form an appropriate corporate culture of big data management for further transition to big data-based decision-making.

Moving to a Data-Driven Public Sector

The OECD classifies a data-driven public sector as one of the six dimensions of the OECD Digital Government Framework (The OECD Digital Government Policy Framework: Six dimensions of a Digital Government, 2020, p. 6), highlighting that a mature digital state is a data-driven state (Ibid, p. 14) and calls for a shift from information-driven states to a data-driven public sector3. In real life, the implementation of big data technologies in public administration faces three main issues:

• technical and practical problems (insufficient amount of quality data, lack of common standards and level of interaction between various public and private ICT systems, etc.);

• limited resources and opportunities (insufficient funding for necessary research, low qualifications of civil servants and digital literacy in society, etc.);

• the presence of barriers in society and the state (institutional, legislative, cultural, etc.) (Ubaldi et al., 2019, p. 53).

To simplify and improve the abovementioned practices, the OECD recommends that states develop strategies to support data-driven public sector development (Strengthening Digital Government, 2019, p. 1). Typically, such a strategy focuses on three main areas: the needs of citizens and society, obtaining, analyzing, and making decisions based on big data, mobility, and speed of sharing big data and decision making (Akatkin et al., 2017, p. 19). In addition, the data-driven state focuses on its use to create social value by predicting and planning possible changes, improving the quality of public services, implementing public policy and responding to public requests, as well as assessing and monitoring its use (The OECD Digital Government Policy Framework: Six dimensions of a Digital Government, 2020, p. 15).

In practice, this means that the development of approaches in which new technologies and big data become instruments for achieving the state’s goals and objectives. At the same time, the very transition to strategic management of big data at the state level, as a rule, presupposes the presence of three major elements:

• nationwide data architecture;

• digital transformation of public administration;

• human resources (The state as a platform. People and technology, 2019, p. 8–11).

In addition, the analysis shows that the shift to data-driven governance can be facilitated by a government strategy focusing on two main areas: leveraging existing data from different sources through aggregation and analysis and building real-time data exchange networks to deliver improved public service (Real-Time, Data-Driven Government. Develop Forward-Thinking, Citizen-Centric Programs, 2021, p. 9).

The change in the format of interaction between the state and society necessitates the creation of a unified state digital ecosystem with data and algorithms in the center. The currently implemented platform approach in public administration4 will significantly increase the speed of provision of public services, and feedback from citizens in real time will significantly increase the requirements for the quality of their provision.

Such a paradigm of relations presupposes a rethinking of the principles of the public administration system, ultimately ensuring the transition from an electronic state to a digital one (The OECD Digital Government Policy Framework: Six dimensions of a Digital Government, 2020, p. 6). The goal of this process should be the transformation of all areas of the state’s activities, including internal processes and standards for the provision of public services, the organizational structure of public administration, internal corporate culture, the necessary skills and competencies of civil servants, the format of interaction with citizens and receiving feedback, etc. As a result, the likelihood of creating values ​​that meet the increased expectations and requirements of citizens regarding the results of their interaction with the state increases. Thus, in the digital era, the digital transformation of the state necessitates the adaptation and use of new technologies and big data in state activities in order to “reset” public administration.

In this regard, World Bank’s “World Development Report 2021: Data for Better Lives” is proposing a new social contract to address data governance challenges that will:

• increase the use and reuse of data to extract more value;

• ensure more equitable access to the benefits that data creates;

• build trust by creating mechanisms to protect citizens from data risks;

• prepare the ground for the creation of an integrated national data system (World Development Report 2021: Data for Better Lives, 2021).

An integrated national data system includes three groups of actors: states and international organizations, individuals and civil society, and the private sector, which cooperate in an environment where data is safely produced, exchanged, and used. Despite the need for close interaction between the parties, such a system does not require an obligatory centralized management system and storage of all data in it (Ibid).

The introduction of the term “unified national data system” into scientific research reflects a new stage in the development of the digital environment. The system is built on the basis of four pillars (Table 1).

Table 1.

Four pillars of a unified national data system

Pillar	Specifications
Infrastructure policy	provides equal access to data for states and people; contributes to the development of internal data infrastructure, which allows for local storage, processing and exchange of data within the state.
Data Regulations	trust in data transactions is supported by strong laws and regulations; factors facilitating the exchange of data are usually more developed for publicly available data, where government policy and legislation establish rules for access and exchange of such data, than for private data, where governments have more limited influence.
Economic policy measures	growing role of data in the business models of digital platforms is changing competition, trade and taxation in the real economy, posing risks for low- and middle-income states; security measures and data processing tools developed by states will have an impact on the real economy (antitrust regulation, tax issues, trade in services based on data); low- and middle-income states often lack the institutional capacity to manage the policy problems posed by data-driven economies.
Data Management Institutes	institutions to manage data have four main functions: strategic planning, setting rules and standards, compliance and enforcement, and gaining the knowledge and evidence needed to understand and address emerging issues; a targeted, participatory approach to data management and oversight can help institutions keep pace with the ever-evolving data ecosystem and enhance their legitimacy, transparency and accountability (World Development Report 2021: Data for Better Lives, 2021).

Both states and organizations that want to be effective during the period of active digital transformation of the economy, when moving to data-centric management, are recommended to take into account the above factors.

The priority of using big data for the state in the Russian Federation is consolidated in regulatory documents. The decree of the President of the Russian Federation of 05/09/2017 N 203 “On the Strategy for the Development of the Information Society in the Russian Federation for 2017–2030” defines the digital economy as “an economic activity in which the key production factor is digital data, processing of large volumes and the use of the analysis results of which, in comparison with traditional forms of management, can significantly increase the efficiency of various types of production, technologies, equipment, storage, sale, delivery of goods and services” (Strategy of the Information Society Development in the Russian Federation for 2017–2030, 2017). On June 3, 2019, by Order of the Government of the Russian Federation No. 1189-r, the Concept for the Creation and Operation of a National Data Management System (NDMS) and a roadmap for the creation of an NDMS for 2019–2021 were approved.

According to the Big Data Association, compared to 2019, the introduction of big data in the Russian Federation under the optimistic scenario can provide 1.8% of GDP growth (1.2% under the baseline scenario), as well as the growth of the big data market to 160 billion rubles (100 billion rubles - under the baseline scenario) by 2024. At the same time, in order to expand the possibilities of using big data, the Association proposes to overcome five groups of barriers (lack of specialists for the mass adoption of big data, limited infrastructure and data availability, difficulties in research and innovation in the field of big data, lack of large-scale implementation of big data in sectors of the economy) (Strategy for the Big Data market development by 2024, 2019).

Currently, in order to be competitive, both states and organizations need to use a wide range of internal and external information resources. This means increasing the volume of different types of data by closer collaboration with different stakeholders (Judah, 2021). It seems that effective data management does not occur in isolation, but in close interaction with other participants of the digital ecosystem, who are collectively responsible for the final result. In this regard, it is necessary to work out in detail the issue of the availability of open and common tools (for example, open standards, APIs, algorithms) that can facilitate overall integration both inside and outside the organization (The Path to Becoming a Data-Driven Public Sector, 2019). Considering the problem from the state point of view, the establishment of partnerships outside the public sector will allow it to:

• take advantage of digital private sector solutions to improve, optimize and modernize public sector data infrastructure (e.g. cloud solutions);

• facilitate the publication of data produced by civil society organizations on government open data platforms or the publication of open government data on non-government data portals;

• support the exchange of data between multiple stakeholders from different sectors and strengthen the control and power of data owners over the exchange and use of their data for common policy objectives (Ibid).

For example, the practice of collecting big data is gaining popularity in order to maintain economic statistics used to monitor the economic situation in the state (in Indonesia, tourism statistics are monitored using data from cellular operators, Azerbaijan uses data from electronic databases of retail for statistical purposes, in Australia statistics on jobs and gross wages are analyzed using payroll data, etc.) (Bernal et al., 2021).

Concluding Discussions

This paper aims to fill the knowledge gap around big data and data-driven public administration in the post-Covid period. The pandemic has intensified the key role of big data in responding to emerging challenges and decision-making that affects states and societies, significantly increasing the role of artificial intelligence and machine learning in big data analytics in the public and private sectors. The author states that the volume of analyzed big data is constantly growing and requires the creation and maintenance of an appropriate technological architecture and ICT infrastructure, as well as an increase in the scale of its storage, analysis, and transmission. At the same time, the fact of having big data does not automatically impy that it can be used to solve real problems. The main value of big data lies not in itself, but in the conclusions, products and services that appear as a result of its processing and analysis (Davenport and Dyché, 2013, p. 30). In fact, organizational productivity improvements and competitive advantage arise from analytic models that identify new opportunities and enable states and organizations to predict and optimize their performance (Barton and Court, 2012). The author stresses that data reliability implies such characteristics as accuracy, consistency, and context. The issue of data quality is one of the key areas of activity of states and organizations and needs to be directly linked to their KPIs. However. the widespread introduction of tools to improve data quality nowadays is constrained by their high cost.

The analysis reveals problems in the field of data protection. The use of big data leads to a reset of the risk management system both within the state and organizations, allowing it to be processed in real time to obtain more accurate results, improve the quality of work and feedback from citizens and counterparties. The main task in the field of big data is finding a balance between emerging risks and its value for the state and organizations in the context of the constant increase in volume and the development of new technologies that ensure its cheaper storage. In addition, the growth in big data can lead to an information gap between the strategic and operational levels of government and corporate governance. In this regard, one of the important consequences of the digital transformation of an organization is the need to build processes that could be managed in real time. Mechanisms for managing big data, on the one hand, have to protect it from factors that can destroy or limit its value, on the other hand, to exclude situations in which states and organizations will not be able to take of advantage of the value of their data (Tallon, 2013 p. 26).

It is noted that a mature digital state is a data-driven state that encompasses three major elements: nationwide data architecture, digital transformation of public administration and human resources. The shift to data-driven governance implies the creation of a unified state digital ecosystem with data and algorithms in the center building real-time data exchange networks to deliver improved public service. Such approach calls for rethinking of public administration principles unltimately ensuring the transition from an electronic state to a digital one to increase its efficiency. Thus, the effectiveness of the use of big data largely depends on the development of the digital environment and the ability to move data quickly and at low cost among a potentially unlimited number of participants in a government-regulated framework.

Funding Information

Not applicable

Declarations

The author has no relevant financial or non-financial interests to disclose.

Conflicts of Interest

Not applicable

Ethics Approval

Not applicable

Informed Consent

Not applicable