A literature review of “lawful” text and data mining.

Version Changes

Revised. Amendments from Version 1

In response to the valuable feedback provided by the reviewers, this paper has undergone revisions to enhance its structure, clarity, and depth of analysis. Firstly, the introduction has been restructured to provide a clearer explanation of key concepts, which are essential to understanding Text and Data Mining (TDM). Similarly, the selection criteria for the literature review have been clarified, including language limitations and the interdisciplinary scope of the review. The regulatory framework section has been expanded to include a broader discussion of the legal challenges surrounding TDM, particularly in relation to data protection. New authoritative sources recommended by the reviewers, such as official guidelines from supervisory bodies, have been included to enhance the discussion on GDPR and other regulatory frameworks. The section on intellectual property rights has been renamed "TDM and Copyright and Related Rights" for greater accuracy. In response to concerns about outdated references, older sources have been justified or removed as appropriate. The data protection section now includes more authoritative sources, and a deeper analysis of GDPR provisions such as data protection by design and default, profiling, and inferred data. Further, the section on fundamental rights has been revised to address issues of bias and discrimination in TDM, with key literature added, while the generalist approach section has been clarified to justify its classification. Footnotes and references have been reviewed and revised for consistency and accuracy throughout the paper.

Abstract

Text and data mining (TDM) is a process, typically automated, that looks for patterns in data that may otherwise remain unnoticed. In a world where data driven solutions play a progressively more important role, TDM has become a vital tool in sectors ranging from medicine, to commerce, gaining widespread attraction. Nevertheless, a variety of regulatory frameworks not always specifically attuned towards regulating TDM continue to apply concurrently. The literature within the context of regulatory frameworks governing TDM is a fragmented piecemeal of valuable insights into what “lawful” TDM resembles.

This literature review adopts a grounded theory approach analysing 88 pieces of literature, collating views regarding “lawful” TDM, ultimately providing a holistic assessment of academics’ and practitioners’ views and opinions regarding the regulatory framework which governs TDM. A total of 7 categories were identified and each of these are analysed. Tables are provided in the Appendix (accessible here: https://doi.org/10.5281/zenodo.12654691)highlighting which scholarly works were used for each section of the literature review, but also how those works were used.

It is ultimately concluded that the regulatory frameworks that apply to users conducting TDM are multifaceted, and ever-changing on a case-by-case basis. There is an ever-growing need for a holistic interpretation of the regulatory frameworks which apply, creating a map which would allow for users conducting TDM to navigate this complex web of legal rules.

Plain language abstract

Text and data mining (TDM) is an automated method used to find hidden patterns in information and is often used to generate new knowledge. TDM is used widely today but the laws that apply to TDM are unclear. Similarly, research examining the laws that apply to TDM offer snippets of valuable insights.

This paper examines 88 pieces of literature which study the laws that apply to TDM, providing a more complete image of academics’ views on this topic.

What this paper shows is that the laws for people doing TDM are complicated and change depending on the situation. There is a growing need for a complete understanding of these laws, to help people doing TDM find their way through this tricky set of laws.

Introduction

• “ Each day on Earth we generate 500 million tweets, 294 billion emails, 4 million gigabytes of Facebook data, 65 billion WhatsApp messages and 720,000 hours of new content added daily on YouTube”. ¹

This “information revolution” has resulted in vast amounts of data, in various formats, being continuously generated and stored and is accurately termed “big data”. ² As per the European Commission “ digital innovation, driven by the combination of Big Data, Cloud Computing, Mobile technologies and Social media, is one the most powerful drivers of change and the best opportunity for Europe to move back to a growth path”. ³ Hence, the utilisation of big data is regarded as valuable to the European economy.

For some the word “data” correlates with “an organised set of values”. ⁴ Nevertheless, data is often messy, and unstructured and cannot effectively be utilised. ⁵ Consequently, it is often the case that to analyse “big data”, unstructured messy data must be transformed into structured data.

Text and data mining (TDM) is a process, typically automated, that looks for patterns in data that may otherwise remain unnoticed, often transforming raw unstructured data into more understandable and useful information. ⁶ Generally TDM comes in a variety of different shapes and sizes varying on the type of data being used, data gathering techniques, analysis techniques employed, and presentation of data, but involves a common underlying objective. That is, utilising vast amounts of data to extract knowledge that would otherwise remain unnoticed. ⁷ Typically, TDM involves a series of stages such as document retrieval, document transformation (transforming the data into machine readable format), data extraction, (extracting the information which seems most useful), and finally mining that information resulting in new knowledge discovery. ⁸

In a world where data driven solutions play a progressively more important role, TDM has become a vital tool in sectors ranging from medicine, ⁹ to commerce, ¹⁰ gaining widespread attraction. Nevertheless, a variety of regulatory frameworks not always specifically attuned towards regulating TDM continue to apply concurrently. As such, compliance with applicable regulatory frameworks is not always obvious, meaning unlawful TDM may be taking place in the shadows of the law.

This literature review forms part of a wider research project which aims to facilitate “lawful" TDM, by highlighting the aspects of regulatory frameworks which concurrently apply to TDM, enabling users to conduct regulatory compliant TDM. This research will ultimately result in the creation of a practical guide and tool enabling users who conduct TDM to navigate the complex web of legal rules which make up the regulatory framework governing TDM.

Just as the regulatory framework, the literature within the context of regulatory frameworks governing TDM is a fragmented piecemeal of valuable insights. This literature review collates views regarding the regulatory frameworks governing TDM, providing a holistic assessment of academics’ and practitioners’ views and opinions regarding the regulatory framework which governs TDM.

To collate the relevant literature, keyword searches, using search operators were used on Google Scholar, Westlaw and LexisNexis. Namely, in October 2023 the keyword search used was ("Text and data mining") OR ((“Text” OR “Data”) AND ("mining" OR "mine")) AND (“law” OR “regulation” OR "legal").

This keyword search was used with the aim of encompassing all published literature which examined the regulatory framework governing TDM. Given the inherently legal nature of the topic, legal libraries like Westlaw and LexisNexis were used. Consequently, this review includes primarily legal literature. Nevertheless, in an attempt to widen this review’s reach in order to provide a holistic approach, some non-legal literature was also analysed as part of this review. For instance, as will emerge from the discussion below, primarily the section examining “Data Protection and TDM”, some computer science literature was also caught by this keyword search. However, given the interdisciplinary nature of this topic, and with the aim of providing a holistic assessment of academics’ and practitioners’ views and opinions regarding the regulatory framework which governs TDM, disciplinary boundaries were not set as selection criteria for this literature review.

The only selection criteria used for this literature review was the search query. One inherent limitation of doing so, is the exclusion of publications, not curated by the aforementioned libraries, which may result in some publications not being collected. Some literature which was not caught by the search query, was recommended by reviewers of this literature review. These pieces of literature were added so as to include as much relevant literature as possible. Additionally, this method of collecting literature to review, also excludes literature that has been published in the official languages of each jurisdiction. Nevertheless, given the difficulties and limitations associated with reviewing translated documents, ¹¹ the English language was set as a selection criterion.

Considering the nature of TDM, it is inevitable that users conducting TDM encounter multijurisdictional regulatory frameworks. As such, Flynn et al. call for a harmonised international approach towards the regulation of TDM, asking for the World Intellectual Property Organization (WIPO) to play a “ constructive role in establishing a consistent international baseline” for the regulation of TDM. ¹² Nevertheless, as it stands users conducting TDM must navigate a complex web of regulatory frameworks spanning across multiple jurisdictions. Given the inherent difficulties of comprehensively examining the inner workings of lawful TDM on a global scale, this literature review will provide scholarly insights from the United States (US), European Union (EU), and United Kingdom (UK) providing insights where relevant from other jurisdictions. ¹³ Similarly, the practical guide and tool aimed at facilitating lawful TDM, created as part of this research’s wider purpose, will take into account and comparatively analyse these jurisdictions.

The collected literature was updated periodically, up until May 2024 using the “sort by date” function identifying any newly published literature. Although literature examining the regulatory frameworks which apply to TDM is a relatively new topic of concern, this yielded a plethora of results. This is attributed to the returning of false positives such as mining in the geological sense. After qualitatively assessing results by examining their title and, if need be, their abreact, determining if the content examined regulatory frameworks which apply to TDM, a total of 88 pieces of literature were collected. These were then analysed manually adopting a grounded theory approach individually analysing and categorising them. For a list of all pieces of scholarly work analysed in this review please see Table 9 in the Appendix (accessible here: https://zenodo.org/records/13880018).

As indicated by Table 1 below, a total of 7 distinct categories were identified. Literature was not always exclusively under one of the categories, meaning that some could fall under two or more of the categories. Tables for each of the category, listing which pieces of scholarly work were used is included in the Appendix (Tables 2 – 8) accessible here: https://zenodo.org/records/13880018.

Table 1. Categories of literature identified and the amount of literature within each of the categories.

Category	Description	Amount of Literature included in each section
Taxonomy of TDM	Identifying what TDM is and what it entails.	9
TDM and Copyright and Related Rights	How TDM interacts with copyright and database rights.	44
TDM and Data Protection	How TDM interacts with data protection regulations.	18
TDM and Fundamental Rights	How TDM interacts with fundamental human rights	6
TDM and Contract Law	Hurdles for TDM caused by contracts and licenses.	11
TDM and Competition Law	How TDM interacts with competition law	2
Generalist Approach	A generalist approach where authors provide a more holistic approach toward the regulatory framework applicable to TDM	8

The structure of this literature review will follow the 7 categories identified. It begins by providing scholars’ views regarding what TDM entails, and how their view of TDM ultimately shapes their respective analysis of the regulatory frameworks This review then presents academics’ and practitioners’ views of the regulatory framework governing TDM. Namely TDM’s interactions with copyright and related rights, data protection and privacy rules, fundamental rights, contract law, and competition law are presented respectively. Finally, several authors adopt a broader, “generalist approach” to the legal analysis of TDM. Rather than focusing on one particular area of law, these pieces of literature explore the intersection of multiple legal regimes and how they collectively influence the practice of lawful TDM. Hence the final section of this literature review presents literature that decided to present the applicable regulatory frameworks more holistically, adopting a so called “generalist approach”.

A taxonomy of TDM

As aforementioned, TDM is a process, typically automated, uncovering knowledge from data that may otherwise remain unnoticed, often transforming raw data into more understandable and useful information. ¹⁴ In the literature, there is an undeniable articulation of the benefits that TDM has to offer, whether that be within the context of libraries, ¹⁵ scientific research and innovation, ¹⁶ medicine, ¹⁷ and generally its ability to yield economic advantages. ¹⁸ Wyner et al., even introduce the idea of using TDM to extract information from case law databases thereby enhancing the legal profession, especially in common law systems which rely on case law. ¹⁹ Margoni argues that “ TDM, together with other types of data-driven analytical tools, deserves its own autonomous methodological classification as computational legal methods”. ²⁰ Nevertheless, there does not seem to be a common understanding of TDM amongst scholars, both within and across different disciplines.

Some scholars make a distinction between text mining and data mining. ²¹ Data mining is sometimes seen as a subset of text mining since text mining is the computational process of analysing unstructured messy data, whilst data mining is the computational process of analysing structured data. ²²

Other scholars focus on a taxonomy and categorisation of TDM as a way of explaining what regulatory frameworks a user conducting TDM might encounter. For instance, Colonna emphasises the fact that data mining involves separate processes, and in different contexts, each of them with varying legal implications. ²³ Similarly, Tóth argues that the process of TDM involves several stages each of which could potentially conflict with regulatory frameworks. ²⁴ Carroll also methodically works through each type of copying that happens in the course of collecting, formatting, processing, and storing data for TDM purposes, analysing relevant case law for each stage. ²⁵ Fiil-Flynn et al. also claim that TDM projects potentially face regulatory restraints at different stages such as digitisation and compilation of data and the subsequent application of algorithms. ²⁶ Equally, Rosati identifies three general TDM processes each of which with their own legal consideration:

“ 1) Access to content,

2) Extraction and/or copying of content, and;

3) Mining of text and/or data for knowledge discovery”. ²⁷

Likewise, Guadamuz and Cabell state that “ it is difficult to generalise on what exactly the method for content mining is, as there are different algorithmic and model structures depending on the subject, the type of database, and the type of analysis being performed”. ²⁸ Nevertheless, the authors argue that most content mining roughly follows these steps:

“ 1. Individual content is created.

2. Content is placed into data set, repository or collection.

3. Miner gains access to the data.

4. Mining tools applied to the data set.

5. Analysis of the processed data.

6. New knowledge”. ²⁹

Guadamuz and Cabell then argue that legal conflicts will arise at stages 3 and 4. ³⁰ Hence, it can be deduced that the literature although divided on the substantive procedural aspects of TDM stages, there is a cohesive undertone in the understanding that TDM involves a series of steps, each of which have their own legal considerations.

Regulatory frameworks which govern TDM

Flynn et al. highlight the importance of being able to conduct TDM without disproportionate legislative barriers but argue that a legislative patchwork of regulatory frameworks spanning various jurisdictions may in fact hinder the progress of TDM. ³¹ This section aims to outline academics’ and practitioners’ views and opinions regarding the different regulatory frameworks which govern TDM. This is intended to act as a steppingstone towards uncovering what “lawful TDM” i.e., TDM which would be considered as compliant with the applicable regulatory frameworks, resembles.

A variety of regulatory frameworks are examined when discussing “lawful TDM” such as copyright and sui generis database rights, data protection and privacy, contract law, competition law and fundamental rights. Some scholars take a generalist approach examining a variety of frameworks which interact with the act of TDM whilst others adopt a more targeted approach examining one regulatory interaction. This review will present authors’ views systematically, presenting interactions with distinct regulatory frameworks separately. Subsequently, it will collectively discuss the views of authors who adopt a generalist approach.

TDM and copyright and related rights

Above, TDM was defined as a process, typically automated, that looks for patterns in data that may otherwise remain unnoticed, often transforming raw unstructured data into more understandable and useful information. Article 2(2) of the Directive on Copyright in the Digital Single Market (CDSM) defines TDM as “ any automated analytical technique aiming to analyse text and data in digital form to generate information such as patterns, trends and correlations”. ³² Likewise, Recital 8 of the CDSM defines TDM as “ the automated computational analysis of information in digital form, such as text, sounds, images or data’ enabled by new technologies”. ³³

From these definitions it can be deduced that the act of TDM may conflict with intellectual property rights where the work or database being reused is protected by intellectual property rights. In the EU, such protection is afforded by Directive 2001/29 (InfoSoc Directive), ³⁴ Directive 2009/24 (Software Directive), ³⁵ or Directive 96/9 (Database Directive). ³⁶ Other jurisdictions have their own regulatory frameworks for affording intellectual property rights. This section aims to present authors’ views, drawing insights from a variety of jurisdictions beginning with the EU approach, as this seems to make up the bulk of scholarly work. Namely 29 out of the 42 pieces of literature analysed under this category examined how TDM interacts with EU copyright and database rights.

It is often argued that since data itself is not protected by intellectual property rights, the act of TDM should in theory not interfere with any intellectual property rights. ³⁷ Recital 9 of the Directive on Copyright in the Digital Single Market (CDSM) acknowledges this by stating that TDM “ can also be carried out in relation to mere facts or data that are not protected by copyright, and in such instances no authorisation is required under copyright law. There can also be instances of text and data mining that do not involve acts of reproduction or where the reproductions made fall under the mandatory exception for temporary acts of reproduction provided for in Article 5(1) of Directive 2001/29/EC, which should continue to apply to text and data mining techniques that do not involve the making of copies beyond the scope of that exception”. ³⁸ This however, is not always the case. In some instances, the act of copying involved with TDM will often encroach upon a rightsholders right to reproduce a work afforded to them through copyright protection. ³⁹ For instance, when creative works are extracted, it is often the case that the entirety of that work is reproduced, resulting in infringement of the rightsholder’s exclusive right of reproduction. ⁴⁰ Nasr et al., were able to show that generative Artificial Intelligence (AI), which utilises data mining mechanisms sometimes regurgitates its training data in its entirety. ⁴¹ Similarly, Baris who examined derivative musical works created by AI, states that “ where an AI-generated output reproduced protectable parts of a work that was used in the training of the AI system”, this would likely amount to copyright infringement. ⁴² On the other hand Senftleben, argues that at an international copyright law level, the act of TDM is not prohibited. ⁴³ Awarding copyright protection is ultimately a balancing exercise of all stakeholders’ interests. As will emerge from the discussion below, this balancing exercise becomes more complex in the realm of TDM.

The act of TDM may in some instances consist of an act of extraction and/or re-utilisation of a database protected in the EU by the sui generis database right. ⁴⁴ As stated by Rosati “ if the content extracted and/or copied is included in a database, then both copyright and the sui generis (database) right – these being two rights that are independent of each other and may subsist together at the same time on the same database) – might come into consideration”. ⁴⁵ Hence, depending upon the circumstances under which data is accessed, how data is processed and presented, it is possible that intellectual property rights are infringed.

The majority of scholarship in this domain emanates from the applicability of copyright exceptions to the act of TDM, most notably with the recent introduction of the two new mandatory copyright exceptions within the EU jurisdictions under Articles 3 and 4 of the CDSM. ⁴⁶

In a pre-CDSM era, several Member States had already introduced their own TDM-specific copyright exceptions. The UK for instance, was the first Member State (in a pre-Brexit era) to have introduced their own TDM-specific exception back in 2014. ⁴⁷ Section 29A of the Copyright, Designs and Patents Act 1988 introduced a text and data analysis exception for the purposes of non-commercial research. ⁴⁸ Similarly, France introduced their own TDM exception applying to both works and databases. ⁴⁹ Estonia and Germany also introduced their own TDM-specific exceptions. ⁵⁰ Greece had also introduced its own rules regulating the act of web harvesting and archiving. ⁵¹ As per Geiger et al., “ this regulatory patchwork […] led to a fragmented legal environment in the EU”, which ultimately led to the introduction of the CDSM exceptions so as to create a harmonised level playing field in the EU. ⁵²

Scholars at this stage had called for the broadest possible EU TDM-specific exceptions so as to “ boost the international competitiveness of its knowledge economy”. ⁵³ Once the dust had settled and the final version of the CDSM was published, Article 3 and 4 introduced two new mandatory exceptions arguably creating a new avenue for “lawful TDM”. Article 3 mandates Member States to allow for cultural heritage and research institutions to freely use protected works for scientific text and data mining research purposes. ⁵⁴ Article 4 necessitates the introduction of a general TDM exception, allowing other entities to develop algorithms without the burden of authorisation and remuneration payment, provided the right has not been expressly reserved by the rightsholder. ⁵⁵ These exceptions were welcomed to a certain extent, for their potential to harmonise a collective EU-wide approach towards “lawful TDM”, ultimately fostering competition. ⁵⁶ For instance Papadopoulos et al., who examined libraries’ views and opinions of web-harvesting (a form of TDM), and found that most libraries already conduct such forms of TDM and those that do not indicate a desire to do so, concluded that the EU CDSM TDM-specific exceptions create “ a favourable legal foundation for the deployment of Web-harvesting and archiving operations through the national libraries of the EU Member States”. ⁵⁷

Nevertheless, authors have not strayed away from criticising these new set of mandatory EU TDM-specific exceptions. For instance, Ducato and Strowel present their arguments in a table which outlines all that these two exceptions got right, wrong, and also those sections which raise more questions. ⁵⁸ They ultimately argue that Article 3 is too narrow with its limited scope of cultural heritage and research institutions whilst Article 4 is limited in the sense that it can be overridden by contracts. ⁵⁹

This limiting of scope in Articles 3 and 4 of the CDSM to content that is “lawfully accessed” is interpreted in Recital 14 as including content that is openly accessible online and materials made available to users through the conclusion of a contractual agreement. ⁶⁰ Whilst Article 3 cannot be overridden by contracts, Article 4 can. ⁶¹ This has resulted in commentators characterising this as giving rise to an opt-out mechanism. ⁶² The European Copyright Society, in its comment addressing selected aspects of the implementation of Articles 3 to 7 of the CDSM, argued that as a result of this provision “ publishers might price TDM into their subscription fees and, as a consequence, few research organisations would be able to acquire licences for all databases that are relevant for a TDM research project”. ⁶³ It may even result in biased AI systems where it becomes more “ economically attractive for developers to train their algorithms on older, less accurate, biased data, or to import AI models already trained on unverifiable data”. ⁶⁴ This, arguably greatly limits the applicability of the exception, putting the EU at a competitive disadvantage when compared to jurisdictions like the US which will be discussed in greater detail below, and undermines the “ widespread assumption that the right to read should be the right to mine”. ⁶⁵ In a blog post, Keller argues that the way in which the proposed, at the time, EU AI Act is being phrased explicitly reinforces this opt-out mechanism, with some of the big names in generative AI also having shown evidence that they intend to implement some form of opt-out mechanisms for rightsholders. ⁶⁶ Keller however, argues that this creates a fragmented framework for rightsholders who have to individually opt out of all new generative AI machines. ⁶⁷ Nevertheless, the author paints the EU’s approach towards regulating AI, which inevitably regulates TDM as balanced, implying that this could in turn result in it being the next global standard. ⁶⁸ Similarly, Senftleben argues that the EU AI Act could serve as a template for other regions promoting a global standard supporting human creativity in the age of AI. ⁶⁹ Namely, Senftleben argues that the primary objective of the EU AI Act is to “ ensure that authors are properly remunerated for the use of their works in AI training processes”. ⁷⁰ Senftleben sees this as a necessary step in order to support human creativity, thereby preventing the market from being overrun by AI-generated content. ⁷¹

Another issue that may be brought back to the limelight by this opt-out mechanism, is the interplay between copyright and technical protection measures (TPMs). TPMs allow for rightsholders to place architectural constraints on their works to regulate access. Under Article 6(1) of the InfoSoc Directive, TPMs are not to be circumvented, whilst under Article 6(4), rightsholders are to make exceptions for those uses which may be able to circumvent TPMs by falling under a copyright exception. ⁷² TPMs may be implemented as a way of utilising a rightsholder’s ability to opt out from TDM. ⁷³ Nevertheless, this will undoubtably create uncertainty as to when TDM is being carried out for the purposes of research, which may inadvertently result in an over-blocking of TDM for research purposes. ⁷⁴ A study conducted by the Max Planck Institute for Innovation and Competition suggests that whilst rightsholders should be able to utilise TPMs to restrict TDM, “ such measures must not go beyond what is required, i.e. technical protection measures must be precluded from making TDM unnecessarily more difficult or even de facto impossible”. ⁷⁵ Havlikova who examined the interplay between the opt-out provision in Article 4 of the CDSM and web scraping, a form of data collection from web sources which is often the starting point of some forms of TDM, found that rightsholders may struggle to reserve their rights. ⁷⁶ Havlikova attributes this to the fact that for content published online, a rightsholder’s reservation must be made using machine-readable means and “ according to Recital 18 of the CDSM Directive, such machine-readable means may include metadata and terms and conditions of a website or a service”. ⁷⁷ Havlikova argues that as a result of such a wide and unstructured means afforded to a rightsholder allowing for them to reserve their right “ could be the Achilles heel of the TDM exception”. ⁷⁸ Havlikova ultimately argues that “ Robots.txt, which is a simple text file containing rules on which crawlers may access which parts of a site”, ⁷⁹ could be one feasible way of reserving one’s rights in a machine-readable means, but that this would also have its drawbacks, especially when it comes to indexing websites. ⁸⁰ As it stands, the author calls for a more standardised method of reserving one’s rights through machine readable means. ⁸¹

As aforementioned, awarding copyright protection is a balancing exercise of all stakeholders’ interests. One way of balancing the extent of protection afforded is through the applicability of exceptions. These mandatory exceptions, whilst a step forward in the context of a harmonised approach towards the regulation of TDM, have their limitations which may ultimately tip the balance in favour of rightsholders.

The limitation of scope of these TDM-specific EU exceptions has been regarded as a major pitfall in the EU’s attempt to become more competitive in the digital era. ⁸² For instance, Rosati argues that they “ might have a negative impact on the (unlicensed) development of AI creativity”. ⁸³ As per Manteghi “ the vague and narrow nature [of the EU CDSM TDM-specific exceptions] could force users to consider alternative solutions to avoid potential infringement claims”. ⁸⁴ One of these solutions, Manteghi claims could be found under Article 35 of the proposed Data Act which “ provides a safe harbour for users needing to access, use or share only databases made of data generated by the use of a product or related service (IoT data)”. ⁸⁵ This avenue towards avoiding potential infringement claims is limited, to certain types of TDM which will utilise certain specific types of databases, meaning it would only apply to potential sui generis database rights infringements and not copyright. Nevertheless, it is possible that certain users conducting TDM may find refuge under this proposed Article. Another alternative may be offered by Article 5(1) of the InfoSoc Directive which allows for transient copies to be made. ⁸⁶ Nevertheless, as per Margoni and Kretschmer whilst Article 5(1) of the InfoSoc Directive “ retains a significant potential for TDM activities and computational uses, the cumulative, occasionally narrow and partially uncertain nature of its conditions and the fact that it only covers temporary reproductions, does not offer a clear and comprehensive solution within which not only science but virtually any human activity employing text and data analytics can operate confidently”. ⁸⁷

The extent of these exceptions’ reach is inevitably dependent upon the national implementation of Member States. Pereira in their examination of the Portuguese implementation of these exceptions, which “ has been faithful” to the CDSM, argues that these exceptions seek to balance the interests of all relevant stakeholders. ⁸⁸ Other commentators have however, called for a wider approach. For instance, Micke called for Finland to take an expansive transposition thereby guaranteeing the freedom to mine. ⁸⁹ Similarly, Calabrese questions why the EU legislature did not exempt TDM users from infringing a rightsholder right of communication to the public in the TDM exceptions, and praises Italy for doing so in their national transposition. ⁹⁰ Calabrese concludes by stating that the Italian approach “ can be appreciated as an attempt of ‘better’—or at least ‘not worse’—harmonization in terms of free communication to the public for TDM, not forgetting that the EU has also to face the regulatory competition of third-country systems which are more open to innovation in this field”. ⁹¹ When stating this Calabrese was referring to the Japanese approach which was the first nation in the world which introduced a TDM-specific copyright exception allowing for TDM researchers to utilise copyright protected works for machine learning purposes. ⁹² Dermawan similarly argued that EU Member States shall consider adopting the Japanese approach towards a more favourable TDM exemption in their national implementation of the CDSM. ⁹³ Nevertheless, Rosati, when examining the extent to which national member states have freedom to diverge from Articles 3 and 4 of the CDSM, stated “ both Articles 3 and 4 require a minimalistic transposition technique on the side of national legislatures, with the result that the normative content of either provision cannot be compromised at the Member State level. In turn, a national [exception] with a broader scope of application than what either provision allows would be incompatible with EU law”. ⁹⁴ At the time of writing this review, Poland had yet to implement the TDM-specific copyright exceptions into national law. The Polish government argues that their delayed implementation of the CDSM enabled a thorough examination of the implications of generative AI for copyright protection, ultimately deciding that training generative AI models on copyright protected works should fall outside the scope of the TDM-specific copyright exceptions. ⁹⁵ How this rationale plays out remains to be seen, but as noted by Paul Keller in a blog post, it is “ clear that any attempt to exclude from the scope of the TDM provision the reproductions made in the context of training generative AI models would, prima facie, result in a non-compliant implementation”. ⁹⁶ Hence, it is unlikely that such a divergence is accepted, but it does highlight the fact that national implementation of these exceptions remains fragmented. In their master’s thesis, André Stéphan, in analysing the national implementation of France, Germany, the Netherlands, Spain and Ireland argues that “ if the objective was to create a digital single market, where stakeholders could benefit from the same rules in each Member State, it has not been reached”. ⁹⁷ This implies that there is still a long way to go before true harmonisation is reached.

Another jurisdiction under which TDM researchers may find refuge is that of the US. There are numerous scholars which approach TDM through the lens of the US’s fair use copyright exceptions. Namely, as did EU scholars, several authors who examine the US’s regulatory framework for TDM assert that TDM is and should be lawful as a matter of US copyright law. ⁹⁸ In other words such authors claim that the US fair use copyright exceptions would encompass the act of TDM. Most justify this reasoning by presenting the lawsuits that commercial authors and publishers lodged against Google, ⁹⁹ and the HathiTrust Digital Library. ¹⁰⁰ These cases involved the digitisation of printed books, allowing for users to search through this collection of literature. In accordance with the Author Guild’s perspective the systematic copying of copyright protected authorial works amounted to copyright infringement. However, Google and HathiTrust argued that their re-use was “fair use” as it was socially beneficial, but also since copyright protected works were not displayed in full. Only small extracts of the copyright protected works were presented to those who used the search function. These arguments ultimately swayed the court to rule in favour of Google and HathiTrust in finding that their reuse amounted to fair use.

Sag argues that TDM would have always found refuge under the US’s fair use exceptions and the US courts have now solidified this position. ¹⁰¹ Sag makes a distinction which the courts failed to do in these cases between a non-expressive use and transformative use. ¹⁰² Sag argues that TDM would fall under the category of a non-expressive use, whereby the act of TDM does not communicate original forms of expression for the purposes of them being read. ¹⁰³ Sag assimilates it to photocopying a magazine and throwing it in a fireplace, which would not be regarded as infringing. ¹⁰⁴ It is asserted that in Google Book saga cases non-expressive reuse was treated as a subset of transformative uses falling under “fair use”. ¹⁰⁵

Carroll, argues that in the US, “ a researcher can legally download all or a portion of the Sci-Hub collection [a website which contains copious copyright infringing academic literature] solely for TDM research”. ¹⁰⁶ Whilst, such an argument may attract greater controversy in that it tips the balance in favour of users to the detriment of rightsholders, it is actually supported by others. ¹⁰⁷ In a recent blog post about an upcoming journal article, Thomas Margoni puts forward this same idea, but within the remits of an EU jurisdiction. ¹⁰⁸ Margoni argues that the lawful access requirement found under Articles 3 and 4 of the CDSM which ultimately creates a TDM realm where content which is accessed needs a license (discussed in greater detail below), should “ only cover the behaviour of the beneficiary of the exception and not extend to the status of the accessed source”. ¹⁰⁹ Whether this liberal interpretation will indeed be the case remains to be seen.

Similar to its EU counterpart, the US allows for TPMs restricting TDM. ¹¹⁰ Nevertheless, Dombrowski and Tilton highlight the historical development of an exception for circumvention of TPMs for research purposes in US. ¹¹¹ The authors state that “ per the final decision, researchers residing in institutions of higher education could bypass DRM for research under a series of conditions. This included staff and students if they are a part of a research team or as a part of teaching. The university must own the source of the data (i.e., not accessing it via a subscription service), and the researchers must take “effective security measures” to protect the data. As a result, researchers at institutions of higher education can conduct TDM with sources such as DVDs and ebooks, which is a significant development for the Digital Humanities”. ¹¹² Nevertheless, the authors argue that while the ruling expands TDM access for “well-resourced” institutions, it falls short when considering wider TDM users and activities. ¹¹³

It can be deduced that authors generally view intellectual property rights as restricting one’s freedom to conduct TDM. Whilst the recent introduction of the EU CDSM TDM-specific copyright exceptions provides a new avenue towards “lawful TDM”, their narrow scope acts as a barrier to free mining. The wider approach of the US arguably creates a more welcoming market for TDM in the US. In a 2022 Research Paper Sean Flynn et al. categorised global copyright exceptions for research purposes focusing on TDM exceptions. ¹¹⁴ The authors provide an informative world map comprising of seven colours categorising countries as having a general research exception, restriction on sharing, restrictions to private reproductions, restrictions to institutional users, restrictions on types of works, TDM being restricted and countries that were not mapped. ¹¹⁵ The authors then comparatively analyse each of their categorisations. They also provide a figure comparing specific TDM exceptions that have been introduced. ¹¹⁶ Sean Flynn et al. ultimately “ show that although every copyright law in the world has at least one exception that promotes research purposes, there is a wide degree of variation between countries” ranging across the spectrum of categorisations they identified with some like the US, being more favourable than others. ¹¹⁷ As per Rosati “ the US copyright regime has been considered more favourable to TDM practices than what appears to be the case under European laws”. ¹¹⁸ This is also evidenced by an empirical study conducted by Handke et al., in 2015 who empirically examined bibliometric data from 43 large economies, including the 15 largest EU Member States, spanning from 1992–2014. ¹¹⁹ The authors found that data mining based articles were more common in countries where mining is “probably allowed”, under copyright law like that of the US, “ which suggests that a more permissive copyright framework is associated with more data mining research”. ¹²⁰ On the other hand, Kretschmer et al., who examined machine learning, natural language processing and computer vision for computer moderation as case studies where TDM is vital, found that US fair use jurisprudence is likely to set global trends. The legal uncertainties created by fair use exceptions, may lead to practices “ where commercial AI developers are told by their legal departments to “mine everything and then destroy the training material” since it will be very difficult to reverse-engineer the trained model, go back to the training material and prove infringement”. ¹²¹ Thus, the balancing of stakeholder interests in the realm where copyright and related rights, and TDM intersect arguably still needs refining.

TDM and data protection

Given the inherent processing of data involved in the act of TDM, it is inevitable that users conducting TDM encounter data protection related regulatory frameworks. ¹²² Nonetheless, how TDM interacts with data protection frameworks seems under-researched. Colonna states that in existing literature “ while the applicability of copyright law to TDM is being considered, there is very little research being done concerning the applicability of data protection law to this new technology”. ¹²³ This section aims to nevertheless, provide an outline of existing research in the field examining the interplay between TDM and data protection regulatory frameworks.

One of the most influential regulatory frameworks in this regard is the EU General Data Protection Regulation (GDPR). ¹²⁴ As per Lynskey, unlike jurisdictions like the US, which is characterised as having a “sectoral” regulation of data protection and privacy, meaning certain aspects of data protection and privacy are regulated on a granular level, the GDPR takes an “omnibus” approach towards the all-encompassing regulation of data protection and privacy. ¹²⁵ Under Article 2, the GDPR regulates the processing of personal data. ¹²⁶ As such “ to fully understand the material scope, one needs to look at the definitions of “personal data”, [and] “processing”.” ¹²⁷

Article 4(1) defines personal data as “ any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person”. ¹²⁸ The notion of “any information,” implies that the term “personal data” should be interpreted as broadly as possible. ¹²⁹ Similarly, the CJEU in case C-434/16, stated that the notion of “relating to” found under Article 4(1) of the GDPR is “ is satisfied where the information, by reason of its content, purpose or effect, is linked to a particular person”. ¹³⁰ As per Bygrave and Tosoni, this “ puts another gaping hole in the few borders of the personal data concept, although the reference to ‘a particular person’ would seem to exclude that data relating to an aggregate of persons (e.g. a household) are personal data, irrespective of the size of the aggregate”. ¹³¹ Hence the concept of “personal data” is arguably rather broad. One must also consider the higher level of protection afforded to so called “special categories of personal data” under Article 9 of the GDPR which restricts the processing of “ personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation”. ¹³² This may prove particularly challenging for users conducting TDM, particularly where inferences can be made using particular data. ¹³³ The GDPR implications caused to users conducting TDM by inferences are discussed below.

Article 4(2) defines processing as “ any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction”. ¹³⁴ This definition, like the definition of “personal data” is also broad, essentially covering “ any data processing operation”. ¹³⁵ Thus, taking a synoptic view of the notions of “personal data” and “processing” it can be deduced that the act of TDM is likely to interact with the GDPR.

Apart from its “omnibus” approach toward the regulation of data protection and privacy, the GDPR also has wide territorial reach. As per Dove “ one might consider the GDPR to be a Europe-centric law of little global consequence […] on the contrary, the territorial scope of the GDPR follows the data that it protects and therefore has direct bearing on the activities of organisations based in countries around the world”. ¹³⁶ This cross-border reach of the GDPR may in fact restrict certain forms of TDM. As stated by Colonna, “ the ability to conduct meaningful research utilising TDM requires not only being able to access data remotely but also being able to share and further process results within the Digital Single Market and beyond” . ¹³⁷ The territorial reach of the GDPR is governed by Article 3 which extends the regulations’ reach beyond the territorial boundaries of the EU. As clearly outlined by Svantesson, “ Article 3 may be broken down into three parts. The first (Article 3(1)), ensures that the GDPR applies to the processing of personal data by a controller or a processor with an establishment in the Union. The second (Article 3(2)), extends the GDPR’s application to a controller or a processor that lacks an establishment in the Union, under certain defined circumstances. The third (Article 3(3)), addresses specific situations where Member State law applies by virtue of public international law.” ¹³⁸ As such, the GDPR not only applies to personal data that is processed in the EU, but also beyond, following the data that it protects. Given the global scale at which TDM is often conducted, it is consequently expected that users conducting TDM will only escape the territorial scope of the GDPR in very limited circumstances.

To legally process personal data, a user conducting TDM would have to abide by the principles relating to lawful processing of personal data under Article 5, meaning the personal data would have to be “ processed lawfully, fairly and in a transparent manner in relation to the data subject” (Article 5(1)(a)), “ collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes” (Article 5(1)(b)), “ relevant and limited to what is necessary in relation to the purposes for which they are processed” (Article 5(1)(c)), “ accurate and, where necessary, kept up to date; every reasonable step must be taken to ensure that personal data that are inaccurate, having regard to the purposes for which they are processed, are erased or rectified without delay” (Article 5(1)(d)), “ kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the personal data are processed” (Article 5(1)(e)), “ processed in a manner that ensures appropriate security of the personal data, including protection against unauthorised or unlawful processing and against accidental loss, destruction or damage, using appropriate technical or organisational measures” (Article 5(1)(f)). ¹³⁹

One issue, raised in the literature examined for this literature review, which users conducting TDM could encounter and is discussed in greater detail below, is the issue that comes along with data minimisation under Article 5(1)(c). Namely, minimising data can sometimes lead to a loss of context, potentially causing more harm than good in certain data mining scenarios. In their book chapter which predates the introduction of the GDPR, Van der Sloot, instead of data minimisation, which was also a requirement in the Data Protection Directive which predates the GDPR, introduce the concept of “data minimumisation”. ¹⁴⁰ Meaning that a minimum set of data “ are gathered, stored and clustered when used in practice”. ¹⁴¹ Van der Sloot argues that doing so, could alleviate some of the data protection and privacy concerns that arise from a loss of context due to minimising data. ¹⁴² This balance between data manipulation and data utility is discussed in greater detail below.

Additionally, to legally process data, users conducting TDM would need to specify a legal basis for the data processing under the closed list of six legal bases found in Article 6 of the GDPR. ¹⁴³ The first legal basis for processing data is that of consent (Article 6(1)(a)). Obtaining consent may be cumbersome for users conducting TDM. For instance, taking the use of social media data as an example, Bremert argues that “ it is more than doubtful, that a clause, hidden within the terms of service or the privacy policy of an online service is a clear indication of the data subject’s wish nor can these terms specify legal consequences for plain actions such as publishing content on a social media platform”. ¹⁴⁴ Bremert ultimately concludes that such consent should be regarded as a “sham consent” thereby not being regarded as “legal basis” for the processing of data. ¹⁴⁵ Similarly, where the data being processed as part of mining activities, is being processed for purposes which differ from the purposes the data was initially collected for, users conducting TDM may struggle to comply with Article 6(4) of the GDPR. ¹⁴⁶ To comply, a compatibility test would have to be performed “ which considers the link between the original purposes and the secondary purpose, the context of processing, the type and nature of the data, the possible consequences of the further processing, and the presence of the appropriate safeguards for processing”. ¹⁴⁷

Notably, under Article 89 the GDPR acknowledges the importance of facilitating different forms of research. ¹⁴⁸ A baseline level of adherence with the GDPR is set for scientific and research purposes whilst balancing the rights and freedoms of the respective data subjects using appropriate safeguards, such as pseudonymisation or anonymisation “ though the list of safeguards mentioned in the article is non-exhaustive”. ¹⁴⁹ It permits derogations from certain data subject rights including, if fulfilling them would impair the research. Article 89 also allows the further processing of personal data for research even if not originally collected for that purpose, and data can be stored for extended periods if solely used for research. Thus, provided TDM is carried out for research purposes, it is possible that the such users may find refuge under Article 89 of the GDPR with exceptions that are designed to balance privacy with the benefits of research.

As aforementioned, one of the first steps for TDM is the collection of data. Arguably, new methods of data collection, namely “data scaping”, also referred to as “web scraping”, which allows users to systematically and efficiently collect data from the web, have allowed for more efficient and wider reaching data collection. ¹⁵⁰ However, it is argued that users who conduct TDM and utilise such data collection methods are likely to encounter restrictions posed by the GDPR. ¹⁵¹ Namely, in the Polish case of Bisnode it was decided that using data scraping, may result in a breach of Article 14 (1–3) of the GDPR which mandates users who scrape publicly available data to provide the data subject with specific details, including the types of data being processed, the legal basis and purposes for the processing, their rights concerning personal data, and the contact information of the data controller. ¹⁵²

While the GDPR provides protections to users whose data are being collected, TDM in some instances may be used to draw inferences and predictions “ about the behaviours, preferences, and private lives of individuals”. ¹⁵³ The GDPR’s application to inferential data i.e., the insights and predictions derived from data mining, remains ambiguous and insufficient. According to Wachter and Mittelstadt, inferences drawn from data are often treated as "economy class" personal data under the GDPR, meaning individuals have little control or oversight over how these inferences are used in decision-making processes. ¹⁵⁴ This “accountability gap” is especially concerning given that inferences can be privacy-invasive, discriminatory, or damaging to one's reputation, yet they are often unverifiable and based on non-transparent algorithms. ¹⁵⁵ As such, Wachter and Mittelstadt argue in favour of a new "right to reasonable inferences" that would require data controllers to justify and disclose the reasoning behind high-impact inferences before they are used. ¹⁵⁶ This gap reveals that while the GDPR aims to regulate the collection and processing of data, it falls short in addressing the broader implications of algorithmic decision-making and profiling in TDM-related activities.

Whilst the GDPR, is generally viewed as an advancement in data protection law, encouraging responsible scientific research by introducing a plethora of stringent pre-processing and procedural requirements such as data protection by design, requirements of reporting data breaches, the appointment of data protection officers, the enforcing of hefty fines, and the right to be forgotten, it is argued that it does not necessarily represent a radical shift in data protection laws. ¹⁵⁷ Nevertheless, it also argued that some businesses face challenges in going from GDPR non-compliance to compliance. ¹⁵⁸ “ Organisations and businesses have been processing personal data of people, in cases without explicit consent, up to the extent required for their businesses, retaining it for longer periods, and even subletting it to other businesses to be used for their own processing purposes”. ¹⁵⁹

In existing literature examining the interplay between TDM and data protection and privacy rules, a lot of emphasis is placed on privacy by design, or privacy preserving measures. ¹⁶⁰ GDPR non-compliance may result in hefty fines being placed. ¹⁶¹ Hence, an emphasis on privacy preserving measures in existing literature may be correlated to an increasing need for GDPR compliance.

Privacy by design is enshrined under Article 25 of the GDPR which asks of data controllers and processors to “ both at the time of the determination of the means for processing and at the time of the processing itself, implement appropriate technical and organisational measures, such as pseudonymisation, which are designed to implement data-protection principles, such as data minimisation, in an effective manner and to integrate the necessary safeguards into the processing in order to meet the requirements of this Regulation and protect the rights of data subjects.” ¹⁶² It is often recommended that techniques for preserving privacy are adopted at early stages of a research project to align with respective regulatory frameworks. ¹⁶³

The European Data Protection Board, in addition to the early adoption of privacy preserving measures, makes a series of further recommendations for adherence to Article 25 of the GDPR. ¹⁶⁴ They encourage the active involvement of a Data Protection Officer, if available. ¹⁶⁵ They also recommend the certification of processing operations as these offer added value and can enhance trust and competitiveness. ¹⁶⁶ If certification is not available, controllers should ensure producers and processors comply with privacy by design requirements through other guarantees. ¹⁶⁷ Specific protections for children and vulnerable groups should be considered, and producers and processors should support controllers in meeting privacy by design obligations by facilitating implementation and staying updated on technological advancements. ¹⁶⁸ Controllers should include contractual clauses to keep up with changes and use performance indicators to assess compliance. ¹⁶⁹ Harmonised, sector-specific guidance should be sought, and transparency in demonstrating effective privacy by design implementation is crucial. ¹⁷⁰ Privacy-enhancing technologies (PETs) should be used as appropriate, and legacy systems must comply with privacy by design obligations or be discontinued. ¹⁷¹ SMEs can facilitate compliance by conducting early risk assessments, starting with small-scale processing, seeking guarantees from producers and processors, and utilizing available guidance and professional advice. ¹⁷² Adopting privacy by design means that rules mandated by regulatory frameworks can be “hardwired” into a TDM process. ¹⁷³

A number of scholars in this domain, attempt to provide practical guides on how to implement privacy by design. For instance, Sangaroonsilp et al., identified “ an emerging need to translate complex privacy concerns set out in regulations and standards into requirements that are to be implemented in software applications”. ¹⁷⁴ The authors consequently “ developed a comprehensive taxonomy of privacy requirements for software systems by extracting and refining requirements from the widely-adopted GDPR and ISO/IEC 29100 privacy framework as well as the newly developed Thailand PDPA and the region-specific APEC privacy framework”. ¹⁷⁵ Hence, Sangaroonsilp et al. took a multijurisdictional approach to the applicability of data protection and privacy related regulatory frameworks developing a holistic taxonomy of the requirements a business which collects and/or processes data will have to adhere to. This could equally apply to a user conducting TDM. The authors broke these requirements down into 7 categories, including themes like lawfulness, purpose limitation, data minimisation, accuracy, storage limitation, integrity and confidentiality, and accountability, with a total of 71 requirements. ¹⁷⁶

Other scholars provide more technical guidance on implementing a privacy by design approach. The majority of such literature predates the GDPR. Nevertheless, they still provide valuable technical guidance, for implementing techniques which would allow for GDPR compliant mining techniques. Clifton and Marks emphasise the need to balance between the benefits of data sharing and the risks associated with data mining, advocating for careful control of access to data to ensure efficient sharing without compromising security and privacy. ¹⁷⁷ Whilst this paper is from the year 2000, the authors acknowledge the evolving nature of the field and call for ongoing research to address emerging challenges in preventing unwanted data mining. ¹⁷⁸

Schermer examines the data mining approaches in profiling which involves discovering correlations in databases to represent individuals or groups. ¹⁷⁹ Schermer explores two data mining approaches: descriptive, which uncovers relations between data objects without a specific target, and predictive, which predicts events based on established patterns. ¹⁸⁰ Schermer identifies an number of privacy related risks associated with profiling using data mining techniques such as discrimination, de-individualization, and information asymmetries. ¹⁸¹ Schermer argues that traditional privacy protection strategies alone are insufficient, and a multidisciplinary approach involving increased accountability, transparency, and algorithmic safeguards is crucial. ¹⁸² Schermer argues that for descriptive data mining, awareness among data subjects and controllers about the possibilities, limits, and risks of data mining is essential. ¹⁸³ Similarly predictive data mining requires transparency to prevent inaccurate applications, emphasizing the importance of distinguishing between direct and indirect indicators in decision-making. ¹⁸⁴ Whilst this paper also predates the GDRP, it highlights, and provides guidance on some of the difficulties highlighted by Wachter and Mittelstadt, which were discussed above. ¹⁸⁵ Namely that GDPR’s application to inferential data remains ambiguous and insufficient.

Clifton et al., in a pre-GDRP era highlight the growing interest in privacy-preserving data mining, but the existence of ambiguity in defining privacy. ¹⁸⁶ Whilst this paper predated the GDPR, it provides some technical guidance on privacy preserving techniques which could be implemented by users conducting TDM today. Clifton et al. underscore the need for diverse measures of privacy preservation, providing four distinct measures for doing so. ¹⁸⁷ First, the concept of bounded knowledge is introduced which involves data obscuration techniques where information about a protected attribute may be revealed within certain bounds. ¹⁸⁸ Clifton et al. provide a metric for doing so which adds noise to certain variables. ¹⁸⁹ Second the so-called “need-to-know” concept is introduced which implies that data should be processed only if necessary for a specific purpose, aligning with legal and privacy regulations. ¹⁹⁰ Secure Multiparty Computation is suggested as a basis for data mining that adheres to the "need to know" standard. ¹⁹¹ “ If the results of the data mining are required to accomplish an allowable task, then learning those results should be allowed under the “need to know” standard under which most privacy regulations allow release of information. Data mining approaches that are Secure Multiparty Computations can be proven not to disclose anything except the results”. ¹⁹² Third Clifton et al. argue that certain items, including individual data items and rules, may need protection from disclosure categorising such techniques for achieving this under the category of “protected from disclosure”. ¹⁹³ Existing techniques in the database security community for inference prevention are discussed, but challenges in defining appropriate thresholds persist. ¹⁹⁴ Finally Clifton et al. present anonymity techniques as a concept for privacy preserving data mining techniques. ¹⁹⁵

Lindell and Pinkas address the challenge of two parties with private databases wanting to collaborate on computing a data mining algorithm without revealing the contents of their databases to each other. ¹⁹⁶ The goal is for each party to only learn information from the output of the data mining algorithm. ¹⁹⁷ Lindell and Pinkas propose the use of decision tree learning. ¹⁹⁸ One concern that is raised is the concept of the so called “semi-honest adversary”, where malicious parties follow the protocol but attempt to learn additional information from the communication transcript, by altering their input, e.g. leaving their input parameter blank returning the data of the other party which they wish to retrieve. ¹⁹⁹

Agrawal and Aggarwal aim to protect user information while still enabling data mining. ²⁰⁰ The proposed approach involves perturbing data values using a known distribution, allowing the reconstruction of aggregate distributions rather than individual records. ²⁰¹ Similarly, in their book, Vaidya, Zhu and Clifton explore techniques such as data perturbation and Secure Multiparty Computation to address potential misuse of data. ²⁰² This highlights the aforementioned balance that must be struck between data manipulation and data utility.

Kantarcioǧlu et al., raise concerns about existing techniques for privacy-preserving data mining, such as adding noise to data or encryption-based approaches. ²⁰³ Kantarcioǧlu et al. emphasise the need to evaluate the privacy impact of data mining models. ²⁰⁴ The authors use a "medical diagnosis" scenario to illustrate the complexity of privacy issues, discussing the challenges of determining whether a classifier violates privacy when using public and private information. ²⁰⁵ The authors propose a framework for evaluating privacy loss in data mining results, offering precise definitions and formal analysis. ²⁰⁶ Section 2 presents a classification model, and Section 3 introduces a metric for privacy loss, providing examples and methods for calculation. ²⁰⁷ Their work aims to contribute to understanding and addressing privacy concerns in the context of data mining models. ²⁰⁸

In conclusion, literature examining the interplay between TDM, and data protection is sparse. There seems to be a focus on creating mining mechanisms which implement privacy by design and several scholars provide practical and technical solutions for achieving this. The common underlying message seems to be the manipulation of private information to avoid data breaches. Nevertheless, there seems to be a correlation between data manipulation and data utility and a balance must be struck between the two to allow for privacy preserving TDM research to be carried out.

TDM and fundamental rights

Another regulatory framework which may come into play when conducting TDM, which is sometimes examined by scholars, is that of fundamental rights. Fundamental rights, in this context, refer to the basic rights and freedoms guaranteed to individuals, typically protected through various legal mechanisms. In the United States, these are primarily enshrined in the Constitution and its amendments. In the European Union, they are protected at multiple levels: through the EU Charter of Fundamental Rights, national constitutions and laws of member states, and the European Convention on Human Rights.

These rights form the cornerstone of democratic societies and often include freedoms such as expression, privacy, and access to information. As TDM technologies continue to advance, they increasingly interact with and sometimes challenge these fundamental rights, raising important questions about the balance between technological progress and individual protections.

Some scholars indirectly examine the interplay of TDM and fundamental rights, by exploring how TDM regulatory frameworks like those examined above, might interfere with fundamental rights. Others, directly examine the act of TDM and how that might interact with fundamental rights. Several human rights may be at risk, depending on the type of TDM being carried out, and under which circumstances. This section aims to present the literature that examines both the direct and indirect impacts of TDM on fundamental rights in the U.S. and the EU.

Indirect. A series of academic papers examine the recent EU CDSM TDM-specific exceptions, presented above, but under the guise of how such exceptions interact with fundamental rights. For instance, Manteghi argues that the balancing mechanisms regulating their scope of application, may restrict one’s freedom of information and research enshrined under the freedom of expression. ²⁰⁹ Manteghi argues that the restriction of Article 3 of the CDSM to public sector entities, excluding private actors like journalists, individual researchers, and businesses raises concerns about restricting the freedom of information and research. ²¹⁰ In an earlier paper, Manteghi argues that Article 3 and 4 of the CDSM which introduce the TDM-specific exceptions “ are likely to create uncertainty about their coverage and scope, which possibly may undermine the users’ right to information”. ²¹¹ Similarly, Geiger argues that the EU’s recent strategies for regulating AI resembles, a “ football team that would be left without any strikers to score successfully and to win any of the competitions with other jurisdictions which may have the advantage of more flexible legal provisions allowing broader TDM activities”. ²¹² Geiger argues that by restricting the scope of Article 3 of the CDSM, it may interfere with journalists’ freedom of expression highlighting the “Panama Papers” scandal as an example which may fall outside the scope of this new exception. ²¹³ Geiger added that the Commission “ risked creating an ineffective and therefore rapidly obsolete provision, in particular regarding the development of artificial intelligence, but also with regards to other activities of essential research and innovation not conducted by public bodies”. ²¹⁴ Scholars’ views regarding the narrow scope given to the EU TDM-specific exceptions were presented above, whereby their application is limited to content that is “lawfully accessed”. Manteghi argues that this concept should be interpreted in light of the term “lawful use” thereby ensuring that any harm caused by this scoping is proportional to the benefits that these exceptions are likely to provide in terms of copyright protection. ²¹⁵ Furthermore, Synodinou argues that “lawful access” requires the adoption of a broad meaning within the confines of TDM extending to “any use which is not restricted by law”. ²¹⁶ Hence, any means by which lawful use has been obtained will inevitably allow for the application of those EU CDSM TDM-specific exceptions, arguably proportionally safeguarding one’s right to information and research. ²¹⁷

Direct. Berendt and Preibusch, conducted an empirical study designed to investigate how people make decisions with the assistance of data mining tools and how discrimination arises in these semi-automated decisions. ²¹⁸ The authors argue that TDM may in some instances create or perpetuate discriminatory outcomes ultimately interfering with one’s freedom from discrimination, which in the EU is enshrined in the GDPR. ²¹⁹ This is one example of how TDM may directly interfere with fundamental rights. Berendt and Preibusch point out that data-driven decisions can impact individuals' rights by reinforcing existing biases or introducing new forms of discrimination, especially when based on sensitive attributes like race, gender, or nationality. ²²⁰ The authors advocate for a combination of algorithmic fairness tools, human oversight, and organisational accountability to reduce discrimination in data mining processes. ²²¹ Similarly, but in the context of the US, Barocas and Selbst, argue that “ algorithms are only as good as the data they work with” and if the data reflects historical biases or social inequalities, the algorithm applied to such data will likely reproduce those biases in its decision-making processes. ²²² In the US, discrimination can be proven through “disparate impact”, which focuses on outcomes rather than intent, and a seemingly neutral practice disproportionately affects a protected class. ²²³ Data mining can cause disparate impact by unintentionally discriminating against vulnerable groups due to biased data or flawed feature selection. ²²⁴

Tien, examining the US jurisdiction, considers the extensive information held by the US government on individuals, stored across various databases, raising concerns about civil liberties such as privacy, freedom of association, and freedom of speech questioning whether the Fourth Amendment, designed to protect against unreasonable searches and seizures, constrains government data mining activities. ²²⁵ “ The Fourth Amendment's basic purpose is to safeguard the privacy and security of individuals against arbitrary invasions by governmental officials”. ²²⁶ The author argues that the identification of patterns by mining across several databases, constitutes a search and thus cannot be knowingly exposed, and therefore, subjects of such searches deserve Fourth Amendment protection. ²²⁷

In summary, several scholars examine how the recent EU CDSM TDM-specific exceptions may interfere with one’s freedom of information and research, whilst another optic angle is provided by authors who assert that TDM itself might interfere with one’s right to privacy or freedom from discrimination. As aforementioned, the interplay between TDM and fundamental rights is potentially multifaceted, depending on the type of TDM being carried out, and under which circumstances. For instance, one’s right to property which acts as the human rights foundation for the protection of intellectual property rights, enshrined in the EU under Article 17(2) of the Charter of Fundamental Rights, which states that “ intellectual property rights shall be protected”, may be interfered with when conducting TDM. Nevertheless, the CJEU clarified the scope of this right in stating that whilst “ protection of the right to intellectual property is indeed enshrined in Article 17(2) of the Charter of Fundamental Rights of the European Union (‘the Charter’). There is, however, nothing whatsoever in the wording of that provision or in the Court’s case-law to suggest that that right is inviolable and must for that reason be absolutely protected”. ²²⁸ Hence, one’s right to intellectual property may be interfered with under certain circumstances. For this to happen, the CJEU requires a “ balance to be struck between the various fundamental rights protected by the European Union”. ²²⁹ Thus, it can be concluded that another area of law which may come into play when conducting TDM is that of fundamental rights. The right interfered is dependent upon the type of TDM being carried out, and under which circumstances.

TDM and contract law

So far, the arguments presented above, to a great extent, assume that no contract applies. It is, however, possible that contracts exist between hosts of databases and datasets and those users which want to access that data for mining purposes. Such contracts may restrict TDM from taking place. ²³⁰ For instance, many scientific publishers offer licenses which explicitly prohibit TDM from taking place. ²³¹ Others apply their contracts through “terms of use” on their websites, which often also restrict TDM from taking place. ²³² This section aims to present scholars’ views on the area where contract law and TDM meet.

The CDSM has arguably brought about changes in this intersection between contract law and TDM. ²³³ Above, author’s views regarding Article 4 of the CDSM were presented which arguably allows for rightsholders to opt-out from TDM activities to a certain extent restricting TDM through contracts. Nonetheless, Article 7(1) of the directive explicitly restricts contracts from overriding the mandatory exception found under Article 3 of the CDSM. ²³⁴ However, as discussed in previous sections and as pointed out by Meys, “ in terms of material scope, the exception is limited to subject matter to which the beneficiaries have lawful access”, which creates a contradiction as it conceptually implies that this exception will only apply to TDM that makes use of content that is freely available online or accessed based on “ open access policies or contractual arrangements such as subscriptions”. ²³⁵ Bearing in mind the discussion above, this lies on the opposite side of the legal spectrum when compared to its US counterpart where Carroll argued that in the US, users would, in theory be able to access, download and use works from Sci-Hub for the purposes of TDM. ²³⁶

Ducato and Strowel, examined the real-world impact contracts have on users who wish to conduct TDM. ²³⁷ The authors examined the terms and conditions of “ 21 online platforms, equally distributed among three sectors: mobility (carpooling and car sharing); accommodation (including services for sharing office space); and food (i.e. initiatives for the recuperation of unsold or unused food, sharing or delivery of home-cooked meals, etc.)”. ²³⁸ Ducato and Strowel found that almost all (20 out of 21) websites examined either expressly or indirectly excluded TDM from taking place. ²³⁹ The authors assert that their analysis “ shows a trend toward the general contractual ban of TDM”. ²⁴⁰

McCracken and Raub who examined the difficulties researchers may face when conducting TDM in the US assert that “ in some ways, TDM licensing still seems like a Wild West, where too many vendors are taking too many approaches, causing too many librarians to have to figure out far too many different licensing options” . ²⁴¹ The authors attempted to establish “ access for a collection of faculty and graduate students from a range of disciplines, departments, and even universities”. ²⁴² When doing so the authors encountered significant and remarkable limitations in how much they were able to achieve with different content providers. ²⁴³

In a blog post Hugenholtz argues that the EU CDSM TDM-specific exceptions “ effectively create and legitimise a derivative market for text and data mining, which right holders may wish to control, license or even entirely prohibit”. ²⁴⁴ Manteghi asserts that this may in turn “ suppress the ability of users to access, share, or express protected content across the public and private spheres”. ²⁴⁵ The author highlights the difficulties involved in obtaining rights clearance, which will indirectly benefit very large commercial companies or internet giants, whilst hindering start-ups and SMEs ability to progress in the field of AI. ²⁴⁶ As per Geiger “ take it or leave it contractual conditions make access conditional upon accepting providers’ terms, while obtaining specific permission from various publishers to carry out TDM research can be extremely complex”. ²⁴⁷ Manteghi highlights the need for Member States to deviate from this opt-out mechanism in a way in which is beneficial for start-ups and SMEs involved in AI development. ²⁴⁸

On the contrary, Senftleben argued that the opt-out mechanism for TDM under Article 4 of the CDSM could be transformed into frameworks that enhance the collection of descriptive and ownership data in favour of rightsholders. ²⁴⁹ Senftleben argues that these provisions provide an opportunity for the EU to enhance licensing opportunities for rightsholders. ²⁵⁰ If Europe fails to develop large-scale, low-cost licensing solutions, AI training may shift to regions with better infrastructure, risking significant revenue loss for the European creative industries. ²⁵¹ Therefore, the creation of a comprehensive and interoperable data infrastructure is urgent to capitalize on these emerging opportunities. ²⁵² Hence, unlike what other scholars have argued, where complex licensing agreements may ultimately stifle TDM research, Senftleben argues that Article 4 of the CDSM provides the opportunity to enhance metadata, streamlining licensing processes, allowing for rightsholders to more easily license their content for TDM purposes. On a similar note but using a different line of argument for a proposed solution, Tylec et al., highlight the difficulties involved with identifying works used in training AI, and ensuring rightsholders are equitably remunerated. ²⁵³ Tylec et al. propose a solution similar to the private copying levy in the InfoSoc Directive. ²⁵⁴ With their proposed solution, AI developers would be allowed to use copyrighted works for training algorithms without needing individual licenses for each work. Instead, they would pay a statutory fee to compensate copyright holders, and the collected fees would be managed by collective management organisations. ²⁵⁵

In conclusion, contracts, and licensing agreements between hosts of datasets and those who wish to access said data for mining purposes, currently act as a barrier to free mining. To conduct lawful mining, users must navigate complex contractual terms and agreement procedures. Whilst some scholars, propose solutions for streamlining licensing agreements for TDM, there is still a gap in formal black letter law in this regard.

TDM and competition law

Vesala proposes that one way in which these contractual hurdles for TDM may be addressed is through competition law. ²⁵⁶ This takes us onto the next section of this literature review which presents author’s views on the interplay between TDM and competition law. Vesala argues that EU competition law can assist in overcoming the obstacles created by contract law “ by requiring copyright holders to provide access to works and grant licences for their use”. ²⁵⁷ Nevertheless, Vesala acknowledges the fact that such legal remedies offered by legal frameworks which regulate competition, may only be sought “ when the practices concerned constitute abuse of a dominant position or a restrictive agreement, which is not automatically or even generally the case when access to or use of copyright protected materials is limited”. ²⁵⁸

Beyond the remedies offered by EU competition law in overcoming contractual hurdles users conducting TDM may encounter, Kochelek examined the potential of using TDM to act anticompetitively, examining how such practices may come into conflict with US antitrust laws. ²⁵⁹ Kochelek argues that when data mining practices are employed against aggregated customer data, thereby enabling the identification of a consumers’ so-called “pain-point” i.e., the most undesirable value at which a party will still engage in a transaction, this in turn enables price discrimination. ²⁶⁰ The author ultimately argues that enforcing US antitrust rules against data-mining based price discrimination aligns with the US Sherman Act principles, which prescribe the rules governing free competition. ²⁶¹

Hence, competition law may act as a bridge in overcoming contractual restrictions for users conducting TDM using datasets held by hosts which abuse their dominant position. Similarly, TDM may be used to enable anticompetitive practices.

Generalist approach

In addition to examining specific regulatory frameworks like copyright and the sui generis database right, data protection, contract law, fundamental rights, and competition law, several authors adopt a broader, “generalist approach” to the legal analysis of TDM. Rather than focusing on one particular area of law, these pieces of literature explore the intersection of multiple legal regimes and how they collectively influence the practice of lawful TDM. Their work often highlights overarching legal challenges such as inconsistencies across legal frameworks, the fragmentation of regulations, or the cumulative impact of multiple legal constraints, faced by those seeking to navigate between “free mining” and lawful, legally compliant TDM. This section presents literature that approaches TDM regulation to a certain extent, more holistically, offering cross-cutting insights into how different legal regimes converge to either facilitate or hinder TDM activities. In contrast to the prior sections, which analyse regulatory frameworks in isolation, these works provide a more integrated view of the legal landscape.

In a pre-Brexit era, Brook highlighted the EU regulatory framework governing TDM drawing examples from the UK’s approach. ²⁶² The author stresses that copyright protection and the subsequent potential for copyright infringement acts as a barrier to users who wish to conduct TDM, examining the recent, at the time, introduction of a TDM-specific UK copyright exception for TDM carried out for the sole purpose of research for a non-commercial purpose, provided copying is accompanied by sufficient acknowledgement, except where this would be impossible. ²⁶³ The author nonetheless, argues that with the introduction of that exception “ numerous constraints have, for UK researchers, been removed at a stroke”. ²⁶⁴ The author then highlights the difficulties database rights create for researchers wishing to conduct TDM. The author whilst only briefly describing the issues associated with database rights concludes that “ as with copyright, the law here is often unclear and acts as a chilling effect and barrier to many”. ²⁶⁵ Brook then moves onto discuss the issues created by complex licensing procedures highlighting the importance of creating open-access scientific literature through the use of creative commons licensing. ²⁶⁶ In an era preceding the CDSM, the author calls for a collective harmonised approach at EU level. ²⁶⁷ Hence, it can be deduced that Brook, in a pre-Brexit and CDSM era, highlighted some of the same difficulties that more recent scholarly work reviewed above, had also discussed.

In a 2014 study funded by the European Commission, when the UK was on the cusp of introducing their TDM specific exception, Triaille et al., examined the copyright issues relevant to TDM in the EU in-depth, providing first a working definition of TDM and access to data choosing to use the term “data analysis” as an all-encompassing term. ²⁶⁸ The authors then analyse the exclusive rights afforded by copyright protection and how TDM might interfere with those, but also the potentially applicable exceptions at the time. ²⁶⁹ Triaille et al. subsequently examine other legal provisions that may be of relevance in the context of TDM, briefly highlighting issues relating to data protection, contract law and unfair competition. ²⁷⁰ The authors conclude by suggesting the introduction of a “new “data analysis exception” applying solely to TDM, which greatly resembles the CDSM exceptions. ²⁷¹

Truyens and Van Eecke provide an analysis of the regulatory frameworks applicable to text mining, distinguishing it from data mining. ²⁷² The authors examine how text mining interacts with copyright law analysing the criteria for assessing originality and the applicability of exceptions. ²⁷³ The authors then describe how text mining might come into conflicts with database rights, and contract law, ²⁷⁴ before providing their assessment claiming that a legal issue within the context of text-mining is fact-dependant and that “ only for some delineated areas (very old texts, legal statutes, texts in the public domain) strong legal certainty can be obtained without case-by-case assessments”. ²⁷⁵ The authors suggest that rigid legal rules in the EU might hinder scientific research and technological developments related to text mining. ²⁷⁶

Guadamuz and Cabel, focus on an analysis of content mining in research institutions and the challenges faced by both users and creators and as aforementioned, claim that legal issues arise at the point at which a user conducting TDM obtains access to their dataset and apply mining tools to said dataset. ²⁷⁷ Guadamuz and Cabel also focus on an analysis of copyright law, database rights and licensing, calling for an increase in open access policies enabling greater access to mining. ²⁷⁸

Meys, examines the existing limitations on data reusability imposed by database rights and contracts, whilst analysing the EU CDSM TDM-specific exceptions. ²⁷⁹ The author ultimately concludes that copyright, sui generis database rights and contracts pose obstacles to data re-usability for AI development. ²⁸⁰

Rosati who as mentioned in the first section of this review, identifies three general TDM processes “ 1) Access to content, 2) Extraction and/or copying of content, and 3) Mining of text and/or data for knowledge discovery” each of which with their own legal consideration, also adopts a generalist approach highlighting several areas of law that may make up the regulatory framework governing TDM. ²⁸¹ Namely, when analysing the first TDM process – “access to content” – Rosati discusses contractual issues that may arise in terms of obtaining a license to mine. ²⁸² The added difficulties that may arise when dealing with orphan works and out-of-commerce works are also discussed. ²⁸³ When examining the second TDM process – “ extraction and/or copying of content” – Rosati highlights the legal concerns that may arise within the context of copyright and sui generis (database) rights. ²⁸⁴ The author then finally notes that other areas of law may be relevant at this stage such as data protection/privacy laws, and contract law. ²⁸⁵ In the third stage of TDM – “ mining of text and/or data and knowledge discovery” – the author provides a procedural analysis of what a typical TDM activity would involve, in terms of knowledge discovery. ²⁸⁶ Upon analysing these TDM processes, Rosati then moves onto a jurisdictional comparison of “ jurisdictions that display a more frequent and advanced use of TDM”, examining the US’s fair use, the UK’s TDM exception and the EU’s CDSM TDM-specific exceptions. ²⁸⁷ Rosati concludes by stating that as EU Member States transpose the CDSM into their laws, caution is urged to ensure that EU businesses engaging in TDM are not disadvantaged compared to non-EU counterparts, considering TDM's potential impact on AI and AI-generated creativity. ²⁸⁸

Hammon et al. discuss the rapid growth of AI applications particularly since the release of ChatGPT in November 2022, emphasising the need for businesses to consider various legal and compliance risks associated with the development of AI. ²⁸⁹ The authors provide an overview of the regulatory frameworks such businesses are likely to encounter, such as contractual protection and AI specific regulations such as the EU’s AI Act discussed above. ²⁹⁰ Hammon et al. then separate their analysis into input and output risks, examining regulatory risks associated with what the AI is fed as input and what output an AI is likely to produce respectively. ²⁹¹ In terms of input risks the authors highlight that there are likely data protection and intellectual property risks. ²⁹² Similarly with regards to output risks the authors highlight the intellectual property risks associated. ²⁹³ Hammon et al. then examine what they call “overarching considerations”, highlighting the environmental risks related to the energy consumption of AI systems, social risks related to the takeover of a business’s workforce by AI, and the importance of ethical policies, audits, and alignment with corporate values in AI adoption is emphasised. ²⁹⁴

Finally, in a recent paper, Sean M Fill-Flynn summarises the key points that have emerged out of the wider project on the Right to Research in International Copyright. First, the author highlights the importance of TDM which involves analysing digitized information and is crucial for various AI tools, including generative AI models whilst it is also widely used in academic research across different fields. ²⁹⁵ Second, the importance of TDM is emphasised, demonstrating how it is linked to fundamental rights like receiving and imparting information and benefiting from culture and science, often referred to as the “ right to research”. ²⁹⁶ Finally, the author exemplifies the barriers that copyright created for TDM research. First, a patchwork of frameworks across multiple jurisdictions create barriers for TDM research especially in cross-border projects. ²⁹⁷ Second, licensing models for regulating access to text and data hinder research and economic activity, particularly affecting resource-dependent researchers and AI industries, especially in developing countries. ²⁹⁸ Open research exceptions could benefit TDM research, and claims of harm to creators from open exceptions lack empirical support. ²⁹⁹ Finally, there is a trend towards open exceptions especially for research purposes and the right to research should be advocated for in international copyright law. ³⁰⁰

Conclusion

Currently, users who conduct TDM must navigate this complex web of legal rules without a map. This research as a whole, which this literature review forms part of, aims to create a practical guide and tool facilitating lawful TDM to take place.

What this literature review makes evident is that literature examining “lawful TDM” is fragmented. Authors have examined how TDM interacts with copyright and the sui generis database right, data protection, contract law, fundamental rights, and competition law each providing valuable piecemeals of insights into what regulatory frameworks users conducting TDM may encounter.

Whilst some attempt to provide a more general interpretation of how one can conduct “lawful TDM”, this literature review illuminates the fact that they are not all-encompassing. The regulatory frameworks that apply to users conducting TDM are multifaceted, and ever-changing on a case-by-case basis, which heavily depends upon the type of mining being conducted, where mining takes place, the dataset involved, how such data is accessed, and how the output of mining processes will ultimately be used. There is an ever-growing need for a holistic interpretation of the regulatory frameworks which apply, creating a map which would allow for users conducting TDM to navigate this complex web of legal rules.

Funding Statement

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 101034403.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data availability

Extended data

Zenodo: Appendix for Review Paper Entitled: “A literature review of “lawful” text and data mining.” ( https://zenodo.org/records/13880018) (Vrakas 2024)

The project contains the following extended data:

Appendix

Data is available under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0).