Capturing complexity: Mixing methods in the analysis of a European tobacco control policy network

Abstract

Social network analysis (SNA), a method which can be used to explore networks in various contexts, has received increasing attention. Drawing on the development of European smoke-free policy, this paper explores how a mixed method approach to SNA can be utilised to investigate a complex policy network. Textual data from public documents, consultation submissions and websites were extracted, converted and analysed using plagiarism detection software and quantitative network analysis, and qualitative data from public documents and 35 interviews were thematically analysed. While the quantitative analysis enabled understanding of the network's structure and components, the qualitative analysis provided in-depth information about specific actors' positions, relationships and interactions. The paper establishes that SNA is suited to empirically testing and analysing networks in EU policymaking. It contributes to methodological debates about the antagonism between qualitative and quantitative approaches and demonstrates that qualitative and quantitative network analysis can offer a powerful tool for policy analysis.

Introduction

In recent decades, there has been an increasing acknowledgement in the social and political sciences of the importance of networks, relationships and complexity when explaining social phenomena (Thompson, 2004). As a result of attempts to develop techniques which capture this complexity, social network analysis (SNA) has emerged as a powerful methodological approach (Emirbayer, 1997). SNA defines a network as a set of nodes (e.g. individuals, organisations or other entities) which display attributes (e.g. gender, size or position on a certain issue) and are connected by one or more types of relationships (e.g. formal or informal relationships, collaboration or information flows) (Knoke & Kuklinski, 1982; Marin & Wellman, 2011; Scott, 1991; Wasserman & Faust, 1994). The particular value of SNA lies in the opportunity that it offers for studying the complex interactions between individuals or organisations, the constraints and opportunities that are a result of the patterned relationships between them and the impact of the structural environment on their attitudes and actions (Emirbayer & Goodwin, 1994).

SNA has been applied to the study of political institutions, actors and processes (Christopoulos, 2006; Christopoulos & Quaglia, 2009; Harris et al., 2008; Luke et al., 2010; Wipfli et al., 2010). Ideas of networks and methods of exploring them seem to correspond to the recent focus on “policy networks”, a concept which postulates that policy is developed by complex, non-hierarchical groups of mutually dependent actors with an interest and some degree of influence over the policy process (Heclo, 1975; Peterson, 2009). The concept is used to understand, describe and explain the complex web of actors involved in modern democratic governance and the intricacy of policymaking (Marsh, 1998; Peterson et al., 2008; Rhodes, 1997). Policy networks (or, according to Sabatier, “subsystems”) can be divided into smaller sets of actors who have a common interest in a policy issue, share similar values, and, by building coalitions, try to jointly influence the policy process in a certain way (Sabatier, 1993). Policy networks and coalitions have been identified as important features of policy processes and as particularly influential in the development of EU policy (Bomberg et al., 2008). Previous research has further postulated that coalitions are crucial in the development and implementation of tobacco control policies (Cairney, 2007). In a handful of articles, policy networks in tobacco control have been identified as consisting of two adversarial coalitions: one coalition which supports effective tobacco control policies and another which opposes respective policies due to its members' economic interests in tobacco consumption (Farquharson, 2003; Read, 1992; Sato, 1999; Smith, 2013). While the importance of networks in policymaking has long been acknowledged, academics have only recently begun to use SNA to systematically analyse them (Lubell et al., 2012). No studies have used SNA to explore the role of policy networks and advocacy coalitions in the development of tobacco control or public health policy.

Drawing on a European Union (EU) tobacco control policy initiative as a case study, this paper explores how a mixed method approach to SNA can improve understanding of the composition, structure, function and complexity of a network of actors involved in EU policymaking. The paper further introduces a novel method of data collection and conversion and describes how textual data from public sources can be extracted, converted into relational data and analysed using plagiarism detection and SNA software. First, the article critically assesses the available literature on the application and triangulation of different methodological approaches to SNA. It then introduces the case study and describes the quantitative and qualitative approach taken. It critically examines the combination of quantitative and qualitative network analysis and discusses the approach of converting textual into relational data described in this paper. The article concludes by outlining implications for future studies which aim to apply the approach to other areas of research.

Mixed methods and triangulation in SNA

Discussions about research methods in the social sciences throughout the last decades have been dominated by a “great divide” between quantitative and qualitative research approaches (Onwuegbuzie & Leech, 2005, p. 375). Studies have often been categorised as either “qualitative” studies, which collect and analyse textual data, or “quantitative” studies, which deal with numerical data (Blaikie, 2000). This methodological divide is also reflected in social network research (Knox et al., 2006): Quantitative network studies gather data in numerical form, transform it into matrices, graphs and network measures and include statistics and the simulation of networks (Börzel & Heard-Laureote, 2009; Crossley, 2010; Edwards, 2010). Qualitative studies, on the other hand, focus on the content of a network, describe and explain what happens within it and explore processes of interaction, actors' accounts, opinions and perception of the network, and the quality of actors' relationships with each other (Börzel & Heard-Laureote, 2009; Crossley, 2010; Edwards, 2010). Both analytical approaches have been hailed and contested for different reasons. Quantitative network approaches systematically collect data about all possible relationships and actors and analyse the position, structural constraints and possibilities of network members. By reducing data and employing standard criteria for analysis, such studies, however, tend to set artificial network boundaries, neglect questions which are of crucial importance to understanding the complexity of social interaction (Edwards, 2010) and have thus been criticised for over-simplifying social phenomena (Crossley, 2010). Qualitative network studies, in contrast, unearth in-depth information about features of a network and increase understanding about network complexity and the context in which they are established (Crossley, 2010), but risk simply reflecting the messiness of the social world and fail to provide compact information.

Social network scholars whose research is situated on either side of the qualitative-quantitative divide often fail to acknowledge the shortcomings of studies which rely on one of the two approaches. Scholars have argued that the division between qualitative and quantitative methods is unnecessarily rigid and that researchers limit their ability to adequately explore a social phenomenon and their freedom to combine different methods by committing themselves to one approach (Blaikie, 2000). In line with these arguments, some social network scholars stress the considerable value which mixed method studies can bring to the field of SNA and highlight that the two approaches can be mutually informative (Crossley, 2010; Edwards, 2010; Jack, 2010). Edwards (2010, p. 2), for example, postulates that mixing methods allows the researcher to combine an “outsider's view” of a network's structure with an “insider's view” of network content and quality and suggests that network structure is better analysed via quantitative methods, whereas the development, change and meaning of a network is more adequately explored via qualitative methods.

Building on previous literature, this paper aims to critically assess the added value that a mixed method approach can bring to the study of policy networks and thus contribute to the methodological debates on the use of quantitative and qualitative approaches to SNA. Applying a mixed method approach to the investigation of a policy network in EU tobacco control policy, the study used quantitative and qualitative methods to analyse qualitative archival data from policy documents and in-depth interviews. It thus follows one of the ways of mixing methods in social network research discussed in detail by Edwards (2010) and resembles sociological studies by Crossley and Edwards which analyse archival data via a mixed-method approach to explore social phenomena (Crossley, 2008; Edwards & Crossley, 2009).

The Case Study: The European Council Recommendation on smoke-free environments

The mixed method approach described in this paper was developed as part of a research project which aimed to investigate the network of actors with an interest in the development of the European Council Recommendation on smoke-free environments, an EU tobacco control policy addressing smoking in public places (Council of the European Union, 2009b). The negotiations about EU smoke-free policy were initiated by a European Commission Green Paper which was issued in January 2007 and outlined the harms caused by second hand smoke, the regulatory environment in the EU and the potential options for smoke-free policy at EU level (Directorate General Health and Consumers, 2007a). The Green Paper marked the start of a public consultation process, giving interested parties the opportunity to comment on policy options and engage in the policy process. A total of 306 stakeholders, including 176 organisations (public authorities, health-related organisations, tobacco-related organisations, social partners and others), submitted responses to the consultation (Directorate General Health and Consumers, 2007b). The policy was negotiated over a period of two years, leading to a European Commission release of the proposal for a Council Recommendation in June 2009 and the adoption of the final policy document on 30 November 2009 by the Council of the European Union (Council of the European Union, 2009a).

The primary objective of the study was to analyse the network of advocates and their coalition-building during the development of the Council Recommendation on smoke-free environments, focusing specifically on those opposing and supporting the initiative. The study thus explored a case of stakeholder engagement in the development of a non-binding EU policy which contained no direct obligations for EU member states to transfer the policy into national law. It further focused on the negotiations and the period leading to the adoption of the policy measure, rather than on policy implementation and enforcement. The study draws on data from policy documents produced in the course of the policy process and the responses of organisational stakeholders to the European Commission consultation, and data from semi-structured, retrospective, in-depth interviews with 35 political actors who had been involved in the development of the Council Recommendation on smoke-free environments. Qualitative thematic analysis and quantitative network analysis were combined in the analysis of the data. In order to investigate whether a mixed method approach to SNA can shed light on a complex policy network, the focus of this paper is to describe, critically examine and reflect on the methodological approach taken.

Quantitative network analysis

Quantitative data on policy networks have previously been collected in various ways, with surveys and structured questionnaires being the most common mode of data collection (Marin & Wellman, 2011; Wasserman & Faust, 1994). A number of limitations are inherent in such individual accounts of relationships and thus of primary data collection methods (Christopoulos & Quaglia, 2009): Respondents might forget to report people with whom they share relations, withhold information, depict themselves as more connected than they really are or be reluctant or unable to report certain types of relationships (Christopoulos, 2008; Knoke & Kuklinski, 1982; Marin & Wellman, 2011). Such problems are exacerbated by the fact that relying on interviews to gather quantitative network data demands substantial engagement on the part of the interviewer and the interviewee and is thus likely to put considerable constraints on sample size, participant selection and the overall scope of a research project (Christopoulos & Quaglia, 2009; Real & Hasanagas, 2005). In order to overcome such difficulties, social network researchers have drawn on secondary data, including public statements, policy documents and legislative texts (Bellotti, 2012; Christopoulos, 2006; Christopoulos & Quaglia, 2009; Crossley, 2010; Wipfli et al., 2010). While providing a wealth of data, the problem with using existent data sources is that they often do not contain data in the required format. Whereas previously, publicly available data have therefore mainly been used to triangulate primary modes of data collection, this study aimed to capitalise on the richness and advantages of publicly available, archival data and provide insight into a network that would have been difficult to capture using primary modes of data collection. The study piloted a novel method of extracting, adapting and converting textual data from consultation submissions and organisational websites into a relational format that could be analysed using SNA software.

Given that the European Commission's consultation provided an early opportunity for stakeholders to engage in the process of developing EU smoke-free policy, it was assumed that a high proportion of political actors with significant interest in the issue would have submitted a response. Focusing on organisations that submitted responses to the consultation¹ thus seemed a suitable approach to capturing the policy network. In a first step, each organisation that submitted a response to the public consultation was registered as a node². Where several organisations submitted a joint response, each organisation was listed as a separate node. Each organisation was assigned a serial number and several attributes which were considered useful in analysing the network and answering the research questions. Attributes included, for example, organisational type and main focus, member state affiliation, geographical location and position on the policy initiative. Attributes were assigned on the basis of categorisations that had been undertaken by the European Commission's Directorate General for Public Health and Consumers when analysing consultation submissions³ (Directorate General Health and Consumers, 2007c) and on a detailed reading of each submission.

With regard to the definition of relationships between the organisations which had been selected as nodes, approaches were piloted on a random 10% sample of all documents. Three independent measures were identified and defined to distinguish different forms of collaboration between network actors: public relationship, shared citations and active relationship. The quantitative analysis was then systematically applied to the entire dataset of 176 organisational consultation submissions.

Public relationship

Organisations were defined as sharing a public relationship if an organisation officially declared that it had a relationship with another organisation that had submitted a response. In order to identify public relationships, all submissions were carefully read. If organisations mentioned a membership, partnership, collaboration or other type of relationship with another organisation, the respective organisation and the submitting organisation were coded as having a public relationship. In addition, websites of all organisations that had submitted responses were searched using the search engine Google and scanned for any indication of a connection with any other organisation that had submitted a response. While English, French, German and Danish websites were thoroughly read, due to limited language skills, websites in other European languages could only be scanned for any obvious references or logos which demonstrated collaboration with other organisations. A matrix was created and data were coded as binary data. Public relationships were interpreted as illustrating the general willingness of organisations to build coalitions and collaborative partnerships.

Shared citations

If organisations cited the same three or more references in their submission, they were defined as sharing citations. In order to identify shared citations, a complete reference list was extracted for each organisation, listing all citations that the organisation mentioned in its response. The number of joint references was then counted for each pair of organisations. Several random samples were checked to confirm that organisations cited the respective literature to underpin their arguments rather than to refute its content. After scrutinising the samples, it was decided that three joint citations were sufficient to indicate that the respective organisations engaged in a similar discourse. A matrix was created and the data were coded as binary data with a cut-off point at three joint citations. Shared citations suggested that organisations employed similar arguments and drew on the same literature to underpin their positions or referred to the same policy documents to remind policymakers of their commitments and obligations.

Active relationship

Organisations were defined as having an active relationship if their submissions resembled each other by 40% or more. Measuring active relationships between organisations involved several steps. First, all responses were converted into word documents. If the consultation questions that had been posed by the European Commission were repeated, these were deleted in order to avoid counting them as overlap between submissions. All Word files were then uploaded into the plagiarism detection software Turnitin (iParadigms LLC) and subsequently checked to locate identical or similar documents on the internet. When a web-document was identified as a submission by another organisation to the consultation process, the percentage similarity of each organisation pair (reported in the Turnitin similarity index of the originality check) was recorded in a matrix table. All submissions which showed similarities with the respective submission of more than 10% were thoroughly read and cross-checked with the original document⁴. Notes were taken about qualitative aspects of the similarities between the documents (e.g. whole paragraphs that had been identified as identical, themes that were raised by both organisations). After discussion among the team of authors and consistent with practices used within the University of Edinburgh to review student papers for plagiarism, it was agreed that 40% similarity would be a justifiable cut-off point, a conservative indicator of a relationship and likely to identify substantive collaboration, rather than spurious or incidental similarities. An active relationship was assumed to suggest that the respective organisations had collaborated regarding the European Commission consultation on smoke-free policy at an operational level by exchanging drafts of their consultation responses.

Analysis

All data were entered into the SNA software UCINet 6 for Windows (Borgatti et al., 2002) and graphically depicted using NetDraw (Borgatti, 2002). In a first step, all relationship matrices were merged and the overall network was visualised to provide an overview of the entire policy network and any connections between actors (figure 1). The three different relationship types were then depicted in isolation, resulting in networks of public relationships (figure 2a), shared citations (figure 2b) and active relationships (figure 2c) and allowing comparison of the different types of relationships.

Figure 1. Policy network of all organisational actors that submitted responses to the European Commission consultation on smoke-free environments and their relationships, all relationships merged; Squares: organisational actors, Lines: relationships.

Figure 2a. Public relationships between organisational actors that submitted responses to the European Commission consultation on smoke-free environments; Squares: organisational actors, Lines: public relationships.

Figure 2b. Shared citations between organisational actors that submitted responses to the European Commission consultation on smoke-free environments; Squares: organisational actors, Lines: shared citations.

Figure 2c. Active relationships between organisational actors that submitted responses to the European Commission consultation on smoke-free environments; Squares: organisational actors, Lines: active relationships.

Several analyses were then conducted on the overall network which incorporated all three relationship types. By applying the Girvan-Newman algorithm (Girvan & Newman, 2002), the main network component was split into two cohesive subgroups. The network and all components were analysed considering different attributes. This analysis showed that each component consisted of different types of organisation and that the organisational members of each component held clearly contradictory positions on the policy initiative. The distinct groups of advocates were separately analysed and compared with respect to size, composition, density, centralisation and core-periphery structure, and individual network measures were calculated for their respective members. The quantitative analysis served as a starting point to investigate the network and was used to gain an overview of and insight into the structure of the network of actors involved in the development of the Council Recommendation on smoke-free environments. It helped to systematically identify central and peripheral actors in the network, the existence and type of relationships between them, the affiliation of actors with a particular group and drew attention to the lack of connectedness or complete absence of some actors. The sociograms helped to visualise the main network features and made the quantitative analyses more amenable to interpretation. The analysis also provided information which proved valuable in the preparation, execution and interpretation of the interviews and served a basis for the in-depth exploration of the role of individual actors, the quality of relationships and other features of the network. The quantitative approach allowed, for example, the systematic comparison of the composition and structure of the two separate groups, shedding light on each group's make-up and distinct characteristics. It also helped to identify central actors, including key European umbrella organisations and other organisations which were strongly connected and thus held crucial roles in the network and were important in coordinating collaborative political action. Attention was further drawn to unexpected and less obvious relationships, including those between commercial sector and civil society organisations and between tobacco industry and other commercial sector representatives. These relationships were subsequently explored in more detail in the qualitative part of the study.

Qualitative network analysis

The second methodological strand involved a review of documentary data and the thematic analysis of interview data in order to qualitatively investigate the policy network. In a first step, the websites of the main EU institutions and Eurlex (a search engine for European law documents) were searched between October 2009 and September 2010 to gather an overview of the development of the Council Recommendation on smoke-free environments. The following search terms were applied: “smoke-free*”, “smoke free*”, “smoking ban*”, “second hand smoke* AND polic*”, secondhand smoke* AND polic*”. The searches focused primarily on documents produced between January 2007 and November 2009, but documents from preceding and subsequent years were included if they seemed relevant for a comprehensive understanding of the policy process. Using the information that had been retrieved via these searches as a starting point, more specific searches were conducted to locate other documentary material which provided more detailed information about the policy process and the actors involved. The searches produced policy drafts, responses and opinions produced by the EU institutions, documents which the Council Recommendation on smoke-free environments referred to, consultation submissions, briefings, reports, surveys, research reports and other documents. Repeated reading of these documents helped to gain an overview of the actors that had engaged in the policy process, their involvement and positions and the debates that had been held, gather background information, and compile a list of potential interviewees. This information was subsequently drawn on to develop the interview topic guide, verify the coding of the interview data and interpret the data, and thus informed the quantitative network analysis.

Following the review of documentary data, 32 semi-structured, in-depth interviews were conducted with political decision makers, interest representatives of Brussels-based and national organisations and other political actors. Interviewees were selected using purposive sampling drawing on a list of 175 potential interviewees who had been identified as involved in the development of EU smoke-free policy via the documentary analysis. The documentary data were used to compile information about the potential interviewee's organisational affiliation, involvement in the policy process and position on the policy initiative. Based on this information, the individual was assigned to one of three groups: (i) key actors of crucial importance in the policy process (e.g. representatives of EU institutions, EU member states, other institutions or organisations which were strongly involved in the development of the policy, n=49), (ii) stakeholders with considerable interest in the policy process (e.g. other actors who had been involved in the development of the policy, n=59), or (iii) individuals who had shown an interest in the topic of EU smoke-free policy but had not directly engaged in the policy process (n=67). Interviewees were mainly recruited from the first category. Efforts were made to obtain a breadth of data by interviewing representatives of various types of organisations, who held different opinions on the policy initiative, had different geographical remits and member state affiliation, and were involved at different stages of the policy process. In order to obtain insight into specific aspects of the network, efforts were made to recruit representatives of organisations which occupied particularly prominent, unexpected or peripheral positions in the network. A small number of actors who had not engaged in the development of the policy were interviewed to explore some of the reasons for non-participation.

48 individuals were contacted. Six (13%) declined the interview, five (10%) did not respond and 35 (73%) were interviewed in 29 one-to-one and three paired interviews. 27 interviews were conducted in person, whereas five interviews were conducted via the telephone. Interviews were conducted with 18 representatives of public health organisations, five political decision makers (i.e. politicians and civil servants), four representatives of the tobacco industry, four representatives of social partner organisations, one representative of the ventilation industry and three representatives of other commercial sectors. 27 interviewees were representatives of organisations that had submitted a consultation response, meaning that their organisations were counted as actors in the quantitative network analysis. The interviews included a narrative part, during which the interviewee provided a personal account of the development of the Council Recommendation, and a semi-structured part, during which interviewee was prompted about specific aspects, including his/her involvement in tobacco control, the development of EU smoke-free policy, the policy network and advocacy coalitions and his/her personal assessment of the policy process and its outcome. A small number of interviewees were asked to comment on the sociograms, which typically served as useful triggers for discussions, thus shedding light on selected features of the network and helping to clarify contradictory data and interpret the quantitative analysis.

All interviews except three were transcribed verbatim⁵. Each transcript was analysed using QSR NVivo, a computer-assisted software aimed at managing, analysing and reporting qualitative data (QSR International, 2007). Based on the objectives of the research project and the insight gained through the documentary analysis, a hermeneutic analytical procedure was developed, similar to that described by Bauer (2000), which involved an iterative process of identifying recurring themes, comparing them across sub-samples and systematically applying a coding framework to the entire data. All interviews were read several times in order to identify thematic clusters and themes that occurred repeatedly throughout the interviews. Following Boyatzis' framework for thematic analysis and code development (1998), all interviews were then systematically coded according to these themes. Particular attention was paid to ensure that quotes were fully contextualised.

The thematic analysis of the interview data corroborated the distinct division of the network into two opposing groups that was identified through quantitative network analysis. The interviewees reported, for example, that the members of the two components were reluctant to interact with each other and were clearly perceived as two rival coalitions. They stressed the strategic importance of the most central actors and the close collaboration of a core group of actors within each of the coalitions (mirroring the active relationships between network actors identified through quantitative analysis, depicted in figure 2c). Interview data mainly helped to interpret an actor's position and role in the policy network, and further added an in-depth perspective by exploring his/her level of engagement and motivation to engage in the policy process and barriers to policy engagement and coalition-building. By talking about the ways in which they had collaborated with other actors, the issues they collaborated on, the meanings they attached to particular relationships and gaps of collaboration, interviewees presented crucial information about the content, quality and meaning of relationships and interaction. Documentary data, on the other hand, were mainly used to explore, compare and contrast actors' opinions on the policy initiative. The review of documentary and thematic analysis of interview data were crucial for understanding actor constellations, relationships and interactions, developing a deeper insight and comprehending the complexity of the policy network.

While the documentary and interview data and the quantitative and qualitative network analysis largely complemented and supported each other, in some instances, findings were inconsistent or even contradictory. This provided a challenge in terms of making sense of differences that emerged, reflecting Mathison's (1988) acknowledgement of the complexities of converging data. In these instances, extensive efforts were made to explain discrepancies by drawing on additional data sources and reflecting on assumptions and interpretation of the data. Incongruities emerged, for example, with regard to the position and coalition-building of actors advocating for ventilation, which the previous literature depicted as tobacco industry allies (Campbell & Balbach, 2011; Drope et al., 2004). While the quantitative network analysis identified representatives of the ventilation industry as isolates, rather than members of one of the advocacy coalitions, the interviews revealed that they had collaborated with members of the coalition which opposed the EU policy initiative. The interview data further disclosed the sector's reluctance to be associated with the members of this coalition and their explicit efforts to publicly distance themselves from the tobacco sector, with one interviewee highlighting that ventilation industry representatives “wanted to avoid that they are, from an image point of view, shuffled onto the side of the cigarette industry”. Discrepancies between the quantitative analysis which had not identified organisations representing the ventilation sector as tobacco industry allies, and qualitative reports which revealed that these organisations had exchanges with tobacco industry representatives, were thus illuminated and interpreted by using a mixed method approach. This thus added considerable value and insight about the policy network which would not have been gained if the network had only been analysed using one methodological approach.

Discussion

Combining quantitative and qualitative network analysis

This article builds on previous literature on mixed methods in SNA which highlights that combining qualitative and quantitative approaches can provide a comprehensive account and in-depth insight of networks in a number of social settings (Crossley, 2010; Edwards & Crossley, 2009). Researchers who employ mixed methods suggest that the researcher's task is to analyse data in their context and skilfully weave together various pieces of evidence of a social phenomenon (Yin, 2003). Throughout this process, researchers have to carefully consider the ways in which they want to combine different techniques and the role of each method within the research project (Crossley, 2010; Edwards, 2010). To the best of our knowledge, this paper is the first to provide a detailed outline of and reflection on a mixed method approach to SNA in the study of policy networks. It describes the combination of different data sources and analytical methods and outlines the added value that such a methodological approach can bring to a research project. Being the first to use SNA to empirically analyse the network and coalitions of actors involved in the development of EU tobacco control policy, this study provides compelling evidence that SNA is a suitable method to analyse stakeholder engagement in EU policymaking and to empirically test and analyse policy networks. The article demonstrates that combining quantitative network analysis and thematic analysis of qualitative data can offer valuable insights into complex political phenomena and can be used to develop a thorough understanding of the structure and the content of a policy network and key network features, including coalition-building and leadership.

Piloting a novel approach to data collection and conversion for quantitative policy network analysis

The paper further responds to problems of data collection that are inherent to analysing complex, medium- to large-sized, unbounded networks of actors with an interest in a policy issue. Previous research has highlighted that it is hard to determine who belongs to a network and who does not (John, 1998). This is even more the case for policy networks, which are fluid, complex and include a variety of different actors (Peterson, 2009). A particular strength of our approach is that by employing an event-based approach, we were able to collect data on a bounded policy network of organisational political actors interested in EU smoke-free policy while avoiding relying on egocentric networks.

An important contribution of the article is that it describes an innovative approach to analysing policy networks which avoids the limitations of relying on surveys and instead draws on publicly available, textual data sources to gather network data. While previous studies have drawn on the population of actors involved in consultations to retrieve a sample of study participants (Christopoulos & Quaglia, 2009) and on websites and academic publications to extract relational data (cf. Bellotti, 2012; Catanese et al., 2011; Lewis et al., 2008), this project, to the best of our knowledge, is the first to draw on consultation submissions as data sources for SNA and employ plagiarism detection software to convert textual data into a relational format. Plagiarism detection software was used to identify similarities between consultation responses of stakeholders in EU smoke-free policy which served as proxies for connections and collaboration between organisational actors and provided a basis to analyse and graphically depict a complex policy network.

The conversion of textual data from consultation submissions and public websites into a relational format and use of plagiarism detection software to determine relationships between political actors can be considered unique. As with any novel approach, a number of limitations have to be acknowledged. Archival data has been widely used in previous studies to quantitatively analyse networks (Bellotti, 2012; Catanese et al., 2011; Edwards & Crossley, 2009; Padgett & Ansell, 1993), including those investigating the development of EU policy (Christopoulos, 2006) and tobacco control policy (Wipfli et al., 2010). Due to their fragmented character (Finnegan, 1996), one of the main limitations of using archives as data sources for quantitative network research is the risk of missing data and its pivotal implications for analysis and interpretation (Kossinets, 2006; 1983). While these caveats apply to all network studies that rely on archival data, they are likely to be exacerbated if not only information about nodes and attributes but also relational data are extracted from archives. There is no reason to assume that data was systematically omitted, but the non-inclusion of the European Commission constitutes a limitation of this study. The fact that organisations that submitted responses to the consultation were allowed to opt for non-disclosure further implies the risk that some actors were omitted. It also needs to be acknowledged that the approach taken to record public relationships might have resulted in an under-representation of this type of relationship. Some organisational websites could not be retrieved in the authors' languages and could therefore not be searched as rigorously as the English, French, German and Danish websites. The possibility that some relationships were not officially declared or had ceased to exist means that not all existing relationships might have been recorded. Regarding active relationships, it needs to be highlighted that the percentage of similarity on which the definition of active relationships was based was proportionate to the length of the document, with longer documents being less likely to show high percentages of similarity than shorter documents. Finally, the nature of the archive did not allow the extraction of more detailed information about actors and their relationships. Given that the qualitative data sources provided detailed information, most of the drawbacks outlined above were alleviated by the triangulation of the quantitative data with interview and documentary data, highlighting once again the advantages of a mixed method approach. Future studies might, however, be able to refine the approach to incorporate some of these dimensions where appropriate data are available.

Implications for future research

While the case study design and the features of the specific case limit the generalisation of the study's findings to other cases, our study suggests that the methodological approach described in this paper can be applied to future research of policy networks. Replications of the study methodology could, for example, clarify whether the triangulation of quantitative network analysis and thematic analysis of qualitative data can provide equally valuable insights into policy networks in other contexts, e.g. networks that form during the negotiation of binding EU policy, in other policy areas, other legislatures or at later stages of the policy process. One major challenge of mixed method SNA projects is the resources that are required to conduct such studies. Equally, the chosen approach, which involved searching and analysing websites and submissions, comparing submissions for similarities, extracting and converting textual information into a relational format, conducting interviews and coding quantitative and qualitative data, required considerable time and resources. Given that decisions which are made during data gathering and conversion are likely to have a considerable impact on the results (cf. Butts, 2009), it is important to link these decisions to the specific research objectives and be reflexive about their potential implications for the interpretation of the results. Accordingly, such studies need to build in sufficient time and resources and include a researcher in the data conversion process who is familiar with the objectives of the study. The mixed method approach also posed particular challenges in the data analysis stage as the researchers had to constantly alternate between two different methods of analysis and diverse data and conflicting accounts had to be brought together to produce a coherent report.

This article introduces a novel approach to data collection and conversion which can help to more fully utilise data from under-used, public archives for SNA. The successful application of plagiarism detection software to identify relationships and analyse a network of stakeholders involved in EU smoke-free policy suggests that the methodological approach might be applicable to future research projects. Interval-level data derived from plagiarism detection software could be used to distinguish different strengths of relationships between network actors and thus provide insight into the intensity of the relationships under investigation. Future studies could also employ plagiarism detection software to compare stakeholders' position papers with policy statements, policy proposals and final versions of legislative acts, compare policy documents in a specific policy area over a period of time or map similarities between policy documents adopted in different jurisdictions⁶. Assuming that the similarities between documents identified in this manner could be considered as proxies for political influence, studies which use plagiarism detection software may advance understanding of stakeholder influence, the evolution and dynamics of policy debates and policy transfer in multi-level governance.

Conclusion

Drawing on a network of actors involved in the development of EU smoke-free policy, this paper provides evidence that mixed methods can provide a comprehensive understanding of a complex policy network. The outlined approach provides opportunities to analyse and graphically depict the structural and contextual complexity and interconnectedness of a multifaceted policy network, something that is likely to be of considerable value for those with an interest in policymaking and policy processes. The systematic quantitative and qualitative analysis might help advocates and political decision makers to better understand the policy environment they operate in, enhance understanding of the development of policies and offer a potentially powerful tool for policy analysis.

By employing a mixed methods approach, the study addresses the dearth of studies which illustrate how mixed method approaches can be employed in social network research (Crossley, 2010) and makes an important contribution to debates about the antagonism which has developed between qualitative and quantitative approaches (Edwards & Crossley, 2009). The article shows that mixed methods are suited to developing insights into the structure and content of policy networks and gaining an in-depth understanding of political phenomena. Social network researchers have argued that by producing “messy results”, triangulation mirrors the “messiness” of social networks (Lievrouw et al., 1987, p. 245). Likewise, this article suggests that the complex and disorganised world of policy networks can only be satisfactorily understood when quantitative and qualitative methods are combined.

Biographies

Heide Weishaar is a postgraduate research fellow based at the School of Social and Political Science of the University of Edinburgh. She has researched stakeholder engagement in the development of public health policy and the influence of transnational tobacco corporations on international, European and national public health policy using qualitative and quantitative methodology. Her primary research interests are in public health policy, tobacco control and social network analysis.

Heide.Weishaar@ed.ac.uk

+44 (0)131 650 3043

Amanda Amos is Professor of Health Promotion at the Centre for Population Health Sciences of the University of Edinburgh. She is a social scientist specialising in qualitative research and also the evaluation of complex interventions. Her primary research interests are smoking and tobacco control.

Amanda.Amos@ed.ac.uk

Tel: +44 (0)131 650 3236

Fax: +44 (0)131 650 6909

Jeff Collin is the director of the Global Public Health Unit and head of Social Policy at the University of Edinburgh. His research focuses on globalisation and tobacco control, including research using corporate documents to analyse how transnational tobacco companies have sought to influence public policy.

Jeff.Collin@ed.ac.uk

+44 (0)131 651 3961