Abstract
Introduction:
Health in all Policies (HiAP) is a valuable method for effective Healthcare policy development. Big data analysis can be useful to both individuals and clinicians so that the full potential of big data is employed.
Aim:
The present paper deals with Health in All Policies, and how the use of Big Data can lead and support the development of new policies.
Methods:
To this end, in the context of the CrowdHEALTH project, data from heterogeneous sources will be exploited and the Policy Development Toolkit (PDT) model will be used. In order to facilitate new insights to healthcare by exploiting all available data sources.
Results:
In the case study that is being proposed, the NOHS Story Board (inpatient and outpatient health care) utilizing data from reimbursement of disease-related groups (DRGs), as well as medical costs for outpatient data, will be analyzed by the PDT.
Conclusion:
PDT seems promising as an efficient decision support system for policymakers to align with HiAP as it offers Causal Analysis by calculating the total cost (expenses) per ICD-10, Forecasting Information by measuring the clinical effectiveness of reimbursement cost per medical condition, per gender and per age for outpatient healthcare, and Risk Stratification by investigating Screening Parameters, Indexes (Indicators) and other factors related to healthcare management. Thus, PDT could also support HiAP by helping policymakers to tailor various policies according to their needs, such as reduction of healthcare cost, improvement of clinical effectiveness and restriction of fraud.
Keywords: Health Policy, Policy-Making, Big Data
1. INTRODUCTION
According to the World Health Organization, the Health in all Policies (HiAP) is a method by which public policies across sectors systematically considers the health implications of decisions, seek synergies, and avoids harmful health impacts to improve population health and health equity (1). Multiple sectors of the government, as well as non-governmental systems that participate in health equity, have to cooperate to put HiAP into practice (2), (Figure 1). HiAP aims to improve population health and promote health equity by highlighting the impact of public policies on health systems, determinants of health and overall wellbeing. Thus, public policy-making modeling must achieve the best possible outcomes. As a concept, it must ensure that national and international laws are followed, governments hold responsibilities towards their people, policymaking is transparent and information can be freely accessed, society actively participates in the development and implementation of policies, that are sustainable and can meet the needs of generations to come, and finally that all government sectors cooperate in the direction of making policies that promote health, equity and well-being (3). It becomes clear, therefore, that HiAP is a rather general concept that includes diverse initiatives. How sectors and approaches are coordinated and combined over time affects the implementation of HiAP (Figure 1). Based on the above, governments are ultimately responsible for the health of their citizens. However, health authorities at all levels and the health sector, in general, are key actors in promoting and applying HiAP. Their role is to present evidence on health and health equity and the impact of policies on health (4). Since assessing the implementation of HiAP depends on how sectors and approaches are coordinated and combined over time and the fact that the collaboration across sectors and levels of government is crucial in supporting policies that promote health, equity and well-being, it becomes evident that effective collaboration among sectors is necessary. The government must support the participation of sectors that have common interests and benefit from such cooperation. The participation of these sectors must be well designed, with clear objectives and responsibilities. Furthermore, the government must ensure that the proposed health policies have high political importance, with laws to support them, with sufficient funding and must be disclosed to the general public. Close monitoring and maintaining the impact of the policy is also important. HiAP requires a shared approach, but vague shared tasks and responsibilities across sectors might impede the implementation and monitoring of HiAP. Other problems, such as low capacity and limited resources are also common. A renowned American political scientist, Kingdon proposed the presence of three non-linear streams in policy-making, problems, policies, and politics, which interact to favor opportunities for policy decisions (Figure 2). Based on this interaction, an issue needs to be recognized first of all as a “problem” by politicians, policymakers and the overall community before it can be considered in the policy-making agenda. Secondly, suggestions for solutions to the problems are required, in other words, “policies”. These policies are frequently developed by public institutions, universities, think-tanks and/or private bodies and provide alternative solutions for the problems. Finally, a change in policy is only possible if the “politics” environment is appropriate. It becomes clear, therefore, that policy-makers must be able to recognize the right moment in politics when a policy change would be most likely to be adopted. In the EU, Health in All Policies (HiAP) was established in the 1992 Maastricht Treaty which stated that “health protection requirements should form a constituent part of the Community’s other policies (5). However, during the second Finnish EU Presidency in 2006, the context of HiAP was presented in more detail (6). In the 8th Global Conference on Health Promotion held in Helsinki, Finland, 10th to 14th June 2013, with the topic “Health in All Policies” (7), the participants decided to:
Prioritize health and equity in the government’s responsibilities towards its people.
Declare that effective health policymaking and well-being are of most importance.
Acknowledge that such actions involve political determination, courage, and strategic foresight.
Figure 1. Framework for HiAP implementation (Adapted from Shankardass et al., 2018).
Figure 2. Schematic representation of Kingdon’s non-linear framework for policy-making.
Therefore, for the above to occur, the governments must assure the following:
That they have health and equity as a priority. To this end, they must embrace the principles of Health in All Policies and act on non-health, i.e. social determinants of health.
Build and maintain structures and processes that can accommodate the implementation of the Health in All Policies across governments and between governments.
Reinforce the capacity of Ministries of Health to involve other sectors of government to achieve better health outcomes.
Develop skills that enable the implementation of Health in All Policies and reveal the determinants of health and inequity and react effectively.
Perform audits for health and equity.
Establish effective safety measures for policies against commercial, financial and personal interests. • Involve the community and social organizations in the development, implementation, and monitoring of Health in All Policies.
Meanwhile, the public and private health sectors are moving towards the use of Big Data and hence are investing in technologies and analytical tools to deal with such data. The public health sector can use the national health-care records/datasets to assess its performance and utilization of health care services (8). Big data are enormous and complex data sets, not easily processed using traditional methods. Big data in health refers to large routinely or automatically collected datasets, which are electronically captured and stored. Big Data is reusable in the sense of multipurpose data and comprises the fusion and connection of existing databases to improve health and health system performance (9). Big Data can be therefore collected from electronic health care records, test results, claims and cost data, biometric data, genomic and pharmaceutical data, clinical trials, even social media, and mobile applications. Governments and public institutions are directed towards Open data policies. Big data and Open data are similar but not identical. Open data is data that everyone can easily access and exploit. Open data must be publicly and freely available for anyone to use, and with a license allowing its re-use. The exploitation and analysis of big data and open data can help transform businesses, governments, and research. Combining the two might prove even more useful since big data provides tools to comprehend, study, and even change the world we live in, while open data guarantees that this power will be shared. With the right framework, the ultimate goal is the re-use of open and big data to make discoveries, drive innovation, growth, and transparency (10). Important developments utilizing big data include the use of genomic data in drug discovery, the disclosure of data from clinical trials, the use of electronic healthcare records, and the increased availability of data from mobile health applications, patient registries, and social media. According to the international literature, the results of big data analysis must be useful and valuable to both individuals and clinicians, so that the full potential of big data is employed. If handled correctly, big data has the power to transform health care, improve health outcomes and ultimately shape lives. It can do this by helping deliver higher-quality, more cost-effective health care. It can help disclose health determinants and propose appropriate care for individuals or populations. Analysis of big data might enable discoveries, affect outcomes, and reduce costs, in other words, better health care outcomes. Analysts can use statistical models and tools on big data to predict risk or trends and to help health care providers make policies that improve outcomes and are cost-efficient. Some of the problems faced when analyzing big data in health care are lack of state-of-the-art data analytics, ineffective infrastructure, insufficient funding, mistrust in databases and unwillingness to share information. Privacy and security concerns, lack of data experts and workforce, and a general lack of tools are also challenging in utilizing the potential of big data. Therefore, there is an urgent need to build a culture in medicine that integrates the use of data in a practical way, so it does not impede the development of innovative data management solutions (11-12).
2. AIM
In the context of the CrowdHEALTH project, we will measure the clinical effectiveness of both inpatient and outpatient health care, for specific ICD-10s referring to non-communicable diseases using Big Data technologies. To this end, specific health analytic tools will be employed, regarding specific storyboards to support Health in all Policies procedure.
3. CROWDHEALTH FUNCTIONALITIES IN THE CONTEXT OF HIAP: NOHS CASE STUDY
The Hellenic National Organization for the Provision of Health Services (NOHS), partner in the CrowdHEALTH project, is the largest public (national) health insurance organization in Greece which aims to cover the healthcare expenditures for Greek insured citizens. NOHS takes part in Healthcare Policy Making since it negotiates contracts and their cost, and remunerates health professionals prescribing the healthcare basket for the beneficiaries which include among others medical treatment, tests, therapies, drugs, consumables, supplements, medical devices, and other healthcare provisions. In order to test the capabilities of PDT, the NOHS story board (inpatient and outpatient health care) utilizing data from reimbursement of disease related groups (DRGs), as well as medical costs for outpatient data will be chosen. The data related to following ICD-10s were collected from 2014 to 2018:
E10-E14 Diabetes mellitus
E85 Amyloidosis
J44 Other chronic obstructive pulmonopathies
I10 Essential hypertension
J84 Other disorders of the interstitial pulmonary tissue
E66 Obesity
E78 Metabolic disorders of lipoproteins and other lipidemias
C43-C44 Melanoma and other malignant skin cancers
C15 Malignant neoplasms of the esophagus
C16 Malignant neoplasms of the stomach
C17 Malignant neoplasms of the small intestine
C18 Malignant neoplasms of the colon
C21 Malignant neoplasms of the anus and anal canal
C22 Malignant neoplasms of the liver and intrahepatic biliaries
C25 Malignant neoplasms of the pancreas
C26 Malignant neoplasms in other ill-defined organs of the digestive system
C50 Breast malignant neoplasms
C73-C75 Malignant neoplasms of the thyroid and other endocrine glands
3.1. Non-communicable diseases / Cancer
Diseases have a significant impact on the person, the health care system and the whole society. This burden is apparent on a clinical, financial and social level, as well as in terms of morbidity and mortality. Under this prism, the recording, understanding and assessing future encumbrance together with monitoring the needs and preferences of the population can contribute to modulating complete interventions and health policies. Most public health policies focus on non-communicable diseases (NCDs). NCDs are the leading cause of death, disease and disability in the WHO European Region. The four major NCDs together, cardiovascular disease, cancer, chronic obstructive pulmonary diseases and diabetes, account for the vast majority of the disease burden and of premature mortality in the Region. Policy development in European countries for facing non-communicable diseases must help improve health outcomes and healthcare systems in Europe. In this context, the need for recording and analyzing the cost resulting from non-communicable diseases are the most important. Cost is not only limited to medical costs, however, gathering data for other cost categories deriving from the level of society is rather difficult. Therefore, we only analyzed data based on medical costs, as recorded on the files of NOHS.
3.2. Disease-related groups (DRGs)
Diagnosis-related groups (DRGs) were developed in the early 80s in the U.S.A as a method to reimburse hospital provided health care (12). DRG is a system to classify hospital cases into one of originally 467 groups (13), with the last group (coded as 470 through 999) being “ungroupable“. The advantages of the DRG payment system are increased performance, efficiency and transparency and reduced average length of stay. The disadvantage, however, of DRG is creating economic motives towards earlier hospital discharges (14). In Greece, KEN-DRGs constitute a general list of hospital activities fees, which apart from defining the cost also define the mean length of stay (LoS) for every diagnosis-related group. These data correspond to the average hospitalizations and are used for the quick and easy pricing of each case, regardless of the exact amount of charges, type of services and the exact amount of hospitalization cost that occurred in reality. KENDRGs is based on the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) (15). Via the DRG system, the care structure of each hospital, the comparison of characteristics of hospitals and clinics over time and geographically, the development of health policies based on results of morbidity, mortality and hence overall the quality of services can be attained.
Health care performance is the maintenance of an efficient and fair health care system. Health care performance is evaluated by the quality and distribution of the benefits and costs of personal and public health care (16). The use of key performance indicators (KPIs) in health care provision systems have been used in public health and the health care sector in the last decades (17, 18). The benefits of using KPIs are (i) assessing the outcomes of health care policies, (ii) comparing equal indicators among different organizations, (iii) enhancing accountability of relevant actors, (iv) advancing transparency by publicly reporting the results, and (v) identifying areas for further investigation (17). DRG-based hospital payment systems have the potential to enhance the efficiency of hospital services. This is because there are motives for hospitals to work more intensely. After all, they are paid according to the number of patients they treat, and also to manage their expenses since they are reimbursed based on prices that are independent of their costs. Although DRG-based payment systems are now mainly used as a reimbursement mechanism, the original idea was to enable performance comparisons across hospitals (19-21). Hospitals are inclined to (i) reduce costs per treated patient by reducing length of stay or quality of services, (ii) increase profits by re-classifying patients, i.e. “up-coding” them to higher-priced DRGs, and (iii) increase the number of patients by changing the hospital’s admission rules. It is uniformly accepted that DRG-based hospital payment affects indicators of efficiency, such as activity and length of stay, although the same caveats apply. Accidental consequences may include saving (on quality mainly), cost-shifting, patient selection or up-coding to higher-priced DRGs (22). In Europe, therefore, there is an ongoing effort to assure adequate payment for outliers and high-cost services. Therefore, most of the countries have developed mechanisms to identify outlier cases and to pay hospitals separately for the extra costs of treating such patients. These include evaluating the coding and treatment of patients by auditing patients’ medical records and financially penalizing hospitals if patients are readmitted for the same problem within 30 days after initial discharge; for these patients, hospitals do not receive a second DRG-based payment (22). Based on the data available from NOHS (2014-2018), it has been observed that several private clinics, to maximize their reimbursement by NOHS, are trying to find ways of maximization. These ways include, among others, cases of extra charging, such as charging systematically DRGs with complications, which are more expensive than those without complications or charging systematically double DRGs.
3.3. Datasets used in CrowdHEALTH
The Greek NOHS maintains two main databases that communicate with each other; KMES for prescription of medicines (pharmacies) and EDAPY for all other healthcare services, such as diagnostic exams, visits to physicians, consumables and medical products, and inpatient hospital care. The information stored in the databases includes all data generated from processed submissions of providers that are contracted to NOHS and pharmacies. Monthly submissions to NOHS are about 27,000 entries that correspond to approximately 5 million medical acts entered into system. It is also worth noting that from the beginning of the organization till today, 900 million medical acts have been registered (more than 120 GB of data). The transition of most health-related institutions in Europe from the plain use of electronic health records to the efficient utilization of big data is a reality that stems from the demand for better patient outcomes, operational efficiency, and innovation in services provided. The insights gained from big data can allow institutions and professionals to solve problems that could not be tackled with traditional software or analytics. In healthcare, these new insights can help policymakers gain a deeper understanding of data to improve the results of policies applied, boost the productivity of healthcare professionals and improve the financial efficiency of the healthcare ecosystem. Big Data can be applied in prospective and retrospective research. From a payment perspective, big data can be used to ensure that providers have solid performance records and are reimbursed based on the quality of patient outcomes rather than the quantity of care delivered. The KPIs that will be used will be generated from data from the years 2014-2018 for non-communicable diseases/cancer and include the pseudonymized Social Security Number, gender, age, the year and month of expense. Cost also has to be taken into account. We will utilize the direct cost of treatment. Direct costs include cost of diagnosis, treatment, use of diagnostic and laboratory examinations. Most commonly, direct costs involve the following: outpatient and in-hospital care, emergencies, medicines and other medical devices (only for inpatient services).
Outpatient services for the ICD-10s to be studied will include the following diagnostic exams:
Blood exams
CT and X-ray Scans
Ultrasound (U/S)
Endoscopies
Cardiology exams (cardiogram)
Spirometry
For treatment, the following will be used:
Surgical, for complete or partial ablation of the tumor
Chemotherapy
Immunotherapy
Bone marrow transplantation
Targeted drugs
Based on ICD-10, neoplasms are classified and coded in categories C00-D48. Categories C00-C97 refer to malignant neoplasms, while category D00-D09 refers to intraepithelial neoplasms, category D10-D36 benign neoplasms and category D37-D48 neoplasms with uncertain or unknown behavior. Table 1 lists the NCDs analyzed along with their ICD10.
Table 1: Non-communicable diseases studied and their ICD-10
E10-E14 Diabetes mellitus
E85 Amyloidosis
J44 Other chronic obstructive pulmonopathies
I10-I15 Hypertensive diseases
J84 Other disorders of the interstitial pulmonary tissue
E66 Obesity
E78 Metabolic disorders of lipoproteins and other lipidemias
C43-C44 Melanoma and other malignant skin cancers
C15 Malignant neoplasms of the esophagus
C16 Malignant neoplasms of the stomach
C17 Malignant neoplasms of the small intestine
C18 Malignant neoplasms of the colon
C21 Malignant neoplasms of the anus and anal canal
C22 Malignant neoplasms of the liver and intrahepatic biliaries
C25 Malignant neoplasms of the pancreas
C26 Malignant neoplasms in other ill-defined organs of the digestive system
C50 Breast malignant neoplasms
C73-C75 Malignant neoplasms of the thyroid and other endocrine glands
The main actors for the health policies that will arise are NOHS, physicians, and hospitals, whereas the stakeholders are the patients and in general the whole population. To analyze the aforementioned data, the CrowdHEALTH’s project Policy Development Toolkit (PDT) will be used as a decision support system for policy makers to support the HiAP. The PDT offers Causal Analysis by calculating the total cost (expenses) per ICD-10, Forecasting Information by measuring the clinical effectiveness of reimbursement cost per medical condition, per gender and per age for outpatient healthcare, and Risk Stratification by investigating Screening Parameters, Indexes (Indicators) and other Factors related to healthcare management.
4. CONCLUSION
This paper presents how a proposed case study related to the health analytic tools usage could manage big data, in order to model and evaluate components related to the policies creation. The ultimate goal is to support HiAP by using the PDT to help policy makers to tailor various policies according to their needs, such as reduction of healthcare cost, improvement of clinical effectiveness and restriction of fraud. The PDT will help gain collective knowledge for the provision of efficient public health policies and services, provide added value health policies and increment data visualization techniques delivering data analytics outcome. With the analytical tools to exploit Big Data, we aim to extract dynamic knowledge from various data sources and medical records, the results of which might enable us to perform predictive risk/causal analysis, with respect to all health determinants. Therefore, using simulations and evidence-based approaches, we can support policy makers to define or tailor specific policies and contribute to HiAP (1,2,23).
Long term, the health policies that will be generated should be directed towards reducing the stage at which various non-communicable diseases are diagnosed and improving survival, as well as decreasing the associated costs. The fact that better performance of non-medical determinants of health is related to better health has led to new policies. Such policies include more effective healthcare for better health outcomes, patient-centered care, real-time learning and monitoring, re-use of existing data, and to provide source of real-world evidence on the effectiveness and cost effectiveness of treatments.
Acknowledgment:
CrowdHEALTH project is co-funded by the Horizon 2020 Programme of the European Commission Grant Agreement number: 727560 – Collective wisdom driving public health policies.
Author’s contribution:
Each author gave substantial contribution in acquisition, analysis and data interpretation. Each author had a part in preparing article for drafting and revising it critically for important intellectual content. Each author gave final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Conflicts of interest:
There are no conflicts of interest to declare.
Financial support and sponsorship:
Nil.
REFERENCES
- 1.WHO Working Definition prepared for the 8th Global Conference on Health Promotion; 10-14 June 2013; Helsinki. [Google Scholar]
- 2.Shankardass K, Muntaner C, Kokkinen L, Shahidi FV, Freiler A, Oneka G, Bayoumi AM, O’Campo P. The implementation of Health in All Policies initiatives: a systems framework for government action. Health Research Policy and Systems. 2018. pp. 16–26. [DOI] [PMC free article] [PubMed]
- 3.Leppo K, Tangcharoensathien V. Health in All Policies: Seizing Opportunities, Implementing Policies. Finland: Ministry of Social Affairs and Health; 2013-2013. The Health Sector’s Role in HiAP in Leppo K et al. [Google Scholar]
- 4.Ollila E, et al. Policy brief. Brussels: Directorate General for Health and Consumers; 2005. [27 December 2018]. Health in All Policies in the European Union and its member states. ( http://ec.europa.eu/health/ph_projects/2005/action1/docs/2005_1_18_frep_a4_en.pdf. [Google Scholar]
- 5.European Commission 1992. Treaty on European Union (Maastricht text). Article 129. Public Health. Official Journal of the European Union, C 191/1.
- 6.Stahl T, et al. Helsinki: Ministry of Social Affairs and Health; 2006. Health in All Policies: prospects and potentials. [Google Scholar]
- 7.WHO Helsinki Statement on Health in All Policies. WHO 8th Global Conference on Health Promotion. 2013.
- 8.Vayena E, Dzenowagis J, Brownstein JS, Sheikh A. Policy implications of big data in the health sector. Bull World Health Organ. 2017;96(1):66–68. doi: 10.2471/BLT.17.197426. 2018 Nov 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Study on Big Data in Public Health, Telemedine and Healthcare. European Commission, Directorate-General for Health and Food Safety Health Programme 2016.
- 10.Szlezak N, et al. The role of big data and advanced analytics in drug discovery, development, and commercialization. Nature. 2014;95(5):492–495. doi: 10.1038/clpt.2014.29. [DOI] [PubMed] [Google Scholar]
- 11.Bernstein SW. Q&A: The US big data report and fully utilising big data within healthcare. eHealth Law & Policy. 2014;1(4) [Google Scholar]
- 12.Fetter RB, et al. Case mix definition by diagnosis related groups. Med care. 1980;18(2):1–53. [PubMed] [Google Scholar]
- 13.Mistichelli J. Diagnosis Related Groups (DRGs) and the Prospective Payment System: Forecasting Social Implications. 1984.
- 14.Mihailovic N, Kocic S, Jakovljevic M. Review of Diagnosis-Related Group-Based Financing of Hospital Care. Health Serv Res Manag Epidemiol. 2016;3:1–8. doi: 10.1177/2333392816647892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.WHO . World Health Organization Geneva; 2004. International Statistical Classification of Diseases and Related Health Problems. 10th Revision; pp. 1–169. [Google Scholar]
- 16.Arah OA, et al. Conceptual framework for the OECD Health Care Quality Indicators Project. International Journal for Quality in Health Care. 2006. pp. 5–13. [DOI] [PubMed]
- 17.HIQA . Health Information and Quality Authority; 2013. Guidance on Developing Key Performance Indicators and Minimum Data Sets to Monitor Healthcare Quality. 2018 6 June. https://www.hiqa.ie/publications/guidance-developing-key-performance-indicators-kpis-and-minimum-data-sets-monitor-health] [Google Scholar]
- 18.Rozner S. A toolkit for Health Sector Management. Health Finance & Governance Project; 2013. Developing and using key performance indicators. [Google Scholar]
- 19.Langenbrunner JC, Cashin C, O’Dougherty S. Washington: The World Bank; 2009. Designing and implementing provider payment systems: how to manuals. [Google Scholar]
- 20.Kobel C, et al. DRG systems and similar patient classification systems in Europe, in Diagnosis-related groups in Europe: moving towards transparency, efficiency and quality in hospitals. In: R. Busse, et al., editors. Maidenhead: Open University Press; 2011. pp. 37–58. [Google Scholar]
- 21.Park M, et al. Geneva: World Health Organization; 2007. Provider payments and cost-containment lessons from OECD countries. [Google Scholar]
- 22.Diagnosis-related groups in Europe . Moving towards transparency, efficiency and quality in hospitals. In: R. Busse, et al., editors. European Observatory on Health Systems and Policies Series. Maidenhead, Berkshire, England: Open University Press; 2011. [Google Scholar]
- 23.Malliaros S, Xenakis C, Moldovan G, Mantas J, Magdalinou A, Montandon L. The Intergrated Holistic Security and Privacy Framework Deployed in CrowdHEALTH Project. Acta Inform Med. 2019 Dec;27(5):333–340. doi: 10.5455/aim.2019.27.333-340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Perakis K, Miltiadou D, De Nigro A, Torelli F, Montandon L, Mantas J., et al. Data Sources and Gateways: Design and Open Specification. Acta Inform Med. 2019 Dec;27(5):341–347. doi: 10.5455/aim.2019.27.341-347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kyriazis D, Autexier S, Boniface M, Engen V, Jimenez-Peris R, Jordan B., et al. The CrowHEALTH Project and the Hollistic Health Records: Collective Wisdom Driving Public Health Polices. Acta Inform Med. 2019 Dec;27(5):369–373. doi: 10.5455/aim.2019.27.369-373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Magdalinou A, Mantas J, Montandon L, Weber P, Gallos P. Disseminating research Outputs. The CrowdHEALTH Project. Acta Inform Med. 2019 Dec;27(5):348–355. doi: 10.5455/aim.2019.27.348-355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wajid U, Orton C, Mogdalinou A, Mantas J, Montandon L. Generating and Knowledge Framework: Design and Open Specification. Acta Inform Med. 2019 Dec;27(5):362–368. doi: 10.5455/aim.2019.27.362-368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.The Use of Big Data in Public Health Policy and Research European Commission, Directorate-General for Health and Consumers Unit D3 eHealth and Health Technology Assessment. 2014.