Abstract
BACKGROUND:
Considering the importance and necessity of establishing a nationwide information system for health grey literature in Iran, this study aimed to identify the main dimensions and components needed for developing a health grey literature information system in Iran and validate them according to experts’ opinions.
MATERIALS AND METHODS:
A mixed-method approach with an exploratory sequential design was used in this study. The research was done in following main steps: (1) conducting a systematic literature review to identify the potential components of the health grey literature information system suggested in the literature, (2) Interviewing 19 experts to explore further components required for designing the health grey literature system for Iran and doing a thematic analysis for analyzing the interviews, and (3) validating the identified components by a Delphi panel in two rounds for finalizing the initially-approved dimensions and components. Descriptive statistical analysis was also used for analyzing the Delphi panel’s data.
RESULTS:
Eight dimensions were identified as necessary for developing Iran’s health grey literature information system (including 31 components and 111 elements). The main dimensions included goals, data sources, minimum data set, data collection techniques, data content management procedures, quality control approaches, stakeholders, and management and policy-making.
CONCLUSION:
Using the identified and validated functional components in this study can be helpful In designing a health grey literature system that is of value for health policymakers and medical researchers as well as health information users.
Keywords: Grey literature, health information systems, health promotion, Iran
Introduction
Grey literature is of valuable and updated information resources that are daily produced by many organizations and individuals throughout the world, and despite their value in scientific communities and having copyright as well as worth being produced, these resources are not formally published in formal communication media, such as books and journals and/or accessible in the trade market throughout the publishing industry.[1,2] They rarely are cataloged and indexed in the main indexing/abstracting databases and libraries. These resources have many formats ranging from conference proceedings to manuscripts to daily notes. However, they are valuable information resources with some advantages. For example, grey literature helps in enriching systematic reviews, conducting accurate meta-analyses, and promoting scientific research.[3,4]
Grey literature is applicable for and beneficial to all scientific fields, including, among others, in health-related fields. Researchers in health-related fields have used these resources for conducting comprehensive studies. Therefore, they are important resources and include the main part of public health information, especially in developing countries.[5] Due to the lack of comprehensive information databases in these countries, researchers tend to use accessible grey literature for their research as possible. The private sector and non-governmental organizations (NGOs) have a main role in producing health grey literature, as they have no formal and integrated forum for publishing their information.
Dimensions and components needed for designing a health grey literature information system were rarely studied in the literature. Two studies focused on a nationwide grey literature information system[6] and the architecture of a grey literature information system in the public health sector.[7] The former proposed some software features (including digital content management, user interface, system and client management, etc.), and the latter focused on an initial public health knowledge management repository architecture (for content, data management, and the user interface) that includes grey literature to support decision making in the public health.
Grey literature is less considered in Iranian organizations, even in academic libraries.[8,9] This ignorance resulted in their dispersed management and organization and absence from bibliographies and indexes. Although few databases are developed and active in the storage and retrieval of some specific types of grey literature, mainly dissertations and conference proceedings, in some research centers such as IranDoc and the National Library of Iran, however, there is not a nationwide integrated information system for health grey literature in Iran. In addition, there are no proper guidance and operational instructions on the grey literature as well as a lack of formal supporting organizations for collecting and organizing these resources in the country.[10] Despite the existence of some limited systems for papers and dissertations, the country suffers from the lack of information systems for other kinds of grey literature, especially in the health field.
Considering the lack of a national information system for health grey literature in Iran, this study can help health policymakers in avoiding repetition and parallel works, as well as making added-values by saving time and effort in developing a centralized system from various decentralized data with its dedicated primary components.
Establishing such a system can be beneficial to access grey literature at the national level and encourage producers for more production. This information system would be the first nationwide one that is developed by experts and its reliable data could support decisions in the health field as well as being used for research and even satisfying the information needs of the public.
Materials and Methods
A mixed-method approach was used in this study. After approval of the study by the Research Ethics Committee in Iran University of Medical Sciences under the license No. IR.IUMS.REC1397.565, the research was done in three steps: (a) identifying the potential components of the health grey literature information system by conducting a literature review, (b) interviewing experts and content analysis of the interviews to explore further requirements for designing the system, and (c) validating the identified components by a Delphi panel in two rounds. It is notable that interviewees were informed of the importance and goals of the research, as well as their rights in protecting their private data and freedom in participating in the study. Figure 1 depicts the schematic steps of the study.
Figure 1.
Schematic steps of the study
Conducting a systematic literature review
For identifying the potential components, a systematic literature review was conducted for retrieving studies. Inclusion criteria were being in English, accessibility, and in line with study objectives. Letters to editors were excluded. The validity of the form was confirmed by some specialists in the related fields. Six main databases, including Scopus, Web of Science, Pubmed, Emerald, Sciencedirect, and Google Scholar, were searched via the keywords grey literature, informal publishing, unpublished, semi published, fugitive literature, no-written material, nonconventional literature, small-circulation literature, unconventional literature, and information systems. The applied search strategies were different in each search database [Appendix 1]. Finally, out of 523 retrieved documents, 17 documents were selected by using preferred reporting items for systematic reviews and meta-analyses (PRISMA) checklist [Appendix 2]. A researcher-made form was used for recording bibliographic information and extracting the main findings of each selected study. The results of this review were used for developing questions of the semi-structured interview. In addition, the results were used as an initial codebook for analyzing the interview data.
Identification of functional components
Semi-structured interviews were conducted to identify functional components of health grey literature.
An interview guide, including semi-structured questions, was used for data collection. It included 16 questions, mostly adopted from reviewing the literature in the previous phase, related to identifying components required for designing the health grey literature information system in Iran. The questions were reviewed and validated by five experts (three library and information specialists and two medical informatics specialists) before conducting the interviews.
The purposeful snowball sampling method was used for selecting interviewees. The sample consisted of 19 experts (eight in purposeful sampling and 11 in snowball sampling) having at least 5 years of working experience in the related fields, including experts, researchers, and specialists affiliated with the libraries and research organizations active in administrating health grey literature, managers from health private organizations active in the production of grey literature, and faculty members on library and information sciences. Interviews continued up to data saturation.
During interview sessions, conditions for brainstorming and finding hidden information by asking some probing questions were provided in the place. The interviews were noted and recorded. The duration of each interview was 56 minutes on average. The interviews were reheard twice to ensure that all main data were included and transcribed verbatim.
Codification and revision of components
In the qualitative phase, following the thematic analysis approach, coding, concept identification, and categorization of main themes were performed in MAXQDA 2018. All codes and themes were revised by an expert panel with expertise in qualitative research and content analysis.
Expert panel for designing a suggested information system
The research population included experts in medical library and information science, managers and expert staff of the Research Deputy of the Ministry of Health and Medical Sciences (MHME) with at least 5 years of working experience in administration or research policy-making in the production, storage, organization, and dissemination of grey literature. Of them, five experts were purposefully selected as research samples. The initial suggested model for Iran’s health grey literature information system achieved via data from the previous steps was discussed based on a viewpoint form. At first, the initial model was presented, and their views on the components were asked. The members’ agreement and disagreement rates were recorded, and their corrections were made to the final proposed model. The criterion of keeping or including a component in the model was the agreement of the majority of the members.
Validation of functional components
A two-rounded Delphi method was used in this step. The Delphi members were experts consulted for validating the initial components identified and approved in the previous steps. In the first and second rounds, the components were reviewed by 19 and 12 experts, respectively. They completed an electronic questionnaire that was sent via e-mail containing 5-point Likert-scale questions. The validity of this questionnaire was achieved using the content validity method. The total agreement rate of at least 75% for a component was accepted in the first round, and the total agreement rate of 50-75% for a component in the first round was reconsidered in the second round. SPSS 23 software package was used for data analysis in this step.
Results
Component identification and development
After transcribing the interviews, 731 initial codes were identified. Then, comparing the codes for merging similar ones and/or deleting repeated ones, 407 conceptual codes were selected. For conceptualizing and reporting meaningful connections at the higher levels, selective coding was done, and all codes were categorized into eight dimensions (34 components and 152 elements). The main dimensions included goals, data sources, minimum data set, data collection techniques, data content management procedures, quality control approaches, stakeholders, and management and policy-making. The Delphi experts’ panel added further, eight elements to the initial framework.
Result of round I
In round I, panelists made their opinions on the initial components that were achieved in the content analysis of the interviews. They added eight further elements for the framework. No components or elements were deleted from the initial framework. Eight dimensions, 31 main components, and 139 elements were accepted for the final framework.
In the dimension of goals, three components and 10 elements were accepted, three elements were deleted, and two remained for round II. In the dimension of data sources, 18 elements were accepted, and one remained for round II. In the dimension of the minimum data set, 51 elements were accepted, five were deleted, and one remained for round II. In the dimension of data collection techniques, all elements were accepted. In the dimension of content management procedures, two elements were accepted, and two elements were reminded for round II. In the dimension of stakeholders, 11 elements were accepted, and only one element was deleted. In the dimension of quality control approaches, all seven elements in the two components were accepted. In the dimension of management and decision-making, 10 elements were accepted, and three elements remained for round II. Only one element was deleted. Five elements that were proposed by the panelists were added for decision-making in round II.
Result of round II
During round II, eight dimensions with 31 components and 111 elements were accepted and finalized as a conceptual framework. Table 1 shows the finalized dimensions, components, and elements.
Table 1.
Dimensions, components, and elements needed for establishing a health grey literature information system for Iran
| Dimensions Components |
Elements |
|---|---|
| Goals | |
| Managerial | (1) Long-term maintenance and guaranteeing continuous access, (2) Centralized and integrated access and retrieval, |
| Policy-making | (3) Helping to make better decisions, (4) Affecting information behavior and fulfilling health information needs, (5) Identifying nationwide research trends in health, (6) Creating knowledge memory in national and organizational levels, (7) Avoiding research repetition and encouraging added-value, (8) Using big data collected by the system, (9) Identifying information producers, (10) Helping to evaluate health information producers |
| Attitudinal | (11) Encouraging policymakers to consider the necessity of establishing national regulations, (12) Considering the necessity of storing, organizing, and accessing health grey literature, (13) Helping in highlighting the activities of small organizations and NGOs |
| Data sources (producers) | |
| Ministry of Health and related organizations | (1) Ministry of Health and affiliated organizations, (2) Medical universities, (3) Medical libraries, (4) Hospitals and medical educational centers, (5) Research institutes, (6) Health scientific associations |
| Governmental section | (7) Health centers in the Social Welfare Organization, (8) National Standard Organization, (9) National Statistics Organization, (10) National Library, (11) National Academy of Medical Sciences, (12) Military medical and educational centers, (13) Health museums and archives |
| Non-governmental section | (14) Health information production and retrieval centers, (15) Producers of medical and health tools, (16) Private hospitals and clinics, (17) Health sector NGOs, (18) Organizations active in health grants |
| Independent researchers | (19) Organizational and non-organizational researchers in health |
| Data collection Techniques | |
| Active | (1) Sent by producers/authors, (2) Data input and output as XML, (3) Mediating software for converting data |
| Inactive | (4) Data mining from websites and repositories |
| Mixed | (5) Using both active and inactive techniques |
| Minimum data sets | |
| Identifiers | (1) Unique identifiers produced by the system, (2) Digital object identifier (DOI) |
| Document types | (3) Reports (annual, research, technical, statistical, grant, and administrative reports), (4) Author works (as monographs, pre-prints, unpublished documents, and papers), (5) Theses (BD, MD, PhD, and Post-doctoral dissertations), (6) Conference proceedings, papers, and posters, (7) Methodological and analytical items (analyses, studies, methods), (8) Promotional and educational items (policies, regulations, guides, instructions, data sets, pictures, films) |
| Resource formats | (9) Text, (10) Image, (11) Multimedia |
| Content types | (12) Scientific, (13) Educational, (14) Technical, (15) Administrational, (16) commercial |
| Core metadata | (17) Title, (18) Author(s), (19) Publication year, (20) Document type, (21) Translator(s), (19) Keywords, (20) Subject category, (21) Abstract, (22) Language, (22) Publisher, (23) Publication place, (24) Copyright, (25) Citation style, (26) URL |
| Content management procedures | |
| Storage | (1) Common standard information storage procedures |
| Organization | (2) Common standard information organization procedures, (3) Indexing with natural and controlled vocabularies, (4) Summarization and abstracting, (5) Subject directories |
| Representation and Retrieval | (6) Simple and advanced searches, (7) Refined searches, (8) Browsing by subjects/alphabetically, (9) Separating messages of the documents based on the kinds of audience, (10) Retrieval based on the types of data producer (11) Retrieval based on accuracy levels of data, (12) Selections based on reporting types, (13) Managerial dashboards |
| Self-achieving | (14) Self-achieving by non-organizational author, (15) Authorized and verified self-achieving by the system |
| Connection to other software packages | (16) Common standard information exchange and retrieval procedures, (17) Common standards for data input and output as XML, (18) A mediating software package for data conversion |
| Stakeholders | |
| Data resources | (1) Ministry of Health and related sections, (2) Governmental health organizations, (3) Non-governmental health organizations |
| The public | (4) Health researchers, (5) Health service users |
| Mass media | (6) Those who are active in media and communication industry |
| Education | (7) Basic education, (8) Higher education |
| Policy-makers | (9) Ministry of Health, (10) The Legislative power, (11) The Executive and Administrative power |
| Quality control approaches | |
| Content quality control | (1) Automated system based on minimum data entering, (2) Controlling by subject specialists, (3) Users’ reports of defects, (4) Quality controls of non-governmental organizations done by some selected governmental organizations, (5) Randomized quality controls |
| System quality control | (6) Periodical opinion surveys, (7) Periodical technical evaluations |
| Management and policy-making | |
| System administrator | (1) Ministry of Health/an approved medical university, (2) A consortium of producers |
| Producers’ cooperation models | (3) Requirement of the Ministry of Health and subordinate units, (4) Signing a memorandum of understanding between the system administrator and content producers to ensure long-term maintenance and access to resources, (5) Allocation of rewards and incentive schemes for the continuous and complete entry of information producers |
| Funding ways | (6) Funding by the Ministry of Health, (7) Funded by a consortium of information producers, (8) Financing through the creation of information value-added services, (9) Financing through the sale of big data of the system, (10) Funding through advertising on the system’s website |
| Access type | (11) Open access, under creative commons Licenses |
| Copyright | (12) Under supervision of the producers |
NGOs=Non-governmental organizations, URL=Uniform resource locator, XML=Extensible markup language, BD=Bachelor’s Degree, MD=Doctor of Medicine, PhD=Doctor of Philosophy
As can be seen in Table 1, the goals of the system (with its associated components such as policy-making, managerial and attitudinal goals) are one of the main dimensions for establishing a health grey literature system in Iran. It directs the activities of the system at hand. The second dimension required for designing a health grey literature information system in Iran is data sources and producers who create raw data as health grey literature. According to the experts who participated in this study, the main data sources and producers included the health ministry and its related deputies, medical colleges, medical university libraries, scientific research institutes, medical research centers, independent researchers, and NGOs.
The minimum data set as core data was identified as the third dimension in this study with its special components such as Identifiers, document types, resource formats, content types, and core metadata that are required for designing a health grey literature information system in Iran. In the fourth dimension, that is, the data collection techniques, three procedures for data collection were identified and validated: active, inactive, and mixed ones. The content management procedures are the fifth dimension identified in our study that is needed for better searching and using data at hand. These procedures include organizing, storing, retrieving, and self-archiving health information. In the data organization, using standardized techniques such as Dublin-core and MARC for cataloging was proposed. The sixth dimension identified for designing a health grey literature information system in Iran is stakeholders. Experts argued that producers are the main stakeholders justifying the existence of the system. The public and patients as well as system users and the educational sectors, are other stakeholders. The seventh component that was proposed in our study was the quality control approaches that guarantee the accuracy of data. The eighth component is management and policy-making, which are the potential objective of designing any information system. The Health Ministry or a medical university on behalf has appeared to be the main administrator for the system.
Discussion
Aiming at exploring the functional dimensions and components required for designing a health grey literature information system in Iran, the study identified goals, data sources, data collection techniques, minimum data sets, content management approaches, stakeholders, quality control procedures, and management and policy-making as the main components. System goals as the first component direct all procedures and processes and justify the cost and protect its continuation. In the policy-making component, user expectations and system perspectives can be depicted. It helps in immediate decision-making and responding to health information users in fulfilling their needs.[7] Research trends at a national level can be explored by policy-making, too. A so-called knowledge memory can be constructed for better decision-making, this knowledge memory was proposed in 2014.[11] Representing research background can be helpful in avoiding research repetition and making added-value decisions. Using metadata in crises[12] and decisions on health management[13] are the goals. Managerial goals are helpful in guaranteeing continuous access to resources as emphasized in some studies.[14,15,16,17,18] This necessitates integrated and concentrated access to the grey literature.[19,20,21] Attitudinal goals help policymakers for awareness of the importance of the system and facilitate their protection of the system and help small private institutes to manifest their activities, and encourage them to more information production.[22,23]
The second dimension identified in the study was data sources and producers. The role of these data producers in producing and using grey literature resources was argued in previous studies.[16,17,24] Other producers such as national archives, independent researchers, and NGOs are the other main producers. Data producers of various kinds are the motivators for further research by providing raw data in the health field.
The minimum data set (or core data elements) was identified as the third dimension. As most systems act based on their own local and national requirements,[17] this main point was considered in proposing the components for Iran’s health grey literature system too by Delphi members in our research. Designing a unique identifier for each grey resource was emphasized by experts as one of the core data elements. Such unique identifiers have been emphasized in some related systems.[19,20] Core metadata such as title, author(s), publication year, document type, translator(s), keywords, subject category, abstract, etc. were also obtained mainly from previous studies[25,26,27] and then validated by the expert panel.
In the fourth dimension, that is, the data collection techniques, three procedures were identified that were active, inactive, and mixed approaches for collecting data. Most worldwide grey literature systems use inactive procedures for data collection[8,18,19,28] by applying some protocols such as Z39.50 and Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). However, taking active approaches into account can enrich the data collection process.
The content management procedures were the fifth identified dimension including components such as organizing, storing, retrieving, and self-archiving health information. In the data organization, using standardized techniques such as Dublin Core and machine-readable cataloging (MARC) for cataloging was proposed. This dimension and its components are more technically related and need more attention and cooperation between librarians and computer specialists.
The sixth dimension was stakeholders. Stakeholders can be encouraged to participate more in system optimization by different approaches. For example, system users can be motivated to send their feedback on their information search experiences and information-seeking behavior.
The seventh component was the quality control approaches that have been heavily emphasized in some studies.[14,29,30] This type of quality control is done by controlling the quality of the content and the quality of the system at hand. An automatic system based on minimum data entering is one of the ways in this regard.[20,31] For system quality control, some ways can be proposed: resource interpretability and authenticity[32] and periodical surveys for measuring user satisfaction and controlling risk factors.[30]
The eighth component was management and policy-making. Information producers could cooperate in data entering by some mandating and/or encouraging approaches. For funding, the Health Ministry or a consortium of producers might be beneficial in this regard. As we found in the subcomponent of copyright, challenges related to copyright of procedures need to be more considered, as emphasized in other studies.[15,24,33]
Conclusion
This study is the first to comprehensively explore the validated dimensions and components for designing a health grey literature information system in Iran. These validated components can be helpful for designing an information system for grey literature in Iran as well as in other countries. Health grey literature policymakers, administrators, researchers, and system developers can use the finding for optimizing the current systems or for designing an appropriate system for managing health grey literature.
The study had some limitations, such as accessing several grey literature systems and databases and unwillingness of some participants to be interviewed. However, the study achieved its objectives as the first study in investigating the potential components of designing a health grey literature information system in Iran. Further work can be done for developing the logical and physical architecture required for designing an operational system for health grey literature. Further research could also be conducted on the feasibility of implementing a health grey literature information system in Iran, especially regarding its facilitating standards and obstacles to copyrights and legislations.
Financial support and sponsorship
This study was part of a Ph.D. dissertation supported by the Iran University of Medical Sciences (Grant No: IUMS/SHMIS_97-3-32-12820).
Conflicts of interest
There are no conflicts of interest.
Acknowledgement
The authors would like to thank all the contributors to this study.
Appendix 1.
Search strategy in different databases and search results
| Database | Search Query | Result |
|---|---|---|
| Pubmed | #1: “grey literature”[Title/Abstract] OR “gray literature”[Title/Abstract] OR “Gray Literature”[Mesh] OR “informal publishing”[Title/Abstract] OR “semi published”[Title/Abstract] OR “fugitive literature”[Title/Abstract] OR “no written material”[Title/Abstract] OR “nonconventional literature”[Title/Abstract] OR “small-circulation literature”[Title/Abstract] OR “unconventional literature”[Title/Abstract] | 5024 |
| #2: “information system*”[Title/Abstract] | 28569 | |
| #3: #1 AND #2 | 50 | |
| Scopus | #1: TITLE-ABS-KEY(“grey literature” OR “gray literature” OR “informal publishing” OR “semi published” OR “fugitive literature” OR “no written material” OR “nonconventional literature” OR “small-circulation literature” OR “unconventional literature”) | 5844 |
| # 2: TITLE-ABS-KEY (“information System*”) | 352189 | |
| #3: #1 AND #2 | 123 | |
| Web of Science | #1: TS = (“grey literature” OR “gray literature” OR “informal publishing” OR “semi published” OR “fugitive literature” OR “no written material” OR “nonconventional literature” OR “small-circulation literature” OR “unconventional literature”) | 4825 |
| #2: TS = (“information system*”) | 118761 | |
| #3: #1 AND #2 | 71 | |
| Emerald | abstract:”grey literature” OR (abstract:”gray literature”) OR (abstract:”informal publishing”) OR (abstract:”semi published”) OR (abstract:”fugitive literature”) OR (abstract:”no written material”) OR (abstract:”nonconventional literature”) OR (abstract:”small-circulation literature”) OR (abstract:”unconventional literature”) AND (abstract:”Information System*”) | 148 |
| Science Direct | Title, abstract, keywords: (“grey literature” OR “gray literature” OR “informal publishing” OR “semi published” OR “fugitive literature” OR “no written material” OR “nonconventional literature”) AND (“Information System”) | 12 |
| Google Scholar | ((intitle:”grey literature” OR intitle:”gray literature” OR intitle:”informal publishing” OR intitle:”semi published” OR intitle:”fugitive literature” OR intitle:”no written material” OR intitle:”nonconventional literature” OR intitle: “small-circulation literature” OR intitle:”unconventional literature”) AND (intitle:”information system” OR intitle:”information systems”)) | 119 |
Appendix 2.
PRISMA (preferred reporting items for systematic reviews and meta-analyses ) flowchart diagram of study selection process in the review
References
- 1.Schöpfel J. Research Outside The Academy. Springer; 2019. Grey Literature and Professional Knowledge Making; pp. 137–53. [Google Scholar]
- 2.Farace DJ, Schöpfel J, editors, editors. Berlin/New York: Walter de Gruyter GmbH; 2010. Grey literature in library and information studies. [Google Scholar]
- 3.Pappas C, Williams I. Grey literature: Its emerging importance. J Hosp Librariansh. 2011;11:228–34. [Google Scholar]
- 4.Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007;18:MR000010. doi: 10.1002/14651858.MR000010.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Crowe J. Grey literature repositories: Tools for NGOs involved in public health activities in developing countries. In: Farace DJ, Schöpfel J, editors. Grey literature in library and information studies. Berlin/New York: Walter de Gruyter GmbH; 2010. pp. 181–99. [Google Scholar]
- 6.technická knihovna S. Software Specification for the NRGL 2012. [Available from: http://repozitar.techlib.cz/record/273?ln=en] [Google Scholar]
- 7.Revere D, Bugni PF, Fuller S. A public health knowledge management repository that includes grey literature. Pub Res Q. 2007;23:65–70. [Google Scholar]
- 8.Safazadeh S, Asnafi AR, Salemi N. Gray literature management in libraries of agricultural research, education and extension organization. J Natr Stud Librariansh Inf Org. 2016;26:55–74. [In Persian] [Google Scholar]
- 9.Kamkar H. Msc [Thesis] Tehran, Iran: Shahid Beheshti university of medical sciences; 2014. Study of the status of gray documents management in shahid beheshti university of medical sciences libraries. [In Persian] [Google Scholar]
- 10.Safazade S. Msc [Thesis] Tehran, Iran: Shahid Beheshti university of medical sciences; 2013. A Survey on collection, organization and dissemination of gray literature in Specialized libraries in Tehran. [In Persian] [Google Scholar]
- 11.Pisa declaration on policy development for grey literature resources. Available from: http://www.greynet.org/images/Pisa_Declaration,_May_2014.pdf. [Last accessed on 2018 April 2] [Google Scholar]
- 12.Freeman JD, Blacker B, Hatt G, Tan S, Ratcliff J, Woolf TB, et al. Use of Big Data and Information and Communications Technology in Disasters: An Integrative Review. Disaster Med Public Health Prep. 2019;13(2):353–67. doi: 10.1017/dmp.2018.73. [DOI] [PubMed] [Google Scholar]
- 13.Chinnaswamy A, Papa A, Dezi L, Mattiacci A. Big data visualisation, geographic information systems and decision making in healthcare management. Manage Decis. 2019;57:1937–59. [Google Scholar]
- 14.Schöpfel J, Le Bescond I, Prost H, editors, editors. Open Is Not Enough: Grey Literature in Institutional Repositories. GL 13: Thirteenth International Conference on Grey Literature: The Grey Circuit from Social Networking to Wealth Creation Washington. 2011 2011 December 5–6; [Google Scholar]
- 15.Myska M, Savelka J. A model framework for publishing grey literature in open access. J Intell Prop Info Tech & Elec Com L. 2013;4:104–15. [Google Scholar]
- 16.Schöpfel J. Access to European Grey Literature. In: Bacuvcik R, editor. Grey Literature Repositories: VeRBuM; 2014. pp. 20–33. [Google Scholar]
- 17.Siegel GE. Institutional Grey Literature in the University Environment. In: Farace DJ, editor. Grey Literature in Library and Information Studies. Munich: De Gruyter Saur publisher; 2010. pp. 69–84. [Google Scholar]
- 18.National Repository of Grey Literature. Available from: https://nrgl.techlib.cz/en. [Last accessed on 2021 Feb 21] [Google Scholar]
- 19.Opengrey. Available from: http://www.opengrey.eu/ [Last accessed on 2021 Mar 21] [Google Scholar]
- 20.CDS Invenio for National Repository of Grey Literature. Available from: http://www.nusl.cz/ntk/nusl-81439. [Last accessed on 2020 Dec 18] [Google Scholar]
- 21.Aloia D, Naughton R, editors, editors. The GreyLit Report: Understanding the Challenges of Finding Grey Literature. Eighteenth International Conference on Grey Literature; 2017 [Google Scholar]
- 22.Záviška M, Maixnerová L. The value of grey literature in repositories: Proceedings. Prague: National Library of Technology; Grey literature in the national medical library; p. 20. Available from: http://repozitar.techlib.cz/record/802/files/idr-802_4.pdf2014 . [Last accessed on 2019 Dec 10] [Google Scholar]
- 23.Shah SA, Ilhaq H. Information system for NGO libraries in Pakistan: A proposed model for organizing the Grey Literature. IFLA Publictions. 2006;123:127. [Google Scholar]
- 24.Murugathas K, Balasooriya H. Developing an institutional repository: Experiences at the library, Faculty of Medicine, University of Jaffna. J Univ Libr Assoc Sri Lanka. 2014;18:39–50. [Google Scholar]
- 25.Turner AM, Liddy ED, Bradley J, Wheatley JA. Modeling public health interventions for improved access to the gray literature. J Med Libr Assoc. 2005;93:487–94. [PMC free article] [PubMed] [Google Scholar]
- 26.Pejšová P, Vaska M. An analysis of current grey literature document typology. Grey J. 2011;7:72–80. [Google Scholar]
- 27.Banieghbal N, Khosravi F, Faal S. A Comparative Survey of the Standards for Describing Non-Book Resources (Print and Non-Print) and presenting a Suggested Plan for a Manual for the Libraries and Archives of Iran. Ganjine-ye Asnad. 2011;21:106–19. [In Persian]. Available from: https://ganjineh.nlai.ir/article_496.html?lang=en . [Google Scholar]
- 28.Chantavaridou E. Open access and institutional repositories in Greece: progress so far. OCLC Syst Serv. 2009;25:47–59. [Google Scholar]
- 29.Tyndall J. How low can you go? Toward a hierarchy of grey literature 2008. Available from: https://dspace.flinders.edu.au/jspui/bitstream/2328 / 3326/1/Tyndall.pdf. [Last accessed on 2018 Feb 18] [Google Scholar]
- 30.Pejšová P, Vaska M. Audit DRAMBORA for trustworthy repositories: A study dealing with the digital repository of grey literature. Grey J. 2012;8:96–105. [Google Scholar]
- 31.Ikeda K, Ohshima T, Gonda M, Nagaya S, Hayakawa M, Mineo Y, et al. Contribution to the improvement of dissemination of grey literature-JAEA Library’s efforts for collecting, organizing and disseminating information on nuclear accidents. Grey J. 2014;10:7–13. [Google Scholar]
- 32.Dobratz S, Schoger A, editors, editors. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Springer; Verlag: 2007. Trustworthy digital long-term repositories: The nestor approach in the context of international developments. [Google Scholar]
- 33.Saleh AA, Ratajeski MA, Bertolet M. Grey Literature Searching for Health Sciences Systematic Reviews: A Prospective Study of Time Spent and Resources Utilized. Evid Based Libr Inf Pract. 2014;9:28–50. doi: 10.18438/b8dw3k. [DOI] [PMC free article] [PubMed] [Google Scholar]