Abstract
After the regulatory approval has been obtained, epidemiological studies are acknowledged scientific medical research methods for a new drug which provide additional knowledge about routine application of the drug in clinical daily routine. These studies are performed according to the recommendations of both international and national expert associations, the recommendations of the higher federal authorities in Germany and according to the recommendations of the associations of the pharmaceutical industry.
Two surveys among the member companies of the Association of Research-based Pharmaceutical Companies investigated the status of the implementation of the recommendations in the years 2008 and 2010 and compared the results with each other. It could be shown that these recommendations were implemented successfully and were fully adhered to during the conduct of non-interventional studies in Germany. The recommendations define a quality standard which justifies a high level of confidence in the validity of the data collected and the results from these investigations.
Keywords: non-interventional studies, observational studies, quality assurance, quality-assurance measures
Abstract
Epidemiologische Studien sind nach der Erlangung der behördlichen Zulassung für ein neues Arzneimittel anerkannte, medizinisch wissenschaftliche Untersuchungsmethoden, die dem zusätzlichen Erkenntnisgewinn bei der routinemäßigen Anwendung des Medikamentes im klinischen Alltag dienen. Die Durchführung solcher Untersuchungen erfolgt gemäß den Empfehlungen der internationalen wie nationalen Fachgesellschaften, den Empfehlungen der Bundesoberbehörden in Deutschland sowie den Empfehlungen der Verbände der pharmazeutischen Industrie.
In zwei Umfragen unter den Mitgliedsunternehmen des Verbandes der forschenden Pharma-Unternehmen wurde der Stand der Umsetzung der Inhalte dieser Empfehlungen in den Jahren 2008 und 2010 untersucht und die Ergebnisse miteinander verglichen.
Es konnte gezeigt werden, dass diese Vorgaben erfolgreich umgesetzt und bei der Durchführung nicht-interventioneller Studien in Deutschland umfänglich berücksichtigt werden.
Diese Empfehlungen definieren einen Qualitätsstandard, der ein hohes Maß an Vertrauen in die Validität der erhobenen Daten und die Ergebnisse aus diesen Untersuchungen rechtfertigt.
Introduction
Within the context of the discussion about the efficacy and safety of innovative drugs it becomes more and more essential to obtain additional scientific data and evidence after the regulatory approval of a drug has been obtained in order to facilitate a comprehensive assessment of a new therapeutic option.
The evidence for the efficacy, besides tolerance and safety of a drug, is delivered by the results from controlled clinical trials of the phases II and III in a limited number of patients under defined treatment conditions. These trials are conducted in line with strict legal requirements as well as by the principles of the Guideline for Good Clinical Practice (GCP) of the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) [1].
The marketing authorisation for a drug is based on a positive assessment of the risk-benefit-ratio of a drug for a defined target population in a specific indication.
In contrast to a clinical trial non-interventional studies are defined by the guideline 2001/20/EG in article 2 (c) [2] and the German Drug Law (AMG) in section 4 subsection 23 [3]. Section 67 subsection 6 of the German Drug Law introduces the term “Anwendungsbeobachtung” (AWB, post marketing surveillance study) synonymously.
After achievement of the approval non-interventional studies (NIS)/AWB may provide additional knowledge of routine administration in daily clinical routine.
Already during the clinical study program a central task of the drug safety unit of a pharmaceutical company is the detection of possible signals of rare adverse events in the target population itself or the identification of possible risks in sub-groups of the target population. If there are sufficient hints for such signals or risks the European legislative requires further diligent follow-up of the signals addressed by a Risk Management Plan (RMP) [4]. An essential part of the RMP after the approval is the observation and assessment of the detected signals and risks in daily routine in a significantly larger patient population than in the previous clinical study program. In this context Post Authorisation Safety Studies (PASS) are especially important. PASS were first defined by Directive 2001/83/EC [5] and have been stipulated as a part of a Risk Management Plan by Volume 9A [4] since 2008 in European law. Depending on the type of the study, the medical objective and the size of the patient population to be observed, PASS can be conducted either as clinical trials of phase IV or as NIS/AWB.
The precondition for the performance as NIS/AWB is that the general quality requirements for epidemiological studies by the expert associations are observed both on the international [6], [7] and national level [8]. However, the principles of the ICH GCP Guideline are not applied for NIS/AWB in accordance with part I chapter 7.1 of Volume 9A of the Guidelines on Pharmacovigilance of the European Commission [4].
Furthermore, in Germany NIS/AWB are based on the regulations and provisions of sections 67 (6) of the Drug Law [3] as well as on the recommendations of the higher federal authorities [9].
The recommendations of the Association of Research-based Pharmaceutical Companies (vfa) [10] together with the German Pharmaceutical Industry Association (BPI) [11] as well as other recommendations [12] define a quality standard and/or a State of the Art in NIS/AWB, which puts the validity of the data collected and the results from these investigations close to those obtained from clinical trials. The implementation of the contents of the recommendations defined and the legal provisions in the member companies of the vfa was investigated in two surveys in 2008 and 2010. In this publication the results of the 2010 survey are presented, discussed and compared with the results from 2008.
Methods
In order to investigate which quality standards and/or quality assurance measures are applied in the research-based pharmaceutical companies when planning, conducting and evaluating NIS/AWB, a working group of the vfa-subcommittee Clinical Research/Quality Assurance and the Clinical Quality Assurance Germany (CQAG) performed a detailed survey among the member companies of the vfa in February and March 2008 [13]. Basically, the questions referred to the status of the implementation of and the adherence to the vfa-recommendations from 2007. In order to investigate the sustainability of the vfa-recommendations and the quality assurance measures which were introduced in 2007 and to check the current standard of the performance of NIS/AWB among the member companies of the vfa, the working group used a comparable, supplemented questionnaire for a new survey in May and June 2010. The questions refer to general issues as well as to
the assessment of the importance of NIS/AWB by the company,
the nature and number of current projects,
the existence of corresponding procedural instructions or Standard Operating Procedures (SOP) and
the training performed or planned for internal and external staff involved concerning these regulations and provisions.
Another question concerned specific measures
during the planning phase of a NIS/AWB (e.g. considerations about the representativeness of the study sites) and
during the evaluation phase (e.g. to data entry, data management, and statistical evaluation).
At the same time, the companies were interviewed for their experience with the Ethics Committee's consultation procedure for NIS, especially concerning AWB, for the average duration of the procedure and its results.
Complex questions were dedicated to the planned and the measures actually applied during the phase of conduct, with special attention to
the selection of study sites,
the analysis of the patient inclusion rate and
the measures planned in case the inclusion rate did not meet the expectations.
Moreover, the participating companies were interviewed in detail about the topics
Informed Consent of the patients in NIS/AWB,
verification of the data collected,
planned quality control and quality assurance methods as well as
audit-related activities.
Measures to be taken to assure transparency, in particular the publication of information about planned and current NIS/AWB projects and the way of publishing the results from these investigations were another part of questions.
The possibility of multiple entries explains the sum of responses exceeding the number of n=28 and/or 100%, respectively, in the Tables 2–6.
Survey
In April 2010 the vfa sent out questionnaires to all 46 member companies asking for their participation in the survey. 36 responses were received early in June 2010. This corresponds to 78% of the companies contacted. From this share 8 companies, mainly biotechnical enterprises, indicated that they did not conduct any NIS/AWB since they had not yet obtained marketing authorisation for any of their products.
The extensive evaluation of the results of this survey refers to the detailed responses of 28 companies which actually performed NIS/AWB or which in principle would have had the possibility to do so. Due to the size of these companies and the therapeutic scope of their research activities described these results can be regarded as representative of all vfa member companies and also as representative of these types of epidemiological research in Germany.
Results
Type and number of non-interventional studies
All responses (n=28) of the companies interviewed defined NIS as “important and essential types of studies after the approval of a drug”. However, in 4 companies this did not apply to the most frequent kind of NIS, namely AWB, but only to the category NIS itself. One company explained that the reason for this was the “poor (public and regulatory) acceptance” of AWB.
At the same time, in the majority of companies (n=21), AWB are part of the Risk Management Plan of a drug.
The number of studies carried out by the member companies interviewed and the type of studies are summarized in Table 1 (Tab. 1). These are compared with the results of the survey from 2008.
Table 1. NIS/AWBs performed.
The absolute numbers in Table 1 (Tab. 1) are to be taken as a snapshot at the time each survey was carried out. Considering the total average number of NIS conducted per company a tendency to increase from 
=8.3 in the year 2008 to =10.1 in 2010 can be stated whereas the average number of AWB carried out per company on average decreases from =7.1 to =5.8 during the same period.
Standard Operating Procedures, procedural instructions
An important part of quality management is the existence of comprehensive, written procedural instructions and/or Standard Operating Procedures (SOP). Ideally, these refer to international as well as to national regulations, guidelines and recommendations on NIS/AWB. 22 companies (79%) stated that they had their own NIS SOP on an international level, in 10 companies (36%) NIS SOP were available on both the international and national level. Specific German AWB SOP were compiled in 21 companies (75%); 6 companies (21%) indicated that the topic and the procedures were adequately covered by their international SOP. 26 companies (93%) took the reference of the SOP systems to the relevant recommendations and regulations in Germany for granted.
One company exclusively commissioned external service providers, Contract Research Organisations (CRO) with the conduct of NIS/AWB that had compiled their own SOP which were applied in this context.
Training measures
In order to provide an understanding of the SOP contents, the companies interviewed used a variety of different methods with actual and/or planned repetition or refresher intervals. In all companies (n=28) the staff of the medical department received training about the SOP contents and the regulations they are based on. Two companies stated that they repeated the training every 6 months, 10 companies stated that they repeated it annually. All other companies planned a training prior to every new project.
The most frequent training method was the presence training (n=22) of the staff involved. In 9 cases training in form of e-learning tools alone or in connection with presence training was chosen. According to the statements of the majority of the companies interviewed, new, amended or updated regulations prompted an appropriate update of their own SOP or procedural instructions, and, as a consequence, allow for new training measures.
In 23 companies no specific training for NIS/AWB for the marketing departments nor for the group of sales representatives was conducted. The following reasons were given: on the one hand the role of the sales representative would not exist in its former form, i.e. the tasks would be assumed by specialised, scientific sales representatives. On the other hand the conduct of NIS/AWB was more and more assigned to external service providers, an intensive project-related training for marketing and sales representatives did no longer seem to be appropriate. Of course, in these cases a previous training of the staff of the service provider dealing with the project would have to be performed.
Responsibility within the company
In all companies interviewed (n=28) the organizational overall responsibility for the preparation of the study plan, the implementation of the project, the timely analysis and the publication of the summary of the study results as well as the budget responsibility for NIS/AWB was with the head of the medical department.
Selection and recruitment of the study centres and distribution of the study documents
Subject to the planned number of study sites and the patients to be observed and depending on the indication investigated different considerations arose for the companies with regard to the selection of appropriate study sites. In studies with a large number of participating sites in a widely spread indication the objective was to obtain a geographic distribution as even as possible. As a rule this could be achieved by a non-restrictive solution of sites without selection criteria applied.
In studies with a rare indication or a low number of specialised sites the basis for the selection of qualified sites was defined as follows: on the one hand company-owned knowledge about experiences of the participating physicians with the indication, on the other hand source of information based on the internet or commercially available data bases for physicians with indication of the medical and therapeutic specialisation of the respective institution or practice.
In 20 companies the documents were distributed via CRO, in 15 companies they were distributed by the staff of the medical department. Nine companies used both options. Seven companies exclusively commissioned CRO with the distribution of the documents via their medical department Table 2 (Tab. 2).
Table 2. Distribution of study documents.
As an example of further efficient selection procedures for qualified sites, questionnaires were quoted which inquired about the general willingness for participation, the treatment frequency of the patient population to be observed as well as the special features of equipment or practice in advance. Thus, it became possible to specifically integrate sites which, with high probability, were able to conduct the study planned in a qualitatively appropriate way.
All companies indicated the end of the study in the observational plan of a NIS/AWB. Likewise, in all companies measures were provided for the cases in which the patient inclusion rate would be below the rate expected.
Transparency in research
In order to ensure transparency in medical drug research, information on controlled clinical trials prior to approval and statements about NIS/AWB performed after granting the approval are published in registries which are publicly accessible. The companies interviewed used different panels as shown in Table 3 (Tab. 3). The summary of the results from NIS/AWB was published in the internet registries listed in Table 4 (Tab. 4).
Table 3. Publication of information about NIS/AWB.
Table 4. Publication of the summary of the results from NIS/AWB.
Procedures of the Ethics Committee
The recommendations of vfa and BfArM (Federal Institute for Drugs and Medical Devices – Bundesinstitut für Arzneimittel und Medizinprodukte)/PEI (Paul-Ehrlich-Institute) require consultation with an independent Ethics Committee prior to the conduct of prospective NIS/AWB. This corresponds to the requirement for consultation prior to the conduct of epidemiological research projects laid down in the professional code of physicians by some State Chambers of Physicians in Germany, e.g. in Bavaria [14].
The Ethics Committee responsible for a research project is determined by the appointment of the scientific lead of the study who assumes the responsibility of the study: 21 companies (75%) appointed external scientific leads. Three companies (11%) appointed an internal medically qualified employee only. In 4 cases (14%) both internal and external scientific leads were appointed. Nineteen companies (68%) answered the question of whether they had experience with negative results of consultation with “no”.
In 9 cases this question was answered with “yes”. The reasons for a negative opinion were mainly conflicting views between the scientific lead of the study submitting the project and the Ethics Committee regarding the non-interventional nature of the NIS/AWB.
Referring to the question with other experiences with the Ethics Committee's procedure and, if applicable, the dialogue with the Ethics Committee the following additional aspects were cited: In 5 cases, the documents submitted were considered not to be subject to the obligation for consultation by the Ethics Committee responsible for the scientific lead of the study. As a consequence, the scientific lead was replaced in order to meet the recommendations of the higher federal authorities and the vfa.
The question of endorsing the opinion already given for the scientific lead by other Ethics Committees could not be clearly resolved to date. In this context, cases were mentioned where the primary opinion was endorsed but it was also decided that a new assessment by the additionally involved Ethics Committees was necessary.
Furthermore, 3 companies stated that the required application documents for an assessment by the Ethics Committee must also include the curriculum vitae (CV) of participating physicians from the purview of the pertinent State Chamber of Physicians. From the companies' point of view no conclusive explanation was given for this requirement.
Patient information and verification of the data
Both vfa-recommendations and the recommendations of the higher federal authorities include a written consent of the patient to participate in a prospective NIS/AWB. The patient's informed consent is mandatory for granting direct access to source documents for verification of the data collected by company or commissioned staff. However, a verification of the data would also be possible by means of a telephone interview or the interview technique between company staff and the participating physician without immediate access to the patient documentation. The survey included detailed questions about this topic, the results are described in Table 5 (Tab. 5) and Table 6 (Tab. 6). The results exceeding n=28 respectively 100% are due to the possibility to give multiple answers.
Table 5. Types of consent.
Table 6. Types of data verification.
Independent measures of quality assurance
Also with NIS which as per definitions [4], [12] do not have to be performed according to GCP criteria, random systematic audit measures by company-internal, independent quality assurance units may serve as a basis for an evaluation of the performance of these study types in conformity with due form and rules. When asked for audit measures in connection with NIS/AWB, 20 companies (71%) answered that they conducted both company-internal system audits and audits of the contracted CRO. Six companies did not plan for such audits, 2 companies provided no information in this respect.
In the survey 8 companies (29%) stated to perform on-site audit activities in the participating study sites. In 2008 only 2 companies (11%) provided these audit measures, therefore an upward trend could be seen. However, it was pointed out that audit measures at study site were not generally and similarly planned in all NIS/AWB but were basically limited to such AWB/NIS which were conducted as PASS due to regulatory requirements.
Compensation
All recommendations and German Drug Law include to “...calculate compensations which are paid to physicians for their participation in studies in that way that no stimulation is given for a preferred prescription and recommendation of specific drugs” [3].
In general, the calculation of the compensation was based on the medical fee schedule. The amount was calculated on the base of the time actually spent with documentation together with the time needed in order to explain the data protection rules with respect to the transmission and processing of the data collected by the sponsor of the NIS/AWB.
None of the companies interviewed provided a compensation in form of a lump sum without considering the real time spent for documentation and information. In 21 cases a compensation was either indicated after fully documented visits or by completed Case Record Forms. In 14 cases both options were used, each depending on the duration of NIS/AWB and the complexity of the documentation.
Furthermore, it was recommended to evaluate the time needed in tests at potential study sites with both the physicians involved and the non-medical staff. The additional time needed at the study site also has to be taken into account for the cases in which the pharmaceutical company plans to perform a quality control of the data collected.
Data management and evaluation of the results
The measures implemented at recording, entry and evaluation of the data collected from NIS/AWB are summarized in Table 7 (Tab. 7).
Table 7. Tools in data management.
In all companies quality assurance tools and methods as well as an analysis plan developed in advance, routine plausibility checks and the use of queries and/or follow-ups in case of ambiguous or missing data were applied in a regular manner.
Seven companies stated that data was recorded by double entry only. Nine companies planned both double and single data entry.
Discussion
The research-based pharmaceutical companies acknowledged NIS/AWB as efficient, epidemiological tools in order to achieve additional knowledge about the application of a drug after its regulatory approval. These study types are accepted by the higher federal authorities in Germany and publicly discussed by experts [15], [16], [17] in order to obtain additional knowledge about the safety of a drug after its marketing approval. Besides, in agreement with national and international surveillance authorities NIS/AWB may be conducted in order to investigate findings about potential safety risks from the clinical development program of a substance in additional post-authorisation safety studies [4].
In view of the scientific character of these studies, the overall responsibility of NIS/AWB in the vfa-member companies lies with the head of the medical department. This central, organisational aspect which had already been documented in the 2008 survey could be confirmed by the results of the current survey 2010.
Furthermore, the measures for quality control and quality assurance, for instance at the time of verification of the data collected, data entry and recording as well as at the time of biometric evaluation justify a high level of confidence in the validity and the results obtained. In addition, the publication of information about the conduct of NIS/AWB in publicly accessible internet portals helps to ensure transparency in this field of research. All companies interviewed used already existing web pages or have developed their own web sites for this purpose. The same applies to the publication of the results according to international standards [18] within 12 months after completion of the NIS/AWB. In July 2010 a check by the authors showed that in no case of the NIS/AWB published on the internet this deadline had been exceeded.
The previous assessment of NIS/AWB by the Ethics Committee in charge of the scientific lead of the study together with the information and informed consent of the patient into the anonymised data collection and transmission thereof constitute further essentials for medically-scientifically and ethically established research.
It can be shown that the recommendations of the pharmaceutical associations vfa and BPI together with the recommendations of the higher federal authorities and further approaches [12] could make an important contribution to improve further the tool of NIS/AWB as part of the scientific epidemiological research with drugs in the sense of Good Epidemiological Practice.
Furthermore, the current survey could confirm that the vfa member companies have been able to successfully implement this self-commitment in all essential aspects since the publication of the vfa recommendations in May 2007. The results of the survey from 2008 that was performed already a few months after the publication of the vfa recommendations could be confirmed and consolidated.
The recommendations of the higher federal authorities together with the recommendations of the pharmaceutical industry associations constitute a basis for applied quality standards in NIS/AWB which come close to the quality standards for clinical studies. Because of the different character of these kinds of research as well as different national legal regulations for clinical studies on the one hand and NIS/AWB on the other hand, quality assurance methods which are appropriate for the relevant kind of research have to be applied. It has to be clearly differentiated between the legal and regulatory requirements of studies in accordance with the principles of Good Clinical Practice and non-GCP studies.
For instance, audits can also constitute appropriate measures by independent quality assurance units in non-GCP studies. However, they always have to meet the requirements of the dimensions and requisitions of NIS/AWB regarding scope and execution. An inspection of the compliant performance of NIS/AWB by external service providers may regularly be regarded as reasonable and necessary during the course of the studies in order to ensure the compliance with company-owned standards by third parties also.
For instance audits in participating study sites require a critical prior judgement concerning the significance of the chosen sample. The same applies to randomly effected visits at study sites as quality control measures following standards in clinical trials. It is crucial to consider very carefully to what extent these measures are suited in order not to compromise the principle of non-intervention in NIS/AWB, as this has been taken up and discussed by A. Koch, J. Windeler and U. Abel [19] in 1996 already.
Conclusions
With its member companies and their subsidiaries and affiliated firms the vfa represents more than two thirds of the German pharmaceutical market and the majority of the research activities in the pharmaceutical business. The responses and data on which this survey is based are given by such pharmaceutical companies which rate among the leading companies in Germany regarding both their research activities in the major fields of therapy and indication as well as the size of the enterprise.
The recommendations of the higher federal authorities in conjunction with the recommendations of the pharmaceutical industry associations provide guidelines for quality standards in NIS/AWB which come close to those applied in clinical trials. These standards justify a high level of confidence into the quality and the validity of the data collected and the results of non-interventional studies.
Notes
Competing interests
The authors are employees of VFA member companies.
References
- 1.International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite: Guideline for Good Clinical Practice E6 (R1) 1996. Available from: http://www.ich.org/lob/media/media482.pdf. [Google Scholar]
- 2. Directive 2001/20/EC of the European Parliament and of the Council of 4 April 2001 on the approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. Official Journal of the European Union. 2001;L 121, 1/5/2001:34–44. Available from: http://www.eortc.be/Services/Doc/clinical-EU-directive-04-April-01.pdf. [PubMed] [Google Scholar]
- 3.Gesetz über den Verkehr mit Arzneimitteln (Arzneimittelgesetz – AMG) Arzneimittelgesetz in der Fassung der Bekanntmachung vom 12. Dezember 2005 (BGBl. I S. 3394), das durch Artikel 1 der Verordnung vom 28. September 2009 (BGBl. I S. 3172) geändert worden ist. Berlin: Bundesministerium der Justiz; 2009. p. 2009. Available from: http://www.gesetze-im-internet.de/amg_1976/BJNR024480976.html. [Google Scholar]
- 4.European Commission. EudraLex – Volume 9A of the Rules Governing Medicinal Products in the European Union: Guidelines on Pharmacovigilance for Medicinal Products for Human Use. 2008. Available from: http://ec.europa.eu/health/files/eudralex/vol-9/pdf/vol9a_09-2008_en.pdf. [Google Scholar]
- 5. European Parliament; Council of the European Union. Directive 2001/83/EC of the European Parliament and of the Council of 6 November 2001 on the community code relating to medicinal products for human use. Official Journal of the European Union. 2001;L 331, 28/11/2001:67–128. Available from: http://www.edctp.org/fileadmin/documents/ethics/DIRECTIVE_200183EC_OF_THE_EUROPEAN_PARLIAMENT.pdf. [Google Scholar]
- 6.International Epidemiological Association (IEA) European Federation. Good Epidemiological Practice (GEP) – IEA Guidelines for proper conduct of epidemiological research. 2007. [Google Scholar]
- 7. International Society for Pharmacoepidemiology (ISPE) Guidelines for Good Pharmacoepidemiology Practices (GPP) Pharmacoepidemiol Drug Saf. 2008;17(2):200–208. doi: 10.1002/pds.1471. Available from: http://dx.doi.org/10.1002/pds.1471. [DOI] [PubMed] [Google Scholar]
- 8.Arbeitsgruppe Epidemiologische Methoden der Deutschen Arbeitsgemeinschaft für Epidemiologie (DAE) Leitlinien und Empfehlungen zur Sicherung von Guter Epidemiologischer Praxis (GEP) 2004. Available from: http://www.gmds.de/publikationen/1b_LeitlinienUndEmpfehlungen_April2004.pdf. [Google Scholar]
- 9.Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM); Paul-Ehrlich-Institut (PEI) Gemeinsame Empfehlungen des BfArM und PEI zur Planung, Durchführung und Auswertung von Anwendungsbeobachtungen [Stand: 7. Juli 2010] 2010. Available from: http://www.pei.de/cln_170/nn_475588/SharedDocs/Downloads/pu/klin-pruef/100707-awb-kommentierung-fachkreise,templateId=raw,property=publicationFile.pdf/100707-awb-kommentierung-fachkreise.pdf. [Google Scholar]
- 10.Verband Forschender Arzneimittelhersteller e.V. (VFA) VFA-Empfehlungen zur Verbesserung der Qualität und Transparenz von nicht-interventionellen Studien [31. Januar 2007] 2007. Available from: http://infomed.mds-ev.de/sindbad.nsf/0/f99eec1f4951d5c6c12572c800335eee/$FILE/VFA-Empf-NIS_070131.pdf. [Google Scholar]
- 11. Sickmüller B, Breitkopf S. "Points to Consider" zu Anwendungsbeobachtungen. Empfehlungen des Bundesverbands der Pharmazeutischen Industrie zur Durchführung von Anwendungsbeobachtungen. Pharm Ind. 2009;71(5):764–769. Available from: http://www.bpi.de/fileadmin/media/bpi/Downloads/Membernet/Geschaeftsfelder/Arzneimittelsicherheit/Aufsaetze__Fachartikel__wiss._Publikationen/2009-05-15%20BPI%20Sickm%C3%BCller,%20Breitkopf%20Points%20to%20Consider%20zu%20Anwendungsbeobachtungen%20Pharm.%20Ind%2071,%20Nr.%205,%20764%E2%80%93769%20%282009%29.pdf. [Google Scholar]
- 12. Theobald K, Capan M, Herbold M, Schinzel S, Hundt F. Quality assurance in non-interventional studies. GMS Ger Med Sci. 2009;7 doi: 10.3205/000088. Available from: http://dx.doi.org/10.3205/000088.Doc29 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Hahn M, Bethke TD, Hecht A, Henn D, Ruppert T, Hundt F. Qualitätssichernde Maßnahmen in nicht-interventionellen Studien: Ergebnisse einer Umfrage unter den Mitgliedsunternehmen des Verbandes Forschender Arzneimittelhersteller. [Quality assurance measures in non-interventional studies: Results of a survey among the members of the Association of Research-Based Pharmaceutical Companies]. GMS Ger Med Sci. 2008;6 (Ger). Available from: http://www.egms.de/en/gms/2008-6/000057.shtml.Doc12 [Google Scholar]
- 14. Bayerische Landesärztekammer. Berufsordnung für die Ärzte Bayerns: Bekanntmachung der Neufassung vom 1. August 2005. Bayerisches Ärztebl. 2005;60(Spezial 2) Available from: http://www.blaek.de/presse/aerzteblatt/2005/SD_Berufsordnung.pdf. [Google Scholar]
- 15.Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM) Die Bedeutung nicht interventioneller Studien für die Bewertung der Wirksamkeit und der Sicherheit von Arzneimitteln. 2006. Available from: http://www.bfarm.de/cln_103/sid_763994FB5078BBFE4E185A4A53F238D6/DE/BfArM/Termine-und-Veranstaltungen/Dialog_und_Sonstige/2006/060829-Dialog.html. [Google Scholar]
- 16. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887–1892. doi: 10.1056/NEJM200006223422507. Available from: http://dx.doi.org/10.1056/NEJM200006223422507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;342(25):1878–1886. doi: 10.1056/NEJM200006223422506. Available from: http://dx.doi.org/10.1056/NEJM200006223422506. [DOI] [PubMed] [Google Scholar]
- 18. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology. 2007;18(6):805–835. doi: 10.1097/EDE.0b013e3181577511. Available from: http://dx.doi.org/10.1097/EDE.0b013e3181577511. [DOI] [PubMed] [Google Scholar]
- 19. Koch A, Windeler J, Abel U. Anwendungsbeobachtungen: zu Begriff und Nutzen. [Therapeutic indications: on the concept and applications]. Med Klin (Munich) 1996;91(2):103–105. (Ger). [PubMed] [Google Scholar]