Abstract
Objectives: To describe and discuss the challenges in evaluation of a participatory action research with street-involved youth.
Methods: A combination of quantitative and qualitative methods were utilized for both process and outcome evaluations. Process evaluation methods included in-depth individual interviews, focus groups, participant observation, and session debriefing forms. Summative evaluation research included focus testing of the harm reduction video and a survey of video users.
Findings: Members of the youth team reported favourably on the experience, citing friendship, skills development, fun, and pride of accomplishment among the key benefits of participation. Political tensions arose because of the focus on reducing harm from drug use rather than encouraging abstension. The heavy demands of participatory research and development, resource constraints and the priority given to product development in these kinds of projects necessarily precludes extensive youth participation in the design, implementation and analysis of additional evaluation research. Even when resources are directed towards evaluation, there is a tendency to focus on data collection, which may limit time and resources for data analysis. Finally, there is an inclination to focus on the product development rather than dissemination and impact of the product.
Interpretation: Despite the challenges inherent in participatory action research and its evaluation, this project was regarded as an empowering experience by the street youth who participated in it. It is worthwhile to direct resources to evaluation which optimally gives proportional attention to data collection as well as data analysis, and focusses not only on product development but also on its dissemination and impact.
Résumé
Objectifs: Décrire et expliquer les défis de l’évaluation d’un projet de recherche active auprès de jeunes de la rue.
Méthode: Nous avons combiné des méthodes quantitatives et qualitatives pour évaluer tant le processus que les résultats du projet. Pour l’évaluation du processus, nous avons utilisé des entrevues individuelles approfondies, des groupes de discussion, l’observation des participants et des questionnaires récapitulatifs après les séances. Pour l’évaluation sommative, nous avons fait visionner à des groupes cibles une vidéocassette traitant de réduction des méfaits en sollicitant leurs commentaires.
Constatations: Les jeunes ont dit avoir apprécié l’expérience, qui leur a permis entre autres de se faire des amis, d’acquérir des compétences tout en s’amusant et de retirer une certaine fierté de leurs accomplissements. Des tensions politiques se sont manifestées lorsque les jeunes de la rue se sont concentrés sur la réduction des méfaits de la consommation de drogues plutôt que sur l’abstention. En recherche active, il est souvent difficile de doser le temps et les ressources consacrés au projet proprement dit et à son évaluation. Même lorsqu’on dispose de ressources d’évaluation, on a tendance à mettre l’accent sur la collecte des données en négligeant leur analyse. On tend aussi à s’attacher à l’élaboration du produit plutôt qu’à sa diffusion et à l’analyse de ses effets.
Interprétation: Malgré les défis inhérents à la recherche active et à son évaluation, les jeunes qui ont participé au projet considèrent qu’il leur a permis de renforcer leur autonomie. Il vaut la peine de consacrer des ressources à l’évaluation, car cela incite à se pencher autant sur la collecte des données que sur leur analyse et à ne pas en rester à l’élaboration d’un produit, mais à travailler à sa diffusion et à l’analyse de ses effets.
Footnotes
Acknowledgements: Members of the CYPHR Youth Team (D. Andrews, S. Banks, A. Campbell, W. Deschamps, M. Gonsalves, C. Meier); Members of “Core Project Team” (J. Greenbaum, B. Fisher, C. Kelly, P. West, Y. Zdanowicz); members of the “Evaluation Research Team” (S. Cunningham, J. Dunleavy, W. Medved, C. Whittaker, J. Everett; and Members of the “Advisory Group” (W. Cavaleri, S. Gaets, D. Long, W. Hillier, S. Hopkins, S. Cooke, K. Positano, T. Potts, S. Rump, J. Toguri, G. Vardy). We also wish to acknowledge funding from the Addiction Research Foundation, and the invaluable participation of many street-involved youth who participated in research conducted by the CYPHR Youth Team.
Dedication: This paper is dedicated to Mario Gonsalves, a CYPHR team member and friend, who passed away in August 1999.
References
- 1.Leffley A, Abbott M, Cashmore M, Knights B. Using volunteer peer educators in public health programs: Recommendations for planning, recruitment, orientation and maintenance. Public Health & Epidemiology Report Ontario. 1999;10(7):130–31. [Google Scholar]
- 2.Leaman M, Lechner C, Sheeshka J. The Community Nutrition Worker Project: A retrospective survey of peer educators. J Can Dietet Assoc. 1997;58(1):34–38. [Google Scholar]
- 3.Gaventa J. Participatory research in North America. Convergence. 1988;21(2/3):19–27. [Google Scholar]
- 4.Hall BL. From margins to center? The development and purpose of participatory research. Am Sociol. 1992;23(4):15–28. doi: 10.1007/BF02691928. [DOI] [Google Scholar]
- 5.Levy R. Participatory Action Research. Montreal, QC: McGill University; 1994. [Google Scholar]
- 6.Pace LA, Argona DR. Participatory action research. Am Behav Scientist. 1989;32(5):552–65. doi: 10.1177/0002764289032005004. [DOI] [Google Scholar]
- 7.Park P, Brydon-Miller M, Hall B, Jackson T, editors. Voices of Change: Participatory Research in the United States and Canada. Toronto, ON: OISE Press; 1993. [Google Scholar]
- 8.Rahman A. People’s Development: Perspectives on Participatory Action Research. London: Zed Books; 1993. [Google Scholar]
- 9.Reason P, editor. Participation in Human Inquiry. 1994. [Google Scholar]
- 10.Society for Participatory Research in Asia. Participatory Research: An Introduction. Toronto, ON: International Council for Adult Education; 1982. [Google Scholar]
- 11.Tobias KJ. Participatory Research: A Handbook for Fieldworkers. Toronto, ON: International Council for Adult Education/Participatory Research Group; 1981. [Google Scholar]
- 12.Whyte WF. Participatory Action Research. Newbury Park, CA: Sage Publications; 1991. [Google Scholar]
- 13.Corcega TF. Participatory research: Getting the community involved in health empowerment. Int Nurs Rev. 1992;39(6):185–88. [PubMed] [Google Scholar]
- 14.de Koning K, Martin M. Participatory Research in Health: Issues and Experiences. London, UK: Zed; 1996. [Google Scholar]
- 15.Flynn BC, Ray D, Rider MS. Empowering communities: Action research through healthy cities. Health Educ Q. 1994;21(3):395–405. doi: 10.1177/109019819402100310. [DOI] [PubMed] [Google Scholar]
- 16.Green LW, George MA, Daniel M, Frankish CJ, Herbert CJ, Bowie WR, O’Neill M. Study of Participatory Research in Health Promotion: Review and Recommendations for the Development of Participatory Research in Health Promotion in Canada. Vancouver, BC: Institute of Health Promotion Research, University of British Columbia & the BC Consortium for Health Promotion Research; 1995. [Google Scholar]
- 17.Laurell AC, Noriega M, Martinez S, Villegas J. Participatory research on workers’ health. Soc Sci Med. 1992;34(6):603–13. doi: 10.1016/0277-9536(92)90188-V. [DOI] [PubMed] [Google Scholar]
- 18.Macaulay AC, Commanda LE, Freeman WL, Gibson N, McCabe ML, Robbins CM, Twohig PL. Participatory research maximises community and lay involvement. BMJ. 1999;319:774–78. doi: 10.1136/bmj.319.7212.774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Nichter M. Project community diagnosis: Participatory research as a first step toward community involvement in primary health care. Soc Sci Med. 1984;19(3):237–52. doi: 10.1016/0277-9536(84)90215-6. [DOI] [PubMed] [Google Scholar]
- 20.Poland B. Knowledge development and evaluation in, of, and for healthy community initiatives. Part I: Guiding principles. Health Prom Int. 1996;11(3):237–47. doi: 10.1093/heapro/11.3.237. [DOI] [Google Scholar]
- 21.Robottom I, Colquhoun D. Participatory research, environmental health education, and the politics of method. Health Educ Res. 1992;7(4):457–69. doi: 10.1093/her/7.4.457. [DOI] [Google Scholar]
- 22.Smith SE, Pyrch T, Lizardi AO. Participatory action research for health. World Health Forum. 1993;14:319–24. [PubMed] [Google Scholar]
- 23.Travers KD. Reducing inequities through participatory research and community empowerment. Health Educ Q. 1997;24(3):344–56. doi: 10.1177/109019819702400307. [DOI] [PubMed] [Google Scholar]
- 24.Bang AT, Bang RA. Community participation in research and action against alcoholism. World Health Forum. 1991;12:104–9. [PubMed] [Google Scholar]
- 25.Bailey D. Using participatory research in community consortia development and evaluation: Lessons from the beginning of a story. Am Sociol. 1992;23(4):71–82. doi: 10.1007/BF02691932. [DOI] [Google Scholar]
- 26.Cousins JB, Earl LM. The case for participatory evaluation. Educ Eval Pol Anal. 1992;14(4):397–418. doi: 10.3102/01623737014004397. [DOI] [Google Scholar]
- 27.Dixon J. Community stories and indicators for evaluating community development. Commun Dev J. 1995;30(4):327–46. doi: 10.1093/cdj/30.4.327. [DOI] [Google Scholar]
- 28.Karim W-J. Evaluation of participatory research in developing community leadership skills. Convergence. 1982;15(4):52–59. [Google Scholar]
- 29.Boutilier M, Mason R, Rootman I. Community action and reflective practice in health promotion research. Health Prom Int. 1996;12(1):69–78. doi: 10.1093/heapro/12.1.69. [DOI] [Google Scholar]
- 30.Cohen Y. Some critical reflections on participatory research. Convergence. 1982;15(3):77–84. [Google Scholar]
- 31.Macaulay A, Delormier T, McComber A, Cross EJ, Potvin LP, Paradis G, et al. Participatory research with native community of Kahnawake creates innovative code of research ethics. Can J Public Health. 1998;89(2):105–8. doi: 10.1007/BF03404399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Maclure R. The challenge of participatory research and its implications for funding agencies. Int J Sociol Soc Pol. 1990;10(3):1–21. doi: 10.1108/eb013094. [DOI] [Google Scholar]
- 33.Mason R, Boutilier M. The challenge of genuine power sharing in participatory research: The gap between theory and practice. Can J Commun Mental Health. 1996;15(2):145–52. doi: 10.7870/cjcmh-1996-0015. [DOI] [PubMed] [Google Scholar]
- 34.Simonson LJ, Bushaw VA. Participatory action research: Easier said than done. Am Sociol. 1993;24(1):27–37. doi: 10.1007/BF02691943. [DOI] [Google Scholar]
- 35.Smart R, Adlaf EM, Walsh GW, Zdanowicz YM. Drifting and Doing: Changes in Drug Use Among Toronto Street Youth, 1990 and 1992. Toronto, ON: Addiction Research Foundation; 1992. [Google Scholar]
- 36.Breland K, Tupker E, West P, Poland B. Let ’em go: How to support youth in creating their own solutions - the Street Youth Harm Reduction Project. Toronto, ON: Addiction Research Foundation; 1998. [Google Scholar]
- 37.Poland B, Medved W, Cunningham S. The Street-Involved Youth Harm Reduction Project: A (Process) Evaluation Research Perspective (interim unpublished internal report) Toronto, ON: University of Toronto and Addiction Research Foundation; 1997. [Google Scholar]
- 38.Green LW, George MA, Daniel M, Frankish CJ, Herbert CJ, Bowie WR, O’Neill M. Study of Participatory Research in Health Promotion (for Royal Society of Canada) Vancouver, BC: UBC Institute of Health Promotion Research and BC Consortium for Health Promotion Research; 1995. [Google Scholar]
- 39.Reason P. Three approaches to participative inquiry. In: Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. Thousand Oaks, CA: Sage Publications; 1994. [Google Scholar]
- 40.Hall BL. From margins to center? The development and purpose of participatory research. Am Sociol. 1992;23(4):15–28. doi: 10.1007/BF02691928. [DOI] [Google Scholar]