An inter-agency expert panel’s prioritisation of clinical quality improvement topics in a paramedic system


quality improvement
outcome based

How to Cite

Campeau A, Huiskamp M, Sykes N, Kriening S, Bourn S, VanAarsen K. An inter-agency expert panel’s prioritisation of clinical quality improvement topics in a paramedic system. Australasian Journal of Paramedicine [Internet]. 2021Jun.15 [cited 2021Jul.23];18. Available from:



Quality improvement (QI) programs have become common in paramedic systems, but they are often limited to individual agencies. Modern paramedicine involves many different agencies and inter-agency QI programs would better reflect their co-operative efforts. Similarly, inter-agency use of clinical outcome measurements can offer system level performance data. This study’s intent was to explore the feasibility of planning an inter-agency QI program that uses outcome measures.


This study used a modified Delphi methodology. A 49-member panel of inter-agency representatives was convened to identify and prioritise clinical outcome-based topics. Over a 3-month period, two online surveys were conducted followed by a 1-day face-to-face meeting.


The study demonstrated very high participation rates. Results progressed from an initial wide range of 38 topics to a final consensus of two: infection/sepsis and patient safety/care pathways, complete with outcome measures.


Inter-agency quality improvement planning is an under investigated area, but this study demonstrates that it is feasible. Additionally, this planning can incorporate clinical outcome measures that inform system level discussions about quality. Other paramedic agencies may draw on the study’s processes when planning their own quality improvement programs.


Maio R, Bronken T. Quality assurance in emergency medical services systems in Michigan: Report of a survey. QRV Qual Rev Bull 1991;12:392-5.

Tello RR, Braude D, Fullerton L, Froman P. Outcome of trauma patients immobilized by emergency department staff, but not by emergency medical services providers: a quality assurance initiative. Prehosp Emerg Care 2014;4:544-9.

Sahu N, Matthews P, Groner K, Papas M, Megargel R. Observational study on safety of prehospital BLS CPAP in dyspnea. Prehosp Disaster Med 2017;6:610-4.

Mears GD, Pratt D, Glickman SW, et al. The North Carolina EMS data system: a comprehensive integrated emergency medical services quality improvement program. Prehosp Emerg Care 2010;14:85-94.

Tsai D, Choi B, Sullivan F, Williams A. Quality improvement in EMS: a unique and challenging necessity. R I Med J 2014;97:17-9.

Redlener M, Olivieri P, Loo GT, et al. National assessment of quality programs in emergency medical services. Prehosp Emerg Care 2018;22:3. doi: 10.1080/10903127.2017.1380094

Karam M, Brault I, VanDurme T, Jacq J. Comparing interprofessional and interorganisational collaboration in healthcare: a systematic review of the qualitative research. Int J Nurs Stud 2018;79:70-83.

Delbridge T, Bailey B, Chew JL, et al. EMS agenda for the future. National Highway Traffic Safety Administration; 1996. Available at:

American Heart Association. Guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Part 4: Systems of care and continuous quality improvement. Circulation 2015;132:S397-413.

Daulelin D, Kulick ER, S’Amore K, et al. The Massachusetts Emergency Medical Service Stroke Quality Improvement Collaborative, 2009-12. Prev Chronic Dis 2013;10:130126. doi:

Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ Clin Res 1995;311:485. doi: 101136/bmj.3117003.485

Matziou V, Vlahioti E, Perdikaris P, et al. Physician and nursing perceptions concerning interprofessional communication and collaboration. J Interprof Care 2014;28:526-33.

Katakam L, Suresh K. Identifying a quality improvement project. J Perinatol 2017;37:1161-5.

Haugland H, Rehn M, Kelpstad P, Kruger A. EQUIPE collaboration group. Developing quality indicators for physician-staffed emergency medical services: a consensus process. Scand J Trauma 2017;25:14. doi: 10.1186/s13049-017-0362-4

Rowe G, Wright G. The Delphi Technique as a forecasting tool: issues and analysis. Int J Forecast 1999;353-75.

Weise J, Fisher KR, Trollor J. Utility of a modified online Delphi method to define workforce competencies: lessons from the intellectual disability mental health core competencies project. J Policy Pract Intellect Disabil 2016;1:15-22.

Chan TM, Yarris LM, Humphre-Murto S. Delving into Delphis. Can J Emerg Med 2019;21:167-9.

Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. 1st edn. West Sussex: Wiley and Blackwell; 2011.

Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi Studies. J Clin Epidemiol 2014;67:401-9. doi:

MacPherson S, Reese C, Wendler C. Methodology update: Delphi studies. Nurs Res 2018;67:404-10. doi: 10.1097NNR.0000000000000297

Watson T. Public relations research priorities: a Delphi study. Journal of Communication Management 2008;12:2:104-23.

Sumsion T. The Delphi technique: an adaptive research tool. Br J Occup Ther 1998;4:153-6.

Agar M. Speaking of ethnography. Newbury Park, CA: Sage, 1986.

Shield T, Campbell S, Rogers A, et al. Quality indicators for primary care mental health services. Qual Saf Health Care 2003;12:100-6.

Kerr G. Apples, oranges and fruit salad: a Delphi study of the IMC educational mix. Journal of Marketing Communications 2009;15:119-37.

Tersine RJ, Riggs WE. The Delphi technique: a long-range planning tool. Business Horizons 1976;2:51-6.

Gustafson DH, Shukla RK, Delbecq A, Walster GW. A comparative study of differences in subjective likelihood estimates made by individuals, integrating groups, Delphi groups and nominal groups. Organ Behav Hum Perform 1973;2:280-91. doi: 10.1016/0030-5073(73)90052-4

Polito C, Isakov A, Yancey A, et al. Prehospital recognition of severe sepsis: development and validation of a novel EMS screening tool. Am J Emerg Med 2015;33:1119-25.

Smyth MA, Brace-McDonnell SJ, Perkins GD. Identification of adults with sepsis in the prehospital environment: a systematic review. BMJ Open 2016:6:e011218.

Borrelli G, Kock E, Ethan S, Shannon L, Rech MA. Early recognition of sepsis through emergency medical services pre-hospital screening. Am J Emerg Med 2019;37:1428-32.

Tangherlini N, Villar J, Brown J, et al. The HOME team: evaluating the effect of an EMS-based outreach team to decrease the frequency of 911 use among high utilizers of EMS. Prehosp Disaster Med 2016;31:603-7.

Bisman V, Prpic J, Michaud S, et al. MP04:rEDirect: safety and compliacne of an emergency department diversion protocol for mental health and addictions patients. Canadian Journal of Emergency Medicine 2019;21(S1):S43-S43.

Quatman C, Mondor M, Halweg J, Switzer JA. Ten years of EMS fall calls in a community: an opportunity for injury prevention strategies. Geriatric Orthopaedic Surgery Rehabilitation 2018;41(1):42-48.

Graefe A, Armstrong JS. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. Int J Forcasting 2016;1:183-95. doi:10.1016/j.ijforecast.2010.05.004

Watson T. Public relations research priorities: a Delphi study. Journal of Communication Management 2008;2:104-23.

Rowe G, Bolger F. Delphi: a re-evaluation of research and theory. Technol Forecast Soc Change 1991;39:235-51.