Abstract
Objective
The objective of this study was to establish the validity and reliability of a new
interdisciplinary teamwork assessment scale, the Perinatal Emergency Team Response
Assessment (PETRA), to assess team dynamics during a simulated obstetric crisis.
Methods
This observational cohort study was conducted using high-fidelity simulation and multidisciplinary
obstetric teams in order to evaluate the validity and reliability of the previously
developed PETRA scale for the assessment of teamwork in the management of obstetric
crises. Two high-fidelity simulations of preeclampsia and postpartum hemorrhage (PPH)
were conducted 50 times; 42 were performed by multidisciplinary teams and eight (four
“good,” four “poor”) were performed by actors. Five raters used the PETRA tool to
assess the simulation video recordings. Three additional raters assessed each performance
without the use of PETRA as “good” or “poor” in order to provide an overall rating
(referred to as the standardized score). The primary outcome measure was the PETRA
score. Cronbach's alpha and intra-class correlation coefficients (2,1) with 95% CIs
were calculated to examine internal consistency of the scale and level of agreement
among raters, respectively. Construct validity was established by comparing the assessments
of the raters with the standardized scores. Generalizability theory analysis was performed
to demonstrate PETRA's reliability and to investigate the sources of variation in
scores.
Results
The simulated emergencies were performed by 119 participants. There was overall high
consistency (Cronbach's alpha [95% CI] 0.984 [0.981 to 0.987]) and moderate agreement
(intra-class correlation coefficients [95% CI] 0.49 [0.35 to 0.63]) among raters.
Significantly higher PETRA scores (mean [standard deviation]) were recorded with “good”
versus “poor” performing teams (real scenarios 3.8 [0.7] vs. 2.9 [0.7]; P < 0.001; acted scenarios 4.7 [0.5] vs. 2.2 [0.7]; P < 0.001), suggesting strong construct validity. The overall PETRA scores were not
different between the PPH (3.7 [0.7]) and preeclampsia (3.7 [0.8]) scenarios (P = 0.49). Generalizability coefficients were 0.83 for PPH and 0.76 for preeclampsia.
Conclusion
PETRA is a valid and reliable scale that may be a valuable tool in the assessment
and training of multidisciplinary teams in their management of obstetric crises.
Résumé
Objectif
L’objectif de cette étude était de déterminer la validité et la fiabilité d’une nouvelle
échelle d’évaluation du travail d’équipe interdisciplinaire, la Perinatal Emergency Team Response Assessment [Évaluation d’une équipe d’intervention en cas d’urgence périnatale], ou PETRA, conçue
pour évaluer la dynamique de groupe durant la prise en charge d’une situation d’urgence
simulée en obstétrique.
Méthodologie
Dans le cadre de cette étude de cohorte observationnelle, nous avons utilisé des simulations
haute fidélité et fait appel à des équipes multidisciplinaires en obstétrique pour
évaluer la validité et la fiabilité de l’échelle PETRA créée précédemment, servant
à évaluer le travail d’équipe durant la prise en charge d’une situation d’urgence.
Deux simulations haute fidélité de prééclampsie et d’hémorragie de la délivrance ont
été réalisées à 50 reprises, soit 42 fois par des équipes multidisciplinaires, et
8 fois par des équipes d’acteurs (4 « bonnes » et 4 « mauvaises »). Cinq évaluateurs
ont noté les enregistrements vidéo des simulations d’après l’échelle PETRA. Trois
autres évaluateurs ont jugé les simulations sans l’échelle PETRA et leur ont attribué
la mention globale « Bonne » ou « Mauvaise » (ou note standard). L’indicateur de résultat
principal était le score PETRA. Le coefficient alpha de Cronbach et le coefficient
de corrélation intraclasse (2,1) et leurs intervalles de confiance à 95 % ont été
calculés pour évaluer respectivement la cohérence interne de l’échelle et le degré
d’accord entre les évaluateurs. La validité conceptuelle de l’échelle a été déterminée
par une comparaison des résultats des évaluateurs avec les notes standard. Une analyse
fondée sur la théorie de la généralisabilité a été effectuée pour démontrer la fiabilité
de l’échelle PETRA et pour repérer les sources de variation dans les notes.
Résultats
Les situations d’urgence ont été simulées par 119 participants. On a observé une forte
cohérence (alpha de Cronbach [IC à 95 %] : 0,984 [0,981-0,987]) et un degré d’accord
modéré entre les évaluateurs (corrélation intraclasse [IC à 95 %] : 0,49 [0,35-0,63]).
Les scores PETRA (moyenne [écart-type]) des équipes ayant obtenu la mention « Bonne
» étaient significativement plus élevés que ceux des équipes ayant obtenu la mention
« Mauvaise » (simulations réelles : 3,8 [0,7] c. 2,9 [0,7]; P < 0,001; simulations par des acteurs : 4,7 [0,5] c. 2,2 [0,7]; P < 0,001), ce qui semble indiquer une forte validité conceptuelle. Aucune différence
n’a été observée entre les scores PETRA globaux pour les simulations d’hémorragie
de la délivrance (3,7 [0,7]) et de prééclampsie (3,7 [0,8]) [p = 0,49]. Les coefficients de généralisabilité se chiffraient à 0,83 pour l’hémorragie
de délivrance et à 0,76 pour la prééclampsie.
Conclusion
L’échelle PETRA est valide et fiable, et peut être un outil précieux pour évaluer
et former des équipes multidisciplinaires devant gérer des situations d’urgence en
obstétrique.
Key Words
Abbreviations:
ICC (intra-class correlation coefficient), PETRA (Perinatal Emergency Team Response Assessment), PPH (postpartum hemorrhage)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Obstetrics and Gynaecology CanadaAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Saving mothers' lives. Reviewing maternal deaths to make motherhood safer: 2006-2008.BJOG. 2011; 118 (discussion 3–24): 1402-1423
- Saving mothers' lives: the 2006-8 anaesthesia perspective.Br J Anaesth. 2011; 107: 119-122
The Joint Commission. Sentinal event data root causes by event type 2004-2014. Oakbrook Terrace, IL: 2015; The Joint Commission. http://www.tsigconsulting.com/tolcam/wp-content/uploads/2015/04/TJC-Sentinel-Event-Root_Causes_by_Event_Type_2004-2014.pdf. Accessed March 15, 2017.
- Anaesthetists' Non Technical Skills (ANTS): evaluation of a behavioural marker system.Br J Anaesth. 2003; 90: 580-588
- Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system.World J Surg. 2008; 32: 548-556
- A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study.Crit Care Med. 2006; 34: 2167-2174
- Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review.Obstet Gynecol. 2010; 115: 1021-1031
Onwochei D, Halpern S, Balki M. Teamwork assessment tools in obstetric emergencies: a systematic review. Simulation Healthc, in press. Simul Healthc 2016; Dec 22. http://www.10.1097/SIH.0000000000000210. [Epub ahead of print].
- Multidisciplinary Delphi Development of a Scale to Evaluate Team Function in Obstetric Emergencies: The PETRA Scale.J Obstet Gynaecol Can. 2017; 39 (434–442.e2)
- Active management of the third stage of labour: prevention and treatment of postpartum hemorrhage.J Obstet Gynecol Can. 2009; 31: 980-993
- Diagnosis, evaluation, and management of the hypertensive disorders of pregnancy. Pregnancy hypertension.J Obstet Gynaecol Can. 2014; 38: 575-576
- Validation of a tool to measure and promote clinical teamwork.Simul Healthc J Soc Med Simul. 2008; 3: 217-223
- Assessment of procedural skills in anaesthesia.Br J Anaesth. 2009; 103: 472-483
- Coefficient Alpha: interpret with caution.Eur J Psychol. 2013; 9: 687-696
- Making sense of Cronbach's alpha.Int J Med Educ. 2011; 2: 53-55
- Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists.Br J Anaesth. 2003; 90: 43-47
- Pregnancy-related mortality in the United States, 2006-2010.Obstet Gynecol. 2015; 125: 5-12
World Health Organization. The World Health Report 2005-Make Every Mother and Child Count. Available at: http://www.who.int/whr/2005/en/. Accessed on January 26, 2015.
- The use of cognitive aids during emergencies in anesthesia: a review of the literature.Anesth Analg. 2013; 117: 1162-1171
- A step-by-step approach to using SAS for factor analysis and structural equation modeling.ed 2. SAS Institute, Cary, NC2013: 12
Article info
Publication history
Accepted:
December 21,
2016
Received:
November 13,
2016
Footnotes
Competing interests: See Acknowledgements.
Identification
Copyright
© 2017 The Society of Obstetricians and Gynaecologists of Canada/La Société des obstétriciens et gynécologues du Canada. Published by Elsevier Inc. All rights reserved.
