Development and Pilot Testing of an Assessment Tool for Performance of Invasive Mediastinal Staging.
Publication Title
The Annals of thoracic surgery
Document Type
Article
Publication Date
4-19-2019
Abstract
BACKGROUND: To develop and evaluate a surgical trainee competency assessment instrument for invasive mediastinal staging, including cervical mediastinoscopy and endobronchial ultrasound (EBUS), a comprehensive instrument was developed utilizing expert review and simulated and clinical pilot-testing: Thoracic Competency Assessment Tool-Invasive Staging (TCAT-IS).
METHODS: Validity and reliability evidence was collected and item analysis was performed. Initially, a 27-item instrument was developed, which underwent expert review with members of the Canadian Association of Thoracic Surgeons (n=86) in 2014-2015 (response rate 57%). TCAT-IS was refined to 29 items in 4 competency areas: pre-op, general operative, mediastinoscopy and EBUS. Further refinements were made based on simulated use. The final version was then employed to assess competency of five thoracic trainees performing invasive mediastinal staging in live patients.
RESULTS: Participants were assessed during 20 mediastinoscopy and 8 EBUS with 47 total assessments completed. Reliability (Cronbach's alpha=0.94), inter-rater reliability (k=0.80) and correlation with an established global competency scale (k=0.75) were high. The most difficult items were "set up and adjust EBUS equipment" and "identify vascular anatomy (EBUS)". Feedback questionnaires from trainees (response rate 80%) and surgeons (response rate 100%) were consistently positive regarding user friendliness, utility as an assessment tool and educational benefit. Participants felt the tool "facilitated communicating feedback to the trainee with specific areas to work on."
CONCLUSIONS: TCAT-IS is an effective tool for assessing competence in invasive staging, and may enhance instruction. This initial test establishes early validity and reliability evidence, supporting the use of TCAT-IS in providing structured, specific, formative assessments of competency.
Clinical Institute
Cancer
Specialty/Research Institute
Oncology
