Validating Imagine-S

Published Date: 19th July 2016

Publication Authors: Sinha A

​

Purpose 

To validate a virtual reality simulator suitable for training Interventional Radiology (IR) skills. Task Analysis (TA) was employed to create a detailed description of a Liver Biopsy procedure, from which critical procedure steps (CPS) were identified. Engineers and computer scientists used the TA and CPS to build a simulator capable of measuring performance.

Materials and Methods 

Hierarchical and cognitive TA was carried out using interviews (n=12) with subject matter experts, and observation and recording of actual procedures (n=4). CPS were identified through interviews (n=12) and questionnaires (n=8). Engineers and computer scientists used the TA and CPS to build a training simulator capable of measuring performance. The discriminant validity of the simulator was investigated by measuring performance on a simulated Liver Biopsy procedure. The validation study was conducted at three UK clinical centres and participants were a convenience sample recruited on site with varying degrees of experience (consultants n=14, trainees n=26).

Results 

The Liver Biopsy TA described174 procedural steps and 22 performance metrics were included in the simulator. Independent t-tests revealed significant differences between consultants (n=14) and trainees (n=26) on 4 performance metrics. ANOVA revealed significant differences between three groups with differing levels of experience in IR (< 1 year experience, 1-2 years, or 3+ years) on 7 performance metrics. Significant performance metrics were: no go area touched; targeting; length of session; probe usage time; total needle distance moved; number of skin contacts; total time in no go area. All 22 performance metrics followed the predicted pattern with level of performance consistently reflecting experience.

Conclusion 

The use of cognitive task analysis when developing training simulators allows the development of metrics that are a valid discriminator of skill between experts and novices. The indication is that simulation could be a useful training tool that can reflect performance levels on numerous procedural steps. Further validation work is needed to demonstrate transfer of training to the real world.

Johnson, S; Hunt, C; Woolnough, H; Crawshaw, M; Kilkenny, C; Gould, D; England, A; Sinha, A. (2010). Assessing performance on a virtual reality simulated liver biopsy procedure: Validating Imagine-S . Journal of Vascular and Interventional Radiology. 21 (2 (supp)), S103.

« Back