Development of an analytical prepCheck-supported approach to evaluate tutor-based assessments of dental students' practical skills

The aim of the present study was to develop an analytical computer-supported assessment concept for the objective evaluation of students' practical skills, and to investigate the assessor-dependent differences by setting tolerance limits applied during the 'glance and grade' evaluation. A sample of 54 conventionally supervised dental undergraduates were given training in the preparation of a macroretentive Class II cavity. The course participants then took a mandatory practical examination. Three (plus one) course instructors performed the visual evaluations of the examination tasks according to predefined assessment criteria, followed by optical impressions (quadrant scan using Omnicam; Cerec; Dentsply Sirona, Wals, Austria) and subsequent assessment of the prepared cavities (tooth 46) using the prepCheck application (Dentsply Sirona). In this course, the maximum permissible deviation was set at ± 10% from the predefined assessment criteria. If no tolerance was applied, the statistical analysis revealed a significant difference between the actually achieved cavity dimensions and the predefined values of a master preparation (P = 0.0001; Student's t-test), thus leading to an overall failure rate of 100%. The application of the initially targeted 10% tolerance led to an insignificant reduction (96.3%), while the stepwise elevation of the permissible deviation up to 35% finally matched with the result of the visual assessment (total failure rate of approximately 20%). Unlike the pronounced subjectivity of the tutors during the assessment of students' practical skills, the utilization of the prepCheck application enables precise evaluations. Further studies are clearly warranted to investigate the possible educational outcome of an advanced interactive computer-supported training with implemented tolerance corridors. Keywords: computer-aided assessment, phantom lab, practical skills, preclinical dental education, prepCheck