Retrospective versus actual pre-course self-assessments

Self-assessments are frequently used to evaluate the effectiveness of medical education programs. The change in learners’ self-ratings preto post-course is attributed to the educational intervention. However, participants may also change their understanding of the dimension being measured and hence, adjust their criteria for self-rating (response shift). This study was designed to compare a conventional prelpost course evaluation design with a retrospective pre/post method used to minimize the effect of the response shift. Participants attending a course in 1987 or 1988 designed to improve interviewing and teaching skills completed a faculty self-rating form (a) one month before the course; (b) after the course, assessing their pre-course skill level (retrospective pre-course); and (c) after the course, assessing their present skill level (post-course). Retrospective pre-course ratings were significantly lower than actual precourse ratings in two out of four factors empirically derived from the assessment instrument in 1987 and three of four in 1988. Greater differences preto post-course were found using the retrospective as opposed to the actual pre-course self-ratings. Our findings indicate that the collection of retrospective pre-course and post-course self-ratings at a single point in time demonstrates greater differences than the conventional pre/post evaluation method. Use of the retrospective precourse method is efficient and may be a more accurate evaluation method than the conventional prelpost evaluation.