Changing Approaches to OSCE Organization at TNMU according to International Standards: Implementation of European Experience within the Framework of the SAFEMED+ Project

In November 2020, the I. Horbachevsky Ternopil National Medical University started work on the implementation of the project from the Erasmus+ program “Simulation in Undergraduate MEDical Education for Improvement of SAFEty and Quality of Patient Care (SAFEMED+).

TNMU cooperates with seven partners from Georgia, Spain, Lithuania, Italy, Armenia, and Ukraine as part of the SAFEMED+ project. Consortium members work together to develop different mechanisms to improve the quality of medical education through the modernization and implementation of the basic clinical skills program according to the World Federation of Medical Education and WHO standards. The main goals of the project are to improve the curriculum, educational, methodological materials, and tools for learning clinical skills according to world standards; create the material and technical equipment for the development of practical skills centers to improve the educational disciplines of universities in the field of patient safety; enhance educational opportunities of universities of participating countries.

The following can be highlighted from trainings described above:

1. Approaches to the OSCE preparation and conduction at TNMU generally correspond to those adopted in European countries. As an example: different types of stations are organized, such as stations involving a standardized patient (questioning the patient), mannequins (technical skills, physical examination), and analysis of clinical information for decision-making (results of laboratory and instrumental examinations, etc.). Many stations are complex, which means that different types of skills are evaluated at one station.

We made a decision to give preference to complex stations during the creation of OSKI scenarios, especially for 4-6 courses of the medical faculty. That allowed us to evaluate various skills during the student’s performance of one task, which brings the situation closer to real clinical practice (for example, communication skills and physical examination skills in one task).

2. The process of preparing for the exam should involve the creation of clinical cases (clinical situations in the extended version), from which scenarios are created as case fragments with a focus on testing certain skills/competencies. It is important to highlight that such work begins a few months before the exam in the medical higher education institutions of Europe and is carried out annually since it is necessary to constantly replenish the bank of cases and scenarios, analyze their relevance and refine them. All this requires the constant involvement of professors and teaching staff, with the following duties distribution: coordinator – responsible for all aspects of the exam; working groap – selection of clinical situations for the exam and scenarios review, determination of competence assessment points value, creation of a competence table (OSCE blueprint); Coordinating Committee (technical support) – organization and design of stations, documentation management, control of scenario changes at stations; the team of authors – the creation of clinical cases and scenarios, development of logistics, selection, and instruction of standardized patients, training of examiners.

There is a similar administrative structure of the exam at TNMU. The teaching staff of the departments acts as a team of authors and examiners when the working group on the organization and implementation of the OSCE represents the academic and coordination committees and examiners’ training. It was decided to approve the work of such a group permanently because of the importance of the constant practice of replenishing the base of scenarios and their review to create a quality product, as well as training the teaching staff in the methodology of creating scenarios and working as examiners, organizing the exam itself.

3. During the training, significant attention was paid to the methods of scenario and checklist creation to test different types of skills. After the speakers presented the theoretical material and the discussion, a practical part was organized, where the participants worked in groups on creating their scenarios. We found out that some OSCE stations at the University of Santiago de Compostela are paired, which means that when the student is going from one station to another and continues to work within the same clinical situation, performing different tasks at the stations. Usually, such tasks represent the stages of patient care (interrogation, physical examination, evaluation of data from additional interventions, determination of the leading syndrome, making specific decisions regarding the further management of the patient, providing the patient with information, prescribing treatment), which corresponds to the structure of OSCE scenarios at TNMU.

Lecturers gave very useful advices on writing paired station scenarios, for example: how the student can logically move from one station to another without losing the data obtained at the previous one and keep up the realism of the clinical situation.

Besides the paired stations, we took into account advice on the wider use of visual images and photos and any available techniques to create integrity and greater realism of the physical examination of the patient stations. Approaches to writing scenarios have also been improved: each scenario should focus on testing several key competencies and include elements of clinical thinking.

In the context mentioned above, TNMU should move away from focusing only on making a clinical diagnosis practice towards a more focused checkup at each station of specific essential competencies and the ability to make a correct decision in a clinical situation.

All the above-mentioned tips are currently being implemented during the script creation for the exam, which is scheduled to be held in the spring of 2023.

4. In developed countries, higher medical education institutions pay a lot of attention to the communication skills of future doctors. Сhecking them at OSCE is a common practice.

Speakers provided a lot of helpful information about practical approaches for such testing and the main criteria for evaluating students.

As a result, TNMU developed a unified approach for testing communication skills at OSCE and significantly expanded the list of different stations where such skills are tested. Such skills are evaluated more deeply at patient questionary stations, where the examiner pays his attention not only to the content of the student’s conversation with a standardized patient but also to how the student begins, conducts, and ends the conversation, his ability to engage the patient in cooperation and provide him with information about his condition health, further examination, treatment, prevention.

In addition to questioning, communication skills are also checked at other OSCE stations but in a slightly smaller way.

5. One of the essential documents of the OSCE exam is the blueprint, which is a base of competencies to be tested on the exam. The percentage of competencies that are checked at all OSCE stations in total should correspond to their value in the blueprint. Such an approach makes this exam structured and standardized.

OSCE scenarios are created based on a blueprint in Spain, as well as in the majority of European countries. For example, in Spain, a unified blueprint has been adopted for all medical higher education institutions in the country. Thus, with relative freedom in creating stations and scenarios, it is possible to achieve unified exam results.

TNMU worked with Bukovyna Medical University on the regulation of OSCE, where the concept of the OSCE blueprint was introduced. TNMU plans to work on the creation of this tool in the future, but its format will depend on the presence or absence of a unified OSKI blueprint for Ukraine.

6. We started to work on the computer-based OSCE station implementation. Speakers shared their experience at a training session in Catania. We also had a chance to observe exam procedures at the University of Santiago de Compostela, where TNMU representatives were also present. This format is optimal for stations where the main task is to evaluate the data of additional examination methods and determine tactics for patient management. The transfer of these stations in the form of computer OSCE will allow avoiding both simple test writing, which reduces the realism of the exam, as well as turning it into an oral exam, which is contrary to the principles of OSCE.

7. The practice of working on virtual clinical cases based on multi-level tests is being implemented in clinical departments to establish such an exam format. It is planned to arrange courses for teachers to acquaint them with the method of creating scenarios and to create the actual scenarios in March 2023. We plan to pilot this exam format for the 5th-year students of the medical faculty in 2023.

The impact of the experience and knowledge gained within the framework of the SAFEMED project is not limited by the improvement of the OSCE organization but also contributes to the quality of the educational process improvement itself. Mandatory testing of communication competence during the exam has led to the fact that communication skills have become a much greater focus of attention of teachers during the conduction of practical classes and have also become part of the continuing professional development courses of the teachers.

The creation of algorithms for the performance of practical skills by students and checklists leads to an increase in teacher responsibility for training students on a proper level so they can complete the tasks at OSKI stations efficiently; contributes to the unification of practical work approaches and teaching material according to world standards at different departments.

The practice of solving clinical cases in practical classes in clinical disciplines is expanding in the teaching process of TNMU. Work on solving tasks based on multi-level tests using visual materials is much more interactive and engaging for students, which makes their involvement better and increases motivation in learning.

A change in approaches to the organization and conduction of OSCE requires teachers to change their vision, moving away from formal testing of knowledge and skills to a comprehensive assessment of competencies. Such changes require creative approaches and happen slowly, but the process has begun, and the SAFEMED project plays a significant role in this.

Information is provided by the SAFEMED+ project team of Ternopil National Medical University