Twelve tips to successfully conducting virtual clinical assessment using ZoomTM Lessons learned from COVID-19 pandemic

The COVID-19 pandemic changed the world of medical education, moving it years ahead of time into the future. Many concepts needed to be changed, so did the methodologies. Nevertheless, clinical assessment remained tricky. This work highlights the outcome of the work of College of Medicine and Medical Sciences at the Arabian Gulf University in Bahrain in performing virtual clinical examination using ZoomTM. Tips in this article are meant to help plan, implement and evaluate clinical assessments virtually. Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 14 June 2020 doi:10.20944/preprints202006.0171.v1


Context
The COVID-19 pandemic changed the world of medical education, moving it years ahead of time into the future. Many concepts needed to be changed, so did the methodologies. Relying on face to face teaching and assessment became a challenge and thus all medical schools started transitioning into more convenient options for teaching and assessment (Ahmed et al 2020; Almarzooq 2020). Nevertheless, clinical assessment remained an area in the shade and schools resorted to many innovative solutions to be able to evaluate the clinical competencies of students (Boursicot 2020;Amin et al 2020). This work highlights the outcome of the work of the College of Medicine and Medical Sciences at the Arabian Gulf University in performing virtual clinical examination using Zoom™ (Zoom Video Communications, Inc., San Jose, CA, USA). Zoom™ was chosen because its features met the needs of virtual clinical examination as being able to visualize the examinees in virtual panels, dividing students in breakout rooms, preventing a cohort from leaving the exam until the next cohort starts, sharing screens between examiners and examinees, invigilating virtual panels with timely technical support if needed, and moving students automatically from one virtual panels/breakout room to the next. Designing and implementing virtual clinical examination occurs in three phases: planning, implementation, and evaluation. All three phases feed into the efficient and fluent examination process. Throughout these three phases a number of tips can come in handy.

Tip 1: Align your vision with stakeholders
It is important to understand that schools not only have systems and bylaws in place, but also they have inherent cultures. It is important to understand what structures exist in place and to identify the scope of the necessary transition (Curran 2010). A consensus needs to be reached with all your stakeholders on this scope and it must be communicated to them openly. For example, an agreement should be reached to conduct virtual clinical assessments replacing long case clinical encounters and traditional OSCEs. This is very important to alleviate any expectations that might exist and are remnants from old well-established systems. A number of meetings need to be held with the concerned parties; department heads, directors of education, school administration, technical personnel, and student representatives. It is important to map your stakeholders well before you start setting out your plan.

Tip 2: Identify the current situation and its limitations
A close look into the limitations of the situation is much required. It is important to understand the context of the situation at hand and what can and cannot be accessed during this time (Youssef 2019). A few questions are needed to be addressed: • Should assessors be examining remotely or from campus?
• Is there any regulation regarding allowed numbers of people in one room?
• Is it manageable to use standardized patients or simulated patients for the OSCEs?
• How can physical distancing be managed?
• Can students come to the University for Support?
• What are the necessary precautions for infection control?
• What are the existing laws and bylaws that might come in play when designing the virtual clinical examination? Modifications can be done to existing regulatory issues to suit the new condition. It is advisable to think strategically of all the possible expected situations that might arise and apply modifications to current regulations or even issue new ones specifically for the virtual examinations.

Tip 3: Understand the nature of your logistics
Appropriate selection of the venue is vital for the effective execution of virtual exams. Proper arrangement of rooms would facilitate logistics and communication. The venue should be equipped with high speed internet connectivity, electric bell, and additional back up rooms with computers, speakers etc. The control room should be strategically placed in such a way, to be able to communicate with all involved people in an effective manner without compromising the confidentiality of the exam process. A technical check is required on all the available logistics, on daily basis, before exam sessions. Using Local Area Network (LAN) is better and more stable when compared to Wi-Fi connection. Using existing personal computers in staff offices after providing them with webcams, microphones, and speakers is much more convenient to purchasing laptops or tablets (Halbherr 2014). These choices need early involvement and consultation of IT and networking personnel in order to plan for procurement of the necessary gadgets necessary for the assessment plan.

Tip 4: Develop a plan for assessment
A few decisions have to be made at this stage: • Decision upon the clinical competencies that should be assessed throughout this assessment process • Agreement on the modified exam blueprint.
• Choosing the proper format of clinical assessment that can be applied virtually.
• Decide on the proper number of the station under each format and for different exams (end of rotation or finals). • Develop a detailed plan for different exams that includes number of students, duration of each exam and the schedule of the exams. Identify the competencies to be assessed, and make sure you have a well-designed exam blueprint where tests intended to be performed on Zoom™ are highlighted (Mubuuke 2014).

Tip 5: Assign a task force
Assigning a dynamic and committed task force is crucial for the success of any innovation (Fernandez 2020). Institution's top administration needs to assign a taskforce that can manage the complex functions needed to run such an exam successfully. As an example, this team might include: (1) the exam general coordinator (meeting host) who operates the main the Master computer, set appropriate Zoom™ settings, and conduct the exam, (2) A core team who plans provides the daily plans for exams, implements the exam and arrange for all logistics, (3) host assistant who double checks that right students are allowed in and out of breakout rooms, assists in smooth transition between breakout rooms, and facilitates communication between host and other members of the team, (4) faculty invigilator who acts as a standby host and visits virtual panels at regular intervals for quality assurance, (5) secretaries of clinical departments who organizes communication with students to ensure their timely admission to their designated panels as well as preparing the usual exam files to be distributed to assessors, (6) pre-exam invigilators who have a vital function checking students' IDs, students' surroundings, quality of internet connectivity, and audio and video of students, (7) post-exam invigilator who gets instantaneous feedback about the exams, addresses concerns if any, and control students' exit from the exam session, (8) technical team who supports examiners regarding any technical issues and are well trained regarding the flow of the exam. It is also beneficial to have a backup for different roles.

Tip 6: Build needed capacities
In addition to leadership support, the implementation of virtual clinical assessment using Zoom™ demands adequate training of multiple stakeholders namely organizing team, secretaries, technical support personnel, assessors, and students (Curioso 2019). Since, the nature of training required for each group is different, focused workshops in small groups should be conducted in which actual process of managing the meeting with breakout rooms demonstrated. One valuable thing to do is to repeat the demonstration whenever improvements or changes are made. This tip is directly linked to two other tips which are: developing the necessary documents and testing the process through multiple mocks. The plan of capacity building cannot be complete without repeated testing of the process. This determines the needed number of personnel to train from each category as well as the depth of training needed. After each mock, any small upgrades made, requires a new task for one of the team members, and subsequently guiding documents and needed training. On-the-job training can also be conducted before, during, and after mocks to make performance better and better. The magnitude of manpower required for the process might obligate the team to train personnel on some tasks that are outside the scope of their original job description which needs acceptability and strong leadership support. Two signs of success of capacity building are: (1) team members apply their roles routinely with minimal need for guidance when time comes for actual application and (2) a sense of ownership of the process is felt when suggestions and innovative ideas for improvements come from multiple team members.

Tip 7: Develop necessary documents and guiding forms
The roles of individuals in the process are bound to be overlapping roles, which will lead to the feeling of insecurity and ambiguity among many. This should be eased by developing exhaustive checklists for specific roles, which also serve as an effective tool for training and standardization of the entire process (Hughes 2018). These lists should be developed among the core/planning team through regular meetings and envisioning the logistics involved in the entire process. The process is then broken down into easily understandable protocols with probable timeline allotted for each activity. Unique checklists with definite tasks are prepared for each team member. These checklists are revised multiple times based on the novel suggestions from team members and they are tested during mocks and refined. All the documents, after the process of refinement and approval, should be shared through emails and other instant messaging applications like WhatsApp (WhatsApp, Inc., Menlo Park, CA, USA) groups to all the team members. j. Rules and regulations for the virtual exam (containing all scenarios of expected problems and their solutions).

Tip 8: Conduct a number of mocks
Sensitization and orientation of the entire team with necessary documents and guiding forms are mandatory prerequisite for mock exams (Robinson 2017). Mock exams should be conducted in two phases, one exclusively for faculty and the other for students and faculty together. In phase one, simulated students can be used (faculty or staff joining from distance). During this phase the time required for discrete tasks like ID check, evaluation of student, and transition between breakout rooms can be calculated. The nature of broadcast message to be displayed during various stages of exam should also be decided without any ambiguity. During phase two, all the students who are scheduled to take up the exams must be involved in multiple small batches along with their faculty assessors. At the end of each mock, feedback should be requested from all the students and faculty, and preferably incorporated later into future mocks and subsequently real exams based on the available resources. A number of meetings can be held for the core team with a thorough analysis of the feedback collected from the mock exams. All necessary suggestions for improvement should be treated with extreme care and studied in relation to the technical capacity of the system, the people and the application. Whenever possible, simplification of the process is advocated for. The less complex the process, the less are the chances for in-exam complications and failure of the systems.

Tip 9: Maintain constant communication with the team during the exam
Proper and constant communication during the different phases of the process is essential to guarantee flawless planning and conduction of the assessment (Ahmed 2019). Various channels of communication involved here are as follows: a. WhatsApp as a supporting application: This is very useful for sequential admission of participants where cohorts of students are added to separate WhatsApp groups to receive the hyperlink for exam session on timely basis. The same groups can be used to check for absent or late students as well as ensuring that they have proper connectivity. b. Small and large WhatsApp groups: To facilitate effective planning, communication, and timely management of emerging issues. Create several WhatsApp groups based on the logistics of each exam, which eventually help to conduct error-free exams. c. Assessors' briefings: Sending emails and documents is not sufficient for some of the busy clinicians to deal with the application properly. Meetings should be conducted after the first mock and it is also necessary to have a pre-exam briefing for assessors not only about the exam content but also about the Zoom™ application and the functions that they will use throughout the exam. A live demonstration of a Zoom™ meeting with breakout rooms feature can be displayed to examiners for 20-30 minutes prior to every exam. d. Attendance sheet circulation: Make sure that student lists handled by the assessors are the final list after taking out absent students to overcome any possible confusion. Thus, it is advised to share student lists timely. e. Printed spreadsheets with colored pictures of students for each exam: These spreadsheets are distributed to host, co-hosts, examiners, and invigilators in pre-exam rooms in order to help ensuring the identity of the examined students throughout the whole process.
f. Actual bell: An actual bell is used to announce the beginning and end of each student encounter to the assessors in order to standardize the timing. g. Briefing support team: End-of-day debriefing and morning briefings, personal communication, or messages through the WhatsApp groups are means of keeping support teams informed of any necessary changes in examination processes or procedures.

Tip 10: Use Zoom™ efficiently
Multiple features of Zoom™ meeting application already meet the needs of the virtual clinical examination. The following are the features of Zoom™ and how they are matched to the examination needs (Major 2020).
− Waiting rooms: This is used to organize the flow of students as well as to ensure the security of the exam. Students are given a certain time to join the meeting and they are not allowed inside the meeting unless they have the correct first and family names. − Recording feature: This can be used to ensure that all stages of the exam are documented in order to make sure that any student claims can be tracked and assessed later on. − Remote camera control: Can be used to limit the possibilities of cheating. − Breakout rooms: This is the most useful feature used in virtual clinical examination and they are used as follows: • Pre-exam rooms: This is where students are admitted for an ID check, where an invigilator checks their IDs and compares the picture to their webcam images. Students receive summary of exam instruction and are asked to show their surroundings. Pre-exam rooms are also used to keep the new cohort of students busy while another cohort of students are having their exams. • Panels: Breakout rooms are created and labeled as virtual panels where each panel has one assessor, two assessors, or an assessor and a role player (according to the nature of the exam). Students are admitted to virtual panels based on a pre-prepared list and shuffled timely from one panel to the next by the exam coordinator (host). Panels are also allowed to self-record which adds more to the quality assurance aspect of the exam. • Post-exam room: Once a cohort of student finishes its exam, they are moved by the host to the post-exam room where a faculty member takes their feedback about the exam and implicitly keeps them busy until the next cohort of students are admitted to the virtual panels. Once the next cohort of students are admitted, students are allowed to leave the meeting (exam). − Co-hosting feature: This can be utilized to create a co-host by a faculty invigilator. A faculty invigilator who has also the recording ability moves in a systematic manner (with turned off audio and video) around the virtual panels and record parts of the exam while he assures the quality of audio and video as well as provide real-time support whenever needed.

Tip 11: Develop a check list-guided executive plan for every examination session
The most critical part of implementation is microplanning for each and every session (Mallon 2019). Named personnel to be assigned for all the roles, and informed through WhatsApp and official emails, preferably from Dean's office. Their availability should be confirmed at least two or three days before the event. The rooms, computers, bells, speakers, webcams, and other related gadgets to be used for the exams should be checked every day at the beginning and the end of the exams. The Zoom™ ID should be shared everyday with the taskforce to avoid confusion. The Zoom™ account name of assessors to be renamed everyday based on the newly assigned panel numbers/names of the assessor. Students are reminded about the exam instructions and regulations through WhatsApp groups before the exam. All students should be instructed to rename their account name with full names as per the official records. The mobile phone/WhatsApp number of a minimum of two or three contact persons should be shared with all students for instantaneous communication. The number of veiled female students who do not prefer to appear virtually along with opposite gender should be identified well in advance and ID check should be performed in a separate breakout room for such students. Assigning students soon after a break should be synchronized well so that all the assessors are back to their panels. It is wise to follow a standard format of communication with added details, throughout mock and real exams. Assigned personnel assure timely dissemination of the daily plan, confirmation of the assigned personnel, and timely trouble-shooting for emerging issues during implementation.

Tip 12: Define the process evaluation indicators
Evaluation is an important component of any project, especially when it is a new one, to gain feedback for further development and reform (Milone 2017). Evaluation should address three important areas: infrastructure adequacy, process, and outcomes. Measurable indicators must be thought of from the early beginning of the project. In such virtual clinical examination, important indicators can include acceptance by both students and faculty, the extent to which virtual clinical examination measures the intended clinical competencies, the validity and reliability of the assessment results, and the user friendliness of the chosen method.
An important indicator is comparing the assessment results of virtual clinical examination to those of traditional OSCE or long-case clinical exam. This would clearly outline the differences and the areas in which virtual clinical examination can fit well and replace the traditional one without much compromise of the outcomes. Planning for evaluation from the beginning is key to system improvement. It is thus recommended to have a multistage evaluation design from the beginning.

Concluding Statement
Although COVID-19 placed clinical examination of students in a big test, yet the use of Zoom™ for virtual clinical assessment sessions can still be conducted putting in mind that a lot of preparation needs to be done in the form of technical specifications, checks, document preparation, training of people, and continuous quality improvement. The tips offered in this article are intended to guide on the process and to make the outcome as valid and reliable as possible. With future involvement of artificial intelligence, we can be expecting to see a wider range of competencies assessed through the virtual platform.