Research Report

The bovine pregnancy diagnosis challenge: evidence-based evolution of a teaching intervention over 8 years of use

Annett Annandale,^a,b Henry Annandale,^b Kate May,^a Dietmar Holm^a

^aFaculty of Veterinary Science, University of Pretoria, Pretoria, South Africa, ^bSchool of Veterinary Medicine, Murdoch University, Perth, Australia

Abstract

This article describes the implementation of an educational intervention, ‘bovine pregnancy diagnosis challenge’ over an 8-year period, and how evidence-based changes were applied to ensure the best learning outcome for bovine pregnancy diagnosis (PD) via transrectal palpation. With the worldwide shift from mainly knowledge-based to competency-based veterinary courses, many new and innovative teaching interventions have been introduced into veterinary education. These teaching interventions include but are not limited to simulators, models, virtual realities, applications, and computer-assisted learning tools; of which, many have been implemented into various veterinary training programs. While these advances are noteworthy, the question still remains as to whether or not the implementation of these teaching interventions is optimal. Do we know the ‘when,’ ‘who’, and ‘how’ of a specific intervention that will optimize the educational outcome? The described evolution of the bovine PD challenge shows how these questions were investigated for transrectal palpation skills, and how changes were implemented to optimize student training. This is an example of an approach that could be applied widely for validation and skills training investigations within veterinary education to optimize learning outcomes.

Keywords: Teaching intervention, bovine pregnancy diagnosis, transrectal palpation, clinical hands-on skills

 

Introduction

By definition, a teaching intervention is ‘a specific program or set of steps to address an academic need.’ In educational settings, these interventions are often focused teaching sessions that are different from existing or conventional teaching practices. With the worldwide shift from mainly knowledge-based to competency-based veterinary courses,^1–8 many new and innovative teaching interventions have been introduced into veterinary education in the past 20 years. These changes have resulted in veterinary schools graduating entry-level veterinarians with advanced skills.^9,10 The increased expectations of veterinary graduates, as highlighted by the ‘day one competency and year one skills’ lists published by veterinary governing bodies, include not only theoretical knowledge but also professional attitudes and clinical hands-on skills.^1–4,9 Traditionally, hands-on experience was ensured through live animal and cadaver exposure, but more recently, skills acquisition opportunities have become limited by welfare and ethical concerns about the use of live animals for training purposes, large student cohort sizes, budget constraints, and difficulty sourcing cadaver materials.^6,9,11–21 The development of a variety of teaching interventions of varying fidelity levels was inspired by balancing the demands of a strongly skills-oriented veterinary education with respecting societal expectations of higher ethical standards. These teaching interventions include but are not limited to simulators, models, skills laboratories, virtual realities, applications, online programs, and computer-assisted learning tools^{9,11–13,15,22,23}; of which, many have been implemented into various veterinary training programs.^24–33 While these advances are noteworthy, the question still remains as to whether or not the implementation of these teaching interventions is optimal. Do we know the ‘when,’ ‘who,’ and ‘how’ of a specific intervention that will optimize the educational outcome? This article describes the implementation of an educational intervention over an 8-year period, and how evidence-based changes were applied to ensure the best learning outcome for bovine pregnancy diagnosis (PD) via transrectal palpation (TRP).

PD via TRP is an important competency for graduating veterinarians planning to work in large animal practice. This skill is, however, challenging to teach, and the competence level of students after TRP training is difficult to assess. Therefore, the ‘pregnancy diagnosis challenge’ (PD challenge) was developed. The aims of this teaching intervention were to establish a way of accurately measuring students’ TRP competence and to incentivize student learning by identifying the PD challenge champion who, based on the PD assessments done on live cows, had the highest PD accuracy (sensitivity and specificity).^34,35 At the same time, a longitudinal research project was implemented to enable the evaluation of the learning outcome as well as various teaching and training interventions aimed at fast-tracking PD competency.^34,36–39

Design and evidence-based evolution of the pregnancy diagnosis challenge and results

The PD challenge was set up to consist of PD via TRP training, followed by a knock-out competition where students were graded on their overall PD accuracy, sensitivity, and specificity in live cow TRPs. The setup of the PD assessment has been described.^34–36 The best participants of the first round took part in the final round of the PD challenge, where they performed more PDs in both beef and dairy cattle within a set time limit, which made it possible to identify the most accurate student under time pressure and crown the PD challenge champion. The University of Pretoria’s 6-year Bachelor of Veterinary Science (BVSc) program includes 9 semesters of didactic preclinical training and 3 semesters of clinical, work-integrated learning during the last 18 months of study.⁴⁰ The PD challenge was part of the preclinical training in 2014 and 2016 and was moved to later stages of the curriculum as part of the work-integrated learning from 2018 onward.

The first 2 PD challenges (2014 and 2016) were designed to determine if PD accuracy could be reliably and efficiently measured in a large group of students, and if so, to establish baseline values for student PD accuracy.^34,35,36 The 2^nd to 4^th PD challenges (2016, 2018, and 2019) were used to answer the questions of ‘when’ the PD challenge should be done (curricular implementation), ‘who’ (which students should participate), and ‘what’ in terms of training was best to optimize the learning outcomes by adjusting student cohorts, training methods, and student exposure.^36,37 More recent iterations of the PD challenge (5^th and 6^th versions, 2021 and 2022, respectively) included the implementation of known methods to improve PD accuracy based on our previous research, and the use of an objective structured clinical examination (OSCE)-like assessment on TRP of non-pregnant cows,³⁵ to tailor 2 parallel programs within the PD challenge for students who are either already competent in bovine TRP or who still need improvement.

The first PD challenge in 2014 was done within 1 semester of the didactic 4^th year veterinary reproduction module, which included the initial student TRP and PD training for the entire 4^th year cohort (n = 138).³⁴ Until 2014, students’ posttraining TRP skill level and PD accuracy had not been measured, and the outcome of the training was unknown. The initial TRP training consisted of palpation of bovine abattoir-derived reproductive organs and university-owned non-pregnant cattle and was followed up by PD training on either Breed’n Betsy^®a rectal examination simulators or on beef cows at a university satellite facility.³⁴ The first PD challenge therefore gave a baseline and insight into our 4^th year students’ PD capabilities and allowed for the evaluation of simulator versus live cow PD training.³⁴ While the 4^th year student TRP competence is not expected to be at the same level as that of experienced large animal practitioners who routinely perform PDs, a specificity (defined as students’ ability to correctly identify non-pregnant cows) of 41% indicated the priority area that needed improvement (Table).

Table. Overall student PD accuracy, sensitivity, and specificity for 4^th, 5^th, and final year student cohorts

	2014	2016	2018	2019
	4th yearstudent cohort	5th yearstudent cohort	Final yearstudent cohort	Final yearstudent cohort
Pregnancy rate (%)	62	49	64	66
Cows ≤ 6 months pregnant (%)	46	70	69	64
Overall student PD accuracy (%)	71	61	82	85
Sensitivity (%)	87	79	88	89
Specificity (%)	41	42	67	79

In order to expose students to additional ‘refresher’ training on rectal examination simulators and TRP in live, non-pregnant cows, the 2016 PD challenge was moved to the 5^th year student cohort (n = 128) after the initial TRP and PD training in their 4th year.^35,36 Interestingly, this additional training did not appear to improve student performance or improve PD accuracy (Table).

From 2018 onward, the PD challenge was organized as a 2-week block. The first week entailed an intense training program dedicated to TRP training and consisted of a mandatory day of supervised bovine TRP and PD training at the teaching institution, supervised and executed by experienced faculty. The training included inspection and palpation of abattoir-derived specimens of bovine female reproductive organs, skills laboratory exposure to rectal examination simulators (Breed’n Betsy^®a and Haptic Cow^b), and exposure to theoretical information on bovine TRP and PD for self-study. Students were then exposed to 2 days of live cow PDs in large animal practices, which was supervised by experienced veterinarians. The training was then followed by the first and final round of knock-out competitions during the second week.³⁷ The PD challenge was also moved from 5^th to final year and was made available only to students selecting the production elective (n = 59) during their 3 semesters of workplace-based training. During the 2018 PD challenge, students were given focused access to live cow palpations during this later stage of their curriculum. Results of PD assessments performed during the 2014, 2016, 2018, and 2019 PD challenges are summarized (Table).

Discussion

Results demonstrated that the various training interventions and advancing from preclinical to clinical student cohorts for TRP and PD training improved students’ PD accuracy. Improvement in specificity is particularly encouraging and shows that in 2014, students would only have recognized 1 out of 3 (33%) non-pregnant cows correctly,³⁴ in 2018, this had advanced to 2 out of 3 (66%),³⁷ and to 4 out of 5 (80%) in 2019. The differing pregnancy rate and range of pregnancy stages in 2014 and 2016 can account for the discrepancies in student PD accuracy and sensitivity across these 2 years. In 2014, the pregnancy rate was 62% compared to 49% in 2016, with 46% and 70% of pregnant cows < 6 months pregnant for 2014 and 2016, respectively (Table). Both sets of results showed that students’ PD sensitivity is higher in cows > 6 months pregnant,^34,36 so the higher percentage of cows < 6 months pregnant in combination with the lower pregnancy rate in general in the 2016 study made a correct diagnosis more difficult and explains the lower student PD accuracy and sensitivity.

The stage-corrected sensitivity (defined as the proportion of pregnant cows in which pregnancy stage was correctly identified by the student) was 43% for final-year students who had received a 1-week intense TRP and PD training program (2018).³⁷ This was higher than the reported 31% for a 5^th student cohort (2016)³⁶ but is an area that could still be improved.

From 2018 onward, only students who had chosen a production animal as elective as part of their 3 final year work-integrated learning semesters were able to enroll in the 1 week intense bovine PD training.³⁷ The student cohorts were therefore smaller (n = 59) and were composed of students with a specific interest in food animal practice, usually with additional hands-on clinical experience in production animals. The majority of these student participants had been exposed to bovine TRPs and PDs during their clinical rotations in reproduction, herd health, as well as private practice placements. Another change from 2018 onward was to provide students with access to live cow palpation opportunities in large animal private practices and to take this training load away from the university.³⁷ Participating veterinary practices were identified and contacted by the PD challenge organizers to ensure real-life workplace-based exposure for PD challenge students. Financial support for student travel was secured through the teaching institution and allocated on an individual student basis.

Placing the PD challenge into the final-year of veterinary training has subsequently been proven successful, and the overall PD accuracy, sensitivity, and specificity were higher in 2018 and 2019 compared to previous student cohorts (Table). The advancement of PD specificity from 41% in 2014 to 79% in 2019 was a major improvement in this important variable and highlighted the importance of ‘when,’ ‘who,’ and ‘what’ for enhancing training outcomes. Some changes in 2018 included the need for students to pass a validated TRP OSCE³⁵ prior to being allowed to proceed to the allocated 2 days of live-cow palpation practice with a private practitioner before the PD challenge. In 2019, the live-cow TRP exposure at private practices was increased from a 2 day to a 4 day period.

In 2020, the PD challenge was cancelled due to the disruptions in the clinical training program brought about by the COVID-19 pandemic; however, in 2021, the PD challenge was resumed in a similar format to that of 2019. Over the years, the small number of finalists identified during the OP PD challenge received additional exposure when they qualified for the final round of the challenge, during which, the PD champion was determined. After the 2021 PD challenge, the organizers realized, however, that those students who did not qualify for the final round are in fact the ones who would benefit most from additional exposure, and the program was therefore again revised for the 2022 season by making 2 pertinent changes. The first change was that students were allowed to spend 1 week in private practice for TRP training prior to the PD challenge week, and this placement week could be from 4 months prior to the PD challenge. This was done in response to comments received from private veterinarians, indicating that the placements did not always happen at a convenient time in their herd health programs, and this change made it possible for students to attend practices during weeks when they could benefit the most from the exposure provided by the practitioner. The second important change made in 2022 was that the 1 week program of the PD challenge was started with an OSCE-type assessment on a live non-pregnant cow, where students were assessed on their TRP skills. This OSCE-type assessment was previously validated and is known to correlate with PD accuracy.³⁵ Based on the outcome of this OSCE-type assessment, students were assigned to 2 groups: ‘challengers’ and ‘qualifiers’. The latter group was those with lower assessment scores, and an alternate program was designed for them to ensure more structured and intensively guided TRP training via ultrasonography on non-pregnant cows to provide immediate and self-driven feedback, and intensive one-on-one PD training on cows in the third trimester of pregnancy in one of the institution’s training herds. The ‘qualifiers’ then had a PD challenge on their own but on the same herd as those palpated by the ‘challengers.’

Some of the challenges encountered during implementation of this teaching intervention included how to accurately measure student skill level for PD via TRP after training, how to collect research data while teaching and how to motivate students to perform at their best to gather good and meaningful research data. To increase student motivation and incentivize students, through industry partner involvement, prestigious prizes were offered to the PD champion.

Summary

The PD challenge was initially set up to include entire student cohorts (4^th and 5^th year student cohorts in 2014 and 2016 with 138 and 128 students, respectively) and moved subsequently to only enable students with an active interest in production animal practice as a career choice as part of the elective program to enroll. This shift allowed for more intense training and exposure of these students and allowed the organizers to overcome the challenge of limited training opportunities outside the veterinary training institution. An iterative process of continuously assessing learning outcomes and modifying our training approach allowed us to answer the following questions for this specific teaching intervention:

‘When’ (curricular implementation): as part of the elective during the final 3 semesters of workplace-based experience;

‘Who:’ final year students with an interest in large animal practice;

‘What:’ an intense 1 week training program including live animal TRP and PD exposure as well as voluntary repeat/refresher model and simulator training.

The PD challenge is now an annual elective event and shows encouraging results in assisting students to become competent in bovine PD via TRP while overcoming some of the previously reported challenges in live animal exposure. The development of this teaching intervention and the changes implemented over a number of years in response to varying challenges could be used as an example for other scenarios to optimize learning outcomes.³⁹ Furthermore, the concept of ‘blending a fun student project with research while teaching’ makes best use of both time and effort in collecting research data while also allowing for teaching and motivating students to perform at their best.³⁹ This approach could be implemented widely for validation and skills training investigations within veterinary education.

Authors’ contributions

Annett Annandale: conceptualization, methodology, investigation, resources, writing, review, and editing.

Henry Annandale: initial teaching intervention idea, conceptualization, methodology, writing, review, and editing.

Kate May: methodology, investigation, writing, review, and editing.

Dietmar Holm: conceptualization, methodology, investigation, resources, writing, review, and editing.

References

1.	Bok HGJ, Jaarsma DADC, Teunissen PW, et al: Development and validation of a competency framework for veterinarians. J Vet Med Educ 2011;38:262–269. doi: 10.3138/jvme.38.3.262
2.	Bok HGJ, Teunissen PW, Boerboom TBB, et al: International survey of veterinarians to assess the importance of competencies in professional practice and education. J Am Vet Med Assoc 2014;245:906–913. doi: 10.2460/javma.245.8.906
3.	May SA, Silva-Fletcher A: Scaffolded active learning: nine pedagogical principles for building a modern veterinary curriculum. J Vet Med Educ 2015;42:332–339. doi: 10.3138/jvme.0415-063R
4.	Read EK, Hecker KG: The development and delivery of a systematic veterinary clinical skills education program at the University of Calgary. J Vet Sci Tech 2013;S4:004. doi: 10.4172/2157-7579.S4-004
5.	Sabin EA, DeHaven WR: Veterinary education: a basis for good governance leading to effective veterinary services. Rev Sci Tech 2012;31:519–531. doi: 10.20506/rst.31.2.2137
6.	Kneebone R, Baillie S: Contextualized simulation and procedural skills: a view from medical education. J Vet Med Educ 2008;35:595–598. doi: 10.3138/jvme.35.4.595
7.	Kelly A, Heider LE, Prasse KW: Renewed directions in veterinary medical education. J Am Vet Med Assoc 2003;222:1511–1512. doi: 10.2460/javma.2003.222.1511
8.	Prasse KW, Heider LE, Maccabe AT: Perspectives in professional education – Envisioning the future of veterinary medicine: the imperative for change in veterinary medical education. J Am Vet Med Assoc 2007;231:1340–1342. doi: 10.2460/javma.231.9.1340
9.	Dilly M, Read EK, Baillie S: A Survey of established veterinary clinical skills laboratories from Europe and North America: present practices and recent developments. J Vet Med Educ 2017;44:580–589. doi: 10.3138/jvme.0216-030R1
10.	Malone E: Challenges & Issues: Evidence-based clinical skills teaching and learning: what do we really know? J Vet Med Educ 2019;46:379–398. doi: 10.3138/jvme.0717-094r1
11.	Baillie S: Utilisation of simulators in veterinary training. Cattle Prac 2007;15:224–228.
12.	Root Kustritz MV: Simulations in veterinary education. Clin Theriogenol 2014;6:607–610.
13.	Scalese RJ, Issenberg SB: Effective use of simulations for the teaching and acquisition of veterinary professional and clinical skills. J Vet Med Educ 2005;32:461–467. doi: 10.3138/jvme.32.4.461
14.	Iki Y, Ito T, Kudo K, et al: Animal ethics and welfare education in wet-lab training can foster residents’ ethical values toward life. Exp Anim 2017;66:313–320. doi: 10.1538/expanim.17-0026
15.	Valliyate M, Robinson NG, Goodman JR: Current concepts in simulation and other alternatives for veterinary education: a review. Vet Med 2012;57:325–337. doi: 10.17221/6261-VETMED
16.	Kneebone R, ApSimon D: Surgical skills training: simulation and multimedia combined. Med Educ 2001;35:909–915. doi: 10.1046/j.1365-2923.2001.00997.x
17.	de Boo J, Knight A: ‘Concepts in animal welfare’: a syllabus in animal welfare science and ethics for veterinary schools. J Vet Med Educ 2005;32:451–453. doi: 10.3138/jvme.32.4.451
18.	Weichbrod RH, (Heidbrink) Thompson GA, Norton JN: editors. Management of Animal Care and Use Programs in Research, Education, and Testing. 2nd edition, Boca Raton; CRC Press: 2017. doi: 10.1201/9781315152189
19.	Martinsen S, Jukes N: Towards a humane veterinary education. J Vet Med Educ 2005;32:454–460. doi: 10.3138/jvme.32.4.454
20.	Wood MW, Hart LA, Weng H-Y: Effective bibliographic searching for animal alternatives in veterinary medical education: The UC Davis web site. J Vet Med Educ 2005;32:468–472. doi: 10.3138/jvme.32.4.468
21.	Pereira GdG, Dieguez FJ, Demirbas YS, et al: Alternatives to animal use in veterinary education: a growing debate. Ank Univ Vet Fak Derg 2017;64:235–239. doi: 10.1501/Vetfak_0000002804
22.	Langebaek R, Eika B, Jensen AL, et al: Anxiety in veterinary surgical students: a quantitative study. J Vet Med Educ 2012;39:331–340. doi: 10.3138/jvme.1111-111R1
23.	Langebaek R, Eika B, Tanggaard L, et al: Emotions in veterinary surgical students: a qualitative study. J Vet Med Educ 2012;39:312–321. doi: 10.3138/jvme.0611.068R1
24.	Baillie S, Crossan A, Brewster S, et al: Validation of a bovine rectal palpation simulator for training veterinary students. Stud Health Technol Inform 2005;111:33–36.
25.	Baillie S, Mellor DJ, Brewster SA, et al: Integrating a bovine rectal palpation simulator into an undergraduate veterinary curriculum. J Vet Med Educ 2005;32:79–85. doi: 10.3138/jvme.32.1.79
26.	Langebaek R, Toft N, Eriksen T: The SimSpay-student perceptions of a low-cost build-it-yourself model for novice training of surgical skills in canine ovariohysterectomy. J Vet Med Educ 2015;42:166–171. doi: 10.3138/jvme.1014-105
27.	Read EK, Vallevand A, Farrell RM: Evaluation of veterinary student surgical skills preparation for ovariohysterectomy using simulators: a pilot study. J Vet Med Educ 2016;43:190–213. doi: 10.3138/jvme.0815-138R1
28.	Greenfield CL, Johnson AL, Schaeffer DJ, et al: Comparison of surgical skills of veterinary students trained using models or live animals. J Am Vet Med Assoc 1995;206:1840–1845.
29.	Gopinath D, McGreevy PD, Zuber RM, et al: Developments in undergraduate teaching of small-animal soft-tissue surgical skills at the University of Sydney. J Vet Med Educ 2012;39:21–29. doi: 10.3138/jvme.0411.044R
30.	Griffon DJ, Cronin P, Kirby B, et al: Evaluation of a hemostasis model for teaching ovariohysterectomy in veterinary surgery. Vet Surg 2000;29:309–316. doi: 10.1053/jvet.2000.7541
31.	Holmberg DL, Cockshutt JR, Basher AWP: Use of a dog abdominal surrogate for teaching surgery. J Vet Med Educ 1993;20:61–62.
32.	Fahie M, Cloke A, Lagman M, et al: Training veterinary students to perform ovariectomy using the MOOSE spay model with traditional method versus the Dowling Spay Retractor. J Vet Med Educ 2016;43:176–183. doi: 10.3138/jvme.0915-150R
33.	Annandale A, Scheepers E, Fosgate GT: The effect of ovariohysterectomy model practice on surgical times for final-year veterinary students’ first live-animal ovariohysterectomies. J Vet Med Educ 2020;47:44–55. doi: 10.3138/jvme.1217-181r1
34.	Annandale A, Annandale CH, Fosgate GT, et al: Training method and other factors affecting student accuracy in bovine pregnancy diagnosis. J Vet Med Educ 2018;45:224–231. doi: 10.3138/jvme.1016-166r1
35.	Annandale A, Fosgate GT, Bok H, et al: Ability of a bovine transrectal palpation objective structured clinical examination to predict veterinary students’ pregnancy diagnosis accuracy. Vet Rec 2019;185:171–171. doi: 10.1136/vr.105022
36.	Annandale A, Fosgate GT, Eksteen CA, et al: Influence of an exercise program, muscle strength, proprioception, and arm length on veterinary students’ bovine pregnancy diagnosis accuracy. J Vet Med Educ 2020;e20190043.
37.	Annandale A, May CE, van der Leek ML, et al: Effect of a high-intensity one-week training programme and student-level variables on the bovine transrectal palpation and pregnancy diagnosis skills of final-year veterinary students. Vet Rec 2020;187:e99. doi: 10.1136/vr.105909
38.	Annandale A, Fosgate GT, Eksteen CA, et al: Electromyographic analysis of muscle activation patterns during bovine transrectal palpation and the development of the bovine pregnancy diagnosis improvement exercise program. J Vet Med Educ 2021;48:686–697. doi: 10.3138/jvme-2020-0039
39.	Annandale A: Thinking differently about clinical skills training: The bovine pregnancy diagnosis via transrectal palpation showcase. Utrecht University: 2020. [cited 2020 July 6] Available from: https://dspace.library.uu.nl/handle/1874/397523.
40.	Irons PC, Holm DE, Annandale CH: Curricular renewal at the southern tip of Africa: the 2016 veterinary curriculum at the University of Pretoria. J Vet Med Educ 2017;44:440–449. doi: 10.3138/jvme.0316-062R