Table 2 Results of the included studies

1

Abdulla et al. 2020 [40]

n = 64 cohort 1 pre and post test

25–33 = 40 34–44 = 21 45–54 = 3

M = 4 F = 60

Knowledge Percentages presented for each question

n = 56 cohort 2 same pre and post-test as cohort 1

25–33 = 28 34–44 = 22 45–54 = 6

M = 3 F = 53

Knowledge Percentages presented for each question

Pre/post results presented for Cohort 1 and 2, but not between cohorts

Significant Diff

2

Alwy Al-Beity et al. 2020 [41]

n = 636

Not stated

Not stated

Knowledge—74.2 mean at pre-training, 89.2 mean immediately after the training. Skill acquisition—increased from 38.2 mean to 85.4 immediately post-training

n = 193 (10-month assessment in a subset of health workers)

Not stated

Not stated

Knowledge—85.4 mean at the 10-month follow-up. Skill acquisition—significant overall decline of skills at 10-month follow-up from 85.4 mean to 80.8

Knowledge: overall scores increased from 78 to 93% (p < 0.0002). Skill acquisition: scores increased from 38 to 83% (p < 0.000)

Significant Diff

3

Ayisi-Boateng et al. 2022 [42]

n = 49

Mean 34.6 (± 6.82)—range 22–50 years

M = 24 F = 25

19.8 (± 4.3) out of 30 Score relates to participants’ knowledge

n = 49

mean 34.6 (± 6.82)—range 22–50 years

M = 24 F = 25

23.2 (± 4.0) out of 30 Score relates to participants’ knowledge

increase in the proportion of participants who had correct answers in all the seven domains (p < 0.01)

Significant Diff

4

Azoulay et al. 2021 [43]

Resident numbers not provided

Not stated

not stated

Number of surgeries: 2/138 (0.1%) hepatectomies during period 1 (first 2 years). No WhatsApp HPB Group in period 1. Conferences = no formal conference/presentations in period 1. Publications = 2

Resident numbers not provided

Not stated

Not stated

Number of surgeries: of 81/188 (43.1%) hepatectomies during period 2. WhatsApp HPB Group in period 2 very active. Conferences = 11. Presentations = 7. Publications = 12

Number of surgeries (representing increase in knowledge): 40-fold increase. Publications/presentations = sixfold increase

Significant Diff

5

Bennett et al. 2022 [44]

n = 160

Not stated

Not stated

CCS Total score (median) = 81 (range 18–90) P-CAT Total Score (median) = 43 (range 13–65)

n = 160

Not stated

Not stated

CCS Total score: median = 85 (range 63–90) P-CAT Total Score: median = 44 (range 29–57)

Difference in CCS Total score = p = 0.007; P-CAT Total Score = P ≤ 0.001

Significant Diff

6

Chicoine 2022 [45]

n = 28

39.1 mean

M = 1 F = 27

Self-efficacy = 7.8 (least square means)

n = 19 (6 month) n = 12 (12 month)

Not Stated

Not Stated

Self-efficacy—6 months = 7.8; 12 months = 7.9

significant changes in self-efficacy at 12-month follow-up (P = 0.0213),

among the nurses who attended more than 25% of the 20-session curriculum

Significant Diff

7

Clancy et al. 2020 [46]

n = 1450

64% aged 51 + years

M = 0 F = 1450

Competency/knowledge Participants reported low to moderate confidence in their ability to recognize or respond to signs/symptoms of early relational trauma in families

n = 734 (post) n = 651 (follow-up at 2–3 months post)

Not stated

M = 0 F = 734 (post) F = 651 (follow-up)

Competency/knowledge Increases in confidence and capability

Competency/knowledge Increases in confidence and capability (p < 0.01)

Significant Diff

8

Dierkes et al. 2022 [47]

Not stated

Not stated

Not stated

HPPD pre-mandate (mean): California = 6.03; other States = 6.03

Not stated

Not stated

Not stated

HPPD pre-mandate (mean): California = 7.90; other States 6.73

Not reported; states p < 0.05

Significant Diff

9

Downing et al. 2016 [48]

n = 27

Not stated

M = 1 F = 26

Confidence/competence = least confident in morphine prescribing (mean = 2.32), models of palliative care (mean = 2.48), end-of-life care (mean = 2.68) and bereavement support (mean = 2.76)

n = 25

Not stated

Unclear

Confidence/competence = least confident in morphine prescribing (mean = 2.32), models of palliative care (mean = 2.48), end-of-life care (mean = 2.68) and bereavement support (mean = 2.76)

Confidence/competence = p < 0.001 for what is palliative care, concept of total pain, models of palliative care provision, basic communication, bereavement support, pain assessment and

management, Morphine prescribing, end of life care, caring for children

Significant Diff

10

Gajewski et al. 2019 [49]

n = 8

Not stated

Not stated

Knowledge demonstrated through increase surgeries; caesarean sections pre results: = 990; post results: 525 (− 47% change); Common surgeries pre results: = 417; post results: 437 (+ 4.8% change)

n = 9

Not stated

Not stated

Knowledge demonstrated through increase surgeries; caesarean sections pre results: = 900; post results: 1037 (15.2% change); Common surgeries pre results: = 508; post results: 483 (− 4.9% change)

In 5 pairs intervention hospitals performed more caesarean sections (p = 0.015)

Significant Diff

11

Gordon et al. 2022 [50]

General navigation n = 163; Documentation n = 158; Order entry = 150; Medications = 150; In-basket = 163; Reports = 132

Not stated

Not stated

PS (baseline mean scores) General navigation 58.5; Documentation 57.7; Order entry 56,4; Medications 57.4; In-basket 52.3; Reports 24.1

General navigation n = 163; Documentation n = 158; Order entry = 150; Medications = 150; In-basket = 163; Reports = 132

Not stated

Not stated

PS (1 month post mean scores) General navigation 72.0; Documentation 70.4; Order entry = 71.0; Medications = 70.1; In-basket = 70.5; Reports = 34.2

Before to after reBoot camp: p < 0.001 for all domains; Sustained at 6 months for all domains

Significant Diff

12

Islam et al. 2020 [51]

n = 1290 (n = 471 face to face; n = 819 step 1 online*) * n = 443 at step 2; n = 307 step 3; n = 253 step 4

Not stated

Not stated

1. Self-perceived levels of importance, knowledge and confidence = individual statistics not provided (in graph format). 2. 4 step assessments (average): Step 1 = 0.93; step 2 = 1.20; step 3 = 1.09; step 4 = 0.78

Post: n = 471 face-to-face 6 months follow-up: unclear

Not stated

Not stated

1. Self-perceived levels of importance, knowledge and confidence: improvement in all areas; 2. 4 step assessments at follow-up only (average): Step 1 = 2.27; step 2 = 2.42; step 3 = 2.36; step 4 = 2.15

1. Self-perceived levels of importance, knowledge and confidence = differences between pre and post scores significant all 3 variables; step 1 = 1.34 p < 0.01; step 2 = 1.21 p < 0.01; step 3 = 1.26 p < 0.01; step 4 = 1.37 p < 0.01

Significant Diff

13

Jafari et al. 2020 [52]

n = 62

Not stated

Not stated

1. Self-efficacy (medians values provided for 15 questions). Frequency of practice behaviours (medians values provided for 11 questions)

n = 62

Not stated

Not stated

1. Self-efficacy (medians for 15 questions): p < 0.05. Frequency of practice behaviours (medians for 11 questions): 8 of 11 questions p < 0.05

Self-efficacy significantly increased across all 15 competencies (p < 0.05). Frequency of 8 out of 11 practice behaviours increased significantly (p < 0.05)

Significant Diff

14

Jedwab et al. 2022 [53]

p = 550

37.89 mean

M = 47 F = 491 Other = 8 Missing = 4

1. Well-being: well-being index = 64.00. Maslach Burnout: exhaustion = 1.67, cynicism = 1.33, reduced efficiency = 1.67. 2. Work engagement: Satisfaction = 7.81. Intention to stay = 8.10. Utrecht Work Engagement Scale—vigour = 3.40, dedication = 4.30, absorption = 4.24. Career trajectory satisfaction = 3.65. Perceived psychological safety = 2.91. Motivation to use technology: perceived confidence = 3.36, perceived external drivers = 0.02

n = 392

39.36 Mean

M = 32 F = 352 Other = 6 Missing = 2

1. Well-being: well-being index = 56.00. Maslach Burnout: exhaustion = 2.00, cynicism = 1.33, reduced efficiency = 2.33. 2. Work engagement: Satisfaction 6.99. Intention to stay = 7.53. Utrecht Work Engagement Scale—vigour = 3.03, dedication = 3.98, absorption = 4.12. Career trajectory satisfaction = 3.34. Perceived psychological safety = 2.98. Motivation to use technology: perceived confidence = 3.57, perceived external drivers = − 0.23

Work satisfaction (r = 0.23, p ≤ 0.001), intention to stay (r = 0.11, p = 0.001) and well-being (r = 0.17, p ≤ 0.001) decreased, perceived competence increased (r = 0.10, p = 0.002) despite decreased autonomy (r = 0.10, p = 0.003). Two of three dimensions of work engagement decreased (vigour r = 0.13, p ≤ 0.001; dedication r = 0.13, p ≤ 0.001), all burnout dimensions increased (exhaustion r = 0.08, p = 0.012, cynicism r = 0.07, p = 0.04 and reduced efficiency r = 0.32, p ≤ 0.001). More burnout symptoms reported (95% CI 4.6–4.7%, p = 0.036), were less engaged (95% CI 49.6–49.9%, p  ≤ 0.001) and career trajectory satisfaction decreased (r = 0.15, p  ≤ 0.001)

Significant Diff

15

Johnston et al. 2020 [54]

n = 144 (at entry to medical program) NOTE: only n = 144 completed pre and post data

Cannot determine n = 144 as data for all entry surveys n = 3851 combined

Cannot determine n = 144 as data for all entry surveys n = 3851 combined

Intention to practice in rural and other underserved areas: n = 144 individual entry data not presented

n = 144 (at exit from medical program)

Cannot determine n = 144 as data for all exiting surveys n = 1187 combined

Cannot determine n = 144 as data for exiting surveys n = 1187 combined

Intention to practice in rural and other underserved areas: n = 144 individual exit data not presented

No significant change in proportion of learners intending to practice in rural areas p = 0.644

Non-significant Diff

16

Martin et al. 2019 [55]

n = 787

Not stated

Not stated

Confidence in skills and knowledge (pre-course): e.g., I have the necessary knowledge to help my clients with tobacco-related issues (51.7% agree);I am confident in my ability to address tobacco use (54.5% agree)

n = 765 (post) n = 416 (follow-up)

Not stated

Not stated

Confidence in skills and knowledge (post-course): e.g., I have the necessary knowledge to help my clients with tobacco-related issues (98.8% agree); I am confident in my ability to address tobacco use (98.6% agree)

Pre–post p < 0.001 for all confidence in skills and knowledge questions

Significant Diff

17

Mikolajczyk et al. 2021 [56]

n = 27

Not stated

M = 10 F = 17

Knowledge: mean percentage of MC questions × 12 correct = 55%

n = 59

Not stated

M = 25 F = 34

Knowledge: significant improvement in self-perceived knowledge across all CLD topics before the intervention cohort’s completion of residency

Knowledge: mean percentage of questions answered correctly by the third-year residents in the intervention cohort was 7.8 out of 12 (65%) compared to 6.8 out of 12 (55%) in the historic cohort (p = 0.04)

Significant Diff

18

Morshed et al. 2017 [57]

n = 201

Equally distributed 30–60 + years; less represented 20–29 years

M = 33 F = 168

NO Advanced degree (mean): 1. Skill = 6.61 2. Importance = 9.92 Advanced degree (mean): 1. Skill = 8.06 2. Importance = 10.11

n = 123

Equally distributed 30–60 + years; less represented 20–29 years

M = 18 F = 105

NO Advanced degree: 1. Skill = 7.40 2. Importance = 9.98 Advanced degree: 1. Skill = 8.03 2. Importance = 10.41

NO Advanced degree: 1. Skill p = 0.016 2. Importance p = 0.736 Advanced degree: 1. Skill p = 0.927 2. Importance p = 0.059

Non-significant Diff

19

Murthy et al. 2020 [58]

LF GROUP n = 107

not stated

M = 82 F = 25

1. CBE exam scores (mean): exam 1 = 5.77; exam 2 = 16.44; exam 3 = 23.79; exam 4 = 23.81

HF GROUP n = 107

Not stated

M = 77 F = 30

1. CBE exam scores (mean): exam 1 = 5.68; exam 2 = 17.30; exam 3 = 23.77; exam 4 = 23.92

Mean difference in exam scores between HF and LF models in exam 1 to 4 was not significantly different

Non-significant Diff

20

Neikrug et al. 2022 [59]

n = 251

Average = 44.7 years

M = 73 F = 177 Decline to state = 1

1. Knowledge = baseline to midpoint (mean difference = 7.6%, t = 10.6, p < 0.0001)

n = 251

Average = 44.7 years

M = 73 F = 177 Decline to state = 1

1. Knowledge = midpoint to post fellowship (mean difference  = 4.23%, t = 5.59, p < 0.0001)

1. Knowledge = Repeated-measures analysis of the percent of correct answers on the knowledge score yielded significant improvement across the entire year (mean difference = 11.8%, t = 15.76, p < 0.0001)

Significant Diff

21

Ortega et al. 2018 [60]

Cohort 1 = 58 Cohort 2 = 111 Cohort 3 = 120 TOTAL = 289

(Majority): Cohort 1 = 72% 41–55 years Cohort 2 = 56% 41–55 years Cohort 3 = 50.8% 26–40 years

Cohort 1 = M = 3 F = 55 Cohort 2 = M = 13 F = 98 Cohort 3 = M = 9 F = 111 TOTAL = M = 25 F = 264

Learner performance pre-test/100%: Cohort 1 = (average) 65% Cohort 2 = 57.5% Cohort 3 = 53.4%

Cohort 1 = 48 Cohort 2 = 83 Cohort 3 = 89 TOTAL = 220

Not specific due to drop outs

Not specific due to drop outs

Learner performance post-test: Cohort 1 = 87% Cohort 2 = 89.3% Cohort 3 = 78.4% Mean total performance on modules: Cohort 1 = 95.4% Cohort 2 = 90.3% Cohort 3 = 89.6%

90% mean score on final exam NOTE: pre- and post-test data, therefore, were not individually matched; statistical significance could not be calculated

Non-significant Diff

22

Ortega et al. 2021 [61]

n-85

Not stated

Not stated

Only HuFSHI change scores provided (see difference between groups results)

n = 85

Not stated

Not stated

Only HuFSHI change scores provided (see difference between groups results)

Assessment and Mental State course:—t(7) = − 6.587, p < 0.000; Decision Making course: t(10) = − 4.411, p < 0.000; Place of Safety course: t(18) = − 4.932, p < 0.000; Crisis Resolution and Home

Treatment Teams course: t(16) =  − 4.737, p < 0.000

Significant Diff

23

Parmar et al. 2022 [62]

n = 161

Average 37 years

M = 59 F = 101

Knowledge and confidence = out of a score of 50 mean score 38.90

n = 161

Average 37 years

M = 59 F = 101

Knowledge and confidence = out of a score of 50 mean score 46.60

Post education scores significantly higher than pre P < 0.0001

Significant Diff

24

Playford et al. 2020 [63]

n = 776 (end of placement measure) as a student M = 124 F = 624; n = 474 (1 year post-graduation)—results presented are for n = 474

Not stated

M = 77 F = 393 (1 year follow-up)

Location of practice—urban or rural = 26% in rural practice; rural background had the strongest  relationship with early rural practice

n = 244 (15–17 year post-graduation)

Not stated

M = 77 F = 161 (15 –17 years follow-up)

Location of practice—urban or rural = most were practising in RA 1 (major cities) locations (193/240), with the remainder in (rural areas) RA 2 (23/240), RA 3 (19/240) and RA 4–5 (5/240). This gave a total of 47/240 (20%) practising rurally

Significant association between region practising 1 year post-graduation and region practising 15–17 year post-graduation (p < 0.001); significantly associated with long term rural practice were location of first job (p < 0.001) and rural background (p < 0.007)

Significant Diff

25

Risendal et al. 2022 [64]

n = 254

Not stated

M = 34 F = 220

Knowledge: percent of correct responses = 25%

n = 218

Not stated

M = 29 F = 189

Knowledge: Percent of correct responses = 46%

Knowledge: Percent of correct answers overall pre- to post-test p ≤ 0.0001. 14/15 were significant

Significant Diff

26

Salehi et al. 2021 [65]

n = 330 (for knowledge) (n = 293 for Confidence) (n = 74 for OSCE)

31.3 (mean)

M = 80 F = 250

Knowledge = 52% ± 11.2 Clinical skills = majority not competent at baseline (66% in physical assessment and 52% in communication)

n = 330 (for knowledge) (n = 293 for Confidence) (n = 74 for OSCE)

31.3 (mean)

M = 80 F = 250

Knowledge = 71% ± 9.2 Clinical skills competency: 96% in physical assessment, 99% in communication, and 100% in emergency

Knowledge = 37% increase, P = 0.000 Clinical skills 14 month follow-up = Physical Assessment 3.7 ± 0.4 (p = 0.1); Communication 3.5 ± 0.4 (p = 0.000); Emergency 3.4 ± 0.6 (p = 0.000)

Significant Diff

27

Sibrian et al. 2022 [66]

n = 50

Not stated

Not stated

Not stated. Nil results presented

n = 50

Not stated

Not stated

Numbers not provided

Not stated

Non-significant Diff

28

Tran et al. 2019 [67]

n = 86

Not stated

M = 6 F = 80

1. Leadership = 4.8 (mean) 2. Management = 3.8 (mean) 3. APN Specific = 4.1 (mean)

n = 86

Not stated

M = 6 F = 80

1. Leadership = 6.1 (mean) 2. Management = 5.6 (mean) 3. APN Specific = 5.8 (mean)

Mean scores p < 0.001

Significant Diff

29

Vesel et al. 2015 [68]

n = 129

Not stated

Not stated

1. Coping skills = average 2.79 2. Perceived stress levels (only post-test) = average 2.48

n = 157

Not stated

Not stated

1. Coping skills = positive and significant diffs from retrospective to post test (p = 0.000); pre = average 2.63, post = 3.23 2. Perceived stress levels (only post-test) average = 2.40

Overall Coping = On average higher coping strategy levels in intervention group (score of 3.23) vs comparison (2.79)—Significant diff (p = 0.000). On Average = lower stress levels in intervention group (score of 2.40) vs comparison (2.48)—Significant diff (p = 0.034)

Significant Diff

30

Zhang et al. 2021 [69]

n = 47

mean 38.3

M = 33 F = 14

(Perceived Stress Scale (PSS)-10,

Generalised Anxiety Disorder (GAD)-7) PRE-COVID: 1. Medical service statistics—Number of OP’s = 41.9 ± 11.9; LOS = 0.4 ± 1.0. 2. Workload—weekly working hours 1884.9 ± 34.1; 3. PSS = PSS-10: 4.3 ± 2.4; 4. GAD = 4.0 ± 2.3

n = 47

Mean 38.3

M = 33 F = 14

AFTER COVID OUTBREAK: 1. Medical service statistics—Number of OP’s = 37.6 ± 11.8 per week; LOS = 3.1 ± 3.9 days; 2. Workload—weekly working hours 2023.5 ± 67.3 h; 3. PSS = PSS-10: 7.5 ± 3.9; 4. GAD = 9.4 ± 4.0

AFTER COVID OUTBREAK: 1. Medical service statistics—p = 0.49; LOS = p = 0.02; 2. Workload—weekly working hours p < 0.001; 3. PSS = p < 0.001; 4. GAD = p < 0.001

Significant Diff