Table 3 Summary of papers by intervention
Citation | Design and quality | Participants and geographic location | Intervention and key contextual information | Outcome measures and findings |
|---|---|---|---|---|
Papers examining supervision | ||||
Lynch and Happell [33] | Qualitative – ‘exploratory’ approach: document analysis and interviews | Nurses (in mental health) | Intervention: Examination of the ‘process and journey’ of a clinical supervision implementation strategy (part I) (face to face) | Primary measure: Service and staff outcomes – factors identified that led to successful implementation of clinical supervision models |
Rural: Service examined has 3,000 registered clients covering 44,000 km2 | Contextual information: Five key stages of implementation were identified: Stage 1 – assessing the organisational culture and exploration of possibilities. Stage 2 – initial implementation strategy (need for leadership via leadership group, addressing issue with organisational culture, engagement of external organisation to provide a four-day supervision course for practitioners (where participants had to contribute to the overall strategic plan) and a one-day course for supervisors. Stages 3 to 5 in second article (below) | Positive impact: Large change of culture within the mental health programme. The estimated 80% of people initially negative and suspicious about clinical supervision was now estimated to be only 15% to 30%. Considerations: Strategies for sustainability developed included: continuity of review programme and leadership team (working group) to oversee actions and to work with senior management. | ||
Level III | Australia | |||
Lynch and Happell [34] | Qualitative – ‘exploratory’ approach: document analysis and interviews | Nurses in (mental health) | Intervention: Examination of the ‘process and journey’ of a clinical supervision implementation strategy (Part II) (face to face) | As above |
Level III | Rural: Service examined has 3,000 registered clients covering 44,000 km2 | Contextual information: Active involvement of staff in programme design and evaluation (dedicated ‘team’ of staff to undertake needs analysis (talking to staff, assessing workplace culture) and take control of decisions and implementation. External training for supervisors and supervisees in supervision; established a strategic plan; marketed the programme (official ‘launch’ of the programme, using a strategic plan to demonstrate organisational commitment); continual reflection and gathering of feedback; formal internal review of programme (demonstrated change, effect, impact on staff; clear leadership) | ||
Australia | ||||
English et al. [35] | Mixed methods – ‘following a thread’ | Multi-disciplinary | Intervention: Secondary analysis of data examining how the ‘inputs’ of supervision, feedback and facilitation affected implementation of best practice (face to face) | Primary measure: Staff outcomes (qualitative) – skills, satisfaction, (change in) attitude, leadership |
Level I | Rural | Contextual information: External support and organisational commitment ( external supportive supervision and local management and clear lines of communication regarding expectations established prior to programme); attributes of educator (facilitators were used within intervention hospitals); active involvement of stakeholders (‘health workers must not only know how to perform a task (for example, prescribing) but be willing to perform it’); networking and relationships (team working and integrated working associated with greater satisfaction) | Secondary measure: (qualitative) service outcomes – resource allocation, improved clinical systems | |
Kenya | Positive impact: A multi- faceted intervention strategy can change provider behaviours and improve the quality of inpatient care across a range of high mortality, target diseases. | |||
Considerations: In all settings, health worker motivation was a challenge | ||||
Xavier et al. [36] | Non-experimental – descriptive pre-and post-intervention evaluation | Psychologists and social workers | Intervention: Training, education and supervision. Clinical supervision and education provided by videoconference from a tertiary metro teaching hospital with individual telephone supervision each month (non-face-to-face: real-time videoconferencing) | Primary measures: Staff outcomes – number of participants, self-reported knowledge gains, self-reported confidence in management of particular conditions; satisfaction with the programme |
Level IV | Australia | Contextual information: Externally organised and supported: site coordinators were available to offer technical assistance at the majority of the locations. An administrative assistant was employed to coordinate the study; needs analysis was undertaken prior to the event | Positive impact: Significant increases in self-reported confidence in the areas covered by the educational component, for example assessing and treating pain in people with cancer (Po0.01). Self-assessment of overall effectiveness in current management of psychological distress from pre- to post-evaluation increased by 25%. Participants indicated that attending the educational sessions increased their knowledge (mean 1⁄4 7.3 out of 10). With regard to telephone supervision, most (80%) were very or extremely satisfied. The feedback indicated that remote supervision was well received and that participants were keen to continue their involvement. Overall: It is feasible and acceptable to provide clinical supervision and education via videoconference | |
Papers examining professional support | ||||
Conger and Plager [37] | Qualitative – phenomenology | Nurses | Intervention: Mechanisms promoting connectedness for masters students in rural areas were identified and explored (combination: face to face, telephone, email) | Primary measure: Staff outcomes – mechanisms that encouraged connectedness in rural areas |
Level II | Rural | Contextual information: Targeted development of support networks (relationships formed during study, other professionals in health centre, collaborative practice, mentoring); targeted development of relationship with large urban or metro health centres; targeted development of relationship with community; access to technology; avoiding mechanisms that promote disconnectedness (lack of relationships with health centres, poor avenues of communication with other health centres, lack of mentoring) | Positive impact: Connectedness enhanced by: development of support networks, relationships with large urban medical centres, availability of electronic communication and connections with the rural community. Graduates who reported a sense of disconnectedness when working in a rural community were less likely to remain in that community | |
USA | Negative impact: mechanisms that promote disconnectedness such as: lack of relationships with other health centres or poor communication avenues with other health-care centres; lack of mentoring (incidentally felt phone calls not enough) | |||
Teasley et al. [38] | Non-experimental – descriptive pre- and post-intervention evaluation | Nurses | Intervention: Nurses were requested to participate in meetings that generated and prioritised a list of interventions for implementation to improve perceptions of workload (face to face) | Primary measure: Staff outcomes – workload perceptions. |
Secondary measure: Staff outcomes – satisfaction and retention. | ||||
Level IV | Rural Kentucky: Community of 5,000, 60 miles from major metropolitan areas | Contextual information: Active involvement of stakeholders in programme design and evaluation; active involvement of staff in change process | Positive impact: Participant engagement in developing and implementing self-identified work environment issues led to improved workload, work satisfaction and intent to remain. | |
USA | ||||
Cameron et al. [39] | Qualitative – collective case study methodology | Medical practitioners, community members, spouses | Intervention: Exploration of community factors that promote physicians to practice and remain in a rural area (face to face) | Primary measure: Staff outcomes – factors that support retention of practitioners (this is also identified as a community outcome) |
Level I | Rural | Contextual information: Active involvement of stakeholders (as evidenced by ‘active support’ theme); networking and relationships (connection and reciprocity themes) | Positive impact: Four themes emerged. Appreciation, connection, active support (for the practitioner and pursuits of the practitioner for example defending health region) and physical and recreational assets were positively related to physician retention. These community factors existed to different degrees but were present in all communities. Reciprocity was a fifth factor that emerged. | |
Alberta, Canada | ||||
Healey-Ogden et al. [40] | Qualitative – interviews | Nursing | Intervention: Implementation of an 80/20 staffing model whereby staff have 20% of salaried time off from direct patient care to pursue professional development activities (face to face) | Primary measures: Staff outcomes – retention, knowledge, personal growth Secondary measures: Service outcomes – team engagement, quality of care, collaboration |
Level III | Rural | Contextual information: Organisational commitment (senior management and other partners on steering committee); external support (university, funded by Ministry for Health); accessible and adequate resources with 20% of time for CPD, training or supervision made available through creation of backfill positions (nurses were paid for their 20% time off clinical duties and could access funding to pay for travel and courses and so on); leadership (project coordinator was hired and utilised); flexibility (timing often mismatched between availability of backfill and course availability) | Positive outcomes: 4,000 hours of professional development and learning activities; positive effect on personal growth and work environment; improved job satisfaction and (unmeasured) intention to remain in job; perceived increase in quality of care; increased collaboration with staff of other hospitals and universities; team development | |
British Colombia, Canada | Considerations: Participants had scheduled professional development time during the summer, but most formal educational opportunities begin in September, hence professional development time and the availability of backfill staff did not always match; opportunities sparse in local area implies need for funding for travel and accommodation | |||
Papers examining training or education | ||||
Arora et al. [41] | Non-experimental – descriptive pre- and post-evaluation | Medical practitioners | Intervention: Use of a ‘telehealth clinic’ bringing together metro specialists and rural community based primary care providers to provide care to hepatitis C sufferers (non-face-to-face: real-time videoconferencing) | Primary measures: Service outcomes (from patient level) – efficiency, access and quality/completeness |
Secondary measures: Quality and completeness of health information and services received by clients | ||||
Level IV | Rural | Contextual information: Needs analysis; external support; financial support (three-year funding grant); regular feedback and evaluation opportunities; accessible and adequate resources (two-day orientation to technology and format of sessions); networking and relationships (development of ‘knowledge networks’ between practitioners of different specialities); application of formal learning strategy (learning loops) | Positive impact: Uniform agreement by participants – benefit to the practice and patients, expanded access to specialists, and the provider’s professional enhancement; significant increase in competency sustained for >12 months; competent to educate others; perceived improvements in patient safety and quality of care | |
New Mexico, USA | ||||
Bennett-Levy et al. [42] | Experimental – randomised controlled trial | Multi-disciplinary (psychologists, social workers, nurses, counsellors, medical practitioners) | Intervention: Online training programme for rural and remote mental health practitioners in cognitive behavioural therapy (CBT) (non-face-to-face: internet, video clips) | Primary measures: Staff outcomes – CBT knowledge, skills, confidence, utilisation and satisfaction with programme |
Level II | Urban, regional, rural | Contextual information: External support; accessible and adequate resources (discounted access to online learning programme provided); networking and relationships (15-min support sessions provided by experienced psychologist after each online learning module completed) | Positive impact: Participants in both groups improved their performance scores from pre-program to post-program and follow-up; supported training group was more likely to finish or very nearly finish (96%) than the independent group (76%) (c2 = 3.93, df = 1, P < .05); program characteristics, including the program design and content, proved highly acceptable; value of the 15-min support sessions was almost unanimously endorsed by the supported training group | |
Australia | ||||
Blattner et al. [43] | Qualitative – thematic analysis of interviews | Nurses and medical practitioners | Intervention: Staff at a rural hospital were trained in using a newly installed point of care test analyser (face to face) | Primary measures: Staff outcomes – change in practice behaviour, job satisfaction, process facilitators and barriers Secondary measures: Service outcomes – sustainability of intervention |
Level III | Remote | Contextual information: Access to training, skills, knowledge for the intervention (including refresher courses in interpreting tests); accessible and adequate resources (point of care test analyser located on ward) | Positive impact: Training and use of point of care testing increased diagnostic certainty and improved confidence in clinical decision-making; transfer decisions could be made earlier than they otherwise would have been and often treatment could begin immediately; reduced need for inter-hospital transfers and increased discharge rate; higher standards of practice; access to continuing professional education (CPE) | |
New Zealand | Negative outcomes: Workload increase – managing patients who would previously have been transferred and who now require more care; can be time-consuming; over-testing may become a problem | |||
Brambila et al. [44] | Quasi-experimental – pre- and post-intervention and control groups | Health practitioners (n = 40) | Intervention: Train the trainer: snowballing of a training intervention where two practitioners from each health district (n = 20 × 2 trainers) undertook training in tools to improve service quality. They then each trained approximately six trainees per health district in the programme (face to face) | Primary measures: Service outcomes (from patient level) – efficiency, access, quality and completeness |
Secondary measures: Service and patient outcomes – quality and completeness of health information and services received by clients | ||||
Level III-3 | Rural: Approximate population served 580,000 individuals | Contextual information: External support, coordination and programme; structure and content of programme; train trainers how to use job tools to improve service quality; train trainers how to train health-care practitioners; motivational and attitudinal change elements built into curriculum; needs assessment (content of programme in response to problem areas); appropriate skills and knowledge | Positive impact: Access to services increased significantly | |
Guatemala | No impact: No reduction in client waiting times or total time spent by clients at facilities | |||
Buckley et al. [45] | Non-experimental – descriptive post-intervention evaluation | Nurses | Intervention: Digital photographs were used to develop treatment plans and assess competency of non-specialist nurses in wound management utilising specialist support (non-face-to-face: telephone, email and digital photography) | Primary measures: Service outcomes – agreement on wound assessment and wound management plan between specialist and non-specialist nurse |
Level IV | Rural | Contextual information: Access to technology (computer, internet, email, digital cameras, IT programmes); correct use of technology, ability to use technology (issues identified around ability to take the ‘right’ picture); information privacy (permission to transmit patient information via email); appropriate use and combination of technology to achieve desired outcomes (intervention needed both verbal and pictorial reporting to improve accuracy of reporting) | Positive impact: Agreement on more basic assessment parameters. | |
USA | Less impact: On average there was poor agreement on more complex parameters. Verbal reports often missed vital signs leading to poor agreement between the specialist and non-specialists. | |||
Church et al. [46] | Mixed methods – pre-, during and post-intervention questionnaire and focus groups | Multi-disciplinary | Intervention: Interprofessional education programme in mental health for practitioners in six rural communities (combination: face to face, videoconferencing) | Primary measures: Staff outcomes – satisfaction, knowledge, skills, confidence |
Considerations: Vision is necessary for accurate diagnosis, potentially not just of the wound but of the home environment also | ||||
Level IV | Rural | Contextual information: External support (programme run and supported by researchers); networking and relationships (professionals from different systems brought together, structure of the programme – small groups, interactive, case-based learning) | Positive impact: Significant increase in confidence for seven of the eight mental health interventions and four of the six mental health issues that had been taught in the programme; more reflective mental health practice, more aware of mental health issues; integrating new knowledge and skills into their work; interprofessional referrals, interagency linkages and collaboration increased | |
Rural Newfoundland and Labrador, Canada | ||||
Cunningham et al. [47] | Qualitative – focus groups | Administrative and clerical staff | Intervention: Mechanisms contributing to effective protected learning time were identified (face to face) | Primary measures: Staff outcomes – satisfaction with, benefits of, advantages and disadvantages of PLT |
Level III | Rural | Contextual information: Organisational commitment; structured learning outcomes; structure of the programme (spending time with other teams and services, spending time with immediate colleagues, centrally organised events) | Positive impact: Useful to do with other teams and team members especially team-building activities Considerations: Increased workload the day after. Needs to include quality educational experiences. May be improved using a learning needs assessment | |
Scotland, UK | ||||
Doorenbos et al. [14] | Non-experimental – descriptive post-evaluation | Multi-disciplinary | Intervention: A series of cancer education sessions were delivered using telehealth technology to rural health-care providers (non-face-to-face: real-time videoconferencing) | Primary measures: Staff outcomes – satisfaction (content and mode), attendance rates |
Level IV | Rural | Contextual information: Active involvement of stakeholders (participants worked with university and clinical experts to develop cancer programme; participating rural health-care providers were also engaged in selecting topics and identifying convenient and feasible dates and times for the videoconference presentations); needs analysis; marketing the programme (the series was publicised and scheduled well in advance to allow providers to plan attendance at the presentations most relevant to them); accessible and adequate resources; external support (university technical staff hosted each presentation and were continually available for troubleshooting technological problems) | Positive impact: Overall satisfaction with telehealth as a mode of delivery; educational session information rated highly; high attendance rates; accessing CPE became a reality for rural health-care providers | |
Washington State, rural Alaska, USA | ||||
D'Souza [48] | Non experimental – cross-sectional questionnaire design post-intervention | Mental health practitioners and medical practitioners (general practitioners (GPs)) | Intervention: Delivery of educational and clinical modules for mental health via telemedicine and videoconferencing facilities (non-face-to-face: real-time videoconferencing) | Primary measures: Staff outcomes – satisfaction with the service and associated outcomes; feelings of isolation, fulfilling of academic needs, relevance to professional development, effect on self-assessed competence with mental health clients |
Level IV | Rural | Contextual information: Access to technology; timing of delivery (during team meeting time); mode of delivery (videoconferencing); structure and content of programme (lecture notes delivered prior to videoconferencing, 60-minute CPD blocks plus interactive discussion time) | Positive impact: High satisfaction scores with the service fulfilling their professional and academic needs. The service helped improve confidence and competence in managing mental illness | |
Australia | ||||
Ellis and Philip [49] | Mixed methods – pre- and post-questionnaire and interviews | Multi-disciplinary | Intervention: Development, delivery and evaluation of a short course in managing mental health emergencies at rural and remote health sites (face to face) | Primary measures: Staff outcomes – skills, satisfaction, attitude |
Level IV | Rural and remote towns in South Australia, Northern Territory, Queensland and Western Australia | Contextual information: External support (conducted by Australian rural nurses and midwives using grant from Department of Health and Ageing); adequate and accessible resources (workbook provided to participants; course delivered in rural and remote sites; mode of delivery – face to face) | Positive impact: Significant improvement between pre and post mental health assessment skills (unmatched comparison); changed attitudes towards mental health; improved communication ability when dealing with mental health clients | |
Glazebrook et al. [50] | Non-experimental – pre- and post-test evaluation | Medical practitioners | Intervention: Outreach ultrasound education workshops held in rural locations – specialist doctors from metro areas delivered workshops with local sonographers (face to face) | Primary measures: Staff outcomes – pre- and post-workshop knowledge tests (unvalidated) |
Secondary: Self-rated levels of knowledge, confidence and expertise in ultrasound | ||||
Level IV | Small rural hospitals | Contextual information: External support and organisation; local support (local experts utilised); funding and travel for outreach experts; structure and content of programme (face-to-face: hands-on workshops) | Positive impact: Significant improvement in knowledge and self-reported confidence with ultrasound by medical practitioners | |
Australia | ||||
Gorsche and Woloschuk [51] | Quasi-experimental – longitudinal, matched, case–control study | Medical practitioners | Intervention: Training programmes run within an ‘enrichment programme’ for rural and remote medics (mode not specified) | Primary measures: staff outcomes – goal attainment and retention. |
Level III-2 | Rural: Any Alberta community more than 50 km from a major metropolitan centre | Contextual information: External support (initiative of the Alberta government); accessible and adequate resources (fully supported to undertake training of choice – for example preceptors were compensated and locums arranged) | Positive impact: 97% of participants achieved training or learning goals; all participants were using their new or upgraded skills at 5 years; after 5 years, 100% in the matched enrichment group remained in rural practice compared with 71% physicians who did not partake in the EP (RR = 1.31; confidence interval: 1.06 to 1.62; P < 0.05). | |
Canada | Only paper to demonstrate a statistical link between supportive context, skill acquisition and retention of rural practitioners. | |||
Haythornthwaite [52] | Non-experimental – descriptive pre- and post-intervention evaluation | Mental health practitioners | Intervention: Simultaneous videoconference sessions presented over 12 weeks (‘Rural Links’ programme). Included fortnightly training sessions accompanied by reading material on topics covered and workbooks for use in-session (non-face-to-face: multi-site real-time videoconferencing) | Primary measures: Staff outcomes – number of participants, knowledge in relation to the training topics, participants’ views of video conferencing as a training modality, participant satisfaction |
Level IV | Rural and remote | Contextual information: Access to technology; resources: workbooks and session exercises; assume externally organised | Positive impact: Varied significant improvements in knowledge gains for particular teaching modules (although not consistent gains for all modules); compared with metropolitan participants, who received face-to-face training, rural participants showed similar levels of improvement in learning for most areas; high levels of participant satisfaction with videoconferencing delivery and programme content | |
Western Australia | ||||
King et al. [53] | Qualitative – critical ethnographic post-intervention | Aboriginal health workers | Intervention: A post-graduate university course undertaken by Aboriginal health workers (developed for nurses and allied health practitioners) to qualify them as diabetes educators | Primary measures: Staff outcomes – perceptions of the course, development as a health practitioner, relevance of the course to self and clients, learning outcomes |
Level II | Rural and remote | Contextual information: Reflection, feedback, evaluating outcome of the course; the course has to be relevant and academically targeted appropriate to the participant | Positive impact: Undertaking a post-graduate diabetes education course can improve confidence and competence in Aboriginal health workers. Course helped the Aboriginal health workers become more confident and competent as health professionals and empowered to learn and impart new knowledge as a practitioner | |
South Australia | ||||
Kelley et al. [54] | Non-experimental – cross-sectional survey design | Palliative carers | Intervention: Information regarding how a training programme was developed, planned and delivered in collaboration with local community partners (face to face) | Primary measures: Staff outcomes – self-reported knowledge and skills of practitioners. Service outcomes – sustainability of the programme, development of palliative care programmes in other agencies or the community |
Level IV | Rural: ‘Towns and municipalities less than 10,000 population and located outside the commuting zone of urban centres larger than 10,000 population.’ Remote: ‘isolated community with limited resources, 80 km distance or four or more hours travel from a major urban centre of >50,000 population’ | Contextual information: Course content was locally relevant; networking opportunities; train-the-trainer approach, emphasis within course material on training participants to translate their knowledge gains to co-workers | Positive impact: Increase in self-reported knowledge, confidence in practice and skills. Sustainable via participants imparting learnt knowledge via mentoring and workshops to co-workers. May increase number of palliative care programmes. | |
Canada | Considerations: Networking and learning about supporting resources were identified as the most important elements of the programme; able to collaborate more as a group to improve services and act as a common voice. Indicated they shared information with other staff via mentoring, meetings, case conferences, formal in-service sessions, formal workshops. | |||
Ideal learning: Off site in nearby town, small groups, interactive | ||||
Koczwara et al. [55] | Non-experimental – descriptive planning phase and post-evaluation | Multi-disciplinary | Intervention: Development, implementation and evaluation of an online educational programme (oncology) for rural health practitioners | Primary measures: Staff outcomes – change in practice, satisfaction with programme, users (attendance) |
Level IV | Rural and remote | Contextual information: Needs analysis (survey and focus groups conducted with rural practitioners); regular feedback and evaluation opportunities; marketing of the programme (programme launch at national conference, online advertising to target audiences); accessible and adequate resources; networking and relationships; employ specific learning approach; skills to deliver the intervention (facilitator employed and trained in online environment and subject matter) | Positive impact: High attendance and completion rates; perceived change in practice as a result of completion of learning programme; learning needs met and achievement of specific learning goals; high satisfaction with online multimedia | |
Australia | ||||
Newman et al. [56] | Non-experimental – cross-sectional survey post-intervention | Multi-disciplinary | Intervention: Use of videoconference facility in different urban and rural settings to deliver a one-off education conference | Primary measures: Staff outcomes – knowledge, socialisation, information exchange, ease of use |
Secondary measures: Numbers of participants, geographic location | ||||
Level IV | Urban, rural and remote | Contextual information: External support and organisation (technical preparation of videoconferencing was by the conferencing and media staff from the lead hospital or health service in liaison with staff and departments from other services; ‘site facilitators’ utilised at each site); adequate preparation (speakers provided with guidelines on etiquette and teaching methods); teaching rehearsals prior to event | Positive impact: Useful for learning and were able to contribute or be part of a learning community | |
Australia | Less impact: Mostly a passive experience. Not overly easy to use | |||
Schoo et al. [57] | Mixed methods – action research (questionnaire and interviews) | Physiotherapists | Intervention: Continuing education programme developed, implemented and evaluated by local physiotherapy practitioners with researchers from a university (face to face) | Primary measures: Staff outcomes – relevance, attendance of programme |
Secondary measures: Staff outcomes – perceived effect on clinical practice | ||||
Level III (Daly) | Regional and rural: ‘Accessible and moderately accessible’ on the Accessibility/Remoteness Index | Contextual information: Location of programme (locally delivered); teacher attributes (highly qualified); needs analysis prior to programme development; active involvement of stakeholders in programme development and evaluation (identification of targets and measures for success prior to intervention, active participant engagement with institutional facilitation); external organisation, input and facilitation (needs assessment, development of programme, evaluation tools) | Positive impact: All targets were reached. Attendance – more than half (57.2%) of physiotherapists in the region attended a minimum of four sessions and 68.6% attended at least one ‘on-site’ workshop. More than two-thirds of the physiotherapists (68.6%) knew of others who attended at least one of the continuing education (CE) functions of the 2004/5 programme and 45.7% of these physiotherapists received useful information from others who attended. Interactive CE programme had a positive influence on perceived clinical skills | |
Australia (ARIA) | ||||
White et al. [58] | Non-experimental – cross-sectional survey post-intervention | Medical practitioners (GPs) | Intervention: Government-run CME workshops (face to face) | Primary measures: Staff outcomes – professional isolation, confidence, commitment to remain in rural practise (retention) |
Level IV | Rural: Rural Remote and Metropolitan Areas (RRMA) classification four to seven locations | Contextual information: Needs analysis; clinician-led content; funding (government department funded travel and accommodation); time relief (locum support or locum rebates available for more remote GPs) | Positive impact: Access to CME contributes to confidence in practicing in remote and rural areas; CME strongly alleviates professional isolation; less likely to remain in practice without access to CME | |
Australia | ||||
Wright et al. [59] | Non-experimental – descriptive pre- and post-evaluation | Medical practitioners | Intervention: Evaluation of an educational support programme for international practitioners practicing in rural areas (combination: simulated face-to-face consultations, workshops, weekly meetings, interactive web-based learning modules) | Primary measures: Staff outcomes – clinical practice and competency, retention (at three months post-intervention); satisfaction with the programme |
Level IV | Regional and rural: RRMA two to five | Contextual information: Needs analysis (via a pre-programme learning needs analysis); regular feedback and evaluation opportunities (post workshop and session evaluations and post programme evaluation); accessible and adequate resources | Positive impact: Needs assessment enabled participants to articulate specific skills and knowledge that would assist them to work more effectively in their current clinical contexts; statistically significant and positive changes were identified post-intervention for (i) technical skills appropriate to current practice; (ii) willingness and effectiveness when teaching or training colleagues and (iii) communication with carers and family. Satisfaction with the programme and development of a learning community in Gippsland | |
Australia | ||||
Papers examining mentoring | ||||
Butcher [60] | Non-experimental – descriptive pre-post-evaluation | Nurses and dietitians | Intervention: Mentoring to upskill or train to become certified diabetes educators or simply to improve knowledge of diabetes (combination: face to face, telephone, email) | Primary measures: Service outcomes – access to quality diabetes services; staff outcomes satisfaction with programme |
Level IV | Remote: Population of 902,195 spread across 147,042 square miles: population density of 6.2 persons per square mile | Contextual information: Needs assessment (learning needs of all enrolled in programme were assessed and matched to course materials and a mentor); external support and coordination (central coordinator designated to programme); resources (lending library for study, mentoring manual for mentors and mentees); structure and content of the programme: mentoring was face to face, telephone and email; observation of mentor in diabetes management also encouraging; combination of mentoring programme structure, content and delivery modes (email, face to face, resources) | Positive impact: 30% of enrolled nurses and dietitians gained certification. Number of educators increased 47% (but unsure if directly related to intervention) | |
USA | ||||
Gibb et al. [61] | Qualitative – focus groups held before and after an action research intervention | Nurses | Intervention: Research officer worked with staff to develop a definition of mentoring, the results of which were converted into questionnaires by the research team eventually becoming a set of guidelines of desired qualities for mentors and mentees and an evaluation tool for monitoring the mentoring relationship (mode: n/a) | Primary measures: Staff outcomes – understanding of mentoring, key qualities in mentors and mentees, success of mentoring strategy |
Level III | Small rural hospitals | Contextual information: Needs analysis (staff perception of mentoring needs); external support (facilitation of action research by university); active involvement of stakeholders in programme design and evaluation (the act of coming to an understanding and a working definition of mentoring in context; action research enabled greater understanding of role of mentoring, which in turn allowed for effective mentoring relationships to develop); conversion of discussion into a questionnaire for evaluation and into a guideline document for mentoring | Positive impact: More structured mentoring practice | |
Australia | Considerations: Qualities of a good mentor were identified, action research enabled greater understanding of role of mentoring which in turn allowed for effective mentoring relationships to develop. Link identified between mentoring and development of clinical competence. Key to successful mentoring was management support | |||
Papers examining a combination of support interventions | ||||
Dalton et al. [62] | Mixed methods – pre- and post-intervention evaluation | Pharmacists | Intervention: Education, training and mentoring. Online preceptor education programme with interactive learning modules and online interactive mentoring via discussion groups (non-face-to-face: real-time videoconferencing, telephone, email) | Primary measures: Staff outcomes – assessment of the programme's implementation, design and delivery from the preceptors’ perspective |
Level IV | Rural: Accessibility/Remoteness Index of Australia (ARIA) categories 1 to 6. | Contextual information: Correct use of technology and ability to use technology; willingness of participant to undertake self-directed learning | Positive impact: Interactive elements of the online programme, such as reflective exercises, were useful for learning | |
Australia | Considerations: Some IT issues. Introductory video would be useful for programme but weekend course or videoconferencing is a better mode of delivery. Telephone helpline would be useful Limitations: Presumed pharmacists were good self-directed learners and had adequate IT skills | |||
Gardner et al. [63] | Non-experimental – descriptive post-evaluation | Nurses | Intervention: Professional support, training and education; supporting nurses in rural areas to understand and conduct research (combination: face to face, videoconferencing, telephone, email) | Primary measure: Staff outcomes – orientation to research |
Level IV | Rural and remote | Contextual information: External support; accessible and adequate resources (all participants had access to necessary resources; textbooks and resource packages were provided as well as access to computers during the workshops); active involvement of participants (content of programme was responsive to the needs of the nurses at the rural and remote sites); networking and relationships (mentorship and collaboration encouraged) | No impact: the survey results do not demonstrate any major changes over time in perceived knowledge of research, research orientation or perceptions of barriers and supports to research. Despite the same structured educational intervention being delivered at two rural sites, clinical nurses at only one site completed the research proposals within the study timeframe | |
Australia | ||||
Hoon et al. [64] | Mixed methods – before and after design, action research | Nurses and medical practitioners | Intervention: Training, education and mentoring; information on how a training programme was developed: planned and delivered in collaboration with local community partners (face to face) | Primary measures: Staff outcomes – knowledge and skills in the delivery of chemotherapy and cancer care education. Service outcomes – connection between local rural health services and one or more of the urban specialist cancer services |
Level IV | Rural | Contextual information: Needs analysis; time relief to attend five-day placement; funding to attend five-day placement (salary costs of rural participants, travel and accommodation, salary funding for mentor for one day of placement); indemnity, legal matters, duty of care, responsibility (hands-on opportunities limited by indemnity issues and issues from metro staff around relinquishing cancer care to practitioners with little time, knowledge or skill) | Positive impact: Post-programme significant improvement in understanding of principles of chemo delivery including some technical details; improved confidence in technical details; knowledge translation to other rural practitioners and organisations; changes in procedures and practices; isolated incidences of improved client care (less travel for clients) Considerations: Programme was limited by unmet expectations; integrating new practices with already demanding practice; quality and safety issues as perceived by metro teachers and mentors; variability in opportunities (for example some hands-on but some not, some mentoring but some not) | |
Australia | ||||
MacKinnon [65] | Qualitative – institutional ethnography | Nurses | Intervention: Professional support, training and education; exploration of nurses’ experiences of learning to provide maternity care in rural settings (mode not specified) | Primary measures: Staff outcomes – behaviour, practice, knowledge, skills, job satisfaction |
Secondary measures: Patient outcomes – safe practice; service outcomes – quality | ||||
Level I | Rural: less than 10,000 people living beyond commuting distance of an urban setting | Contextual information: External support (example of funding provided to one participant to upskill in maternity care in a regional centre); accessible and adequate resources; networking and relationships | Difficult to learn about maternity in small rural hospitals, in an environment where few staff members are available and little education is provided; concerns expressed about remaining ‘experienced’ and retaining newly acquired skills; experienced nurses had been mentored to ‘learn maternity’ by an experienced maternity nurse; however, birth rates and staffing levels have changed and such practices as mentorship were no longer available for new RNs; going to a big city to learn maternity nursing ‘does not work’ because a rural hospital nurse is not able to access all the ‘fancy teams’ and high-tech equipment available to RNs working in the city; family commitments made it difficult for them to leave their community for CPE | |
Mitchell et al. [66] | Mixed methods – post-intervention, action research | Mental health practitioners | Intervention: Professional support, training, education and supervision; telemedicine network established to deliver and receive educational material via videoconferencing facilities (non-face-to-face: multi-site real-time videoconferencing) | Primary measures: Staff outcomes – accessing the network, participation in the network, useful sessions, benefits (networking, peer support) |
Level III (Daly) | Rural and remote | Contextual information: Access to technology (type of technology – videoconferencing units; ensuring availability of units, ensuring adequate IT support, ensuring organisational support); organisational commitment and support; ensuing funding; ensuring time available for setup; timing of programme: ensuring flexibility of delivery for staff | Positive outcomes: ability to access second opinions; ability to access specialists; ability to book teleconsultations; ability to access supervision from Adelaide; improved networking and peer support; improved efficiency and travel costs; improved health service efficiency (due to enhanced knowledge), retention Considerations: Impediments included competition with other services for use of equipment; equipment breakdown; time required to set up a session; staff on rotating rosters not being available at a set time; difficulties with local organisational processes, including approvals; imperfect synchronisation of lip movement and audio in videoconferencing sessions; high cost of sessions involving multi-site videoconferencing | |
Australia | ||||
Owen et al. [67] | Non-experimental – pre- and post-intervention descriptive evaluation | Mental health practitioners | Intervention: Professional support, training and education; intermittent outreach service provided by metro mental health specialist practitioners to rural and remote areas – includes joint patient care sessions, education sessions and peer support (face to face) | Primary measures: Staff outcomes – clinical skills gained; success of education sessions; knowledge gained; attitudes. Service and client outcomes – admission rates from each town to a regional centre and transfer of clients for care to regional centres; prescription rates of psychotropic drugs from 18 months prior and during the project via Pharmaceutical Benefits Scheme data |
Level IV | Rural and remote | Contextual information: Active involvement of stakeholders in programme design and implementation (a representative steering committee to finalise teaching topics and oversee project comprising rural health staff, metropolitan health and education staff, rural health administration; clinics organised by local contact); external organisation of the project and intervention (research officers from the university coordinated and organised the project; project lead was a visiting specialist with a vested interest in the programme being successful); marketing of programme (flyers sent to promote education sessions; project promoted in multiple mental health venues; CVs of visiting team circulated); funding (transport costs were met by the project but salaried visiting staff were ‘donated’ to the project) | Positive impact: Education session evaluation – perceived increase in knowledge by most participants; content was perceived as relevant, appropriate. Regional admission rate increased and prescriptions increased (admission rates and prescription rates not controlled statistically for any other factors so cannot attribute to the intervention per se). | |
Far west New South Wales, Australia | Less impact: Knowledge assessment – correct responses to mental health statements same prior to and after intervention (no change from baseline – but possibly using a poor measurement tool); before and after skills assessment (clinical vignettes); small improvement in ability to diagnose psychiatric conditions | |||
Schopp et al. [68] | Non-experimental – descriptive pre-and post-test evaluation | Psychologists | Intervention: Professional support, training and education; specialist one-on-one support and training for remote generalist psychology clinicians through telehealth videoconferencing and website support for families (non-face-to-face: real-time videoconferencing) | Primary measures: Staff outcomes – knowledge gains: rural clinicians undertook a pre-test on issues related to TBI that was matched to the training content Patient outcomes – client satisfaction, family access via structured interview |
Level IV | Rural: Mid-Western rural communities | Contextual information: Access to technology (with ability to encrypt and decrypt data for patient confidentiality); externally supported and organised (participating rural practitioners, technology, content of sessions); attributes of teacher (approachable) | Positive impact: Significant pre- and post-test scores for clinicians for knowledge gain (and self-reported confidence) (means not given). Patients found trained clinicians helpful and knowledgeable. Compared to the 11 patients who chose not to use the trained clinician, authors report trained providers were perceived as more helpful and more knowledgeable than untrained providers – this was reported as significant (the statistical analyses of patient responses when comparing trained with untrained clinicians is flawed, thus we cannot rely on these results) | |
USA | ||||
Sullivan et al. [69] | Non-experimental – descriptive post-intervention evaluation | Medical practitioners (GPs and psychiatrists) | Intervention: Training, education and mentoring; shared care strategies between expert mentor and GP via telephone combined with monthly education sessions and joint clinical consultation (combination: face to face and telephone) | Primary measures: Staff outcomes – identify key success factors to shared care in this manner – measured one year after the pilot project |
Level IV | Rural | Contextual information: Attributes of teacher (relaxed, expert did not take on teacher role, mentor, approachable); needs analysis; accessible resources (funding for travel to education sessions, time to attend sessions) | Positive impact: Mentoring: All physicians viewed mentoring as highly valuable and a preferred method for accessing advice; allowed them to continue their own clinical interventions confidently, which they would not be able to support otherwise. Education: More satisfied if content relevant and if teacher utilised a relaxed approach to teaching | |
Canada | ||||
Tumosa et al. [70] | Non-experimental – descriptive pre- and post-evaluation | Multi-disciplinary | Intervention: Mentoring, training and education; evaluation of a geriatric scholar programme for rural primary care providers consisting of education and training in geriatrics and gerontology and in quality improvement (combination: face to face (clinical practice), webinars, audio conferences) | Primary measures: Staff outcomes – practice behaviour, knowledge, skills, ‘usefulness of programme’ Secondary measures: Service outcomes – quality improvement, perceived impact on patient care |
Level IV | Community of 5,000, 60 miles from major metropolitan areas. | Contextual information: Active involvement of stakeholders; organisational commitment; needs analysis (educational needs assessment); external support (financially and organisationally supported by networks of services with a ‘hub site’ located in a metro centre); accessible and adequate resources (intranet web-based platform to share resources as a learning community); networking and relationships; ongoing evaluation and feedback opportunities (identification of additional learning resources) | Positive impact: Improvements in self-reported competence and self-confidence in geriatric skills, topics and knowledge (and a resulting perceived change in practice); decline in continuing need for further education; high completion rates of QI projects; development of a rich learning community | |
Rural Kentucky, USA | ||||
Papers specifically examining mode of delivery | ||||
Gagnon and Minguet [71] | Non-experimental – pre- and post-test pilot study evaluation | Medical practitioners | Intervention: Professional support, training and education; use of internet for delivery of online courses and collaboration with online tutorial sessions delivered twice weekly (non-face-to-face: virtual classes, collaborative web conferencing, real-time chat) | Primary measures: Staff outcomes – gain in knowledge |
Secondary measures: Staff outcomes – level of comfort with IT | ||||
Level IV | Rural and remote | Contextual information: Access to technology (internet, computer, interactive IT programmes, webcam, microphone, software development, running and analysis); attributes of coach (availability); structure and content of programme (two tutorial sessions per week with real-time conversations online, virtual classes with real-time chatting and asynchronous exchange); correct use of technology and ability to use technology; external support and organisation | Positive impact: Reported the experience had brought them out of isolation and enabled very productive contacts with peers; participants likely to gain pedagogic knowledge and to maintain this knowledge over time | |
Canada (Quebec); France | Less impact: Perception of level of comfort with information and communication technologies was unlikely to change | |||
Stewart and Carpenter [72] | Qualitative – action research | Physiotherapists | Intervention: Twice weekly iChat with mentor and monthly videoconferencing with mentor and other mentees for three months (non-face-to-face: asynchronous chat, iChat, email, real-time videoconferencing) | Primary measures: Staff outcomes – effectiveness of mentoring using this medium; experience with technology |
Level III | Rural | Contextual information: Active involvement of stakeholders in programme design and implementation (measuring success and evaluating effectiveness of programme at key intervals and changing programme in response to feedback ); access to technology (Mac laptops with appropriate programmes; IT support; easy to use); mentor and mentee attributes (relationship between mentor and mentee); externally organised and supported | Positive impact: Improved communication (iChat sessions replicated the colleague interaction that was generally missed in sole positions); improved clinical reasoning, confidence and knowledge translation | |
Canada | ||||
Chipps et al. [73] | Non-experimental – descriptive pre- and post-evaluation | Medical practitioners (psychiatry) | Intervention: Videoconference-based psychiatry registrar training programme (non-face-to-face: real-time videoconferencing) | Primary measures: Attendance; familiarity with videoconferencing; cost and time savings; appropriateness of content and mode, technical issues |
Level IV | Urban and rural | Contextual information: Accessible and adequate resources; networking and relationships (videoconferencing was real time with participant interaction) | Positive outcomes: Improved access to education (increased attendance and reduction in travel resulting in time and cost savings); videoconferencing perceived as appropriate educational tool (and as effective as face-to-face teaching); videoconferencing gave satisfactory interaction | |
South Africa | Considerations: technical issues audio quality | |||
Brownlee et al. [15] | Qualitative – thematic analysis of interviews | Social workers | Intervention: Perception of utility of different technologies for supporting social work practice in rural areas (non-face-to-face: chat, email, internet, videoconferencing) | Primary measures: Staff outcomes – use of technology, change in practice behaviour, connectedness |
Level III | Rural and remote: Practitioners from areas where the population densities are well below 400 people/km2 | Contextual information: Accessible and adequate resources (internet access, email, caseload database systems, phone systems, for example, telehealth) | Positive impact: Professional networking; clinical feedback; supervision and access to services seem to have increased with the availability and use of the internet | |
Canada | Considerations: Not all use internet – language barriers; cumbersome and confusing; not all challenges of rural practice have been remedied, or even affected, by the internet – for example dual relationships in small rural towns | |||