The benefits of international volunteering in a low-resource setting: development of a core outcome set

Abstract

Background

Qualitative narrative analysis and case studies form the majority of the current peer-reviewed literature about the benefits of professional volunteering or international placements for healthcare professionals. These often describe generalised outcomes that are difficult to define or have multiple meanings (such as ‘communication skills’ or ‘leadership’) and are therefore difficult to measure. However, there is an interest from employers, professional groups and individual volunteers in generating metrics for monitoring personal and professional development of volunteers and comparing different volunteering experiences in terms of their impact on the volunteers. In this paper, we describe two studies in which we (a) consolidated qualitative research and individual accounts into a core outcome set and (b) tested the core outcome set in a large group of global health stakeholders.

Method

We conducted a systematic review and meta-synthesis of literature to extract outcomes of international placements and variables that may affect these outcomes. We presented these outcomes to 58 stakeholders in global health, employing a Delphi method to reach consensus about which were ‘core’ and which were likely to be developed through international volunteering.

Results

The systematic review of 55 papers generated 133 unique outcomes and 34 potential variables. One hundred fifty-six statements were then presented to the Delphi stakeholders, of which they agreed 116 were core to a wide variety of healthcare professional practice and likely to be developed through international experiences. The core outcomes (COs) were both negative and positive and included skills, knowledge, attitudes and outcomes for healthcare organisations.

Conclusions

We summarised existing literature and stakeholder opinion into a core outcome set of 116 items that are core to healthcare professional practice and likely to be developed through international experiences. We identified, in the literature, a set of variables that could affect learning outcomes. The core outcome set will be used in a future study to develop a psychometric assessment tool.

Background

Volunteering, or temporarily working in low-resource settings, is often seen solely as a means of helping those in poorer economies [1]. Many professionals find it difficult to obtain support to volunteer and report lack of recognition upon return, which is disincentive to volunteerism [2]. Furthermore, health professionals that volunteer abroad predominantly do so using annual leave, rather than recognised study leave for continued professional development [3, 4]. The notion that those from high-income countries (HICs) are altruistically offering ‘help’ to those in low- and middle-income countries (LMICs) can also lead to a distortion of the partnership relationship between high- and low-income partners in health partnerships. The low-income partners can be seen as beneficiaries and the high-income partners seen as donors [5,6,7]. Furthermore, a tension often exists between UK healthcare professionals and local international staff, as the intentions or role of healthcare professionals and students is often not explicit to the teams with whom they are working. However, the donor-recipient relationship is becoming increasingly contested in recent literature and policy and mutual benefits realised [8, 9].

There is an imperative, therefore, to fully understand the learning outcomes that are possible for HIC health professionals working in low-resource settings and, in particular, to help recognise these activities as educational development [2, 3]. Understanding ‘what’ is gained would allow specification of intended learning outcomes for training and continuing professional development and to make the gain for the HIC more explicit. Understanding under what circumstances learning outcomes occur would result in an understanding of how to maximise that gain.

Literature that explores what and how healthcare professionals learn from temporarily working or volunteering in a low-resource setting tends to report anecdotes or single reports, which provide a lower level of evidence [4, 10]. Furthermore, benefits are detailed in broad categories, with ‘leadership’, ‘communication’ and ‘cultural awareness’ being frequently reported [3, 11,12,13], with a focus on one of these skill sets in depth or a list of outcomes under umbrella terms, such as communication or leadership skills [3, 14]. These broad labels make an assessment of the learning outcomes difficult as they might contain multiple underpinning knowledge, skills, practice and attitudes. Self-assessment of broad terms is not well associated with objective performance [15]; individuals struggle to assess themselves in relation to ambiguous or ill-defined traits [16, 17]. Specifying learning at this broad level means that the more granular levels remain unspecified. A higher-level group might contain a wide range of lower-level outcomes and might not contain others, which would reduce the content validity of an assessment.

Understanding the metrics of health professional volunteerism would have a significant impact on current continued professional development (CPD) policy because international experiences could be evidenced as beneficial to personal and professional development. Numerous policy documents about future health workforce highlight the importance of skills such as leadership, communication and adaptability [18, 19]. Such skills have been described as key outcomes of international placements in LMICs, but have yet to be quantified to enable comparison with other learning opportunities [3].

In a systematic review of the evidence of the benefits to the United Kingdom of health partnership work, Jones et al. reported 40 individual benefits grouped within seven key domains (communication and teamwork, clinical skills, management skills, patient experience and dignity, policy, academic skills and personal satisfaction and interest). There were a number of features of this review that makes it insufficient for the purposes of measuring learning outcomes from international volunteering. Firstly, this review focused only on health partnerships, a specific type of health link, and not all types of volunteering or international placements. Secondly, the findings were categorised broadly, with the difficulties of broad measurement specified above. Thirdly, the professions in their search terms were only doctors and nurses. Finally, it did not extract factors that may affect learning outcomes. For the purpose of measurement, we needed to include literature from a broad range of experiences, extract outcomes at a granular level, include all healthcare staff groups and extract variables that may affect these outcomes.

The outcomes for health professionals are not always positive and the costs of international placements in literature have included health consequences, skills degradation and financial cost [3, 20, 21], reputational, health and opportunity [3]. Research has explored the costs and benefits of international placements [13, 20, 22] and barriers to volunteering, but no research has yet listed all reported negative outcome [23].

Many aspects of LMIC placements are different from working in a HIC. Relationships between outcomes and these aspects have been proposed, for example that individuals learnt from the opportunity to interact with more patients or conditions than in the United Kingdom [10, 24] and that longer stays may be more beneficial than shorter stays [25, 26]. These variables have not been systematically reported.

This current paper presents two studies: a meta-synthesis and a Delphi. The meta-synthesis aimed to (a) detail the personal and professional development outcomes of international work, at a granular level, i.e. ‘knowledge about procedures rarely conducted in the United Kingdom’ (rather than at a too broad level, i.e. clinical skills or too specific level, i.e. experience conducting vesico-vaginal fistula surgery); (b) to report the variables that influence these personal and professional development outcomes; and (c) to explore if the review including all types of UK health professional placement and all cadres of staff found the same categories as the most recent review of Jones et al. [3]. The Delphi study aimed to gather consensus from those with knowledge and expertise in international health professional learning and development, to refine a set of agreed core outcomes.

Method

Study 1: Meta-synthesis study design and sample

The systematic review of peer-reviewed literature, published in academic journals, was conducted between September and November 2014. Inclusion criteria included that (1) participants must not be in receipt of their full UK salary (a stipend or living allowance was permissible), thus excluding those in permanent employment overseas; (2) health professionals or health professional students (students were included, as much research has been conducted about educational outcomes in students); (3) activities must be health-focused to ensure outcomes were related to clinical work; (4) some participants must have departed from the United Kingdom and be UK citizens (papers that included a partial UK sample were included); (5) some participants must only have travelled to a LMIC; and (6) the paper must reference something that is perceived as a benefit, cost or potential variable, (7) there were no date restrictions. Guidelines for inclusion were used to ensure consistency.

Each paper was screened by one team member (NT) to ensure that it met the inclusion criteria. A second team member (JC) independently checked the first 20% of the included papers to ensure agreement of implementation of inclusion criteria. This was then discussed in a meeting. Disagreements would have been resolved using discussion and refining inclusion criteria for greater specificity; however, the reviewers agreed on all of the papers for inclusion (Table 1).

Table 1 Inclusion criteria

Data sources and study selection

A standard set of terms were used to search 11 databases for peer-reviewed literature between the earliest date indexed and the time of the review. This included five columns of synonyms relating to outcomes and variables, international volunteering placements, health professionals, the United Kingdom and LMICs (see Additional file 1). The databases were medical and generic databases: Cochrane Economic Evaluations, Health Management Information Consortium, Health Business Elite, Web of Knowledge/Social Sciences Citation Index, PsycINFO, CINAHL, AMED, International Bibliography of Social Sciences, Social Services Abstracts and Sociological Abstracts, Global Health and JSTOR.

The abstracts and titles of each result of the electronic database search were screened, papers that did not meet inclusion criteria were removed and retained papers were rescreened to confirm inclusion.

Citation mapping

Reference lists of all included papers were assessed. Any papers that were of relevance were assessed against the inclusion criteria.

Quality assessment

We chose to include papers that were peer-reviewed but did not present empirical findings; therefore, the Cochrane risk of bias tool was not applicable to this research [27]. We categorised the papers using a quality framework [28].

Data extraction

We took a thematic synthesis approach to data extraction [29], which consists of three stages: line-by-line coding of text, development of descriptive themes and generation of analytical themes. We did not undertake the third stage as our purpose was the extract outcomes as a low level and the third stage has been criticised for being open to the judgement of the researcher [29, 30].

Each study that met the inclusion criteria was read, and any text (related to variables or positive/negative outcomes, at an individual, national or institutional level) was coded according to both content (explicitly stated in the papers) and meaning (inferred by the researcher). Outcomes were defined as anything that happens to UK health professionals as a result of volunteering/international placements (at an individual, national or institutional level), both positive and negative. Variables were any factors that reported influence outcomes, both implicitly and explicitly.

Using Nvivo, a node was created at a ranked level for each component of descriptive theme. For example, the outcome experience conducting ‘vesico-vaginal fistula surgery’ was coded within the second-order theme of ‘greater knowledge of procedures not used in the United Kingdom’ within the higher-order theme of ‘Increased awareness of and knowledge about conditions and procedures rarely encountered in the United Kingdom’. We decided that the lowest level of specificity would be applicable to all/most professions and generalisable across situations. As each paper was coded, the nodes were adapted, developed and generated. Two researchers (NT, JC) independently reviewed the first 20% of papers and then met to develop a coding framework together. There were no disagreements as we were not looking to categorise, but rather develop a matrix of emerging codes; therefore, any differences in extraction occurred only when one reviewer had overlooked an outcome cost or variable. The second reviewer verified the extraction of the data from a further 20% of papers.

Study 2: Stakeholder Delphi

Design

We used the Delphi method, an iterative process of rounds in which data are collected and condensed into a group consensus [31]. A series of virtual questionnaires record participant’s agreement with statements concerning a particular topic. Delphi is often used to develop COS in health research [32, 33]. As we were creating a core outcome set, this stage of the process only included the outcomes extracted in study 1; variables were not included.

In round 1, we held a face-to-face discussion group with stakeholders to generate outcomes. Subsequent rounds were online (with paper version emailed if there were technical difficulties). Participants were asked to indicate to what extent they agreed or disagreed each outcome was a core outcome of international placements and volunteering.

Participants

Participants were people who were volunteering health professionals; coordinators of international health professional volunteers, responsible for intended learning outcomes (ILOs) for health professionals; coordinators of health partnerships; study health professional education and international development; educational commissioners and NHS stakeholders. Participants were recruited for an initial workshop from a global health network, to ensure that participants from each of the stakeholder groups were invited and represented. Non-attendees were invited to participate online. After this event, a snow-ball sampling technique was used to reach further stakeholders from each group for online rounds; participants were asked to recommend interested individuals.

Instrumentation round 1: Stakeholder face-to-face discussion and pilot

In order to generate a list of outcomes, any new data generated from round 1 was added to the existing coding framework (see Additional file 1). Outcomes were then generated by presenting the highest-order theme as the outcome and any relevant lower-order themes as examples within brackets to add context. We input outcomes from the meta-synthesis and any additional outcomes from round 1 of the Delphi, into the hosting software. We piloted round 2 with seven members of the research team, who commented on structure, grammar, wording, level of specificity and technical issues. With the addition of items from the Delphi round 1 and comments from the pilot (and separation of some outcomes into two unique outcomes), the 133 outcomes from the meta-synthesis were converted into a list comprising of 156 outcomes to go forward to round 2.

Rounds 2–4: The online rounds of the Delphi

Two team members divided the 156 outcomes into three categories (see Table 2): knowledge, skills and attitudes (n = 115); organisational outcomes (n = 8); and negative outcomes (n = 33). Statements were presented alongside a 7-point Likert-type scale, regarding agreement as to whether each statement should be “considered a ‘core outcome’ of international placements that should be measured in a toolkit”. The scale used the following numbers to represent agreement: 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = no preference, 5 = slightly agree and 6 = agree, 7 = strongly agree. For emphasis, the phrase ‘core outcome’ was presented in bold and the definition was repeated in numerous emails, instructions and synopsis. A core outcome was defined in the following way:

A core outcome is something that is common, important and applicable across a wide range of settings. It can be a benefit or cost, but it must be something that would be more likely to happen to an individual on international placement rather than somebody working in the UK.

Table 2 The three questions presented to stakeholders

For each round, participants had 14 days to respond. Email reminders were sent to invitees frequently. However, as the initial questionnaire was particularly long, some participants requested an extension of the deadline by 10 days and 2 days at round 3. In round 4, participants who had not responded in round 3 (but had in round 2) were invited to re-join the study; many stakeholders worked internationally and had limited internet access at certain periods. In round 4, the expressions of some statements were changed in light of the comments from previous rounds to improve clarity.

The statements with at least 70% consensus in the previous round were retained and not re-presented to the group. Therefore, by round 4, a much smaller group of non-consensus statements were presented. In rounds 3 and 4, participants were asked to use the same Likert scale and reconsider their answers from the previous round (displayed) in light of the group median and any anonymised comment gathered in the previous round.

Results

Study 1: Meta-synthesis

Data sources

The search of the electronic databases generated 521 hits including duplicates, i.e. 384 unique papers. Twenty-two papers met inclusion criteria. Citation mapping revealed a further 33 papers which were included. Therefore, the total number of papers from which data was extracted was 55. The main reasons for exclusions of papers were (1) not concerning the subject of interest, (2) non-British populations, (3) no health focus, (4) only placements in HIC, 5) only including paid/permanent staff and (6) reporting no benefits, outcomes or costs.

No papers included fell within the top two quality categories proposed by Benzies et al.: randomised controlled trials [28]. Some papers included qualitative or quantitative data (23/55, 42%), but the majority of papers reported no primary data.

Positive outcomes were extracted from 96% (53/55) of the papers, whilst negative outcomes were extracted from only 49% (27/55). Potential variables that could affect these outcomes were extracted from 90.91% (50/55) of papers. None of the papers explicitly reported or explored how variables were thought to affect outcomes (Fig. 1 and Table 3).

Fig. 1

PRISMA flow chart to show number of papers included and excluded

Table 3 Factors which influence outcomes

Extracted outcomes

We found 133 unique outcomes, including 28 negative outcomes. The outcomes extracted could be categorised within NHS professional development terminology; there were 24 items about knowledge, 44 about skills and 20 about attitudes [34]. Six were organisational benefits and 29 negative; 10 were categorised as ‘other’. Organisational outcomes were deliberately separated, as organisation-specific outcomes were identified in addition to the general positive effect of staff with developed knowledge, skills and attitudes. Only 29 (22%) of the outcomes stated in the literature were negative, suggesting an overall positive attitude towards international placements from the authors (Fig. 2 and Tables 4 and 5).

Fig. 2

Example coding matrix (communication was not a theme, but it highlights how it was used in past research)

Table 4 Percentage of papers containing positive or negative outcomes

Table 5 How the data extracted was coded, including higher-level outcomes, lower-level outcomes and examples from the data

Study 2: Delphi

Participants

Fifty-one participants attended the round 1 workshop. Invitations were sent to 259 participants for the online Delphi, and 78 (30%) accepted. Once enrolled in the study, response rates remained high: round 2, n = 58/78 (74%); round 3, n = 49 (63%); and round 4, n = 45 (58%). More than half of the participants were involved in global health policy, and one third of the participants had volunteered.

After round 2, 98 of the 156 statements (63%) were retained; this meant over 70% of the stakeholders agreed or strongly agreed these 98 statements were core outcomes. After re-considering their own vote in round 2, the group median and anonymous comments regarding each statement, 13 additional statements were retained in round 3. Finally, after readdressing the above items for the second time, an additional five statements met consensus and were retained in round 4. Of the items that met consensus, 99 were positive and eight were negative. Positive outcomes were of educational benefits to the British health professionals and negative outcomes were drawbacks, costs or negative effects (Tables 6, 7, 8, 9 and 10).

Table 6 Number of statements retained at each stage with 70% consensus being met

Table 7 Applying our results to the current knowledge: our core learning outcomes presented within the existing domains from [3]

Table 8 Examples of COs that fell within a number of categories

Table 9 Examples of core learning outcomes that did not fit within the categories

Table 10 List of all outcomes and those that met consensus (those that met consensus were included in the core outcome set)

Conclusion

This study aimed to generate a list of core learning outcomes which might be developed through international placements and variables which might affect their development. We found 55 peer-reviewed papers and extracted 133 outcomes and 34 variables Table 3. The most recent research to summarise learning outcomes [3] found 40 individual benefits in seven domains: clinical skills, management skills, communication and teamwork, patient experience and dignity, policy, academic skills and personal satisfaction and interest. Our results support the domains but present the outcomes at a more granular level. For example, the previous review reports ‘management skills’ as a domain, which includes the outcome of ‘leadership and management’. We extracted more granular knowledge, skills and attitudes which would map into the domain of ‘management’, such as ability to manage self, ability to lead by example and ability to manage risk. These more specific outcomes would lend themselves more to measurement due to the reported difficulties with assessment of domains [15, 16]. By extracting outcomes at a granular level, we were also able to highlight many outcomes that do not fit neatly into any of the pre-defined categories of previous research such as ‘ability to cope’ or those that fit into more than one, i.e. ‘ability to disseminate best practice globally’. Our study is the first to summarise the variables which have been assumed or proposed to influence learning in international placements, which will allow for hypothesis testing in the future. The outcome set provides a framework of personal and professional learning across healthcare professional groups. This is important as previous literature has tended to focus on specific professional cadres, so this COS would allow comparison and collation across professional groups [35, 36].

Our study generated a list of 28 potential negative outcomes. It is interesting that only eight of these were retained in the Delphi, i.e. stakeholders were in agreement that these negative outcomes were either not likely to happen or likely to happen to a range of healthcare professionals. Only one negative outcome was considered core: ‘health consequences’. This indicates that stakeholders believe almost all negative outcomes do not happen on many or most placements. There is much less consensus about the negative aspects of placements.

The literature contains stated or implied variables which might influence learning on international placements, and this study has synthesised these, finding 33 variables. This provides a framework for future research that aims to study the interactions between variables and outcomes by empirically testing some of the hypotheses reported or assumed in the literature.

Historically, international volunteering has been conceptualised as a benefit to the LMIC and a loss to the HIC [8, 9]. Recent policy documents explicitly discuss the benefit to UK health professionals in terms of personal and professional development and the necessity to develop competencies to be used in training curricula [9]. This study will facilitate the specification and exploration of learning outcomes and so in the future help in addressing the imbalanced discourse of the “benefitting LMIC” and the “donor HIC”. Additionally, a recent Royal College policy describes what competencies paediatricians need to work globally, or with a global population in the United Kingdom [37]. Many of the competencies described map onto the core outcome set suggesting that international placements themselves may provide a vehicle for developing these necessary competencies. In fact, the core outcome set maps onto policy documents such as the Health Education England (HEE) Framework 15: 2014–2029, which suggests the future NHS workforce needs to be flexible, open to innovation and change and life-long learners (all components of the COS) [18]. The core outcome set provides a way of framing and evidencing the NHS benefits. Future work will focus on how the core outcome set can be used as a tool to measure outcomes. The research has also influenced the production of the Health Education England Global strategy, which aims to embed global learning opportunities into NHS training [38].

In summary, there is a broad range of learning outcomes which we have synthesised into a set of 116 core outcomes agreed by a group of 45 stakeholders from various invested groups that could be used in future assessment of learning and testing of hypotheses about what leads to or detracts from learning. We also extracted 33 variables from the literature. We reported a list of negative outcomes, as well as every variable that has been reported (implicitly or explicitly) to affect learning. The core outcome set and variables will enable the development of assessments of health professional learning in international placements, which has implications for how international placements are created and on the support for international placements amongst UK healthcare organisations.

Limitations

This study has a number of limitations. Firstly, we did not update the systematic review because this was the first stage of the outcome set development, and therefore, new outcomes could not be added. We conducted a scoping search using the same search strategy in March 2018 and found 23 new papers had been published. We read these papers and did not find any new outcomes or variables reported. Secondly, the papers included in the meta-synthesis included both those with primary data and those which did not. Formal risk of bias assessment, using standard tools, was therefore not possible. However, it is important to note that the papers included and the findings of the Delphi indicate an overall positive attitude towards international placements, with 96% of papers in the review reporting positive outcomes as opposed to 49% reporting negative outcomes. It is possible that there is publication bias, in which reports of negative experiences are less likely to be written and/or accepted for publication. In the Delphi, participants agreed most of the positive outcomes were core and very few negative. It may be that Delphi participants (particularly those who choose to dedicate hours of their own time) feel more positively about the outcomes than those that were invited but chose not to participate. This represents a risk of bias both in terms of an underreporting of negative outcomes and an inconsistent reporting of variables, with variables influencing outcomes being reported by people whose outcomes had been positive.

Future research and recommendations

The core outcome set could be developed into a tool to assess outcomes. Measurement of learning outcomes is not straightforward, and self-report of learning is fraught with difficulties, including people not knowing what they do not know and people not being aware of what has changed for them at a particular time [39]. Nonetheless, metrics and standard indicators are useful for policy and decision-making [40], and this COS could facilitate quantification and the variables could facilitate hypothesis testing.