- Open Access
- Open Peer Review
The benefits of international volunteering in a low-resource setting: development of a core outcome set
Human Resources for Health volume 16, Article number: 69 (2018)
Qualitative narrative analysis and case studies form the majority of the current peer-reviewed literature about the benefits of professional volunteering or international placements for healthcare professionals. These often describe generalised outcomes that are difficult to define or have multiple meanings (such as ‘communication skills’ or ‘leadership’) and are therefore difficult to measure. However, there is an interest from employers, professional groups and individual volunteers in generating metrics for monitoring personal and professional development of volunteers and comparing different volunteering experiences in terms of their impact on the volunteers. In this paper, we describe two studies in which we (a) consolidated qualitative research and individual accounts into a core outcome set and (b) tested the core outcome set in a large group of global health stakeholders.
We conducted a systematic review and meta-synthesis of literature to extract outcomes of international placements and variables that may affect these outcomes. We presented these outcomes to 58 stakeholders in global health, employing a Delphi method to reach consensus about which were ‘core’ and which were likely to be developed through international volunteering.
The systematic review of 55 papers generated 133 unique outcomes and 34 potential variables. One hundred fifty-six statements were then presented to the Delphi stakeholders, of which they agreed 116 were core to a wide variety of healthcare professional practice and likely to be developed through international experiences. The core outcomes (COs) were both negative and positive and included skills, knowledge, attitudes and outcomes for healthcare organisations.
We summarised existing literature and stakeholder opinion into a core outcome set of 116 items that are core to healthcare professional practice and likely to be developed through international experiences. We identified, in the literature, a set of variables that could affect learning outcomes. The core outcome set will be used in a future study to develop a psychometric assessment tool.
Volunteering, or temporarily working in low-resource settings, is often seen solely as a means of helping those in poorer economies . Many professionals find it difficult to obtain support to volunteer and report lack of recognition upon return, which is disincentive to volunteerism . Furthermore, health professionals that volunteer abroad predominantly do so using annual leave, rather than recognised study leave for continued professional development [3, 4]. The notion that those from high-income countries (HICs) are altruistically offering ‘help’ to those in low- and middle-income countries (LMICs) can also lead to a distortion of the partnership relationship between high- and low-income partners in health partnerships. The low-income partners can be seen as beneficiaries and the high-income partners seen as donors [5,6,7]. Furthermore, a tension often exists between UK healthcare professionals and local international staff, as the intentions or role of healthcare professionals and students is often not explicit to the teams with whom they are working. However, the donor-recipient relationship is becoming increasingly contested in recent literature and policy and mutual benefits realised [8, 9].
There is an imperative, therefore, to fully understand the learning outcomes that are possible for HIC health professionals working in low-resource settings and, in particular, to help recognise these activities as educational development [2, 3]. Understanding ‘what’ is gained would allow specification of intended learning outcomes for training and continuing professional development and to make the gain for the HIC more explicit. Understanding under what circumstances learning outcomes occur would result in an understanding of how to maximise that gain.
Literature that explores what and how healthcare professionals learn from temporarily working or volunteering in a low-resource setting tends to report anecdotes or single reports, which provide a lower level of evidence [4, 10]. Furthermore, benefits are detailed in broad categories, with ‘leadership’, ‘communication’ and ‘cultural awareness’ being frequently reported [3, 11,12,13], with a focus on one of these skill sets in depth or a list of outcomes under umbrella terms, such as communication or leadership skills [3, 14]. These broad labels make an assessment of the learning outcomes difficult as they might contain multiple underpinning knowledge, skills, practice and attitudes. Self-assessment of broad terms is not well associated with objective performance ; individuals struggle to assess themselves in relation to ambiguous or ill-defined traits [16, 17]. Specifying learning at this broad level means that the more granular levels remain unspecified. A higher-level group might contain a wide range of lower-level outcomes and might not contain others, which would reduce the content validity of an assessment.
Understanding the metrics of health professional volunteerism would have a significant impact on current continued professional development (CPD) policy because international experiences could be evidenced as beneficial to personal and professional development. Numerous policy documents about future health workforce highlight the importance of skills such as leadership, communication and adaptability [18, 19]. Such skills have been described as key outcomes of international placements in LMICs, but have yet to be quantified to enable comparison with other learning opportunities .
In a systematic review of the evidence of the benefits to the United Kingdom of health partnership work, Jones et al. reported 40 individual benefits grouped within seven key domains (communication and teamwork, clinical skills, management skills, patient experience and dignity, policy, academic skills and personal satisfaction and interest). There were a number of features of this review that makes it insufficient for the purposes of measuring learning outcomes from international volunteering. Firstly, this review focused only on health partnerships, a specific type of health link, and not all types of volunteering or international placements. Secondly, the findings were categorised broadly, with the difficulties of broad measurement specified above. Thirdly, the professions in their search terms were only doctors and nurses. Finally, it did not extract factors that may affect learning outcomes. For the purpose of measurement, we needed to include literature from a broad range of experiences, extract outcomes at a granular level, include all healthcare staff groups and extract variables that may affect these outcomes.
The outcomes for health professionals are not always positive and the costs of international placements in literature have included health consequences, skills degradation and financial cost [3, 20, 21], reputational, health and opportunity . Research has explored the costs and benefits of international placements [13, 20, 22] and barriers to volunteering, but no research has yet listed all reported negative outcome .
Many aspects of LMIC placements are different from working in a HIC. Relationships between outcomes and these aspects have been proposed, for example that individuals learnt from the opportunity to interact with more patients or conditions than in the United Kingdom [10, 24] and that longer stays may be more beneficial than shorter stays [25, 26]. These variables have not been systematically reported.
This current paper presents two studies: a meta-synthesis and a Delphi. The meta-synthesis aimed to (a) detail the personal and professional development outcomes of international work, at a granular level, i.e. ‘knowledge about procedures rarely conducted in the United Kingdom’ (rather than at a too broad level, i.e. clinical skills or too specific level, i.e. experience conducting vesico-vaginal fistula surgery); (b) to report the variables that influence these personal and professional development outcomes; and (c) to explore if the review including all types of UK health professional placement and all cadres of staff found the same categories as the most recent review of Jones et al. . The Delphi study aimed to gather consensus from those with knowledge and expertise in international health professional learning and development, to refine a set of agreed core outcomes.
Study 1: Meta-synthesis study design and sample
The systematic review of peer-reviewed literature, published in academic journals, was conducted between September and November 2014. Inclusion criteria included that (1) participants must not be in receipt of their full UK salary (a stipend or living allowance was permissible), thus excluding those in permanent employment overseas; (2) health professionals or health professional students (students were included, as much research has been conducted about educational outcomes in students); (3) activities must be health-focused to ensure outcomes were related to clinical work; (4) some participants must have departed from the United Kingdom and be UK citizens (papers that included a partial UK sample were included); (5) some participants must only have travelled to a LMIC; and (6) the paper must reference something that is perceived as a benefit, cost or potential variable, (7) there were no date restrictions. Guidelines for inclusion were used to ensure consistency.
Each paper was screened by one team member (NT) to ensure that it met the inclusion criteria. A second team member (JC) independently checked the first 20% of the included papers to ensure agreement of implementation of inclusion criteria. This was then discussed in a meeting. Disagreements would have been resolved using discussion and refining inclusion criteria for greater specificity; however, the reviewers agreed on all of the papers for inclusion (Table 1).
Data sources and study selection
A standard set of terms were used to search 11 databases for peer-reviewed literature between the earliest date indexed and the time of the review. This included five columns of synonyms relating to outcomes and variables, international volunteering placements, health professionals, the United Kingdom and LMICs (see Additional file 1). The databases were medical and generic databases: Cochrane Economic Evaluations, Health Management Information Consortium, Health Business Elite, Web of Knowledge/Social Sciences Citation Index, PsycINFO, CINAHL, AMED, International Bibliography of Social Sciences, Social Services Abstracts and Sociological Abstracts, Global Health and JSTOR.
The abstracts and titles of each result of the electronic database search were screened, papers that did not meet inclusion criteria were removed and retained papers were rescreened to confirm inclusion.
Reference lists of all included papers were assessed. Any papers that were of relevance were assessed against the inclusion criteria.
We chose to include papers that were peer-reviewed but did not present empirical findings; therefore, the Cochrane risk of bias tool was not applicable to this research . We categorised the papers using a quality framework .
We took a thematic synthesis approach to data extraction , which consists of three stages: line-by-line coding of text, development of descriptive themes and generation of analytical themes. We did not undertake the third stage as our purpose was the extract outcomes as a low level and the third stage has been criticised for being open to the judgement of the researcher [29, 30].
Each study that met the inclusion criteria was read, and any text (related to variables or positive/negative outcomes, at an individual, national or institutional level) was coded according to both content (explicitly stated in the papers) and meaning (inferred by the researcher). Outcomes were defined as anything that happens to UK health professionals as a result of volunteering/international placements (at an individual, national or institutional level), both positive and negative. Variables were any factors that reported influence outcomes, both implicitly and explicitly.
Using Nvivo, a node was created at a ranked level for each component of descriptive theme. For example, the outcome experience conducting ‘vesico-vaginal fistula surgery’ was coded within the second-order theme of ‘greater knowledge of procedures not used in the United Kingdom’ within the higher-order theme of ‘Increased awareness of and knowledge about conditions and procedures rarely encountered in the United Kingdom’. We decided that the lowest level of specificity would be applicable to all/most professions and generalisable across situations. As each paper was coded, the nodes were adapted, developed and generated. Two researchers (NT, JC) independently reviewed the first 20% of papers and then met to develop a coding framework together. There were no disagreements as we were not looking to categorise, but rather develop a matrix of emerging codes; therefore, any differences in extraction occurred only when one reviewer had overlooked an outcome cost or variable. The second reviewer verified the extraction of the data from a further 20% of papers.
Study 2: Stakeholder Delphi
We used the Delphi method, an iterative process of rounds in which data are collected and condensed into a group consensus . A series of virtual questionnaires record participant’s agreement with statements concerning a particular topic. Delphi is often used to develop COS in health research [32, 33]. As we were creating a core outcome set, this stage of the process only included the outcomes extracted in study 1; variables were not included.
In round 1, we held a face-to-face discussion group with stakeholders to generate outcomes. Subsequent rounds were online (with paper version emailed if there were technical difficulties). Participants were asked to indicate to what extent they agreed or disagreed each outcome was a core outcome of international placements and volunteering.
Participants were people who were volunteering health professionals; coordinators of international health professional volunteers, responsible for intended learning outcomes (ILOs) for health professionals; coordinators of health partnerships; study health professional education and international development; educational commissioners and NHS stakeholders. Participants were recruited for an initial workshop from a global health network, to ensure that participants from each of the stakeholder groups were invited and represented. Non-attendees were invited to participate online. After this event, a snow-ball sampling technique was used to reach further stakeholders from each group for online rounds; participants were asked to recommend interested individuals.
Instrumentation round 1: Stakeholder face-to-face discussion and pilot
In order to generate a list of outcomes, any new data generated from round 1 was added to the existing coding framework (see Additional file 1). Outcomes were then generated by presenting the highest-order theme as the outcome and any relevant lower-order themes as examples within brackets to add context. We input outcomes from the meta-synthesis and any additional outcomes from round 1 of the Delphi, into the hosting software. We piloted round 2 with seven members of the research team, who commented on structure, grammar, wording, level of specificity and technical issues. With the addition of items from the Delphi round 1 and comments from the pilot (and separation of some outcomes into two unique outcomes), the 133 outcomes from the meta-synthesis were converted into a list comprising of 156 outcomes to go forward to round 2.
Rounds 2–4: The online rounds of the Delphi
Two team members divided the 156 outcomes into three categories (see Table 2): knowledge, skills and attitudes (n = 115); organisational outcomes (n = 8); and negative outcomes (n = 33). Statements were presented alongside a 7-point Likert-type scale, regarding agreement as to whether each statement should be “considered a ‘core outcome’ of international placements that should be measured in a toolkit”. The scale used the following numbers to represent agreement: 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = no preference, 5 = slightly agree and 6 = agree, 7 = strongly agree. For emphasis, the phrase ‘core outcome’ was presented in bold and the definition was repeated in numerous emails, instructions and synopsis. A core outcome was defined in the following way:
A core outcome is something that is common, important and applicable across a wide range of settings. It can be a benefit or cost, but it must be something that would be more likely to happen to an individual on international placement rather than somebody working in the UK.
For each round, participants had 14 days to respond. Email reminders were sent to invitees frequently. However, as the initial questionnaire was particularly long, some participants requested an extension of the deadline by 10 days and 2 days at round 3. In round 4, participants who had not responded in round 3 (but had in round 2) were invited to re-join the study; many stakeholders worked internationally and had limited internet access at certain periods. In round 4, the expressions of some statements were changed in light of the comments from previous rounds to improve clarity.
The statements with at least 70% consensus in the previous round were retained and not re-presented to the group. Therefore, by round 4, a much smaller group of non-consensus statements were presented. In rounds 3 and 4, participants were asked to use the same Likert scale and reconsider their answers from the previous round (displayed) in light of the group median and any anonymised comment gathered in the previous round.
Study 1: Meta-synthesis
The search of the electronic databases generated 521 hits including duplicates, i.e. 384 unique papers. Twenty-two papers met inclusion criteria. Citation mapping revealed a further 33 papers which were included. Therefore, the total number of papers from which data was extracted was 55. The main reasons for exclusions of papers were (1) not concerning the subject of interest, (2) non-British populations, (3) no health focus, (4) only placements in HIC, 5) only including paid/permanent staff and (6) reporting no benefits, outcomes or costs.
No papers included fell within the top two quality categories proposed by Benzies et al.: randomised controlled trials . Some papers included qualitative or quantitative data (23/55, 42%), but the majority of papers reported no primary data.
Positive outcomes were extracted from 96% (53/55) of the papers, whilst negative outcomes were extracted from only 49% (27/55). Potential variables that could affect these outcomes were extracted from 90.91% (50/55) of papers. None of the papers explicitly reported or explored how variables were thought to affect outcomes (Fig. 1 and Table 3).
We found 133 unique outcomes, including 28 negative outcomes. The outcomes extracted could be categorised within NHS professional development terminology; there were 24 items about knowledge, 44 about skills and 20 about attitudes . Six were organisational benefits and 29 negative; 10 were categorised as ‘other’. Organisational outcomes were deliberately separated, as organisation-specific outcomes were identified in addition to the general positive effect of staff with developed knowledge, skills and attitudes. Only 29 (22%) of the outcomes stated in the literature were negative, suggesting an overall positive attitude towards international placements from the authors (Fig. 2 and Tables 4 and 5).
Study 2: Delphi
Fifty-one participants attended the round 1 workshop. Invitations were sent to 259 participants for the online Delphi, and 78 (30%) accepted. Once enrolled in the study, response rates remained high: round 2, n = 58/78 (74%); round 3, n = 49 (63%); and round 4, n = 45 (58%). More than half of the participants were involved in global health policy, and one third of the participants had volunteered.
After round 2, 98 of the 156 statements (63%) were retained; this meant over 70% of the stakeholders agreed or strongly agreed these 98 statements were core outcomes. After re-considering their own vote in round 2, the group median and anonymous comments regarding each statement, 13 additional statements were retained in round 3. Finally, after readdressing the above items for the second time, an additional five statements met consensus and were retained in round 4. Of the items that met consensus, 99 were positive and eight were negative. Positive outcomes were of educational benefits to the British health professionals and negative outcomes were drawbacks, costs or negative effects (Tables 6, 7, 8, 9 and 10).
This study aimed to generate a list of core learning outcomes which might be developed through international placements and variables which might affect their development. We found 55 peer-reviewed papers and extracted 133 outcomes and 34 variables Table 3. The most recent research to summarise learning outcomes  found 40 individual benefits in seven domains: clinical skills, management skills, communication and teamwork, patient experience and dignity, policy, academic skills and personal satisfaction and interest. Our results support the domains but present the outcomes at a more granular level. For example, the previous review reports ‘management skills’ as a domain, which includes the outcome of ‘leadership and management’. We extracted more granular knowledge, skills and attitudes which would map into the domain of ‘management’, such as ability to manage self, ability to lead by example and ability to manage risk. These more specific outcomes would lend themselves more to measurement due to the reported difficulties with assessment of domains [15, 16]. By extracting outcomes at a granular level, we were also able to highlight many outcomes that do not fit neatly into any of the pre-defined categories of previous research such as ‘ability to cope’ or those that fit into more than one, i.e. ‘ability to disseminate best practice globally’. Our study is the first to summarise the variables which have been assumed or proposed to influence learning in international placements, which will allow for hypothesis testing in the future. The outcome set provides a framework of personal and professional learning across healthcare professional groups. This is important as previous literature has tended to focus on specific professional cadres, so this COS would allow comparison and collation across professional groups [35, 36].
Our study generated a list of 28 potential negative outcomes. It is interesting that only eight of these were retained in the Delphi, i.e. stakeholders were in agreement that these negative outcomes were either not likely to happen or likely to happen to a range of healthcare professionals. Only one negative outcome was considered core: ‘health consequences’. This indicates that stakeholders believe almost all negative outcomes do not happen on many or most placements. There is much less consensus about the negative aspects of placements.
The literature contains stated or implied variables which might influence learning on international placements, and this study has synthesised these, finding 33 variables. This provides a framework for future research that aims to study the interactions between variables and outcomes by empirically testing some of the hypotheses reported or assumed in the literature.
Historically, international volunteering has been conceptualised as a benefit to the LMIC and a loss to the HIC [8, 9]. Recent policy documents explicitly discuss the benefit to UK health professionals in terms of personal and professional development and the necessity to develop competencies to be used in training curricula . This study will facilitate the specification and exploration of learning outcomes and so in the future help in addressing the imbalanced discourse of the “benefitting LMIC” and the “donor HIC”. Additionally, a recent Royal College policy describes what competencies paediatricians need to work globally, or with a global population in the United Kingdom . Many of the competencies described map onto the core outcome set suggesting that international placements themselves may provide a vehicle for developing these necessary competencies. In fact, the core outcome set maps onto policy documents such as the Health Education England (HEE) Framework 15: 2014–2029, which suggests the future NHS workforce needs to be flexible, open to innovation and change and life-long learners (all components of the COS) . The core outcome set provides a way of framing and evidencing the NHS benefits. Future work will focus on how the core outcome set can be used as a tool to measure outcomes. The research has also influenced the production of the Health Education England Global strategy, which aims to embed global learning opportunities into NHS training .
In summary, there is a broad range of learning outcomes which we have synthesised into a set of 116 core outcomes agreed by a group of 45 stakeholders from various invested groups that could be used in future assessment of learning and testing of hypotheses about what leads to or detracts from learning. We also extracted 33 variables from the literature. We reported a list of negative outcomes, as well as every variable that has been reported (implicitly or explicitly) to affect learning. The core outcome set and variables will enable the development of assessments of health professional learning in international placements, which has implications for how international placements are created and on the support for international placements amongst UK healthcare organisations.
This study has a number of limitations. Firstly, we did not update the systematic review because this was the first stage of the outcome set development, and therefore, new outcomes could not be added. We conducted a scoping search using the same search strategy in March 2018 and found 23 new papers had been published. We read these papers and did not find any new outcomes or variables reported. Secondly, the papers included in the meta-synthesis included both those with primary data and those which did not. Formal risk of bias assessment, using standard tools, was therefore not possible. However, it is important to note that the papers included and the findings of the Delphi indicate an overall positive attitude towards international placements, with 96% of papers in the review reporting positive outcomes as opposed to 49% reporting negative outcomes. It is possible that there is publication bias, in which reports of negative experiences are less likely to be written and/or accepted for publication. In the Delphi, participants agreed most of the positive outcomes were core and very few negative. It may be that Delphi participants (particularly those who choose to dedicate hours of their own time) feel more positively about the outcomes than those that were invited but chose not to participate. This represents a risk of bias both in terms of an underreporting of negative outcomes and an inconsistent reporting of variables, with variables influencing outcomes being reported by people whose outcomes had been positive.
Future research and recommendations
The core outcome set could be developed into a tool to assess outcomes. Measurement of learning outcomes is not straightforward, and self-report of learning is fraught with difficulties, including people not knowing what they do not know and people not being aware of what has changed for them at a particular time . Nonetheless, metrics and standard indicators are useful for policy and decision-making , and this COS could facilitate quantification and the variables could facilitate hypothesis testing.
Core outcome set
Continued professional development
Intended learning outcome
Low- and middle-income country
MOVE Project, Measuring the outcomes of volunteering for education
National Health Service
Crisp N. Mutual learning and reverse innovation--where next? Glob Health. 2014;10:14. https://doi.org/10.1186/1744-8603-10-14.
Smith C, Pettigrew LM, Seo H-N, Dorward J. Combining general practice with international work: online survey of experiences of UK GPs. JRSM Short Rep. 2012;3:46.
Jones FA, Knights DP, Sinclair VF, Baraitser P. Do health partnerships with organisations in lower income countries benefit the UK partner? Glob Health. 2013;9(38). https://doi.org/10.1186/1744-8603-9-38.
Dean E. Tanzania changed me. Nurs Stand. 2013;27:16–7 http://www.ncbi.nlm.nih.gov/pubmed/23980965. Accessed 29 Dec 2014.
Gilbert BJ, Miller C, Corrick F, Watson RA. Should trainee doctors use the developing world to gain clinical experience? The annual Varsity Medical Debate - London, Friday 20th January, 2012. Philos Ethics Humanit Med. 2013;8(1). https://doi.org/10.1186/1747-5341-8-1.
Leather AJM, Butterfield C, Peachey K, Silverman M, Sheriff RS. International health links movement expands in the United Kingdom. Int Health. 2010;2:165–71. https://doi.org/10.1016/j.inhe.2010.04.004.
Banerjee A. Medical electives: a chance for international health. J R Soc Med. 2010;103:6–8. https://doi.org/10.1258/jrsm.2009.090266.
Crisp N. Turning the world upside down : the search for global health in the twenty-first century: London: Royal Society of Medicine Press; 2010.
Tropical Health Education Trust. In our mutual interest. 2017. https://www.thet.org/wp-content/uploads/2017/08/In-Our-Mutual-Interest.pdf.
Sikka N. All aboard with impact India. Br Dent J. 2009;207:401–3. https://doi.org/10.1038/sj.bdj.2009.932.
Young P, Smith C, Pettigrew L, Seo H-N, Blane D. International work and leadership in UK general practice. Leadersh Health Serv. 2014;27:87–103. https://doi.org/10.1108/LHS-01-2013-0003.
Norton D, Marks-Maran D. Developing cultural sensitivity and awareness in nursing overseas. Nurs Stand. 2014;28:39–43. https://doi.org/10.7748/ns.28.44.39.e8417.
Peiying N, Goddard T, Gribble N, Pickard C. International placements increase the cultural sensitivity and competency of professional health students: a quantitative and qualitative study. J Phys Ther Educ. 2012;26(61) https://espace.curtin.edu.au/handle/20.500.11937/41580. Accessed 23 Aug 2014.
Thompson MJ, Huntington MK, Hunt DD, Pinsky LE, Brodie JJ. Educational effects of international health electives on U.S. and Canadian medical students and residents: a literature review. Acad Med. 2003;78:342–7.
Dunning D, Heath C, Suls JM. Flawed self-assessment: implications for health, education, and the workplace. Psychol Sci Public Interes. 2004;5:69–106. https://doi.org/10.1111/j.1529-1006.2004.00018.x.
Dunning D, Meyerowitz JA, Holzberg AD. Ambiguity and self-evaluation: the role of idiosyncratic trait definitions in self-serving assessments of ability. J Pers Soc Psychol. 1989;57:1082–90.
Suls J, Lemos K, Stewart HL. Self-esteem, construal, and comparisons with the self, friends, and peers. J Pers Soc Psychol. 2002;82:252–61.
Health Education England. Framework 15: Health Education England Strategic Framework, 2014–2029. 2014. https://www.hee.nhs.uk/sites/default/files/documents/HEE%20strategic%20framework%202017_1.pdf Strategic Framework 2015 Refresh Final document.pdf. Accessed 20 Jan 2016.
NHS England. 5 Year Forward View. 2014. https://www.england.nhs.uk/ourwork/futurenhs/.
Wyler N, Green S, Boddington N, Davies C, Friedli K, Lankester T. Travel related illness in short-term volunteers from the UK to developing countries. Travel Med Infect Dis. 2012;10:172–8. https://doi.org/10.1016/j.tmaid.2012.04.002.
Banatvala N, Macklow-Smith A. Integrating overseas work with an NHS career. BMJ. 1997;314:2–2. https://doi.org/10.1136/bmj.314.7093.2.
Ackers HL, Ackers-Johnson J, Chatwin J, Tyler N. Healthcare, frugal innovation, and professional voluntarism. Cham: Springer International Publishing; 2017. https://doi.org/10.1007/978-3-319-48366-5.
Chatwin J, Ackers L. Organisational barriers to the facilitation of overseas volunteering and training placements in the NHS. BMC Health Serv Res. 2018;18:69. https://doi.org/10.1186/s12913-018-2853-z.
Baguley D, Killeen T, Wright J. International health links: an evaluation of partnerships between health-care organizations in the UK and developing countries. Trop Dr. 2006;36:149–54. https://doi.org/10.1258/004947506777978181.
Thompson K, Boore J, Deeny P. A comparison of an international experience for nursing students in developed and developing countries. Int J Nurs Stud. 2000;37:481–92.
Evans R, Dotchin C, Walker R. Maximising the value from the elective experience: post-elective workshops. Clin Teach. 2013;10:362–7. https://doi.org/10.1111/tct.12033.
Hannes K. Critical appraisal of qualitative research. In: Supplementary guidance for inclusion of qualitative research in Cochrane systematic reviews of interventions; 2011.
Benzies KM, Premji S, Hayden KA, Serrett K. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid-Based Nurs. 2006;3:55–61.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. https://doi.org/10.1186/1471-2288-8-45.
Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, et al. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003;56:671–84 http://www.ncbi.nlm.nih.gov/pubmed/12560003. Accessed 11 Feb 2015.
Devane D, Begley C, Clarke M, Horey D, OBoyle C. Evaluating maternity care: a core set of outcome measures. Birth. 2007; https://www.sciencedirect.com/science/article/abs/pii/S1477893912000646. Accessed 16 Mar 2015.
Boers M, Brooks P, Simon LS, Strand V, Tugwell P. OMERACT: An international initiative to improve outcome measurement in rheumatology. Clin Exp Rheumatol. 2005;23(5 SUPPL):39.
Prinsen CAC, Vohra S, Rose MR, King-Jones S, Ishaque S, Bhaloo Z, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a “core outcome set”. Trials. 2014;15:247. https://doi.org/10.1186/1745-6215-15-247.
NHS Employers. Simplified knowledge and skills framework (KSF). NHS England. 2014. http://www.msg.scot.nhs.uk/wp-content/uploads/Short-Guide-to-KSF-Dimensions.pdf
Kiernan P, O’Dempsey T, Kwalombota K, Elliott L, Cowan L. Evaluation of effect on skills of GP trainees taking time out of programme (OOP) in developing countries. Educ Prim Care. 2014;25:78–83 http://www.ncbi.nlm.nih.gov/pubmed/24593970. Accessed 22 Jul 2015.
Button L, Green B. The impact of international placements on nurses’ personal and professional lives: literature review. J Adv. 2005; https://www.ncbi.nlm.nih.gov/pubmed/15811111. Accessed 22 Jul 2015.
Williams B, Morrissey B, Goenka A, Magnus D, Allen S. Global child health competencies for paediatricians. Lancet. 2014;384:1403–5. https://doi.org/10.1016/S0140-6736(14)61128-4.
Health Education England. Global Engagement Strategy: Strengthening the NHS workforce through global learning, vol. 2018.
Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987;40:171–8. https://doi.org/10.1016/0021-9681(87)90069-5.
Li RM, National Research Council (U.S.). Committee on Advancing Social Science Theory: The Importance of Common Metrics., National Research Council (U.S.). Committee on Social Science Evidence for Use., Workshop on Advancing Social Science Theory: The Importance of Common Metrics (2010 : National Academies (U.S.)). The importance of common metrics for advancing social science theory and research: a workshop summary. National Academies Press; 2011. https://www.nap.edu/catalog/13034/the-importance-of-common-metrics-for-advancing-social-science-theory-and-research. Accessed 28 Sep 2018.
We would like to thank Anya Ahmed, Louise Ackers, Natalie Tate, Eileen Cunningham, Hassan Osman and James Ackers-Johnson for being involved in the project and helping with the piloting of the Delphi.
We would like to thank all of the stakeholders involved in the Delphi for their significant contribution of time and effort.
The full title of the study from which this analysis was derived was Measuring the outcomes of volunteering for education (MOVE). The study was funded by Health Education England (Global Health Exchange). The research team was independent from the funding agency. The views expressed in this publication are those of the authors and not necessarily those of Health Education England or the Department of Health.
This work was supported by a project development grant from Health Education England (HEE), through the Global Health Exchange (GHE), grant ref. NURA54.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The approval for the study was obtained from the Ethical Research Committee, University of Salford, and the University of Manchester Research Ethics Committee.
Consent for publication
Professor Ged Byrne is the Director of Global Engagement for Health Education England. The other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Systematic review search criteria. Systematic review instructions for screeners. Systematic review results: table of literature included in the review. Systematic review and meta-synthesis results: table of outcomes. Systematic review and metasynthesis results: table of variables that may affect outcomes. List of core outcomes after Delphi study: percentage of consensus, positive (include)/negative (exclusive) and the overall rank in terms of stakeholder agreement. Descriptive statistics for each statement in the Delphi across the three rounds. (DOCX 143 kb)