The benefits of international volunteering in a low-resource setting: development of a core outcome set

Background Qualitative narrative analysis and case studies form the majority of the current peer-reviewed literature about the benefits of professional volunteering or international placements for healthcare professionals. These often describe generalised outcomes that are difficult to define or have multiple meanings (such as ‘communication skills’ or ‘leadership’) and are therefore difficult to measure. However, there is an interest from employers, professional groups and individual volunteers in generating metrics for monitoring personal and professional development of volunteers and comparing different volunteering experiences in terms of their impact on the volunteers. In this paper, we describe two studies in which we (a) consolidated qualitative research and individual accounts into a core outcome set and (b) tested the core outcome set in a large group of global health stakeholders. Method We conducted a systematic review and meta-synthesis of literature to extract outcomes of international placements and variables that may affect these outcomes. We presented these outcomes to 58 stakeholders in global health, employing a Delphi method to reach consensus about which were ‘core’ and which were likely to be developed through international volunteering. Results The systematic review of 55 papers generated 133 unique outcomes and 34 potential variables. One hundred fifty-six statements were then presented to the Delphi stakeholders, of which they agreed 116 were core to a wide variety of healthcare professional practice and likely to be developed through international experiences. The core outcomes (COs) were both negative and positive and included skills, knowledge, attitudes and outcomes for healthcare organisations. Conclusions We summarised existing literature and stakeholder opinion into a core outcome set of 116 items that are core to healthcare professional practice and likely to be developed through international experiences. We identified, in the literature, a set of variables that could affect learning outcomes. The core outcome set will be used in a future study to develop a psychometric assessment tool. Electronic supplementary material The online version of this article (10.1186/s12960-018-0333-5) contains supplementary material, which is available to authorized users.


Background
Volunteering, or temporarily working in low-resource settings, is often seen solely as a means of helping those in poorer economies [1]. Many professionals find it difficult to obtain support to volunteer and report lack of recognition upon return, which is disincentive to volunteerism [2]. Furthermore, health professionals that volunteer abroad predominantly do so using annual leave, rather than recognised study leave for continued professional development [3,4]. The notion that those from high-income countries (HICs) are altruistically offering 'help' to those in low-and middle-income countries (LMICs) can also lead to a distortion of the partnership relationship between high-and low-income partners in health partnerships. The low-income partners can be seen as beneficiaries and the high-income partners seen as donors [5][6][7]. Furthermore, a tension often exists between UK healthcare professionals and local international staff, as the intentions or role of healthcare professionals and students is often not explicit to the teams with whom they are working. However, the donor-recipient relationship is becoming increasingly contested in recent literature and policy and mutual benefits realised [8,9].
There is an imperative, therefore, to fully understand the learning outcomes that are possible for HIC health professionals working in low-resource settings and, in particular, to help recognise these activities as educational development [2,3]. Understanding 'what' is gained would allow specification of intended learning outcomes for training and continuing professional development and to make the gain for the HIC more explicit. Understanding under what circumstances learning outcomes occur would result in an understanding of how to maximise that gain.
Literature that explores what and how healthcare professionals learn from temporarily working or volunteering in a low-resource setting tends to report anecdotes or single reports, which provide a lower level of evidence [4,10]. Furthermore, benefits are detailed in broad categories, with 'leadership' , 'communication' and 'cultural awareness' being frequently reported [3,[11][12][13], with a focus on one of these skill sets in depth or a list of outcomes under umbrella terms, such as communication or leadership skills [3,14]. These broad labels make an assessment of the learning outcomes difficult as they might contain multiple underpinning knowledge, skills, practice and attitudes. Self-assessment of broad terms is not well associated with objective performance [15]; individuals struggle to assess themselves in relation to ambiguous or ill-defined traits [16,17]. Specifying learning at this broad level means that the more granular levels remain unspecified. A higher-level group might contain a wide range of lower-level outcomes and might not contain others, which would reduce the content validity of an assessment.
Understanding the metrics of health professional volunteerism would have a significant impact on current continued professional development (CPD) policy because international experiences could be evidenced as beneficial to personal and professional development. Numerous policy documents about future health workforce highlight the importance of skills such as leadership, communication and adaptability [18,19]. Such skills have been described as key outcomes of international placements in LMICs, but have yet to be quantified to enable comparison with other learning opportunities [3].
In a systematic review of the evidence of the benefits to the United Kingdom of health partnership work, Jones et al. reported 40 individual benefits grouped within seven key domains (communication and teamwork, clinical skills, management skills, patient experience and dignity, policy, academic skills and personal satisfaction and interest). There were a number of features of this review that makes it insufficient for the purposes of measuring learning outcomes from international volunteering. Firstly, this review focused only on health partnerships, a specific type of health link, and not all types of volunteering or international placements. Secondly, the findings were categorised broadly, with the difficulties of broad measurement specified above. Thirdly, the professions in their search terms were only doctors and nurses. Finally, it did not extract factors that may affect learning outcomes. For the purpose of measurement, we needed to include literature from a broad range of experiences, extract outcomes at a granular level, include all healthcare staff groups and extract variables that may affect these outcomes.
The outcomes for health professionals are not always positive and the costs of international placements in literature have included health consequences, skills degradation and financial cost [3,20,21], reputational, health and opportunity [3]. Research has explored the costs and benefits of international placements [13,20,22] and barriers to volunteering, but no research has yet listed all reported negative outcome [23].
Many aspects of LMIC placements are different from working in a HIC. Relationships between outcomes and these aspects have been proposed, for example that individuals learnt from the opportunity to interact with more patients or conditions than in the United Kingdom [10,24] and that longer stays may be more beneficial than shorter stays [25,26]. These variables have not been systematically reported.
This current paper presents two studies: a meta-synthesis and a Delphi. The meta-synthesis aimed to (a) detail the personal and professional development outcomes of international work, at a granular level, i.e. 'knowledge about procedures rarely conducted in the United Kingdom' (rather than at a too broad level, i.e. clinical skills or too specific level, i.e. experience conducting vesico-vaginal fistula surgery); (b) to report the variables that influence these personal and professional development outcomes; and (c) to explore if the review including all types of UK health professional placement and all cadres of staff found the same categories as the most recent review of Jones et al. [3]. The Delphi study aimed to gather consensus from those with knowledge and expertise in international health professional learning and development, to refine a set of agreed core outcomes.

Study 1: Meta-synthesis study design and sample
The systematic review of peer-reviewed literature, published in academic journals, was conducted between September and November 2014. Inclusion criteria included that (1) participants must not be in receipt of their full UK salary (a stipend or living allowance was permissible), thus excluding those in permanent employment overseas; (2) health professionals or health professional students (students were included, as much research has been conducted about educational outcomes in students); (3) activities must be health-focused to ensure outcomes were related to clinical work; (4) some participants must have departed from the United Kingdom and be UK citizens (papers that included a partial UK sample were included); (5) some participants must only have travelled to a LMIC; and (6) the paper must reference something that is perceived as a benefit, cost or potential variable, (7) there were no date restrictions. Guidelines for inclusion were used to ensure consistency. Each paper was screened by one team member (NT) to ensure that it met the inclusion criteria. A second team member (JC) independently checked the first 20% of the included papers to ensure agreement of implementation of inclusion criteria. This was then discussed in a meeting. Disagreements would have been resolved using discussion and refining inclusion criteria for greater specificity; however, the reviewers agreed on all of the papers for inclusion (Table 1).

Data sources and study selection
A standard set of terms were used to search 11 databases for peer-reviewed literature between the earliest date indexed and the time of the review. This included five columns of synonyms relating to outcomes and variables, international volunteering placements, health professionals, the United Kingdom and LMICs (see Additional file 1). The databases were medical and generic databases: Cochrane Economic Evaluations, Health Management Information Consortium, Health Business Elite, Web of Knowledge/Social Sciences Citation Index, PsycINFO, CINAHL, AMED, International Bibliography of Social Sciences, Social Services Abstracts and Sociological Abstracts, Global Health and JSTOR.
The abstracts and titles of each result of the electronic database search were screened, papers that did not meet inclusion criteria were removed and retained papers were rescreened to confirm inclusion.

Citation mapping
Reference lists of all included papers were assessed. Any papers that were of relevance were assessed against the inclusion criteria.

Quality assessment
We chose to include papers that were peer-reviewed but did not present empirical findings; therefore, the Cochrane risk of bias tool was not applicable to this research [27]. We categorised the papers using a quality framework [28].

Data extraction
We took a thematic synthesis approach to data extraction [29], which consists of three stages: line-by-line coding of text, development of descriptive themes and generation of analytical themes. We did not undertake the third stage as our purpose was the extract outcomes as a low level and the third stage has been criticised for being open to the judgement of the researcher [29,30].
Each study that met the inclusion criteria was read, and any text (related to variables or positive/negative outcomes, at an individual, national or institutional level) was coded according to both content (explicitly stated in the papers) and meaning (inferred by the researcher). Outcomes were defined as anything that happens to UK health professionals as a result of volunteering/international placements (at an individual, national or institutional level), both positive and negative. Variables were any factors that reported influence outcomes, both implicitly and explicitly.
Using Nvivo, a node was created at a ranked level for each component of descriptive theme. For example, the outcome experience conducting 'vesico-vaginal fistula surgery' was coded within the second-order theme of 'greater knowledge of procedures not used in the United Kingdom' within the higher-order theme of 'Increased awareness of and knowledge about conditions and procedures rarely encountered in the United Kingdom'. We decided that the lowest level of specificity would be Table 1 Inclusion criteria The inclusion criteria for the systematic review were peer-reviewed literature, where: 1) Individuals are either volunteers (i.e. not in receipt of full salary) or students on international placements. 2) Activities have a health focus 3) The individuals must be from the UK travelling to a lower income or lower-middle income country 4) There is reference to (individual, institutional or national) benefits or costs or the variables that moderate/mediate outcomes 5) English Language only applicable to all/most professions and generalisable across situations. As each paper was coded, the nodes were adapted, developed and generated. Two researchers (NT, JC) independently reviewed the first 20% of papers and then met to develop a coding framework together. There were no disagreements as we were not looking to categorise, but rather develop a matrix of emerging codes; therefore, any differences in extraction occurred only when one reviewer had overlooked an outcome cost or variable. The second reviewer verified the extraction of the data from a further 20% of papers.

Study 2: Stakeholder Delphi Design
We used the Delphi method, an iterative process of rounds in which data are collected and condensed into a group consensus [31]. A series of virtual questionnaires record participant's agreement with statements concerning a particular topic. Delphi is often used to develop COS in health research [32,33]. As we were creating a core outcome set, this stage of the process only included the outcomes extracted in study 1; variables were not included.
In round 1, we held a face-to-face discussion group with stakeholders to generate outcomes. Subsequent rounds were online (with paper version emailed if there were technical difficulties). Participants were asked to indicate to what extent they agreed or disagreed each outcome was a core outcome of international placements and volunteering.

Participants
Participants were people who were volunteering health professionals; coordinators of international health professional volunteers, responsible for intended learning outcomes (ILOs) for health professionals; coordinators of health partnerships; study health professional education and international development; educational commissioners and NHS stakeholders. Participants were recruited for an initial workshop from a global health network, to ensure that participants from each of the stakeholder groups were invited and represented. Non-attendees were invited to participate online. After this event, a snow-ball sampling technique was used to reach further stakeholders from each group for online rounds; participants were asked to recommend interested individuals.
Instrumentation round 1: Stakeholder face-to-face discussion and pilot In order to generate a list of outcomes, any new data generated from round 1 was added to the existing coding framework (see Additional file 1). Outcomes were then generated by presenting the highest-order theme as the outcome and any relevant lower-order themes as examples within brackets to add context. We input outcomes from the meta-synthesis and any additional outcomes from round 1 of the Delphi, into the hosting software. We piloted round 2 with seven members of the research team, who commented on structure, grammar, wording, level of specificity and technical issues. With the addition of items from the Delphi round 1 and comments from the pilot (and separation of some outcomes into two unique outcomes), the 133 outcomes from the meta-synthesis were converted into a list comprising of 156 outcomes to go forward to round 2.

Rounds 2-4: The online rounds of the Delphi
Two team members divided the 156 outcomes into three categories (see Table 2): knowledge, skills and attitudes (n = 115); organisational outcomes (n = 8); and negative outcomes (n = 33). Statements were presented alongside a 7-point Likert-type scale, regarding agreement as to whether each statement should be "considered a 'core outcome' of international placements that should be measured in a toolkit". The scale used the following numbers to represent agreement: 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = no preference, 5 = slightly agree and 6 = agree, 7 = strongly agree. For emphasis, the phrase 'core outcome' was presented in bold and the definition was repeated in numerous emails, instructions and synopsis. A core outcome was defined in the following way: A core outcome is something that is common, important and applicable across a wide range of settings. It can be a benefit or cost, but it must be something that would be more likely to happen to an individual on international placement rather than somebody working in the UK.
For each round, participants had 14 days to respond. Email reminders were sent to invitees frequently. However, as the initial questionnaire was particularly long, some participants requested an extension of the deadline by 10 days and 2 days at round 3. In round 4, participants who had not responded in round 3 (but had in round 2) were invited to re-join the study; many stakeholders worked internationally and had limited internet access at certain periods. In round 4, the expressions of some statements were changed in light of the comments from previous rounds to improve clarity. The statements with at least 70% consensus in the previous round were retained and not re-presented to the group. Therefore, by round 4, a much smaller group of non-consensus statements were presented. In rounds 3 and 4, participants were asked to use the same Likert scale and reconsider their answers from the previous round (displayed) in light of the group median and any anonymised comment gathered in the previous round.

Study 1: Meta-synthesis Data sources
The search of the electronic databases generated 521 hits including duplicates, i.e. 384 unique papers. Twenty-two papers met inclusion criteria. Citation mapping revealed a further 33 papers which were included. Therefore, the total number of papers from which data was extracted was 55. The main reasons for exclusions of papers were (1) not concerning the subject of interest, (2) non-British populations, (3) no health focus, (4) only placements in HIC, 5) only including paid/permanent staff and (6) reporting no benefits, outcomes or costs.
No papers included fell within the top two quality categories proposed by Benzies et al.: randomised controlled trials [28]. Some papers included qualitative or quantitative data (23/55, 42%), but the majority of papers reported no primary data.
Positive outcomes were extracted from 96% (53/55) of the papers, whilst negative outcomes were extracted from only 49% (27/55). Potential variables that could affect these outcomes were extracted from 90.91% (50/55) of papers. None of the papers explicitly reported or explored how variables were thought to affect outcomes ( Fig. 1 and Table 3).

Extracted outcomes
We found 133 unique outcomes, including 28 negative outcomes. The outcomes extracted could be categorised within NHS professional development terminology; there were 24 items about knowledge, 44 about skills and 20 about attitudes [34]. Six were organisational benefits and 29 negative; 10 were categorised as 'other'. Organisational outcomes were deliberately separated, as organisation-specific outcomes were identified in addition to the general positive effect of staff with developed knowledge, skills and attitudes. Only 29 (22%) of the outcomes stated in the literature were negative, suggesting an overall positive attitude towards international placements from the authors ( Fig. 2 and Tables 4 and 5).

Study 2: Delphi Participants
Fifty-one participants attended the round 1 workshop. Invitations were sent to 259 participants for the online Delphi, and 78 (30%) accepted. Once enrolled in the study, response rates remained high: round 2, n = 58/78 (74%); round 3, n = 49 (63%); and round 4, n = 45 (58%). More than half of the participants were involved in global health policy, and one third of the participants had volunteered.  "This support is, by necessity, mostly provided by the host supervisor, and home medical schools in effect delegate their duty of care to the host." (Lumb, 2014) "Students should be exposed to a variety of nursing experiences within the host country. This would give them a broad spectrum for comparisons between cultures, nursing practice and health care delivery in those cultures" (Button, 2005) Benefits for host organisation Donations Material/financial benefits Payment for supervision "In order to transform a process favouring the trainee into an equitable exchange, each trainee must recognise the need for reciprocity when a community contributes to his or her education. This might manifest through the provision of resources, such as books and surgical supplies, of teaching and new ideas, or of money, which could be reallocated to meet local need." (Banatlava, 1998)

Income of host country Low Middle High
"They therefore concluded that there was no significant difference in level of knowledge and skill gained by going to a developed or developing country" (Button, 2005) Commitment of local staff to project Staff time pressures Empowerment of local staff Involvement of hospital leaders Project use local experts Local perceptions of volunteers Value of volunteer opinions "It was reported that some overseas staff are wary of offering constructive criticism, not wishing to appear ungrateful. There is a move among many links to address this problem through structured appraisal and evaluation for each visit. One had begun to use anonymous feedback forms to learn from visits and improve the quality and effectiveness of health links." (Baguley, 2006) "As this host explains, two prominent negative aspects are insufficient input and time" (Pearson, 2014) "It may have been helpful to obtain more input from similar programs at an earlier stage of planning, and it would be helpful in the future to establish formal links between programs or a forum for discussion" (Kinnear, 2013) 'degree of developing country ownership' (Smith, 2012)   'the opportunity to work in complicated, poorly resourced and challenging environments' (Kiernan, 2014) After round 2, 98 of the 156 statements (63%) were retained; this meant over 70% of the stakeholders agreed or strongly agreed these 98 statements were core outcomes. After re-considering their own vote in round 2, the group median and anonymous comments regarding each statement, 13 additional statements were retained in round 3. Finally, after readdressing the above items for the second time, an additional five statements met consensus and were retained in round 4. Of the items that met consensus, 99 were positive and eight were negative. Positive outcomes were of educational benefits to the British health professionals and negative outcomes were drawbacks, costs or negative effects (Tables 6, 7

Conclusion
This study aimed to generate a list of core learning outcomes which might be developed through international placements and variables which might affect their development. We found 55 peer-reviewed papers and extracted 133 outcomes and 34 variables Table 3. The most recent research to summarise learning outcomes [3] found 40 individual benefits in seven domains: clinical skills, management skills, communication and teamwork, patient experience and dignity, policy, academic skills and personal satisfaction and interest. Our results support the domains but present the outcomes at a more granular level. For example, the previous review reports 'management skills' as a domain, which includes the outcome of 'leadership and management'. We extracted more granular knowledge, skills and attitudes which would map into the domain of 'management' , such as ability to manage self, ability to lead by example and ability to manage risk. These more specific outcomes would lend themselves more to measurement due to the reported difficulties with assessment of domains [15,16]. By extracting outcomes at a granular level, we were also able to highlight many outcomes that do not fit neatly into any of the pre-defined categories of previous research such as 'ability to cope' or those that fit into more than one, i.e. 'ability to disseminate best practice globally'. Our study is the first to summarise the variables which have been assumed or proposed to influence learning in international placements, which will allow for hypothesis testing in the future. The outcome set provides a framework of personal and professional learning across healthcare professional groups. This is important as previous literature has tended to focus on specific professional   'Increased understanding of the importance of culture in health care and the degree of variability in the countries they visited' [25]   Ability to be innovative with clinical skills

Use of innovative techniques New ways of working)
'Innovation in healthcare delivery and use of resources' [3] Ability to use a broader range of clinical skills Enhancing existing skills and acquiring new clinical skill 'Clinical skills were better and that the trainee had a broader range of skills' [35] Ability to apply clinical skills to another context A more challenging environment or a low resource setting 'They gained hands-on experience of care and developed a keen awareness of how the principles of nursing were applied in contexts very different from that to which they were used'. [25] Ability to work with limited resources Being more resourceful Ability to target resource Ability to find solutions despite limited resources Ability to work without reliance on technology Ability to manage in a low resource setting Understanding the reasons behind lack of resources 'The nurses and doctors there are resourceful with what they have to use. I have learnt a lot and it has made me think differently. [4] Ability to 'get the best out of people' Encouraging people to work together Empowering people to recognise their own strengths and to take possession of their own work/projects Ability to assess the capability of others Encouraging people to work together 'Empowering them to recognise their strengths and not deskilling them' (workshop participant) Ability to manage risk Manage risk in advance Evaluation of environment Understanding the clinical importance of risk management Understanding the wider implication of poorly managed risk 'To manage risks they would not normally be exposed to' (Morgan, 2012) Ability to negotiate with multiple stakeholders 'Improved skills of negotiation with multiple stakeholders' [3] Ability to make independent clinical decisions Ability to make an urgent decision in an emergency Dealing with uncertain outcomes  Ability to improve service Including renewed enthusiasm for service improvement 'Service improvement' [11] Ability to transfer skills and knowledge to another context 'Applying those skills in a different context' (workshop participant) Ability to work towards solutions Solution focused approach 'Solutions despite resource constraints' [36] Ability to find facts to solve problems 'They all recognised improvements in their ability to problem solve' (Longstaff, 2012) Ability to make decisions Understanding who the decision is for Taking action on decision Make judgements 'Better able to make decisions and take action' [36] Ability to co-operate 'Enhancing their own cooperation and communication skills' [24] Ability to work as part of a team Ability to adapt social norms to meet needs of another culture Change behaviour to fit with social norms 'Transcultural adaptation' [37] Ability to lead by example 'Leading by example with consistency and perseverance Ability to manage self Own expectations Self-reliance Self-management Self-assurance 'Self-management' (Lumb, 2014) Ability to manage projects 'I gained significant experience in report writing, project planning, managing budgets and particularly human resources'. [11] Ability to think through problems in a logical way  Organisational outcomes

Increased staff knowledge and skills
Increased staff knowledge of lowcost healthcare More knowledgeable staff Staff able to discover better ways of doing things Staff more aware of waste reduction 'Makes people more adaptable when they come back because in some areas if you have not move ward for twenty years, it is trauma just to be asked and work in ward X in the same hospital is not it? If you have got somebody that has been exposed to a range of environment, they are more likely to cover shifts'. 'Reputational development' [3] NHS becomes a more attractive employee (If offers staff opportunity to volunteer) 'Link attracts potential staff' [24] Increased patient satisfaction Staff better able to respond to UK multicultural populations Staff have greater relationships with multicultural patient population 'Patient experience and dignity: understanding of patients from different areas' [3]  Increased workforce productivity 'Increased workforce productivity' [3] Reduction in NHS drop outs Increased staff retention 'Attraction & retention of (more/better quality) workforce' [3] Increased international reputation (of the United Kingdom) '96 per cent of health professionals interviewed for the study thought that the reputation of the NHS could only be enhanced by involvement in international health links'.  Culture shock 'Culture shock due to the contextual differences and challenges faced in resource poor settings'. [3] Environmental and infrastructural risk 'Physical risk to person-environment, infrastructure' cadres, so this COS would allow comparison and collation across professional groups [35,36].
Our study generated a list of 28 potential negative outcomes. It is interesting that only eight of these were retained in the Delphi, i.e. stakeholders were in agreement that these negative outcomes were either not likely to happen or likely to happen to a range of healthcare professionals. Only one negative outcome was considered core: 'health consequences'. This indicates that stakeholders believe almost all negative outcomes do not happen on many or most placements. There is much less consensus about the negative aspects of placements.
The literature contains stated or implied variables which might influence learning on international placements, and this study has synthesised these, finding 33 variables. This provides a framework for future research that aims to study the interactions between variables and outcomes by empirically testing some of the hypotheses reported or assumed in the literature.
Historically, international volunteering has been conceptualised as a benefit to the LMIC and a loss to the HIC [8,9]. Recent policy documents explicitly discuss   Table 7 Applying our results to the current knowledge: our core learning outcomes presented within the existing domains from [3] Domain in [3] Number the benefit to UK health professionals in terms of personal and professional development and the necessity to develop competencies to be used in training curricula [9]. This study will facilitate the specification and exploration of learning outcomes and so in the future help in addressing the imbalanced discourse of the "benefitting LMIC" and the "donor HIC". Additionally, a recent Royal College policy describes what competencies paediatricians need to work globally, or with a global population in the United Kingdom [37]. Many of the competencies described map onto the core outcome set suggesting that international placements themselves may provide a vehicle for developing these necessary competencies. In fact, the core outcome set maps onto policy documents such as the Health Education England (HEE) Framework 15: 2014-2029, which suggests the future NHS workforce needs to be flexible, open to innovation and change and life-long learners (all components of the COS) [18]. The core outcome set provides a way of framing and evidencing the NHS benefits. Future work will focus on how the core outcome set can be used as a tool to measure outcomes. The research has also influenced the production of the Health Education England Global strategy, which aims to embed global learning opportunities into NHS training [38]. In summary, there is a broad range of learning outcomes which we have synthesised into a set of 116 core outcomes agreed by a group of 45 stakeholders from various invested groups that could be used in future assessment of learning and testing of hypotheses about what leads to or detracts from learning. We also extracted 33 variables from the literature. We reported a list of negative outcomes, as well as every variable that has been reported (implicitly or explicitly) to affect learning. The core outcome set and variables will enable the development of assessments of health professional learning in international placements, which has implications for how international placements are created and on the support for international placements amongst UK healthcare organisations.

Limitations
This study has a number of limitations. Firstly, we did not update the systematic review because this was the first stage of the outcome set development, and therefore, new outcomes could not be added. We conducted a scoping search using the same search strategy in March 2018 and found 23 new papers had been published. We read these papers and did not find any new outcomes or variables reported. Secondly, the papers included in the meta-synthesis included both those with primary data and those which did not. Formal risk of bias assessment, using standard tools, was therefore not possible. However, it is important to note that the papers included and the findings of the Delphi indicate an overall positive attitude towards international placements, with 96% of papers in the review reporting positive outcomes as opposed to 49% reporting negative outcomes. It is possible that there is publication bias, in which reports of negative experiences are less likely to be written and/or accepted for publication. In the Delphi, participants agreed most of the positive outcomes were core and very few negative. It may be that Delphi participants (particularly those who choose to dedicate hours of their own time) feel more positively about the outcomes than those that were invited but chose not to participate. This represents a risk of bias both in terms of an underreporting of negative outcomes and an inconsistent reporting of variables, with variables influencing outcomes being reported by people whose outcomes had been positive.

Future research and recommendations
The core outcome set could be developed into a tool to assess outcomes. Measurement of learning outcomes is not straightforward, and self-report of learning is fraught with difficulties, including people not knowing what they   Ability to work within a system with unfamiliar power dynamics 2 88 + 20 Ability to adapt social norms to meet needs of another culture (e.g. change behaviours to fit into another culture, being aware of own social norms and adapting them) Ability to exchange ideas with those from another culture 2 88 + 20 Increased self-awareness (e.g. understanding own skills and limitations, how to challenge own beliefs and importance of reflecting on own situation) 2 8 8 + 2 0 Patience and tolerance (e.g. accepting and working at other peoples pace, more tolerant) 2 88 + 20 Proactivity (e.g. thinking on feet, using initiative, efficiency, get on with things rather than look for someone to blame) 2 8 8 + 2 0 Ability to work with resources available in specific contexts (i.e. understanding the reasons behind lack of resources) 2 8 8 + 2 0 Ability to work towards solutions (e.g. solution focused approach) 2 88 + 20 Understanding that speed and language competency affect communication (e.g. awareness of how speed affects comprehension, understanding language differences and checking recipient comprehension, ability to use an interpreter) 2 8 6 + 3 3 Increased awareness of/knowledge about the importance of community participation in health (e.g. understanding the community and social influences on health, the role of the community in health, public health and the importance of community work) 2 8 6 + 3 3 Ability to use a broader range of clinical skills (e.g. enhancing existing skills and acquiring new clinical skills, greater all round competence) 2 8 6 + 3 3 Understanding that changing behaviour is complex (e.g. understanding how to make small changes and not to force your perspective onto others,) 2 8 6 + 3 3 Ability to improve service (e.g. renewed enthusiasm for service improvement) 2 86 + 33 Increased staff knowledge and skills (e.g. increased staff knowledge of low cost healthcare, more knowledgeable staff able to cover more areas, to discover better ways of doing things and more aware of waste reduction) Appreciation of having the right tools and equipment to be able to do the job (i.e. resources: technical equipment, disposal equipment, cleaning products and protective equipment) Appreciation of excellent human resource in the NHS (e.g. multidisciplinary teams, HR structures, appreciation of own profession, understanding hierarchy and the importance of each person within it) 2 8 3 + 4 3 Improved emotional intelligence (e.g. changed engagement with self, knowledge and world) 2 83 + 43 Ability to identify and anticipate potential problems (e.g. identify problems when setting up a 2 83 + 43 Ability to make independent clinical decisions (e.g. ability to make an urgent decision in an emergency, dealing with uncertain outcomes, evaluating risks to patients and self) 2 8 1 + 5 1 Understanding own potential to empower people 2 81 + 51 Ability to work as part of a team (e.g. understanding team group norms, perception of roles within the group, managing personal objectives within a group) do not know and people not being aware of what has changed for them at a particular time [39]. Nonetheless, metrics and standard indicators are useful for policy and decision-making [40], and this COS could facilitate quantification and the variables could facilitate hypothesis testing.

Additional file
Additional

Ethics approval and consent to participate
The approval for the study was obtained from the Ethical Research Committee, University of Salford, and the University of Manchester Research Ethics Committee.

Consent for publication
Not applicable.

Competing interests
Professor Ged Byrne is the Director of Global Engagement for Health Education England. The other authors declare that they have no competing interests.