Development of the WHO eye care competency framework

Abstract

Background

The eye care workforce, particularly in lower resource settings, face challenges of limited integration into the health system, limited workforce capacity, mismatch of workforce to population need and poor quality of care. In recognition of these challenges, coupled with a gap in existing tools, provides a strong rationale for the development of the Eye care competency framework (ECCF).

Methods

A mixed methods approach was utilised to develop and validate the ECCF. Content was developed by extracting relevant components of existing frameworks used both within and outside of eye care. A diverse technical working group provided feedback and guidance on the structure, design, and content to create a preliminary draft. Competencies and activities were validated using a modified-Delphi study, and the framework was then piloted at four sites to understand how the tool can be implemented in different settings.

Results

The final version of the ECCF included eight outcomes, nine guiding principles, and content of each of the key elements, including the six domains, 22 competencies, 21 activities, 193 behaviours and 234 tasks, and the knowledge and skills that underpin them. 95/112 participants from the six WHO regions completed the modified-Delphi study, yielding an average of 96% agreement across the competencies and activities in the ECCF. The pilot showcased the versatility and flexibility of the ECCF, where each of the four sites had a different experience in implementing the ECCF. All sites found that the ECCF enabled them to identify gaps within their current workforce documentation.

Conclusions

The ECCF was developed using a collaborative approach, reflecting the opinions of participants and stakeholders from all around the world. The comprehensive competencies and activities developed in the ECCF encompass the diverse roles of eye care workers, and thus encourage multi-disciplinary care and better integration into the health system. It is recommended that eye care workforce planners and developers use the ECCF, and adapt it to their context, to support workforce development and focus on the quality and scope of eye care service provision.

Background

Globally, at least 2.2 billion people have a vision impairment or blindness, of whom at least 1 billion have a vision impairment that could have been prevented or has yet to be addressed. The eye care workforce, particularly in limited resource settings, face challenges of limited integration into the health system, limited workforce capacity, mismatch of workforce to population need and poor quality of care [1]. A key strategy to address the burden of vision impairment is to improve the eye care workforce’s availability, accessibly and quality, to meet the eye care needs of the population [2].

The tools available for workforce planning and development in eye care globally have been limited [3]. Tools that do exist, commonly support the setting of workforce targets [4,5,6], rather than supporting workforce development, including the quality and scope of eye care services provision. While popular, the approach has limitations as it does not take into account factors, such as the location and distribution of the current workforce, the population structure, disease epidemiology and the public demand, service regulations and importantly, quality standards [7,8,9]. In addition, they often isolate the specific occupational groups, further limiting multi-disciplinary care and the integration of eye care into the health system.

A competency framework can be defined as a model that communicates the expected performance and activities of the health workforce across professions, and specializations, to enable quality care and service delivery [10]. They provide a useful tool for workforce planning and development, and serve various roles including outlining characteristics of a competent workforce, facilitating mobility, and analysing or assessing expertise [11]. While competency frameworks have been widely used in many fields, it is important to acknowledge their limitations and consider the applicability of such frameworks in different settings. Competency frameworks do not directly calculate workforce targets; however, they still offer benefits by emphasizing the necessary quality of care and scope of practice to meet the eye care needs of the population. Ultimately, competency frameworks are a tool to strengthen the alignment of the workforce with population needs.

A review of 52 existing competency frameworks and models used in eye care established the lack of a comprehensive global workforce competency framework that encompasses all competencies and activities relevant to eye care [3]. In 2022, Yu et al. proposed the need for a competency framework that encapsulates the roles of the diverse eye care workforce while providing a harmonized language to cut across the occupational groups in eye care and be better integrated into the health system. The value of such a framework would be greatest when designed to be applied to any context, particularly to low–middle income countries, where tools for workforce development are limited [3]. This paper presents the development process of the World Health Organization (WHO) Eye care competency framework (ECCF) [7], including the collaborative content creation, validation and pilot of the ECCF.

Methods

The study design approach was a mixed methods study conducted in an exploratory sequential design, where the qualitative component was conducted first, followed by the quantitative component. The qualitative component involved collaborative development of the ECCF through a technical working group (TWG) and the creation of the preliminary framework, while the quantitative component involved validation of the competencies and activities through a modified-Delphi study and piloting of the framework at four different sites.

Collaborative development

To ensure the development of the ECCF was collaborative, international eye care organisations and professional associations were involved and collaborated in a TWG (n = 14). The appropriate candidates were selected based on their experience, geographic location, and the occupational group they represented. The TWG comprised of 14 experts (Male: 5, Female: 9) from different countries across the six WHO regions. They represented different occupational groups in eye care: ophthalmology; optometry; and allied ophthalmic personnel (AOP) (Table 1). Their expertise included in-depth knowledge of the eye care sector with experience in workforce planning and development, and integration into health systems. The use of a TWG was necessary to ensure that the perspectives of each occupational group and of different geographic areas and cultures were considered in the development of the ECCF. The TWG was engaged over a 12-month period from April 2021 to April 2022.

Table 1 Demographic characteristics represented in the ECCF TWG

Framework design and structural formation

To create the blueprint and outline of content areas for the new framework, existing frameworks, models and scopes of practice relevant to eye care were collated and analysed, including recognised frameworks used in the health sector. The proposed framework outline was presented to the TWG, who provided feedback that helped shape its design, and the final structure was then collectively approved [3].

Content formation of the preliminary framework

The preliminary draft of the ECCF was developed through identification of competencies, activities, knowledge and skills drawn from existing competency frameworks, models and scopes of practice relevant to eye care. Content from the existing frameworks was extracted, organised and incorporated into the approved outline. The outcomes and guiding principles were also created to give purpose and guidance (respectively) to the ECCF. The lead author (M.Y.), who was the project leader, reviewed the preliminary draft with TWG through a consultative process, where feedback was received via email communication. After the TWG reviewed the outcomes, the guiding principles, and the key elements of the ECCF, including the competencies, activities, knowledge and skills, feedback was collated, analysed and incorporated in the draft. Any further changes discussed during the subsequent online video meeting were also integrated.

Validation of the competencies and activities

During a 2-month period from September to October 2021, a modified-Delphi study was carried out to build consensus among a wider group of global eye care workers on the competencies and activities of the preliminary framework. The classic Delphi method was modified to employ binary responses (agree/disagree) to decide whether the statements should be kept or changed, paired with a free-text comment box to note what changes were required to reach the best agreement [10]. The modified-Delphi study was implemented through Survey Monkey and participants were given 4 weeks to complete the survey. The snowball sampling method was used to recruit participants that were identified and referred by the TWG members and were selected based on a similar criterion to the TWG. Participants were asked whether they agreed or disagreed with each competency, activity, the behaviours and tasks expressed across four levels of proficiency, and corresponding knowledge and skills. They were also asked to provide a comment under each competency and activity. Results of the modified-Delphi study was analysed through extracting the data from Survey Monkey and importing it into an Excel spreadsheet. Each comment was inputted line by line against each competency and activity to ensure accurate categorization and tracking of feedback. This systematic approach allowed for a comprehensive and detailed analysis of the data. A review and analysis of the data were conducted to identify common themes and patterns within the feedback. All feedback was considered and integrated into the validated version of the preliminary framework. The results of the modified-Delphi study were shared with the TWG.

Progress to the final framework

The validated preliminary framework underwent additional development to include the explanatory text. The framework underwent graphical design, particularly to format the complex and extensive tables containing the competencies and activities. To ensure the framework’s comprehensiveness and relevance to the specific context of human resource development in the public health eye care sector, key personnel at WHO and the TWG reviewed it. It should be noted that during this stage, the key components of the ECCF remained unchanged. However, the review process helped refine explanatory text and design of the framework, which led to the penultimate version of the ECCF.

Piloting the framework

Eye care workforce developers at four sites were consulted over a period of 5 months, to conduct a pilot on the implementation of the ECCF. The sites were identified through members of the TWG with the criteria being that each provided a different setting, i.e., tertiary eye hospital setting; professional body representing a specific occupational group locally and globally, and teaching institutes. Each site also varied from a low–middle income to a high-income setting. The aim of the pilot was to have a clearer understanding of how the ECCF would be utilised in each setting, and to identify any additional resources required to assist in the application of the ECCF. Following the completion of the pilot, each site was asked to complete a brief questionnaire to detail the process of how they applied the framework, and to identify any lessons learnt.

Results

Construction of preliminary framework

The preliminary framework included the construction of the outcomes, the guiding principles, and the key elements of the ECCF, including the domains, competencies, activities, knowledge and skills. The ECCF presents the competencies and activities that encompass the eye care worker through six domains: Practice, Professionalism, Learning and Development, Management and Leadership, Community and Advocacy, and Evidence [3]. Within these domains, knowledge and skills underpin each competency and activity. Competencies are described through behaviours and applicable to all eye care workers, regardless of their scope of practice. On the other hand, activities are described through tasks, and are selectively applicable to eye care workers, depending on their scope of practice within their specific context. Behaviours and tasks for each competency and activity (respectively) are expressed at four different levels of proficiency (Level 1: Introductory; Level 2: Intermediate; Level 3: Advanced; and Level 4: Expert) (Fig. 1). The levels represent an increase in autonomy and decision making and enable the ECCF to be applied across a diverse eye care workforce. In total there were 22 competencies, 21 activities, 193 behaviours and 234 tasks that were developed into the ECCF.

Fig. 1

The structural arrangement of the ECCF comprised of six domains, each accompanied by competencies and activities

The eight outcomes for the ECCF that were developed clearly highlight the aims of the framework (Table 2). These outcomes will illustrate to workforce planners how the ECCF can assist in planning, and potentially be used to advocate for the implementation of the ECCF within a setting. Nine guiding principles (Table 3) of the ECCF were adopted and adapted from the WHO Global strategy on human resources for health 2030 [2].

Table 2 Eight outcomes of the ECCF

Table 3 Nine guiding principles of the ECCF

Validation process: a modified-Delphi study

Of the 112 participants invited to participate in the modified-Delphi study, 95 (85%) completed the study. The gender distribution was female 51.6%. The participants comprised of eye care practitioners, educators, managers, researchers, policy makers and programme implementers. There was representation of participants from all six of the WHO geographic locations, with the highest from African region and Region of Americas, each at 20.7%, and the lowest representation from Eastern Mediterranean Region at 7.3% (Fig. 2).

Fig. 2

Geographic location of participants of the modified-Delphi study, by WHO region

The participants also represented several occupational groups within eye care, ophthalmology 40.2%, optometry 46.3%, AOP/mid-level eye care personnel 15.5%, vision technician/refractionist 3.7%, and community eye care worker/community health worker 1.2%.

Of the participants, 59.8% stated that they were highly involved in eye care workforce planning and development, and 62.2% of participants stated they had a strong ability to implement and/or influenced competencies for eye care workers within their respective settings.

The study participants reviewed and provided their feedback on the key components of the ECCF, eliciting approximately 3000 comments. Most of the feedback received from the participants focused on the complexity of the language used in the framework, with many suggesting that it could be simplified further to enhance its accessibility and usefulness. The Practice domain was a particular area of contention, with some participants expressing disagreement with the categorization of eye care interventions across the different levels of proficiency. There was an average of 96% agreement across the competencies and activities in the ECCF (Fig. 3). The study was conducted in one round, given the high agreement percentage which was well above the target of 80% agreement [13,14,15]. All results, including comments, were reviewed with the TWG, and minor amendments were made.

Fig. 3

Percentages of agreement and disagreement across the competencies and activities of six domains

Formulation of the final framework

The final framework included the development of the explanatory text, such as the executive summary; background and rationale; overview of the ECCF and its components; definitions; and description of the eye care workforce. The outcomes, the guiding principles, and the key elements of the ECCF, including the domains, competencies, activities (Table 4), knowledge and skills, were also finalised. These elements combined to establish the ECCF. The graphical design process presented the ECCF in format that made it easier for users to read.

Table 4 List of the competencies and activities across the six domains

Implementation of the ECCF: a pilot study

The pilot was conducted at four sites, each showcasing a unique context illustrating the versatility and flexibility of the ECCF. At all the pilot sites, the ECCF enabled the identification of gaps within the existing workforce documentation, such as the topics in the curriculum, or the competencies expected from the current workforce. The pilot highlighted the need to adapt the ECCF to make it suitable for the context and identified the importance for an accompanying guidance document to assist users of the ECCF. The questionnaire following the completion of the pilot, yielded lessons learnt from each site (Table 5).

Table 5 Lessons learnt from each of the pilot sites [16]

Discussion

A mixed methods approach was used to produce the ECCF, combining both qualitative and quantitative data. Maximum efforts were made to ensure that viewpoints and perspectives from a variety of frameworks, occupational groups, and people from around the world were included in the development process of the ECCF, because it is intended to be a tool that is universally applicable for the eye care workforce globally. The diverse viewpoints gained from the collaborative approach, further directed the development of the guiding principles, which serve to define the values and tenets of the framework. It is envisaged that these principles will be extended into the workforce, where the ECCF is being implemented.

Despite the fact the eye care sector is relatively mature in many countries, the adoption of competency frameworks is still novel. The use of the ECCF can be particularly beneficial in settings where the eye care workforce is still emerging or in instances where there is a labour supply shortage, and tasks need to be distributed efficiently and effectively to optimise access to care [10, 15]. This aspect was taken into account when developing the ECCF, whose key components encompassed the behaviours and tasks expressed over four levels of proficiencies. The TWG was instrumental in guiding and determining how the behaviours and tasks were described across the four levels of proficiency, to capture the scope of performance represented in the eye care workforce. The knowledge and skills included were not exhaustive, but rather a sample suggestive list for users of the ECCF to consider and further expand in their own contexts.

The Delphi technique was chosen for this study, because it is widely used to develop consensus on group opinion and is frequently used in the development of competency frameworks for personnel in a particular field [8, 18]. While the modified-Delphi technique used in this study deviated from traditional approaches in some ways, author’s acknowledged the importance of its foundational principles, including anonymity, iteration, regulated feedback, statistical group response, and structure [19, 20]. In particular, the study aimed to address issues such as groupthink and dominance bias that can arise in face-to-face group meetings, by conducting the survey online. It should be noted that the study only utilized one consensus approach.

In addition, the modified-Delphi survey consisted of participants entering in the responses to questions asked in a binary format. Binary responses were preferred than other formats, such as a Likert scale, as they captured the opinions of the participants in a straightforward and concise manner, and this approach has been shown to be effective in previous research [10, 21,22,23]. Furthermore, as the survey was based on the comprehensive competency framework and consisted of 265 questions, the use of binary responses was advantageous in that it allowed participants to complete the lengthy survey with greater ease and efficiency.

The snowball sampling method was used to recruit participants for the modified-Delphi survey, relying on referrals from existing TWG members. This approach was found to be effective, as many of the TWGs were part of professional associations and networks with extensive reach and connections within the public health eye care sector, facilitating the identification of a diverse group of individuals with the necessary expertise for the survey [24]. To reach these individuals, other recruitment methods such as open call outs through social media or other platforms would have been less effective due to the limited pool of qualified individuals and the potential challenges in reaching them. Therefore, targeting these individuals through a snowball sampling method was a decision that produced favourable results within the project timeline.

The high agreement percentage (96% agreement) of the modified-Delphi study may be attributed to several factors. First, the study focused on a narrow and specific topic, which made it easier for the participants to reach a consensus. Second, the participants were selected based on their expertise and experience in the field, which ensured that they had a good understanding of the issues and were able to provide informed responses. Third, the binary response format simplified the decision-making process and eliminated the ambiguity that often arises in rating scales. The reasons for disagreements that arose was shown in the comments section. The few points of disagreement that surfaced in the comments section were analysed and revealed that the Practice domain was an area, where participants had slightly diverging opinions. Specifically, some participants expressed dissatisfaction with the categorization of eye care interventions across different proficiency levels, citing inadequate definition and differentiation. Others argued that certain interventions could be classified into multiple categories. Similar concerns were raised during the TWG discussions in the development of the draft, and such comments were expected in the modified-Delphi survey, given the intentional broad scope of the competency framework to make it relevant and usable globally. Despite these disagreements, the feedback overall was constructive and valuable in assisting to refine and improve the framework.

The pilot process was beneficial for testing the ECCF in a practical environment. The common result of each site identifying gaps within the current workforce documentation can be attributed to the comprehensive nature of the ECCF, which has been limited in previous workforce planning tools within eye care. Each site’s pilot experience was different, and each site applied the ECCF differently. It is crucial to acknowledge that the ECCF, like all competency frameworks, cannot be applied universally and must be tailored to meet the specific needs and context of each setting. Moreover, it is important to acknowledge that competency frameworks are not a one-size-fits-all solution, and further discussion and critique on their validity and limitations in different contexts, including in eye care, would be beneficial for advancing their development and use.

The pilot highlighted the need for a guidance paper to make it easier for users to adapt and apply the ECCF in their settings. It was recommended that the learnings from the pilot process be converted to case studies, as it would be beneficial to show potential users on how they could apply the ECCF in their setting. A further recommendation would be to include these case studies within the guidance document that also shows users, a step-by-step process on how to adopt and adapt the ECCF for their individual settings. In addition, the pilot process gave valuable insights on how the ECCF can be used in conjunction with other existing approaches in workforce planning and development. By emphasizing the necessary competencies and activities required for high-quality comprehensive eye care services, the ECCF can serve as a valuable reference tool for improving workforce quality and integrating eye care services into the broader health system.

The development of the ECCF aims to facilitate eye care workforce planning and development by providing a comprehensive set of competencies and activities that encompass the diverse roles represented by eye care workers. This paper complements already published ECCF materials [7, 16] by presenting the thoroughness of the development process, and rationale for an accompanying guidance document. Given the lack of eye care workforce tools particularly for low–middle income resource settings and multi-disciplinary services [3], the ECCF can be a useful reference tool to assist in identifying gaps in current education programmes, workforce planning documentation, and frameworks used to describe the competencies for occupational groups within eye care. For the ECCF to be implemented successfully, a variety of stakeholders, including governments, the WHO, non-governmental organisations, and the private sector, will need to work together to invest in eye care workforce development while also establishing stronger links between eye care education programmes, service providers, and regulatory bodies.

Limitations

Users of the ECCF should consider four key limitations when applying the framework. First, an extensive range of eye care professionals from a variety of nations, including both high- and low-income settings, were represented among the study's stakeholders and participants. Despite this, there were still only small number of participants representing the East Mediterranean region in the modified-Delphi study which may result in some components of the framework not applicable to the workforce in this region. Second, although the ECCF is targeted at both government and non-government institutions, a government setting was not selected for the pilot, as it would have been challenging to implement within the project timeline. Given this limitation, it might be more difficult to understand how the framework can be applied in a government setting. Third, the recruitment of participants for the modified-Delphi study was carried out using the snowball sampling method, where referrals were made from the TWG. Although this assisted in reaching the target audience within the study’s time frame, this may have introduced selection bias. Fourth, the use of binary responses and the increased length of the modified-Delphi survey may have introduced limitations to the study. The binary response options may have led to oversimplification of responses, potentially resulting in less nuanced feedback. In addition, the length of the survey may have led to participant fatigue and rushed responses, potentially affecting the quality of the data collected. These limitations should be considered when interpreting the study findings.

Conclusions

A mixed methods study was used in the development of the ECCF. This approach was chosen as it aligned to views of stakeholders and participants from across the world, including workforce experts from the diverse occupational groups found within eye care, which further enabled the ECCF to have a harmonized language to cut across these groups. The pilot provided an opportunity to learn from applying the ECCF in a practical situation and determine how to use it most effectively. An accompanying guidance document to the ECCF would be beneficial in its implementation as it will provide a step-by-step guidance on how to adapt and adopt the ECCF to make it suitable to each setting [16]. Planners and educators of the eye care workforce are encouraged to use the ECCF to ensure they include the diverse competencies and activities of eye care workers within their programmes, so that the workforce in the eye care sector is in line with population need.