Items in each domain, shown in Table 37.1, measure the extent to which that criterion has been fulfilled using a rating on a 4-point scale.
The item scores are added to achieve a total score for the particular domain.
It should be emphasised that the six domain scores are independent and therefore not aggregated into a single score.
Although domain scores may be useful for comparing guidelines and for decision making, it is important to note that it is not possible to set thresholds for the domain score that would indicate a ‘good’ or a ‘bad’ guideline.15 The instrument comes with detailed instructions, and it is recommended that a minimum of two or ideally four appraisers perform the guideline assessment.
JWBK089-37 JWBK089-Boulton April 24, 2006 19:59 Char Count= 0
APPRAISING GUIDELINES USING THE AGREE INSTRUMENT 407
APPRAISING GUIDELINES USING THE AGREE INSTRUMENT: A PRACTICAL EXAMPLE The following is a practical example of an appraisal of a selection of guidelines using the AGREE instrument. Five well-known guidelines were chosen from a wide range available for the management of the diabetic foot.
r Type 2 Diabetes Prevention and Management of Foot Problems by the National Institute for Clinical Excellence in the United Kingdom (NICE).16 r Management of Diabetes by the Scottish Intercollegiate Guidelines Network (SIGN).17 r Clinical Guidelines and Evidence Review for Type 2 Diabetes: Prevention and Management of Foot Problems by the Royal College of General Practitioners in the United Kingdom (RCGP).18,36 r Management of Type 2 Diabetes by the New Zealand Guidelines Group (NZGG).19 r Practical Guidelines on the Management of the Diabetic Foot based upon the International Consensus on the Diabetic Foot (ICG).20
Following the AGREE instrument instructions for use, the above guidelines were evaluated by four appraisers , a professor in health care research science, a podiatrist and the two authors of this chapter. The results are shown in Table 37.2.
Results Scope and purpose
All guidelines have scores that indicate that the appraisers found all the relevant information.
Stakeholder involvement
For all the guidelines, concern on pre-testing prior to issuing the guidelines is reflected in the scores. The lower scores for RCGP and ICG guidelines are due to lack of clarity on whether patients’ experiences and expectations of health care had been taken into consideration.
Rigour of development
The appraisers gave the lowest scores in this domain to the NICE and the ICG guidelines. Separate analysis of the individual item scores for NICE showed a lack of clarity on the criteria for including/excluding evidence, on the method by which final decisions were made and whether health benefits, side effects and risks of the recommendations in the guidelines have been clearly presented. The results for the ICG guidelines reflect a lack of clarity of the criteria for including/excluding evidence and on the explicit link between the recommendations and the supporting evidence.
JWBK089-37
JWBK089-Boulton
Table 37.2 An example of using the AGREE tool to evaluate five well-known practice guidelines for the management of the diabetic foot April 24, 2006
AGREE domain scores (%) 19:59 Year of Scope and Stakeholder Rigour of Clarity and Editorial Guideline publication purpose involvement development presentation Applicability independence 408 NICE 2004 100.0 72.9 55.9 81.2 72.2 33.3 SIGN 2001 80.5 68.7 83.3 81.2 25.0 54.2 RCGP 2000 97.2 56.2 83.3 66.7 33.3 41.7 Char Count= 0 NZGG 2003 97.2 75.0 84.5 83.3 33.3 100.0 ICG 1999 75.0 54.2 51.2 68.7 58.3 45.8
JWBK089-37 JWBK089-Boulton April 24, 2006 19:59 Char Count= 0
IMPLEMENTING A CLINICAL PRACTICE GUIDELINE 409
Clarity and presentation
All guidelines scored high on this item. The RCGP guidelines scored lower than the three national guidelines mainly because the guidelines are not supported by tools for application. The ICG guidelines have a lower score mainly due to a lower average score on most of the items in this domain compared to the other guidelines.
Applicability
The results for the SIGN, RCGP, NZGG and ICG guidelines express to varying degrees a level of doubt as to whether points such as potential organisational barriers, cost implications and criteria for monitoring and/or auditing purposes are clearly addressed.
Editorial independence
All but the NZGG received low scores in this particular domain. The ultimate choice of guideline for implementation purposes, which would normally follow an appraisal process, should be based on the scores, taking into account the relative importance of the various domains and items to the health care setting in question.
IMPLEMENTING A CLINICAL PRACTICE GUIDELINE A flowchart of the key components of the CPGs’ implementation process is presented in Figure 37.1. These key components are discussed below.
Stakeholders Management and organisational environment
Delivering an adopted guideline into current practice is likely to be a challenging process.
It is important to realise that at least some change in behaviour and policy as well as in resource allocation will most probably be required.
Implementing CPGs nearly always requires some organisational changes and substantial time and resources.
These are important reasons for consulting senior management as a threshold step to gain their support at the outset, if the decision to implement the guideline does not originate from them.
Effective managerial, clinical relations, where managers understand what clinicians value and where clinicians think like managers, have been shown to facilitate implementation of innovations requiring change in practice.21 The speed of adoption is influenced by the degree to which the new practice requires changes in the organisational culture.22 Ensuring that the implementation project fits with the organisa- tion’s overall strategy and resource commitments will eliminate possible organisational issues that could act as barriers.8 Compatibility with the existing organisational norms, values and ways of working is likely to be a determinant of successful assimilation.23
JWBK089-37 JWBK089-Boulton April 24, 2006 19:59 Char Count= 0
410 PRIMARY CARE: DELIVERY/TRANSLATION OF GUIDELINES INTO PRACTICE
Figure 37.1 An overview of the implementation process for guidelines
Attempts should be made to identify key issues in the organisational culture and environ- ment immediately at the outset of the process.
This assessment is essential in order to fully comprehend the resources and changes essential for success.
All possible implications on cost and resources need analysis and clarifying.
Resources include materials, technologies, monies and networks as well as people required for a successful implementation.
Lack of such re- sources and time are recognised barriers to change, and restraining forces that are likely to require resolution include increased workload, lack of time, poor communication, traditional working practices and resistance to change.12,24 Therefore, identifying workload implications and discussing and addressing these at the onset have been shown to be important considera- tions for successful implementation.11 A key element to success may be ensuring that relevant clinical and administrative systems are in place to facilitate adherence to the CPGs, and be- cause it is likely that new administrative systems are required, administrative staff should be involved and invited to provide input.25 Restructuring of medical records to provide either detailed care plans or at least prompts about actions required during the consultation has been recommended.20 Systematic reviews indicate that prompts appear to be consistently effective in achieving change of practice.
Their frequency and proximity to the point of clinical decision making may influence the size of their impact.26
The health care professionals
Preference should be given to considering professional group specific issues at the outset. Encouraging a feeling of ownership may create a more positive attitude to the guideline and facilitate the process for addressing possible barriers and necessary changes in behaviour