A retrospective review of medical records can provide data on the frequency and type of AEs.4 One method, known as the global trigger tool (GTT), initially developed by the Institute of Healthcare Improvement (IHI) in 2003 for use in hospitals,5 has been modified for primary care in the United Kingdom6–9 and North America.10 There are no published studies using a GTT in an Australian general practice environment.
GTTs use a series of triggers to screen the medical record for a potential AE. The presence of a trigger then leads to a more in-depth review of the record. AEs are then coded according to their type and an incident rate is calculated. The original IHI version of the GTT has been shown to have high specificity, moderate sensitivity and favourable inter-rater and intra-rater reliability using hospital-based records.11
GTTs can be used to identify and prioritise types of AEs, such as medication errors, which may subsequently be addressed with quality improvement processes. Data collected at one practice can potentially be aggregated to regional and national levels.12
The aim of this study was to explore whether general practice records contain information on AEs, and to conduct a pilot study on the type and frequency of AEs in general practice in Australia, using a GTT.
Methods
Definitions
Definitions for a patient safety incident, AEs and harm were derived from the International Classification for Patient Safety (ICPS).13
Triggers
A two-stage Delphi process was used to develop the triggers, using GPs and practice nurses who were members of a local Division of General Practice Clinical Leadership Group (CLG) as experts. Twenty-eight candidate triggers, modified from primary care GTTs developed in Scotland6 and England,8 were the starting point and a final list of ten triggers (Table 1) was produced.
Table 1. Final list of 10 triggers
|
Trigger
|
Trigger grouping
|
|---|
|
Were there three contacts with the practice in any given period of a week (this includes telephone calls, consultations with nurse/GP or home visits)?
|
Consultations and attendance
|
|
Had the patient been admitted to a hospital for any intervention, management or procedure (including elective surgery) for at least one night?
|
Consultations and attendance
|
|
Did the patient have a consultation out-of-hours or attend emergency or an ACIS callout/attendance?
|
Consultations and attendance
|
|
Was the patient prescribed opioid analgesia?
|
Medication
|
|
Was the patient prescribed a benzodiazepine?
|
Medication
|
|
Was the patient prescribed an antipsychotic?
|
Medication
|
|
Was the patient prescribed NSAID/COX2 inhibitors?
|
Medication
|
|
Were there any INR readings >4.5?
|
Laboratory
|
|
Was the eGFR ≤60 mL/min/1.73 m2?
|
Laboratory
|
|
Was the haemoglobin level ≤100 g/L?
|
Laboratory
|
|
ACIS, Acute Crisis Intervention Service; COX2, cyclooxygenase 2; eGFR, estimated glomerular filtration rate; INR, international normalised ratio; NSAID, non-steroidal anti-inflammatory drug
|
|---|
Development of a data collection tool
A paper-based data collection tool was then developed, which recorded the presence of triggers and whether an AE occurred for each record review. For those record reviews with an AE, preventability (low, medium, high), level of harm14 and AE source (primary or secondary care) were recorded. A free-text field to describe the type of AE was also included. The patient’s gender and frequency of general practice service use were collected for denominator purposes. A user guide outlining definitions, sampling method and data collection methods was also developed.
Recruiting
Five practices were recruited from within the Southern Adelaide – Fleurieu – Kangaroo Island Medicare Local (SAFKIML) region and each consented to collect data on 100 patients each. The only inclusion criterion was that practices used electronic clinical software. Each practice nominated a practice nurse to undertake the reviews and one researcher (PH) trained the nurses in the method.
Sampling and reviewing
Records were sampled from patients aged 75 years or older as the CLG provided advice that this group may be more vulnerable to harm if exposed to a patient safety incident. Records were reviewed from randomly sampled patients who had attended the practice three or more times in a 6-month period (January–July 2012). Patient records were reviewed for the presence of triggers and, subsequently, for AEs if a trigger was present. No patient identifiers were recorded. Data collection forms were then securely mailed to the research office where they were entered into a Microsoft Access database and analysed.
Ethics
Ethics approval was sought and received from the Royal Australian College of General Practitioners (RACGP) National Research and Evaluation Ethics Committee (NREEC 12-001).
Results
 |
Figure 1. Average time taken to review
medical records |
Records of 428 patient (225 females and 203 males) were reviewed. There was a total of 4117 patient consultations (9.6 per patient), the majority (73%) being in-room consultations with the GP. Three practices did not complete 100 records because of either an insufficient number of eligible patients or missing forms (Table 2). The average time taken to review a medical record was 4.5 minutes (Figure 1).
Table 2. Number of medical record reviews by practice
|
Practice number
|
Number of medical record reviews
|
Reason for practice not completing 100 records
|
|---|
|
1
|
86
|
Insufficient number of eligible patients
|
|
2
|
98
|
Two missing forms
|
|
3
|
100
|
|
|
4
|
100
|
|
|
5
|
44
|
Insufficient number of eligible patients
|
|
Total
|
428
|
|
Triggers and AEs
Table 3 shows the number of AEs by triggers and medical records reviewed. 273 medical records (64%) were coded as having one or more positive triggers (range 1–7) and of these 44 AEs were detected in 41 records (Table 4).
Table 3. Number and percentage of AEs by trigger and medical records reviewed
|
Trigger
|
Medical records in which a trigger was positive
|
AE
|
|---|
|
|
na
|
%
|
nb
|
%c
|
Positive predictive valued
|
|
Three contacts in 1 week?
|
110
|
26
|
19
|
4.4
|
17
|
|
Admitted to hospital?
|
83
|
19
|
17
|
4.0
|
20
|
|
After-hours consult?
|
38
|
9
|
7
|
1.6
|
18
|
|
Opioid analgesia?
|
52
|
12
|
5
|
1
|
10
|
|
NSAID/COX2 inhibitors?
|
32
|
7
|
2
|
0.5
|
6
|
|
eGFR ≤60 mL/min/1.73 m2?
|
102
|
24
|
2
|
0.5
|
2
|
|
Benzodiazepine?
|
42
|
10
|
1
|
0.2
|
2
|
|
Antipsychotic?
|
6
|
1
|
1
|
0.2
|
17
|
|
Haemoglobin level ≤100 g/L?
|
5
|
1
|
1
|
0.2
|
20
|
|
INR >4.5?
|
1
|
0
|
0
|
0
|
0
|
|
AE, adverse event; COX2, cyclooxygenase 2; eGFR, estimated glomerular filtration rate; NSAID, non-steroidal anti-inflammatory drug
a = number of medical records with positive triggers
b = number of medical records with an adverse event
c = b/428 x 100 (428 = number of medical records reviewed)
d = b/a x 100
|
|---|
The percentage of patients with an AE was 9.6%. Primary care was the point of origin in 26 AEs (59%), and 18 AEs (31%) occurred in other levels of care. Reviewers deemed four AEs (10%) as highly preventable, nine (21%) moderately preventable and 29 (69%) low (with two not scored). Most low preventability AEs (21/29, 72%) were medication incidents, usually with unknown adverse reactions.
Table 4. Adverse events by ICPS incident type
|
ICPS incident type
|
n
|
%
|
|---|
|
Medication
|
29
|
66
|
|
Healthcare-associated infections
|
6
|
14
|
|
Clinical management
|
4
|
9
|
|
Assessment/diagnosis
|
3
|
7
|
|
Fall
|
1
|
2
|
|
Pressure ulcer
|
1
|
2
|
|
Total
|
44
|
100
|
Discussion
Main findings
Using a consensus-based approach, this study modified an existing tool for an Australian general practice environment to develop a set of triggers designed to detect AEs. Trained staff then reviewed records for triggers and AEs. The method found an AE rate of 9.6 per 100 patients over a 6-month period; 66% of AEs were medication-related. This is consistent with a similar study in Scotland that found a rate of 9.4% with 59% being related to medication.6 Sampling strategies in this study (patient age >75 years) were different from the study in Scotland6 (patients age >18 years). Another study in primary care in North America found medication-related AE rates of 15%.10
Balancing sensitivity and feasibility
Although retrospective review of medical records is recognised as one of the more sensitive methods of collecting AEs, it can be resource-intensive and time-consuming.4 This is a major barrier to GPs or organisations undertaking their own reviews.
Only five triggers had a positive predictive value of more than 15% and two of these (haemoglobin and antipsychotic medications) were associated with low numbers of AEs. Those triggers that occurred frequently and had a high positive predictive value (such as three general practice contacts in one week, a hospital admission and an after-hours consultation) yielded higher numbers of AEs and require further research. Another avenue for further research into a more time-efficient method may be the use of electronically detected triggers. Medication-related AEs9,10,15 and diagnostic errors16 have been detected using electronic methods.
Policy implications
A national discussion paper entitled Patient Safety in Primary Health Care2 found that one of the four key themes emerging from a consultation process was a lack of knowledge and understanding of patient safety risks in primary healthcare, and limited coordinated systems to identify, analyse and respond to AEs. The paper proposed ‘developing a systematic and coordinated identification, reporting and monitoring system for patient safety incidents in primary care. It was suggested that any activity in this area should include infrastructure and resources to implement this at the local level’.2 GTTs may contribute to such a system; however, other sources such as incident reports, guideline/standards compliance audits, collections of quality indicators, consumer experience feedback and coroners' reports would also be necessary for an integrated and comprehensive framework of safety and quality data in primary care.12
Limitations
Only five general practices were involved in this study and they were recruited by a convenience method, thereby limiting the generalisation of findings more broadly in Australia. There is likely to have been variation between medical record reviewers in the interpretation and thresholds for deciding whether an AE was present, and also in the rating of preventability and level of harm. Inter-rater reliability was not formally tested and requires more research.
Another source of potential bias is the use of reviewers examining medical records from their own practices. The option of using external reviewers was explored but clinicians expressed concern about external reviewers accessing patient-level data. For larger studies of this nature, the protection of statutory immunity may be an option.
Implications for general practice
The study found that significant levels of information about AEs exist in general practice medical records. Rates of harm are broadly in line with a similar study internationally, although with our small sample size, a national rate should not be inferred.
Although time taken to review records was relatively low, more research is necessary in the Australian general practice environment to determine the cost- and time-effectiveness and reliability of GTTs, and to validate their use in a larger number and variety of general practices, including those in rural and remote areas. This could lead to development and publication of an Australian general practice GTT version with associated definitions, data collection tools and training resources. The use of such a tool could be linked to the Safety and Quality (Standard 3.1) RACGP Standards for general practices (currently in 4th edition).17 GTTs are a form of clinical audit, which is an acceptable activity under the RACGP’s Quality Improvement and Continuing Professional Development (QI&CPD) program.18
Tools such as GTTs should be supported by a policy and funding framework, and incentives that allow time off the ‘fee-for-service treadmill’, as the current cost of many high-quality safety and quality activities is potentially prohibitive. For example, in this study GPs were paid to backfill staff to undertake the reviews.
Comprehensive frameworks for systematically understanding the nature of safety and quality issues at general practice, regional and national levels, and provide incentives and resources for continuous improvement within general practice and, more broadly, within primary health care, are necessary for a deeper understanding of and response to safety priorities.
Competing interests: None.
Provenance and peer review: Not commissioned, externally peer reviewed.