The joy of life

January/February 2017

Audit and feedback by medical students to improve the preventive care practices of general practice supervisors

Volume 46, No.1, January/February 2017 Pages 65-69

Lucy A Gilkes

Helena Liira

Jon Emery

Background

Medical students benefit from their contact with clinicians and patients in the clinical setting. However, little is known about whether patients and clinicians also benefit from medical students. We developed an audit and feedback intervention activity to be delivered by medical students to their general practice supervisors. We tested whether the repeated cycle of audit had an effect on the preventive care practices of general practitioners (GPs).

Methods

The students performed an audit on topics of preventive medicine and gave feedback to their supervisors. Each supervisor in the study had more than one student performing the audit over the academic year.

Results

After repetitive cycles of audit and feedback, the recording of social history items by GPs improved. For example, recording alcohol history increased from 24% to 36%.

Discussion

This study shows that medical students can be effective auditors, and their repeated audits may improve their general practice supervisors’ recording of some aspects of social history.

Medical students in their clinical placements represent the most junior members of the healthcare team. Previous studies have assessed patients’ perspectives on having medical students at consultations and the observations are mostly positive. The benefits observed by patients include that the:

  • consultations are longer1
  • quality of the consultation may be improved2
  • general practitioner (GP) might learn more about their condition.3

However, little is known about the effect that medical students have on their general practice supervisors and on patient care.

This study was designed to measure the effect of an audit and feedback educational activity carried out by medical students on patient care. The educational activity consisted of clinical case note audits of preventive care markers. Medical students performed these audits during their general practice clinical placement.

We chose preventive care as the target of our study because its success in primary care is fundamental to the whole healthcare system. However, there are many barriers to its effective implementation in general practice, including limitations of time and human resources.4 A Cochrane review concluded that audit and feedback generally led to small but potentially important improvements in professional practice.5 Clinical audit is a time-consuming activity and, in the Australian fee-for-services healthcare system, there are limited incentives for clinical audit activities.

In order to test the audit and feedback activity on preventive care, we carried out a study where medical students performed the clinical audits for their general practice supervisors. The students gave feedback from the audit to their supervisors. In this study, we report the results of the changes in preventive care practices of general practice supervisors observed over a period of repeated audit and feedback cycles.

Methods

Design and setting

This study was a non-randomised, before–after interventional study that observed the effect of a cycle of repeated audit with feedback over time. Medical students performed audits on the preventive practices of their general practice preceptors. The results of all patients from all general practices who completed at least one cycle of audit were combined. A longitudinal log analysis was performed over four audit periods to determine if there were changes in the proportion of patients with certain preventive health parameters documented over time.

Procedures

Students were given training on the audit process before auditing the case notes of 20 patients who attended the practice during their eight-week general practice term. They then submitted their results through the internet survey tool SurveyMonkey. Students were given a copy of their audit results, which they discussed with their supervisor during their usual teaching session.

The GPs were prepared for the audit through distribution of written information, and were invited to an information session. The audit questionnaire included a wide range of markers of preventive health activities, including documentation of social history risk factors, assessment of cardiovascular risk factors and cancer screening tests. The preventive activities investigated in this study are recommended in The Royal Australian College of General Practitioners’ (RACGP’s) Guidelines for preventive activities in general practice,6 with the exception of prostate-specific antigen (PSA) testing.

Patients gave written consent for their data to be used in the audit. For each preventive health parameter, the combined data from the group of GPs were analysed to determine if there was a trend to change over time.

The effect of this cycle of audit with feedback over time was assessed using longitudinal log analysis. The results were compiled into three groups of preventive health activities:

  • Cardiovascular risk factor assessment
  • Recording social history
  • Cancer screening tests.

Ethics approval was obtained from the University of Western Australia’s Human Research Ethics Committee (reference RA/4/1/2230).

Results

In total, 3076 patients were recruited by medical students. One hundred and fifty-six students performed the audit. A total of 58 GPs were involved in the audit over four audit periods (42 weeks). Thirty-five GPs completed more than one audit with feedback, and their data were included in the longitudinal analysis. This represents 2608 patient audit questionnaires, which is 85% of the total sample.

Table 1 provides a summary of the preventive health data collected from all patient case note audits. Patients’ demographics of age and gender were compared with Bettering the Evaluation and Care of Health (BEACH) data7 (Table 2). The results of the audit were also compared with known Australian Bureau of Statistics (ABS) health data for the Australian population to assess the fidelity of the data collection. The four health parameters examined were smoking, hypertension, alcohol consumption and mammogram screening (Table 2). Data from this study and nationally were not significantly different for the parameters of patients’ age and gender, patients who smoke, patients who drink alcohol above recommended levels, or patients with hypertension.

Table 1. Responses to case note audit questions for all patient audits over 42 weeks

Question

Number of audits (n)

Yes, n (%)

No, n (%)

Not recorded, n (%)

Is weight recorded?

3,076

1,739 (57)

1,337 (43)

 

Is blood pressure recorded in the past three years?

(All patients aged >18 years)

2,810

2,393 (85)

417 (15)

 

Is cholesterol recorded in the past five years?

(All patients aged >50 years)

1,707

1,481 (88)

226 (13)

 

Does the patient smoke?

3,063

395 (13)

1,400 (46)

1,268 (41%)

Is alcohol consumption recorded?

(Excluding children aged <18 years)

2,853

924 (32)

1,929 (68)

 

If recorded, is alcohol consumption above recommended levels?

1,022

190 (19)

832 (81)

 

Has the patient had a Pap smear in the past two years?

(Women aged 18–69 years)

1,547

744 (48)

803 (52)

 

Has the patient had a mammogram in the past two years

(Women aged 50–69 years)

909

351 (39)

558 (61)

 

Has the patient ever had a PSA test?

(Men aged >50 years)

719

548 (76)

171 (24)

 

Is family history recorded?

3,063

986 (32)

2,077 (68)

 

Low-density lipoprotein-cholesterol level (LDL)

3,076

785 (26)*

1,139 (37%)

1,152 (37%)

*LDL >2.5 mm/L; LDL <2.5 mm/L

 

Table 2. Case note audit results compared with ABS and BEACH data

Health or demographic parameter

ABS/BEACH 2007–08

Data from this study

P value

Female

57%

62%

 

Aged 15–49 years

32%

36%

 

Aged 50–69 years

28%

31%

 

Smokers

19%

22%

0.21

High systolic blood pressure

22%

24%

0.20

Alcohol consumption above recommended levels

21%

19%

0.06

Women aged 50–69 years with a mammogram in past two years*

55%*

39%

<0.01

*Breast Screen Australia annual report 2007–2008

Table 3 shows data from the first audit for all GPs and the fourth-term results. The first audit period for all GPs is an indicator of baseline compliance. The fourth‑term results are an indicator of the final outcomes of the audit cycle.

There was a statistically and clinically significant increase in recording patients’ alcohol consumption, which increased from 24% to 36% (odds ratio [OR]: 1.19; 95% confidence interval [CI]: 1.10–1.29). While there was no increase in the proportion of patients who had a family history recorded overall, there was a significant increase in the proportion of patients who had a more detailed family history documented. GPs increased their recording of family histories of type 2 diabetes (23% to 32%), early ischaemic heart disease (24% to 33%), breast cancer (21% to 32%) and colorectal cancer (20% to 30%; Table 3).

There was no significant change in recording weight, cholesterol or blood glucose level over the period of the four audit cycles. There was a small, yet statistically significant reduction in blood pressure recording over the four audit periods (86% to 82%). There appeared to be a small increase in recording smoking status (56% to 60%), but this was not statistically significant (P = 0.051; Table 3) .

There was no significant change in recording of Papanicolaou (Pap) test or PSA testing. There was a significant reduction in the recording of mammogram rate over the course of the audit from 46% to 36% at the end of the audit.

Table 3. GPs’ documentation of risk factor assessment in case note audits over time

Risk Factor

Baseline recording

Term 4 recording (42 weeks)

 

 

 

 

n

n (%)

n

n (%)

 Odds ratio* (95% CI)

Cardiovascular risk factors

 

 

 

 

 

Weight

643

274 (43%)

545

247 (45%)

0.98 (0.91–1.06)

Blood pressure

643

551 (86%)

545

446 (82%)

0.90 (0.82–0.99)

Cholesterol

643

443 (69%)

545

377 (69%)

1.01 (0.93–1.09)

Blood glucose

643

450 (70%)

545

384 (70%)

1.01 (0.93–1.09)

Social history risk factors

 

 

 

 

 

Smoking

641

360 (56%)

543

326 (60%)

1.08 (1.00–1.06)

Alcohol

632

154 (24%)

545

196 (36%)

1.19 (1.10–1.29)

Family history overall

642

200 (31%)

545

167 (31%)

1.02 (0.95–1.10)

Family history diabetes

558

126 (23%)

338

108 (32%)

1.22 (1.11–1.34)

Family history colorectal cancer

557

113 (20%)

336

102 (30%)

1.25 (1.14–1.39)

Family history heart disease <60 years of age

559

137 (24%)

338

112 (33%)

1.22 (1.10–1.33)

Family history breast cancer

557

119 (21%)

337

109 (32%)

1.26 (1.14–1.38)

Cancer screening

 

 

 

 

 

Pap smear (aged 18–69 years)

336

169 (50%)

294

148 (50%)

0.99 (0.90–1.09)

Mammogram (aged 50–69 years)

199

 92 (46%)

186

67 (36%)

0.87 (0.76–0.99)

PSA (>50 years years of age)

191

140 (73%)

158

117 (74%)

0.97 (0.84–1.13)

*Odds ratio (OR) estimated using longitudinal log analysis

P value for details of family history and alcohol is <0.0001

P value for smoking is 0.051

PSA, prostate-specific antigen 

Discussion

Our study has two main messages: audit and feedback by a medical student were feasible procedures, and these created clinically important increases in the general practice supervisors’ recordings of social and family history. The changes were significant in areas with the lowest baseline levels. Recording alcohol consumption increased from 24% to 36%. Similarly, recording patients’ family history of type 2 diabetes, early ischaemic heart disease, and breast or colorectal cancer increased by 10% on average. This result is consistent with the findings of the Cochrane review by Ivers et al on the effect of audit with feedback.5 These authors identified low baseline compliance as a factor associated with greater effect of audit with feedback.

Overall, the preventive health parameters measured varied widely in their compliance with guideline recommendations. Some areas of preventive care showed high compliance with guideline recommendations (eg recording the cardiovascular risk factors and measuring cholesterol in adults >50 years of age). In areas of high baseline compliance, we were not able to demonstrate improvement, which can be explained by a ceiling effect.8 In fact, rates of blood pressure recording dropped from 86% to 82%. The most likely explanation for this observation is a normal fluctuation of adherence around a peak of performance.

The results concerning cancer screening were diverse. We observed a significant reduction in mammogram rate over the course of the audit, which is difficult to explain. There are multiple external factors involved in the breast cancer screening program in Australia, and it is unlikely that the change occurred as a result of the GP audit. By contrast, recording social history is entirely within the influence of the GP and, intuitively, we can understand that identifying a low level of adherence in this area could result in an immediate change in practice.

Another interesting finding was the PSA testing rate, which was high (75% of men >50 years of age) despite the fact that it is not recommended in key general practice guidelines.6 This high rate of testing may be attributable to media coverage of PSA testing and increased patient expectations.4,8

The strengths of our study include our study design, which provided a pragmatic, innovative approach to data collection in Australian general practice. A total of 3076 patient audits were completed while students were learning about preventive healthcare, with the only extra costs being academic time and postage costs. Many studies that have shown a significant change with audit required a very labour intensive, and therefore expensive, audit and feedback process.

Another strength of the study is the rather brave challenge given to medical students in asking them to break out of the hierarchical roles and critique the care given by their supervisors. This was potentially challenging for the students and GPs. However, all the GPs who responded to the post-audit questionnaire indicated that they were comfortable with students performing an audit on their patients, so the difficulties may be more a perception than a reality.

The pragmatic nature of this study created some methodological limitations. We were unable to have a control group as all students needed to complete this audit activity. We used very small cross-sectional samples of patients with the assumption that they are a representative sample of that GPs’ patient population.

An audit has minimal effect unless it is associated with feedback to the clinician.9 The analysis method used in this study assumed that all GPs had equal exposure to the audit and feedback. We were unable to determine the quality of the feedback given by the students. The post-audit questionnaire sent to GPs designed to help assess the quality of feedback had a very low response rate (15 out of 58; 26%).

We did not account for potential clustering effects in our analyses. As an observational study, the observed changes may have been due to other factors within the practices that are unrelated to the study. The small cross-sectional samples of patients may have included patients who also saw other doctors in the practice who were not audited and not given feedback.

This study has shown a small but measurable effect of a student educational activity on the quality of documentation of certain disease risk factors by their supervising GP. This differs from the studies by Gould et al10 and Doyle et al.11 Their studies assessed the impact of more substantial student-led quality improvement activities, which represented a significant body of work for the students; most medical curricula would only expect students to perform one of these activities. This study showed the effect of a small, brief educational activity with multiple students participating at multiple time points. Medical curricula could expect students to perform multiple activities of this size if they were found to have educational validity. It is possible that many of the educational activities in which students participate by reviewing patients in detail and discussing their care with a treating doctor could trigger some reflection by the treating doctor on the quality of their care. This is an area that could deserve further study.

The limitation of time within the consultation and limited human resources are universally cited as barriers for delivering preventive care outcomes in primary care. We require a paradigm shift in the way we approach preventive health in general practice and should look to the whole practice team, including the medical students, to assist through measures such as improving practice systems to record preventive health information.

Implications for general practice

This study has shown that a student‑led audit with feedback can result in improvement in some parameters of preventive care, and that students could be considered as potentially useful members of the primary healthcare team in the campaign for health promotion and disease prevention in the Australian population.

Authors

Lucy A Gilkes MBBS, FRACGP, MClinRes, Senior Lecturer, School of Primary, Aboriginal and Rural Health Care, Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia, Crawley, WA. lucy.gilkes@uwa.edu.au

Helena Liira MD, PhD, MBA, Professor of General Practice, Head of School, School of Primary, Aboriginal and Rural Health Care, Faculty of Medicine, Dentistry and Health Sciences, The University of Western Australia. Crawley, WA

Jon D Emery DPhil, MA, MBBCh, BA, Herman Professor of Primary Care, Cancer Research, Department of General Practice, Melbourne Medical School, Faculty of Medicine, Dentistry & Health Sciences, The University of Melbourne and Western Health, Carlton, Vic

Competing interests: None.

Provenance and peer review: Not commissioned, externally peer reviewed

Acknowledgements

We would like to thank all the medical students and their general practice supervisors who collaborated with us for this study. With statistics we received highly appreciated help from Dr Sharon Evans.

References

  1. Price R, Spencer J, Walker J. Does the presence of medical students affect quality in general practice consultations? Med Educ 2008;42(4):374–81.
  2. Cooke F, Galasko G, Ramrakha V, Richards D, Rose A, Watkins J. Medical students in general practice: How do patients feel? Br J Gen Pract 1996;46(407):361–62.
  3. Walters K, Buszewicz M, Russell J, Humphrey C. Teaching as therapy: Cross sectional and qualitative evaluation of patients’ experiences of undergraduate psychiatry teaching in the community. BMJ 2003;326(7392):740.
  4. Rubio-Valera M, Pons-Vigués M, Martínez-Andrés M, Moreno-Peral P, Berenguera A, Fernández A. Barriers and facilitators for the implementation of primary prevention and health promotion activities in primary care: A synthesis through meta-ethnography. PLOS One 2014;9(2):e89554.
  5. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;(6):CD000259.
  6. The Royal Australian College of General Practitioners. Guidelines for preventive activities in general practice. 8th edn. East Melbourne: RACGP, 2012.
  7. Britt H. General practice activity in Australia 2011–12: BEACH, Bettering the Evaluation And Care of Health. Sydney: Sydney University Press, 2012.
  8. Gattellari M, Young JM, Ward JE. GP and patient predictors of PSA screening in Australian general practice. Fam Pract 2003;20(3):294–303.
  9. Jamtvedt G, Young J, Kristoffersen D, O’Brien M, Oxman A. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006;(2):CD000259.
  10. Gould BE, Grey MR, Huntington CG, et al. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Acad Med 2002;77(10):1011–18.
  11. Doyle DB, Burkhardt MA, Copenhaver J, Thach S, Sotak D. Health professions students as research partners in community oriented primary care. J Community Health 1998;23(5):337–46.

Correspondence afp@racgp.org.au

Yes     No

Declaration of competing interests *

Yes No

Additional Author (remove)

Yes No

    

 

 

 

 

Competing Interests: 

Your comment is being submitted, please wait

 

Download citation in RIS format (EndNote, Zotero, RefMan, RefWorks)

Download citation in BIBTEX format (RefMan)

Download citation in REFER format (EndNote, Zotero, RefMan, RefWorks)

For more information see Wikipedia: Comparison of reference management software