Research indicates that a patient’s transition from hospital care to the care of their general practitioner (GP) carries significant risk of unforeseen adverse effects, including emergency department re-admissions, disability and even death.1,2 It has been reported that almost half (49%) of the patients discharged from hospital experienced at least one adverse event in their continuing care because of incorrect information contained in their hospital discharge summary.3 Therefore, high‑quality hospital discharge communication is essential in helping reduce adverse discharge-related events.4 Also, from a hospital-based perspective, effective discharge summaries, which enable effective clinical handover, are required for accreditation under the Australian National Safety and Quality Service Standards (Standards). These Standards, derived from Australian and international research, set out to establish a consistent set of evidenced-based processes to be used across healthcare services.5 Australian hospital discharge documents based on the ‘eDischarge summary’6 are similar to those reported in international literature in regards to providing the information considered to be essential for successful continuity of care.
International and Australian studies concerning discharge summaries have addressed:
- GP satisfaction with the quality and timeliness of electronic discharge summaries7–9
- audits of the accuracy of, and GP satisfaction with, medications outlined on discharge summaries10,11
- ranking discharge information options by GPs in order of importance12,13
- validation of a scale to measure the quality of hospital discharge summaries for older patients from a GP perspective14
- examination of the reliability, effectiveness, accuracy and timeliness of information transfer from the hospital to the GP.15–19
Furthermore, Middleton et al investigated patients’ knowledge of their hospitalisation and perceived readiness to leave in comparison with GPs’ attitudes towards the usefulness of discharge communications.20 A consistent theme in the literature has been the significant scope for improvement in the quality of discharge-related communication.
We are only aware of one Australian study undertaken to identify and rank what GPs believed were the essential elements required in a discharge summary to enable successful post-hospital continuity of care.12 That study was conducted in a single, Western Australian metropolitan location and concerned patients who had undergone total hip or knee replacement.12 We are aware of extensive research concerning perceptions of adequate discharge planning in Australia and the US.14,21 However, there remains a pressing need for research to directly inform improvements in the discharge summary instrument itself. Therefore, the aim of this study was to develop and pilot a discharge summary quality assessment tool for use in Australia, including assessment of the components that a sample of Australian GPs identified as being most important for safe transfer of care.
Methods
Materials
Development of the pilot instrument was informed by drawing on core themes and findings from a comprehensive review of international literature pertaining to key components of effective discharge summaries. PubMed, MEDLINE, Cochrane Library, CINHAL, SCOPUS and Web of Science databases were searched using the following search terms: ‘GP OR general practice OR primary care AND secondary care OR post-hospital care OR inpatient OR outpatient AND discharge communications OR discharge summary systems OR discharge summary improvements OR hospital discharge summary OR quality assurance AND information transfer OR communication discontinuity OR inaccurate information OR data accuracy OR data quality OR timeliness OR data reliability’. The literature had to be published in English between January 2000 and December 2013.
After screening the abstracts, 64 articles were considered to be relevant to the study and retained for review. A further four papers were included following screening of the reference lists of the retained articles. The papers were summarised and tabulated to facilitate synthesis. In an iterative process, the research team identified 16 components of discharge summaries from the literature that were reported to be important for discharge summary quality. These components were included as items for the pilot instrument (Table 1).
Table 1. Instrument items
a.
|
Reason for admission or presentation to hospital
|
b.
|
Physician examination findings on presentation to hospital
|
c.
|
Results of diagnostic tests done in hospital
|
d.
|
Treatment in hospital
|
e.
|
Progress during hospital admission
|
f.
|
List of medications on admission
|
g.
|
List of medications on discharge
|
h.
|
Reasons for any changes in medications
|
i.
|
List of diagnoses on discharge
|
j.
|
Patient condition or functional status on discharge
|
k.
|
Details of follow up arrangements
|
l.
|
Format
|
m.
|
Prioritising of reported pathology results
|
n.
|
Patients psychological/emotional responses to their hospital stay
|
o.
|
Information given to patient and family
|
p.
|
Patients preferences regarding management
|
Part A of the pilot instrument invited participants to rank the importance of each component using five-point Likert-type response items (1 = ‘Very unimportant’ and 5 = ‘Very important’) in order to assess the relevance of these components for Australian GPs. Part B invited participants to rank their satisfaction with each component in reference to discharge summaries they had received, also using five-point Likert-type response items (1 = ‘Very unsatisfied’ and 5 = ‘Very satisfied’).
In addition to these items, study participants were asked to indicate their preference for the way in which they received discharge summaries (paper or electronic), and other demographic information, including age and gender. They were also invited to respond to an open-ended question at the end of the survey: ‘Do you have any further comments regarding discharge summaries?’. Trialling of the survey within the research team indicated the survey would take 10–15 minutes to complete.
Recruitment
Ethics approval was obtained from the University of Wollongong’s Human Research Ethics Committee (reference HE13/471). In 2014, general practice registrar members of Coast City Country General Practice Training (CCCGPT), GP members of the Illawarra Shoalhaven and Tasmanian Medicare Locals, and a randomised national sample of GPs (available from a commercial database) were invited by email to take part in the pilot. GPs and general practice registrars who volunteered to take part in the pilot study were subsequently sent, via email, a link to the online survey instrument. A reminder email prompt to complete the instrument was sent two weeks after the initial email. A hardcopy of the survey instrument with a reply-paid envelope was sent to participants who did not wish to complete the instrument online.
Analysis
The data were examined and instruments with more than 50% of missing data were excluded from further analysis. The responses to the importance of items were dichotomised (‘Very unimportant/unimportant/neither unimportant or important’ and ‘Important/very important’) to facilitate ranking. Cronbach’s alpha was used to assess the internal reliability of the importance and satisfaction scales respectively. The free text responses were independently coded by two researchers and agreement on themes reached by consensus among the research group. The themes were then compared with the importance and satisfaction responses to inform assessment of the content validity of the instrument.
Results
From 1236 invitations, 121 instruments were returned, and 118 were retained for further analysis (9.5% response rate). The average age of respondents was 52.2 years (standard deviation [SD]: 12.1 years; range: 27–90); 41.5% were female and the average number of years since graduating was 27 (range: 3–62). The majority of respondents completed their medical degree in Australia (n = 81, 68.6%) and spoke English as their first language (n = 98; 83.0%). A majority of respondents preferred to receive discharge summaries electronically (n = 75; 63.6%).
Part A: Ranking of the 16-item hospital discharge summaries
Seven of the items had >90% of respondents, indicating that the described discharge summary component was either ‘Important’ or ‘Very important’. These items were:
- list of medications on discharge
- reason for admission
- treatment in hospital
- details of follow-up arrangements
- list of diagnoses on discharge
- results for diagnostic tests done in hospital
- reasons for any changes in medications.
The only item to have <50% of respondents who rated it as ‘Important/very important’ was item ‘n’, the psychological response of the patient to the hospital stay (44.4%). Table 2 outlines the frequency of responses and rankings of the combined ‘Important/very important’ frequencies.
Table 2. Participant ratings of the importance of the 16 items for hospital discharge summaries
|
Very unimportant n (%)
|
Unimportant n (%)
|
Neither unimportant or important n (%)
|
Important n (%)
|
Very important n (%)
|
Rank
|
|
a.
|
Reason for admission or presentation to hospital
|
2 (1.7)
|
0 (0)
|
1 (0.8)
|
19 (16.1)
|
96 (81.4)
|
2
|
b.
|
Physician examination findings on presentation to hospital
|
3 (2.6)
|
7 (6.0)
|
12 (10.3)
|
58 (49.6)
|
37 (31.6)
|
10
|
c.
|
Results of diagnostic tests done in hospital
|
2 (1.7)
|
2 (1.7)
|
4 (3.4)
|
45 (38.1)
|
65 (55.1)
|
6
|
d.
|
Treatment in hospital
|
2 (1.7)
|
0 (0)
|
2 (1.7)
|
46 (39.0)
|
68 (57.6)
|
3
|
e.
|
Progress during hospital admission
|
1 (0.9)
|
3 (2.6)
|
28 (24.3)
|
57 (49.6)
|
26 (22.6)
|
12
|
f.
|
List of medications on admission
|
5 (4.3)
|
10 (8.5)
|
31 (26.5)
|
45 (38.5)
|
26 (22.2)
|
15
|
g.
|
List of medications on discharge
|
2 (1.7)
|
0 (0)
|
0 (0)
|
10 (8.5)
|
106 (89.8)
|
1
|
h.
|
Reasons for any changes in medications
|
1 (0.8)
|
1 (0.8)
|
6 (5.1)
|
27 (22.9)
|
83 (70.3)
|
7
|
i.
|
List of diagnoses on discharge
|
2 (1.7)
|
0 (0)
|
3 (2.5)
|
23 (19.5)
|
90 (76.3)
|
5
|
j.
|
Patient condition or functional status on discharge
|
2 (1.7)
|
1 (0.9)
|
9 (7.7)
|
51 (43.6)
|
54 (46.2)
|
8
|
k.
|
Details of follow up arrangements
|
2 (1.7)
|
1 (0.8)
|
1 (0.8)
|
42 (35.6)
|
72 (61.0)
|
4
|
l.
|
Efficient format
|
2 (1.7)
|
2 (1.7)
|
14 (12.0)
|
47 (40.2)
|
52 (44.4)
|
9
|
m.
|
Prioritising of reported pathology results
|
2 (1.7)
|
3 (2.6)
|
26 (22.4)
|
53 (45.7)
|
32 (27.6)
|
11
|
n.
|
Patients psychological/emotional responses to their hospital stay
|
5 (4.3)
|
15 (12.8)
|
45 (38.5)
|
46 (39.3)
|
6 (5.1)
|
16
|
o.
|
Information given to patient and family
|
3 (2.6)
|
5 (4.3)
|
37 (31.6)
|
48 (41.0)
|
24 (20.5)
|
13
|
p.
|
Patients preferences regarding management
|
3 (2.6)
|
8 (6.8)
|
34 (29.1)
|
56 (47.9)
|
16 (13.7)
|
14
|
Part B: Satisfaction ratings of the 16-item hospital discharge summaries
The majority of respondents were ‘Very unsatisfied’ or ‘Unsatisfied’ with four items in hospital discharge summaries they received:
- reasons for any changes in medications (65.5%)
- prioritising reported pathology results (56.7%)
- format (46.9%)
- patient’s condition or functional status on discharge (41.7%).
Nearly three-quarters of respondents were ‘Satisfied’ or ‘Very satisfied’ with the reason for admission or presentation to hospital (71.6%), and two-thirds with the list of medications on discharge (66%; refer to Table 3).
The internal reliability of the importance and satisfaction scales were assessed separately using Cronbach’s alpha. For the 16-item importance scale, Cronbach’s alpha was 0.87. This improved to 0.91 with the removal of item ‘n’. Cronbach’s alpha for the 16-item satisfaction scale was 0.92, marginally reducing to 0.91 with removal of item ‘n’. All of these results indicated a high level of internal reliability for each scale.22
Table 3. Participant ratings of their satisfaction with information provided on hospital discharge summaries
|
|
Very unsatisfied n (%)
|
Unsatisfied n (%)
|
Neither unsatisfied or satisfied n (%)
|
Satisfied n (%)
|
Very satisfied n (%)
|
a.
|
Reason for admission or presentation to hospital
|
3 (2.6)
|
16 (13.8)
|
14 (12.1)
|
72 (62.1)
|
11 (9.5)
|
b.
|
Physician examination findings on presentation to hospital
|
5 (4.2)
|
21 (18.3)
|
39 (33.9)
|
44 (38.3)
|
6 (5.2)
|
c.
|
Results of diagnostic tests done in hospital
|
6 (5.2)
|
28 (24.3)
|
25 (21.7)
|
48 (41.7)
|
8 (7.0)
|
d.
|
Treatment in hospital
|
5 (4.3)
|
22 (19.0)
|
35 (30.2)
|
50 (43.1)
|
4 (3.4)
|
e.
|
Progress during hospital admission
|
8 (6.9)
|
27 (23.3)
|
42 (36.2)
|
36 (31.0)
|
3 (2.6)
|
f.
|
List of medications on admission
|
6 (5.3)
|
22 (19.5)
|
47 (41.6)
|
37 (32.7)
|
1 (0.9)
|
g.
|
List of medications on discharge
|
8 (7.0)
|
21 (18.3)
|
10 (8.7)
|
61 (53.0)
|
15 (13.0)
|
h.
|
Reasons for any changes in medications
|
23 (19.8)
|
53 (45.7)
|
24 (20.7)
|
14 (21.1)
|
2 (1.7)
|
i.
|
List of diagnoses on discharge
|
7 (6.1)
|
22 (19.1)
|
31 (27.0)
|
50 (43.5)
|
5 (4.3)
|
j.
|
Patient condition or functional status on discharge
|
9 (7.8)
|
39 (33.9)
|
40 (34.8)
|
25 (21.7)
|
2 (1.7)
|
k.
|
Details of follow up arrangements
|
10 (8.7)
|
25 (21.7)
|
40 (34.8)
|
36 (31.3)
|
4 (3.5)
|
l.
|
Format
|
26 (22.6)
|
28 (24.3)
|
30 (26.1)
|
28 (24.3)
|
3 (2.6)
|
m.
|
Prioritising of reported pathology results
|
21 (18.9)
|
42 (37.8)
|
32 (28.8)
|
15 (13.5)
|
1 (0.9)
|
n.
|
Patient psychological/emotional responses to their hospital stay
|
12 (10.6)
|
23 (20.4)
|
67 (59.3)
|
10 (8.8)
|
1 (0.9)
|
o.
|
Information given to patient and family
|
20 (17.2)
|
26 (22.4)
|
58 (50.0)
|
11 (9.5)
|
1 (0.9)
|
p.
|
Patient preferences regarding management
|
13 (11.8)
|
22 (20.0)
|
65 (59.1)
|
8 (7.3)
|
2 (1.8)
|
Thematic analysis of participants’ responses to the open-ended question
Three key themes were identified from the 68 free-text responses to this question (58% of the total sample). The first theme illustrated concerns regarding the quality and content of current discharge summaries. Thirty-four of the 68 free-text responses were included in this theme. Some respondents suggested that discharge summaries were at times unwieldy and had irrelevant content, possibly reflecting cutting and pasting from electronic medical records by junior doctors.
… overloaded with useless and irrelevant stuff, obscuring the really important bits. Often, RMOs do a free-type segment, it is often the most informative bit … – Female GP, 64 years of age, regional New South Wales
… often written by a junior doctor who has not actually seen [the patient].
– Female GP, 51 years of age, regional Western Australia
The second theme centred on the timeliness of the discharge summaries (32 of 68 responses), with particular concerns regarding how this could affect the quality of the patients’ continuity of care.
The main problem with discharge summaries is that you often don’t get one and have to ring medical records to obtain a copy, whilst the patient is there in front of you and, not uncommonly, there isn’t one completed yet.
– Female GP, 54 years of age, regional New South Wales
Timeliness is essential. Often, I get electronic notices saying ‘patient has been discharged’, but no discharge letter, no details. – Female GP, 28 years of age, regional New South Wales
The format of the current discharge summaries was the third theme identified (35 of 68 responses), which suggested that currently available discharge summaries were too time-consuming, complex and difficult to read, making it difficult to extract important information.
They tend to use ‘in-house’ abbreviations without explanation … are long and contain lots of useless paragraphs. The diagnosis is not obvious and one has to carefully read a long document to tease out the diagnosis.
– Male GP, 61 years of age, metropolitan Victoria
… electronic version often has unimportant template information on mass and you have to search in the small spaces given for information like actual diagnoses! – Female GP, 48 years of age, rural New South Wales
Discussion
This paper describes the initial development and results of piloting an Australian discharge summary quality assessment tool. The participating GPs had a high level of agreement on the most crucial components of discharge summaries included in the tool. These items were directly related to the reason for admission and immediate post-discharge care, including discharge medications, diagnoses and follow-up arrangements. It was reassuring to note that nearly three-quarters of GPs surveyed were satisfied with the reason for admission and, to a slightly lesser extent, medications on discharge. The importance and satisfaction scales had high levels of internal reliability. The open-text responses supported the content of the items included in the instrument, but highlighted that while the instrument sought to ascertain and assess important components of quality discharge summaries, it did not have an item relating to timeliness.
Our findings regarding the participants’ more significant concerns about the quality, content and timeliness of discharge summaries are supported elsewhere in the literature.15,18,23 These issues need to be further investigated and addressed, as poor transfer of information has the potential to compromise patient safety by increasing their risk of adverse events. Similarly, dissatisfaction with the format of the currently available discharge summaries also needs to be addressed in order to reduce the risk of error and to optimise patients’ continuity of care during transfer.
Our selection of items that are most important for GPs was based on a review of the international literature and supported through ranking by more than 100 Australian GPs. Other means of validating the importance of these items, such as a Delphi process or nationally representative survey, would provide triangulation of these findings and will be considered in further development of the tool. The tool requires internal reliability and test–retest reliability analysis across different healthcare contexts in Australia, and assessment of sensitivity to change after quality improvement initiatives.
Limitations
While the results of this pilot study provide useful data to assist in evaluating the performance of the tool, the low response rates from national samples mean that they should be generalised with caution. In order to increase the response rate for future work using this tool, researchers should consider incorporating follow-up emails and letters to the GPs to encourage them to complete the survey.24
It is possible that GPs with polarised views concerning discharge summaries may have been more likely to respond, creating bias in the results. This may have been amplified in the written sections as completing an open-response item requires additional effort. The small sample size does not permit more than descriptive statistics to be presented.
Conclusions
This pilot Australian discharge summary quality assessment tool was developed from a theoretical base and appears to address critical areas in discharge summaries as judged by this sample of Australian GPs. Piloting of the instrument produced results in keeping with international and Australian research, and the tool demonstrated favourable psychometric properties. The item relating to the patient’s psychological response appears to be less important to this sample of Australian GPs, and as its removal did not impair the reliability of the instrument, it can be removed. In response to the piloting, we also recommend inclusion of ‘timeliness’ as an item. Following the piloting we have described, as a tool to measure the quality of discharge summaries as judged by GPs and hopefully also measure improvements, only the satisfaction scale items will be required. We would welcome further research using the pilot tool, administered by PHNs, Local Health Districts (LHDs) and academic groups, to further refine it and act as a stimulus for improving discharge communication. The pilot tool is presented in Table 4. The pilot tool would appear ideal for use in discrete LHD and PHN catchments where assessment using it could be used to guide quality assurance activities, followed by re-administration of it to gauge changes in discharge summary quality.
Table 4. An Australian discharge summary quality assessment tool
Please indicate how satisfied you have been with the information you have received for the following elements of a discharge summary over the last (period of time to be assessed)
|
|
Very unsatisfied (1)
|
Unsatisfied (2)
|
Neither unsatisfied or satisfied (3)
|
Satisfied (4)
|
Very satisfied (5)
|
Timeliness of receipt of the discharge summary
|
|
|
|
|
|
Reason for admission or presentation to hospital
|
|
|
|
|
|
Physician examination findings on presentation to hospital
|
|
|
|
|
|
Results of diagnostic tests done in hospital
|
|
|
|
|
|
Treatment in hospital
|
|
|
|
|
|
Progress during hospital admission
|
|
|
|
|
|
List of medications on admission
|
|
|
|
|
|
List of medications on discharge
|
|
|
|
|
|
Reasons for any changes in medications
|
|
|
|
|
|
List of diagnoses on discharge
|
|
|
|
|
|
Patient condition or functional status on discharge
|
|
|
|
|
|
Details of follow up arrangements
|
|
|
|
|
|
Format
|
|
|
|
|
|
Prioritising of reported pathology results
|
|
|
|
|
|
Information given to patient and family
|
|
|
|
|
|
Patients preferences regarding management
|
|
|
|
|
|
Implications for general practice
- This pilot discharge summary assessment tool was developed from a comprehensive literature review and piloting with a sample of Australian GPs.
- This sample of Australian GPs considered core clinical and management information to be the most important components of discharge summaries, which is consistent with international studies.
- This pilot discharge summary assessment tool may be useful for evaluating and improving the quality of discharge summaries within LHDs and PHNs.
- The data collected from this pilot study will assist in refining and validating the pilot discharge summary assessment tool, which may be of significant value to health services in their quality improvement initiatives.
- This study forms the first step of a planned, ongoing project to improve communication between community and hospital-based health services.
Authors
Carl Mahfouz MBBS (Hons), AMC (Sydney), FRACGP, General Practitioner, Shellharbour, NSW; Regional Community Academic leader: Illawarra, Graduate School of Medicine, University of Wollongong, NSW; Illawarra and Southern Practice Research Network (ISPRN); Career Medical Officer, Port Kembla Hospital, NSW. cmahfouz@uow.edu.au
Andrew Bonney MBBS (Hons), MFM (Clin), PhD DRANZCOG, FRACGP, Roberta Williams Chair of General Practice; Director: Illawarra and Southern Practice Research Network (ISPRN); Graduate School of Medicine, University of Wollongong, NSW
Judy Mullan BPharm, BA , FSHP, AACP, PhD, Deputy Director, Illawarra and Southern Practice Research Network (ISPRN); A/Prof Graduate School of Medicine, University of Wollongong, NSW
Warren Rich DipT, BEd, GradDip Tch Lib, GradDipSci, MEd (ChildLit), MEd (IT Ed), MAppSci, ScEdD, Lecturer, Research and Critical Analysis; School of Medicine, Faculty of Science, Medicine and Health, University of Wollongong, NSW
Competing interests: None.
Provenance and peer review: Not commissioned, externally peer reviewed.
Acknowledgements
The authors would like to thank the practices and participants for their involvement; Ms Bridget Dijkmans-Hadley and Ms Alyssa Horgan for her support with the research project; and CCCGHPT for their $5000 contribution in funding this project.