Immunisation is one of the core functions of public authorities in Australia. Similar to other public health units in New South Wales (NSW), routine work of the immunisation team in the Northern Sydney Public Health Unit includes providing general advice and information on adult and childhood immunisation to healthcare providers and members of the public, following up reports of adverse reactions to vaccinations, facilitating the neonatal hepatitis B program, providing mandatory education to immunisation providers and delivering the NSW Health school-based vaccination program.1,2 The Northern Sydney Local Health District (NSLHD) is aligned with the Sydney North Primary Health Network (PHN) and provides health services for approximately 950,000 residents in a metropolitan setting.
Following the introduction of mandatory reporting of vaccinations to the Australian Immunisation Register (AIR),3 immunisation reports are provided to public health authorities to assess local immunisation coverage.4 These reports are provided to the relevant authorities by the Chief Executive Medicare under the Health Insurance Act 1973 following its Amendment Bill 1996. The reports identify individual children who are overdue for at least one of their childhood immunisations in order to monitor and improve vaccination rates.
The rate of fully immunised children aged five years is a population health indicator for PHNs.5 Immunisation coverage assessment across all levels, including PHNs, relies on the accuracy of AIR data.6 Previous studies at national and state levels6–9 have identified an underestimate of actual vaccination rates when comparing AIR reports and data from individual and provider surveys. Major problems impacting AIR data have been documented and include issues associated with automated data transmission from practice software directly to the AIR, failure to capture data on overseas-vaccinated children and data entry errors.10 A recent study in NSW concluded that under-reporting by healthcare providers to the AIR remains an important issue impacting AIR data.8
Strategies to improve immunisation coverage and accuracy are identified in the National immunisation strategy 2019–2024.11 Nationally and within states, policies to improve vaccination rates through incentives to primary healthcare providers have been implemented with considerable benefits.12 Although immunisation rates are relatively high in Australia, it is important that high rates are maintained and increased, where possible, with the ultimate aim of eliminating vaccine-preventable diseases. A 2009 study by Ali et al evaluated an intervention that aimed at supporting practices with less than 90% vaccination coverage,13 with one of the recommendations being that periodic practice visits for reminders and supporting practices could improve coverage.
In 2017, the NSLHD implemented an intervention to assess and improve AIR recordings of vaccination status using 11A reports, which identify individual children who are due or overdue for vaccination by locality, postcode and age range.4 In this intervention, a public health nurse visited selected general practices and assisted the practice nurse and/or general practitioner to review the identified children’s vaccination records to confirm whether they were truly overdue. For truly overdue children, the public health nurse facilitated the general practice to send a vaccine reminder to the parent or carer.
The aims of the present study were to assess: (1) the accuracy of AIR vaccination records of children overdue for routine childhood vaccinations in the selected practices; and (2) the impact of the public health nurse’s assistance to the general practices on correcting AIR records about previously administered vaccinations and subsequent vaccination for overdue children.
Methods
Recruitment of the intervention and comparison groups
Monthly lists of children aged five years or younger residing in the NSLHD who were overdue for at least one National Immunisation Program vaccine14 were downloaded from the AIR for the period November 2017 to July 2018. The definition of ‘overdue’ was based on AIR due and overdue rules for immunisation.15 Based on the November 2017 initial report, the 23 practices with the highest number of overdue children were identified. Children whose last immunisation provider was one of these practices were assigned to the intervention group.
A comparison group was obtained by taking a random sample of children aged five years or younger from a list of all overdue children residing in the NSLHD from the 11A reports4 for the same time period, November 2017 – July 2018. Children already assigned to the intervention group were excluded from the comparison group. Monthly reports during this period were reviewed and samples were taken to ensure a similar number of comparison children was selected each month to the number of children in the intervention group for that particular month.
Intervention
A public health nurse visited each practice between November 2017 and July 2018 and reviewed individual medical records of children identified as overdue by the 11A report with practice staff to validate what vaccinations had been given and whether the child was not up to date with vaccinations at the time of entry into the study. Following this review, children on the original 11A report who could not be confirmed as being up to date for their vaccinations were considered truly overdue. Information about whether the child was still a current patient of the practice or had moved outside the NSLHD was also collected. Practices sent reminders to parents and guardians of children identified as being current patients and truly overdue according to practice records. This took the form of a telephone call, short message service (SMS) or letter sent by mail, depending on the practice’s preference. The number of reminders sent was also determined by the practice depending on their capacity. The comparison group did not receive any intervention and it was assumed they followed their normal general practice follow-up protocols.
Follow-up and outcome assessment
Completing the primary course and the fourth dose of the tetanus, diphtheria and pertussis (DTPa) vaccine (scheduled for two, four, six and 18 months) and having had one or two doses of the measles, mumps and rubella (MMR) vaccine (scheduled for 12 and 18 months) were the primary outcomes of interest. Because the 11A report contains overdue vaccinations as antigen rather than vaccines, the diphtheria antigen was used as a proxy for the DTPa vaccine and the measles antigen was used as a proxy for the MMR vaccine. This outcome could have been achieved either through recording vaccines already given prior to the initial visit (ie correct an administrative error on the AIR) or through the child receiving additional vaccinations. The outcome was assessed by checking the vaccinations recorded on AIR for the child as at 20 June 2019. This was done through matching of children’s given names, surnames and dates of birth against a consolidate AIR file that included all recorded vaccines for children in NSW. This resulted in children being followed up for between 11 and 19 months. Children no longer on the AIR or identified by practices as not living in NSW were excluded from the analysis.
Analysis
The completeness of AIR data for the intervention group was analysed by determining the proportion of children who were truly overdue according to practice record reviews. A child was deemed truly overdue if they had not received all the vaccines recommended for their age based on the NSW childhood immunisation schedule.14 The intervention’s impacts on both updating AIR records for vaccines already administered but not recorded and the vaccination of truly overdue children were analysed. This was done for the following groups with age calculated at the time of the initial practice visit: completion of the primary DTPa course for children aged over seven months; administration of the 18-month (fourth dose) DTPa vaccine for children aged over 19 months; administration of the first MMR vaccine for children aged over 13 months; and administration of the second MMR vaccine in children aged over 19 months.
Intervention and comparison groups were compared using Χ2 tests for both AIR record updating and subsequent vaccination of truly overdue children.
Ethical considerations
Ethics approval for the study was granted by the NSLHD Human Research Ethics Committee in its capacity as a lead NSW public health system ethics approval body (reference no. 2019/ETH12912).
Results
Intervention practices and vaccination status at initial visit
At the initial visit to the 23 practices selected for the intervention group, most overdue children identified through 11A reports were known to the practices (89.3%); however, 28.3% of the total cohort (410/1448) were incorrectly classified as overdue according to practice records, with 61% of children considered truly overdue by practices. The proportion of truly overdue children varied across practices from as low as 37.5% (Practice L) to as high as 88% (Practice K), with a mean of 68% (Table 1).
| Table 1. General practices visited and proportions of children who were truly overdue |
| |
Initial visit date |
Overdue children according to AIR (a) |
Truly overdue (b) |
No longer at practice (c) |
% Truly overdue (b/[a – c] × 100) |
| Practice visited |
| A |
8 November 2017 |
69 |
34 |
5 |
53.13 |
| B |
29 November 2017 |
78 |
53 |
6 |
73.61 |
| C |
30 November 2017 |
77 |
43 |
20 |
75.44 |
| D |
12 December 2017 |
47 |
35 |
1 |
76.09 |
| E |
13 December 2017 |
63 |
52 |
2 |
85.25 |
| F |
14 December 2017 |
62 |
26 |
13 |
53.06 |
| G |
18 December 2017 |
57 |
36 |
8 |
73.47 |
| H |
19 December 2017 |
112 |
83 |
5 |
77.57 |
| I |
21 December 2017 |
77 |
18 |
29 |
37.50 |
| J |
22 December 2017 |
53 |
28 |
0 |
52.83 |
| K |
4 January 2018 |
25 |
21 |
1 |
87.50 |
| L |
5 January 2018 |
32 |
11 |
2 |
36.67 |
| M |
9 January 2018 |
86 |
57 |
7 |
72.15 |
| N |
10 January 2018 |
171 |
108 |
16 |
69.68 |
| O |
12 January 2018 |
35 |
16 |
0 |
45.71 |
| P |
18 January 2018 |
82 |
63 |
1 |
77.78 |
| Q |
19 January 2018 |
37 |
23 |
0 |
62.16 |
| R |
16 April 2018 |
43 |
11 |
26 |
64.71 |
| S |
24 April 2018 |
97 |
62 |
6 |
68.13 |
| T |
23 May 2018 |
42 |
30 |
0 |
71.43 |
| U |
6 June 2018 |
33 |
24 |
1 |
75.00 |
| V |
4 July 2018 |
41 |
33 |
0 |
80.49 |
| W |
10 July 2018 |
29 |
16 |
6 |
69.57 |
| Total |
1448 |
883 |
155 |
68.29 |
Vaccination status on AIR in June 2019: Intervention and comparison groups
In the intervention group, there were 714 (51%) boys and the mean age was 33.9 months at recruitment (age range 3.9–60 months). This was similar to the comparison group, in which there was a total of 783 (52%) boys and the mean age was 32.6 months (age range 3.6–60 months). Figure 1 schematically describes the results obtained for the study using the completion of the primary DTPa vaccination course as an example.
Figure 1. Schematic of study design, tetanus, diphtheria and pertussis (DTPa) primary course. Click here to enlarge
AA child was considered up to date if they had a record of having received the vaccine they were overdue for prior to entry into the study according to the Australian Immunisation Register (AIR) as at June 2019.
Table 2 compares the proportion of truly overdue children who were subsequently vaccinated by vaccine course and intervention status. The comparison group achieved a higher proportion of overdue children being vaccinated by June 2019 compared with the intervention group, except for the primary DTPa course, where ‘catch-up’ was very low for both groups.
| Table 2. Vaccination outcome in June 2019 by vaccine initially overdue for |
| Vaccines overdue for |
Outcome June 2019 |
| No. vaccinated (%) |
RR (95% CI) |
P-value |
No. record updated (%) |
RR (95% CI) |
P-value |
| Triple antigen (primary course) |
| Intervention |
21/196 (10.7) |
1.05 (0.61, 1.81) |
0.86 |
185/381 (48.6) |
1.46 (1.22, 1.74) |
<0.001 |
| Comparison |
26/255 (10.2) |
127/382 (33.2) |
| Triple antigen (fourth dose) |
| Intervention |
169/478 (35.4) |
0.73 (0.60, 0.86) |
<0.001 |
215/693 (31) |
1.45 (1.20, 1.74) |
<0.001 |
| Comparison |
242/503 (48.1) |
137/640 (21.4) |
| MMR (first dose) |
|
| Intervention |
53/321 (16.5) |
0.67 (0.49, 0.92) |
<0.012 |
168/489 (34.4) |
1.5 (1.20, 1.90) |
<0.001 |
| Comparison |
80/327 (24.5) |
96/423 (22.7) |
| MMR (second dose) |
| Intervention |
198/231 (85.7) |
0.94 (0.88, 1.0) |
0.04 |
118/349 (33.8) |
1.45 (1.13, 1.85) |
0.003 |
| Comparison |
229/250 (91.6) |
76/326 (23.3) |
| CI, confidence interval; MMR, measles, mumps and rubella; RR, risk ratio. |
In contrast, the intervention group consistently achieved a 10–15% higher proportion of children initially identified as overdue by 11A reports being corrected to up to date for a particular vaccine course by June 2019 (Table 2). Between 31% and 49% of children identified as overdue by the AIR prior to the initial visit were actually up to date prior to the initial visit but had not had all the vaccines they had received recorded on the AIR.
Discussion
This study’s finding that a significant proportion of children reported as overdue according to 11A reports from the AIR are not truly overdue is consistent with reports from others.16–19 This varies to some degree with vaccines and scheduled age of administration (eg 59% of children reported as overdue for their primary DTPa course being truly overdue versus 71% for their second MMR vaccine). By June 2019, 48% of children originally reported as overdue by the AIR from the intervention group were identified as having had vaccinations that would have resulted in them being up to date prior to the practice visit. The comparable figure for the comparison group was 33%, or 15% lower, and this might suggest the intervention significantly improved retrospective reporting to the AIR. Despite this finding, the intervention did not result in higher vaccination rates among the truly overdue children.
Overall, a modest 37% of truly overdue children from the intervention group were recorded as up to date by June 2019, and this was lower than that achieved by the comparison group (44%). The interpretation of this finding relies heavily on the appropriateness of the comparison group. Because children in the comparison group were selected using a random sample drawn from practices without the greatest number of overdue children, it could be that the practices they attended had better follow-up procedures than the practices from which the intervention group was selected. Consequently, the intervention might have led to some children being vaccinated who would otherwise not have been, but not to the extent to be superior to the number achieved by the practices who contributed children to the comparison group. Nonetheless, it appears the intervention needs to be modified to achieve substantial increases in vaccination rates among those children truly overdue for routine childhood vaccinations.
Vaccination rates at the 12-month and two-year milestones are high,8 and it is likely that influencing parents of those children who remain not up to date will be challenging. The intervention used in this study was a combination of text, telephone and letter recalls depending on the practice’s preference. The study was not designed to compare these modalities, and it might be that the more resource-intensive strategies, such as using a combination of these approaches, are more effective.
The main strength of the present study is that it was conducted in a day-to-day general practice setting, mainly using existing general practice staff and resources. The advantage of this approach is that the findings are likely to be generalisable to other general practices.
However, there are other limitations of the study, in addition to the selection of the comparison group, with one of the major limitations being the completeness of the data sourced from the AIR. This is particularly important because register data were used to determine whether a child was up to date in 2019 and, although the intervention itself is likely to have improved reporting of vaccines administered to the intervention group, there is still the potential for administered vaccines not to have been reported. A further data-related issue, documented previously by others,10 is allowing for children who have moved overseas, and these data could not be readily obtained for the present study.
The follow-up period for the study was relatively short, ranging from 11 to 20 months, and children might still have been vaccinated after this period. It is difficult to determine how much of an issue this might have been, although given that the study looked at vaccines that should have been administered by six, 12 and 18 months, it would be reasonable to assume this period of follow-up would have been sufficient in most cases.
Conclusion
AIR data are increasingly being used to assess vaccination rates10 and assist public health practitioners to target interventions to improve these rates. The importance of having a universal register with high data quality and completeness is becoming increasingly important in settings where vaccination rates are high because the need for highly focused approaches to encourage the last few per cent of unvaccinated children to be immunised is vital. This study demonstrates that improvements in completeness of AIR data can be achieved but, at the same time, shows that general practice-based interventions to improve vaccination need to go beyond providing reminders to parents or guardians that their children are overdue.