Australian Family Physician
Australian Family Physician


Volume 45, Issue 9, September 2016

Can we measure integration of healthcare from administrative and clinical records?

Ian McRae Paresh Dawda Michelle Banfield Anne Parkinson
Download article
Cite this article    BIBTEX    REFER    RIS

A major component of integrated care is shared information. Computer-based clinical and administrative systems, particularly in multidisciplinary environments, provide an opportunity to directly measure the degree of integration.
The objective of this article is to explore the viability of automated measurement of integration within a multidisciplinary healthcare centre.
Computerised clinical systems provide opportunities for exploring integration of care if they include a range of care providers and all relevant fields are always completed. The latter condition will always be difficult to achieve.
Extracted patient records can be used to understand integration to the degree that communication is recorded, but at significant expense to the practices and researchers. Automated systems are practical to the degree that clinicians complete all relevant identifying fields.

National chronic disease strategy.1 Considerable efforts are also being made to measure components of care integration.2,3 However, there are many variations in the definition of integration;4 the most common concepts or terms are case management, care coordination, collaborative care, or a combination of these.5 Singer et al argued that although integration and coordination are frequently considered synonymous, complete integration requires ‘patient-centeredness’, where patients are informed and involved in decision making.6

Addressing the full range of issues around integration requires a wide range of data. Most measures of integration are survey-based, although a small proportion use automated register data.3 There is also considerable information available in national clinical2 and practice clinical databases. The latter are examined in this study. While clinical data sets are not designed to address questions of integration, it may be possible to directly measure aspects of integration, particularly in relation to information sharing and continuity, if this information can be automatically analysed.7 Previous Australian studies of information continuity have concluded that the availability of information is necessary ‘but not sufficient to ensure continuity for the patient or coordination from a system perspective’.8,9

The purpose of this paper is to explore the degree to which administrative and clinical records can be used in an automated manner to assess:

  • whether the levels of communication necessary for integrated care are met
  • if it is possible to assess the levels of joint decision making required for integrated care.


The ethical aspects of this study were approved by the Australian National University’s Human Research Ethics Committee (Protocol number 2014/651).

This study was part of a pilot project exploring the nature of integrated primary healthcare within a clinic (Site 1) that provides multidisciplinary services at a single site using a shared electronic record system. Patient records were also extracted from a second site run by the same organisation (Site 2), with a more limited shared system. The clinic records manager from Site 2 selected one to two years of complete patient notes for four patients with chronic conditions. Records were carefully de-identified by practice staff, and analysed by the researchers to assess the information flow between general practitioners (GPs) and other providers. We also aimed to observe indications of joint decision making, noting that a lack of reporting on joint decision making does not mean this did not occur.

Next, we assessed the scope for automated data extraction. Information on the system at Site 1 was obtained by discussion with practice staff, observing use of the system and by interrogating records for 19 consenting patients.

Investigation of these two different formats of the data provided opportunities to explore differences in the ease of access and nature of the data stored in different ways.


Both clinics used the Best Practice clinical management system and PEN CS clinical audit system.

Patient notes from Site 2

Practice staff extracted notes for four patients with chronic conditions who needed support from healthcare providers in addition to their GP. These patient notes were provided as portable document formats (PDFs) and included consultation notes, correspondence and notes of telephone conversations. De-identification of the notes required considerable time and multiple passes by two staff members. Therefore, this methodology would not be practical for a larger scale study. A more practical solution would be to obtain patients’ permission for researchers to access identified notes.

Analyses of the notes were also extremely time consuming. While some electronic searching of text in PDF documents was possible, response letters, diagnostic imaging reports and pathology reports were scanned as image files, which could not be automatically searched. This would create significant time demands if the methodology were to be applied to a larger sample.

Analyses of the notes showed considerable communication but almost no reporting of joint decision making, except for reports on a small number of telephone conversations. Communications included requests and reports for diagnostic imaging and pathology, and requests for diagnostic services and treatment from specialists and allied health practitioners (AHPs). There were very few responses to referrals to AHPs, except for one psychologist with whom the practice had close links. Specialists mostly responded by letter to referrals for clinical services. When there was no response to a referral, it was not clear if this was failure of the health professional to report or if the patient did not attend, unless there was information recorded in subsequent GP consultations.

The online systems at Site 1

All healthcare providers at the multidisciplinary clinic used a shared system, which recorded appointments, consultation notes, correspondence, and telephone and other conversations that clinicians chose to document. AHPs and GPs could access notes from consultations with all providers for shared patients. This meant that while referrals were required for Medicare Benefits Schedule (MBS) and health insurance purposes, formal responses were less necessary.

Nineteen patients gave consent for researchers to access their records. The system did not readily enable access to consenting patients only; therefore, a practice nurse opened the files of consenting patients for the researchers. This was practical for a pilot study, but would be problematic for a larger project, emphasising the value of more automated systems.

The components of the database that were relevant to consideration of an automatic extraction of data on integration comprised two folders for each patient:

  • one containing separate files of notes for each consultation/telephone conversation for all providers
  • the other containing separate sub-folders for correspondence out and in.

Consultations were identified by provider name and date, but any information on the content of the consultation required reading the notes. Given that the clinical roles of providers within the clinic were known, the usage of different providers by patients with particular conditions could be measured directly from the system. Some information on informal contacts (eg telephone, ‘corridor’ meetings) was included with notes of a consultation or via the internal messaging system. However, practice staff noted that informal contacts were not reliably recorded; therefore, the system could not fully capture shared patient care. Correspondence was contained in two sub-folders:

  • ‘Out’ sub-folder – mainly referrals or requests for tests
  • ‘In’ sub-folder – comprising letters and reports to the GP.

The referrals ‘out’ were identified by the name and clinical role of the addressee, topic of referral and date. While name and date were always present, clinical role and topic were frequently omitted. For practitioners within the clinic, the clinical role could be inferred, enabling automatic assessment of whether patients were referred to providers treating the chronic conditions and patients’ attendance at these consultations. However, for analysis of external practitioners, assessment would require a comprehensive table of specialists and AHPs. Response to referrals also frequently omitted important identifying information, which makes tracing the complete episode of care challenging.

Automated exploration of the database could be used to identify patients with chronic conditions who are likely to benefit from integrated care. It could then, in principle, be used to identify referrals from the GP to other providers within the clinic classified by clinical role and whether the patient attended that provider. These data could also be supplemented by data on information flows from an in-house messaging system to gain further understanding of how communication is used to support integrated primary healthcare.

Observation of the 19 patients suggests information on the clinical role of external providers may be available only in relatively few cases. It is not clear whether a referral is to treat the chronic condition(s) of interest or an unrelated problem. It is possible to automatically identify responses from the external providers to GP referrals by matching names, but it would be difficult to interpret the nature of the communication without more sophisticated systems to match names to specialities.


Our examination of patient records in two well-structured clinical data systems suggests that these data could be used to develop an understanding of the nature of information continuity and its potential role in supporting coordination and integration of care. However, there are considerable risks and barriers.

With patient consent, access to full patient records would allow assessment of information continuity7 subject to data integrity. However, this does not inform researchers on how providers use the available information.8 The degree of shared decision making between healthcare professionals is unlikely to be fully assessable from patient records, as there is inconsistent reporting of informal communication, and is not available in automated assessments.

Automated assessment using aggregate counts of aspects of integration would be more efficient than manual methods once set-up costs are met and would not require patient approval. Some automated assessments of communication are viable with the Best Practice system for internal communication in a multidisciplinary clinic, although the degree to which providers draw upon information from other consultations is not known. The communication flows are, in principle, measurable for external communication but would only be readily analysed if clinical roles could be identified. The limited reporting of clinical roles observed in our pilot study suggests this would be a major barrier. While the capacity to assess the role of informal communication is limited, there are opportunities to use automated systems to investigate information continuity.

Integrated care is increasingly seen as essential to care delivery.10 One of the dimensions of integrated care is functional integration; a critical enabler for this is high-quality information systems.11 These information systems need to support the different levels of integration from linkage to coordination and full integration. The same modules should have the functionality to enable measurement systems to support integrated service delivery and measure integration. The measures examined in this pilot project would be of more value if all information were completed. However, as most systems are designed for clinical and administrative purposes, users complete what is necessary for the primary purpose and not always what is most useful to the researcher.

While information continuity is necessary for successful care coordination, which is itself necessary for integration of care, clinical systems can measure care coordination but cannot directly measure integration. Integrated care, with its many definitions, shares a common element alongside the patient perspective as the organising principle of service delivery.12 Integration is the combined set of methods, process and models that seek to bring about improved coordination of care. Information systems with the capacity for automated measurement offer an opportunity to measure some aspects of integration, although this capacity may be limited because the systems are mainly designed for clinical management rather than measurement of other dimensions of practice.


This pilot project examined two practices and a relatively small number of patients. However, the purpose was to explore potential processes and not to provide actual outcomes in relation to the success of integration. This study was limited to one clinical management system and other systems may be more or less easily interrogated, so care must be taken in extrapolating these findings, although they provide strong indications of the opportunities and problems that may be found elsewhere. Selection of patients may not be representative for the purposes of exploring the nature of the data stored and how it can be accessed, but this was unlikely to have caused biases. Findings were consistent with previous work on coordination in primary healthcare, which also found that systems are sensitive to the way in which information is entered by providers and office staff. Data entry requires considerable training and maintenance of accurate entry of the relevant information.8 The project did not attempt to prepare tools to extract particular data sets, but rather took advice on what was practical.

Implications for general practice

While general practices have a subjective view of the degree of information continuity and care coordination in their practices, information continuity is rarely measured. There are opportunities for automated measurement subject to how well the data are entered. This information can then inform practices on how they are meeting their objectives with respect to care coordination and integration.


Ian McRae BSc, MSc, BA, PhD, Visiting Fellow, Research School of Population Health, Australian National University, Canberra, ACT. ian.s.mcrae@anu.edu.au

Paresh Dawda MBBS, DRCOG, DFRSH, FRACGP, FRCGP(UK), Regional Medical Director, Ochre Health, Calwell, ACT; Visiting Fellow, Research School of Population Health, Australian National University, Canberra, ACT; Adjunct Associate Professor, University of Canberra, Bruce, ACT

Michelle Banfield BSc, BA (Hons), PhD, Research Fellow, Centre for Mental Health Research, Research School of Population Health, Australian National University, Canberra, ACT

Anne Parkinson BA (Hons), PhD, Research Fellow, Department of Health Services Research and Policy, Research School of Population Health, Australian National University, Canberra, ACT

Competing interests: None

Provenance and peer review: Not commissioned, externally peer reviewed.


This research is a project of the Australian Primary Health Care Research Institute, which is supported by a grant from the Australian Government Department of Health. The information and opinions contained in it do not necessarily reflect the views or policy of the Australian Primary Health Care Research Institute or the Australian Government Department of Health.


  1. National Health Priority Action Council. National chronic disease strategy. Canberra: Department of Health and Ageing, 2006. Search PubMed
  2. Chen LM, Ayanian JZ. Care continuity and care coordination: What counts? JAMA Intern Med 2014;174(5):749–50. Search PubMed
  3. Strandberg-Larsen M. Measuring integrated care. Dan Med Bull 2011;58(2):B4245. Search PubMed
  4. Armitage GD, Suter E, Oelke ND, Adair CE. Health systems integration: State of the evidence. Int J Integr Care 2009;9:e82. Search PubMed
  5. Nolte E, Pitchforth E. What is the evidence on the economic impacts of integrated care? Policy summary 11. Copenhagen: World Health Organization, 2014. Search PubMed
  6. Singer SJ, Burgers J, Friedberg M, Rosenthal MB, Leape L, Schneider E. Defining and measuring integrated patient care: Promoting the next frontier in health care delivery. Med Care Res Rev 2011;68(1):112–27. Search PubMed
  7. Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: A multidisciplinary review. BMJ 2003;327(7425):1219–21. Search PubMed
  8. Banfield M, Gardner K, McRae I, Gillespie J, Wells R, Yen L. Unlocking information for coordination of care in Australia: A qualitative study of information continuity in four primary health care models. BMC Fam Pract 2013;14:34. Search PubMed
  9. Gardner K, Banfield M, McRae I, Gillespie J, Yen L. Improving coordination through information continuity: A framework for translational research. BMC Health Serv Res 2014;14:590. Search PubMed
  10. Kasteridis P, Street A, Dolman M, et al. Who would most benefit from improved integrated care? Implementing an analytical strategy in South Somerset. Int J Integr Care 2015;15:e001. Search PubMed
  11. Feachem RG, Sekhri NK, White KL. Getting more for their dollar: A comparison of the NHS with California’s Kaiser Permanente. BMJ 2002;324(7330):135–41. Search PubMed
  12. Shaw S, Rosen R, Rumbold R. What is integrated care? London: Nuffield Trust, 2011. Search PubMed
Download article PDF

Issues by year


Australian Family Physician RACGP

Printed from Australian Family Physician - https://www.racgp.org.au/afp/2016/september/can-we-measure-integration-of-healthcare-from-admi
© The Australian College of General Practitioners www.racgp.org.au