Artificial intelligence (AI) scribes

What is an ‘artificial intelligence scribe’?

An artificial intelligence (AI) scribe is a tool that can automate parts of the clinical documentation process for a medical practitioner. AI scribes can convert a conversation with a patient into a clinical note, summary, or letter that can be incorporated into the patient’s health record.¹ AI scribes are also referred to as digital scribes, virtual scribes, ambient AI scribes, AI documentation assistants, and digital/virtual/smart clinical assistants.

An AI scribe cannot completely replace the work a general practitioner (GP) undertakes to prepare clinical documentation. The output of an AI scribe must be carefully checked for accuracy by a GP, as it can produce errors and inconsistencies. GPs are ultimately responsible for ensuring that the patient health record is accurate and up-to-date.

How do AI scribes work?

Rapid advancements in AI, automatic speech recognition (ASR), and natural language processing (NLP) over the last five years have seen AI scribes evolve from relatively simple speech-to-text services into sophisticated tools to assist with the preparation of clinical notes, discharge summaries, treatment summaries, and referral letters.

AI scribes use a microphone to capture speech that is taking place during a clinical encounter, then convert the audio data into text. The audio data should not be stored by the software vendor, and therefore it cannot be accessed by either the medical practitioner or the patient later. Using the text transcription and instructions received by the user, the AI scribe leverages the connections between the words and concepts on which it was trained to construct the clinical documentation for use in the patient’s health record.²

GPs should carefully review the output prepared by an AI scribe for false positives and negatives and edit the text as required (adding any missing information or omitting incorrect information). The GP can then add their own notes and observations, and in some cases attach documents, before signing off on the documentation.

What are the potential benefits of AI scribes?

AI scribes are often advertised as a solution to many issues inherent to the use of the electronic health record. In most instances, these claims are not yet backed by substantial research evidence. Some have suggested that using an AI scribe might:

• Reduce administrative task burden for GPs³
• Allow the GP to focus on the patient during the consultation, instead of a computer⁴
• Improve patient satisfaction⁵
• Reduce doctor burnout.⁶

What are potential problems associated with AI scribes?

This is an emerging field. There are a number of known issues with AI products that may affect AI scribes. Unforeseen legal problems might also arise as their use increases.

Clinical issues

• There is limited data on the clinical utility/validity⁷ and patient safety⁸ of AI scribes. Many large companies with AI scribes on the market have not published these data in academic journals.⁷
• In any patient consultation, there will be relevant clinical information that is not explicitly discussed, such as that contained within recent hospital discharge summaries, pathology/diagnostic imaging reports, and other elements of the electronic health record. Information from other sources, such as nonverbal cues from the patient or data from medical devices, is also not captured in the aural record. Without integrating this information, the clinical output generated by the AI scribe is limited in its quality and efficacy.³
• AI scribes can make errors that affect the meaning and accuracy of clinical information, such as:
  • Filtering out pertinent information, classifying it as irrelevant
  • ‘Mishearing’ the names of symptoms/medicines/conditions as a result of the speaker’s accent or use of slang terms
  • Incorrectly categorising data (eg, confusing historical and current symptoms).

These errors might be carried over into other areas of the patient health record or to other medical practitioners’ clinical information systems.⁹

• As noted above, GPs will be liable for errors within the patient health record even if they are generated by the AI scribe. GPs must ensure that the output prepared by the AI scribe constitutes an accurate record of a patient consultation. They must correct any errors and address omissions before signing off on the documentation and entering it into the patient health record.
• As these tools gain popularity and their use increases, there is potential for GPs to become over-reliant on their use and pay less attention to critical clinical details or forgo the vital process of checking the output generated by the AI scribe, resulting in errors that could affect patient safety.¹⁰
• The process of writing clinical documentation might itself be clarifying or elucidatory for the GP in developing a clinical formulation,¹¹ and the effects of removing this process are as yet unknown.

Privacy and security issues

• GPs must obtain consent from a patient to use an AI scribe in the consultation, and some medical defence organisations (MDOs) require that GPs obtain the written consent of the patient.¹² Recording a private conversation without consent is a criminal offence in some Australian jurisdictions. GPs using an AI scribe must abide by state and territory laws regarding the use of surveillance devices.¹²
• A data breach might occur if audio recordings, text transcripts, or clinical documentation prepared by the AI scribe are intercepted or otherwise compromised. The software vendor offering the AI scribe should provide assurances about how the data is encrypted, stored, and destroyed. General practices should also have their own policies and procedures for managing information security to help prevent data breaches, such as enabling multifactor authentication on AI scribes and other applications.
• Software vendors might use collected data for secondary purposes, such as for training AI models in order to improve the output of the digital scribe or to develop other AI products. They might also on-sell data to a third party (and have an obligation to advise the user of this). GPs considering purchasing an AI scribe should carefully review the terms and conditions of the user agreement to determine whether collected data will be used for secondary purposes by the vendor or a third party. GPs should consider whether any secondary use of data specified in the user agreement is appropriate and acceptable before purchasing the product.
• GPs should check whether data collected by the AI scribe will be processed or stored outside of Australia. If data is processed or stored for any amount of time overseas, vendors must ensure that the country where it is stored has similar privacy standards and protections as that of Australia¹³ and should provide assurances to potential users that their product meets their Australian legal obligations in this regard. If the vendor cannot provide such assurances, their products should be avoided.
• AI scribes might be subject to regulatory assessment before and after being brought to market in Australia and GPs are encouraged to investigate if the AI tools they wish to use function appropriately to meet their own needs and to support patient care.

Workflow and practice issues

• Although the GP might spend less time typing during the patient consultation when they are using an AI scribe, the process of checking the output and integrating other relevant information into the clinical documentation will also require time. These new workflows might decrease or negate the anticipated time-saving benefit of the AI scribe.^{11, 15}
• GPs might require training in the use of an AI scribe prior to their use which is an investment of time and money for general practices, and potentially unremunerated time out of practice for GPs.³

What do GPs need to consider in deciding whether to use an AI scribe?

The potential benefits of using an AI scribe must be weighed against the risks described in this document.

If you are a practice owner, it would be prudent to develop a policy on whether AI scribes can be used by GPs in the practice and procedures that outline how they should be used. If you are an employee or independent GP working within a general practice, you should consult your contract regarding the appropriate use of AI tools as well as the practice’s policies and procedures.

To decide whether an AI scribe is right for your practice, consider the following:

1. Are you satisfied you have been provided with enough evidence of the product’s safety, efficacy, and applicability for use in an Australian general practice context? For example:
   1. Is there published data on the clinical utility/validity/safety of the AI scribe?
   2. Have GPs been involved in the development of the AI scribe to ensure it is fit-for-purpose?
   3. Can the software vendor offering the AI scribe provide assurances that the tool is compliant with relevant Australian legislation in how it manages and stores data?
   4. Is the data collected by the AI scribe processed and stored in Australia or overseas?
   5. Can data collected by the AI scribe be used (and/or on-sold) for secondary purposes under the terms and conditions determined by the software vendor?
2. Is the AI scribe easy to use?
3. How will I adjust clinical workflows to account for the time needed to check the clinical documentation generated by the AI scribe, remove inaccurate or irrelevant information, and add other clinically relevant information?
4. Have I informed patients about the use of the AI scribe at the practice, for example through the use of signage, patient enrolment forms, and/or at the beginning of the consult?
5. How will a patient’s consent to the use of the AI scribe be documented in the record?

It is also prudent to consult with your MDO before purchasing and using an AI scribe.

Are there particular AI scribes that are recommended by the RACGP?

The RACGP recommends that GPs proceed with caution and apply their own judgement in deciding to use an AI scribe, or indeed any AI tool, in their practice. The RACGP does not recommend particular commercial products.

Relevant RACGP resources

• Position statement: Artificial intelligence (AI) in primary care
• Privacy and managing health information in general practice
• Information security in general practice
• Three key principles for the secondary use of general practice data by third parties