×
The Training Management System (TMS) is undergoing scheduled maintenance on Tuesday 3 October 2023 from 8:00pm to 9:00pm AEDT. During this time, the application will be unavailable. We apologise for any inconvenience caused.

A toolkit for effective and secure use of mobile technology

Phase 2: Initial trial

Step 7. Establish an evaluation plan

mHealth in general practice
Download PDF

Step 7. Establish an evaluation plan

Why?

An evaluation plan allows your practice to assess the effectiveness and success of its mHealth strategy, and helps to decide whether the strategy should be continued and/or there are any changes that can be made to improve it. 

How?

You should document your evaluation plan as soon as the mHealth trial begins in order to effectively measure its impact. Considering questions such as, ‘How will I know the trial is on track to achieve success?’ or ‘What will success look like at the end of the trial?’ will help identify performance measures.

Assessing mHealth strategy for appropriateness, effectiveness and efficiency

Determine the key evaluation questions you need to answer in order to measure the success of the planning process, implementation process and outcomes:

Planning process:

  • What processes worked well?
  • Was there adequate time and resources for planning?

Strategy implementation:

  • What could have been done differently?
  • Did we adequately identify and manage risks?
  • Did we have enough time and resources?

Outcomes:

  • What has changed in terms of policy in our practice?
  • Has the increased use of mobile devices increased the adoption of  other mHealth technologies?
  • How has the relationship between team members and patients changed?

Decide what data you want to collect and for what time period it will be collected:

  • Will you collect qualitative data (views, opinions), quantitative data (number of interactions with patients in which a mobile device was used), or both?
  • What methods of collecting data (observation, surveys, questionnaires, listening to staff members’ views during meetings) will you use?
  • Which staff members will be responsible for data collection?
  • What timelines will be set for data collection?

Case study

Home-based telemonitoring services

The Home Telemonitoring (HTM) service provides remote monitoring for patients. Daily recording of observations aims to promote self-management and stabilisation of the individual’s chronic disease, as well as decreasing the incidence of hospitalisations. Patients with these chronic conditions are eligible for the service: chronic heart failure, chronic obstructive pulmonary disease (COPD) and diabetes (type 1 and type 2).

The HTM service includes:

  • installation of a telemonitoring station in a patient’s home for a period between one and six months
  • patient education in the use of the home telemonitor for daily recording of record temperature, blood pressure, heart rate, heart rhythm, weight, oxygen saturation, and blood glucose and respiratory function
  • online monitoring of daily observations (Monday to Friday) by a clinician
  • appropriate response to problems identified by the clinician.

Patient benefits include:

  • supporting the referring clinician’s current health management plan
  • helping to self-manage chronic disease more effectively and increase confidence
  • monitoring the effects of, and response to, specific therapy, and helping to stabilise medical conditions
  • reducing the frequency of hospitalisation and emergency department (ED) presentations.8

Initial trial checklist

This checklist will help you define details regarding users, devices and systems required for the successful rollout of your mHealth strategy. 

Develop a business plan that incorporates a cost–benefit analysis

  • Vision – define the role of mobile devices and how they will support the practice’s overall vision of increasing access to care and efficiency
  • Objectives – define success metrics and clarify the specifics of the vision. For example, your clinic may want to ensure that GPs and nurses can keep records of their outreach visits (ie ensure 100% of GPs and nurses using mobile devices for outreach visits can securely record consultation details, including photos of the progress of a patient’s condition)
  • Expenses and budget – clarify how expenses will be covered and describe funding sources
  • Ownership and future steps – clarify responsibilities and plan execution; outline who will be responsible for identifying requirements, purchasing devices, and implementation and training

Build operational processes

  • Determine how devices will be set up to access the necessary software for mobile use
  • Policy for lost or stolen devices
  • Communication protocols during a patient consultation (eg should staff members answer their phone while in a consultation?)
  • Overall communication processes (eg should staff members reply to messages from patients? What are the escalation rules?)

Trial participants

  • Select staff members across your practice to participate in the trial
  • Find a clinical leader to champion the efforts of establishing an mHealth strategy, as this will help overcome obstacles more easily

Establish an evaluation plan 

  • Provide training
  • Determine the best way to train staff members participating in the trial
  • Training modes include one-on-one, webinars and a ‘champion’ within the team who can help with training needs
  • Ensure support is readily available to answer questions that are likely to come up during the trial and troubleshoot any technical issues

Communicate consistent messages

  • Consider the key messages to be communicated to the team before implementing your strategy
    • What can actually change?
    • How will the practice team be involved in final decision-making?
    • What level of input is required from the team?
    • What are the benefits for the team?
This event attracts CPD points and can be self recorded

Did you know you can now log your CPD with a click of a button?

Create Quick log

Advertising