Computerised screening can increase detection rates of health risk behaviours in primary care7 and mobile phones (commonly used in Australia) can facilitate this approach within daily life. We developed a mobile phone mental health assessment and management tool, the Mobile Tracking Young People’s Experiences (mobiletype) program,8,9 with the aim of assisting clinicians in detecting, assessing and managing youth mental health problems. This study aimed to evaluate the utility, usability and feasibility of the mobiletype program in clinical settings with adolescents and adolescent paediatricians.
Methods
Participants
Patients were recruited by six participating paediatricians from the general adolescent outpatient health clinic of the Royal Children’s Hospital (RCH) in Melbourne, Victoria. All participating paediatricians had a special interest and/or training in adolescent health, and screening for and detecting mental health issues was a routine part of their workload. However the clinic was not a specific mental health clinic and patients presented with a range of medical disorders. Patients were referred to the clinic by GPs or other specialists. Patients were 14–24 years of age, fluent in English, and provided informed consent (participants aged 14–17 years were deemed mature minors).
This study had Royal Children’s Hospital Human Research Ethics Committee approval.
Procedure and quantitative measures
Participating paediatricians were given a study reference manual, completed 30 minutes training in the mobiletype procedure and had access to the mobiletype website. The website included adolescent friendly clinical supports, including referral details of allied health professionals and services; internet, email and phone support services; and psycho-education handouts on a range of mental health problems. Participating paediatricians briefly outlined the study to their patients, and referred interested patients to the research assistant. Participants gave consent, completed a baseline assessment consisting of demographic information and the Depression, Anxiety, Stress Scale (DASS10), and were instructed in how to use the mobiletype program.
Participants were lent a ZTE F851 JAVA MIDP 2.0 mobile phone with the mobiletype application and $50 phone credit. Between the initial and follow up medical review (2–4 weeks) a reminder SMS asked participants to complete a mobiletype entry randomly during four time intervals (8:00–10:59 hours, 11:00–15:29 hours, 15:30–19:59 hours, 20:00–23:59 hours). Participants were asked to complete at least one entry per day and were given a card for school stating that they were participating in a Royal Children’s Hospital research study, requiring the use of the mobile phone. The researcher’s contact details were also provided for authentication. Information regarding the development and testing of the mobiletype program has been previously published.9 Each mobiletype entry assessed current location, activity, companions, mood, recent stressful events and responses to them. Once daily, the mobiletype program assessed alcohol and cannabis use, eating, exercise and sleeping patterns.
Encrypted monitoring data was sent to the mobiletype server via SMS, collated, and processed into individualised reports written by the first author including graphs of daily mood and tables containing recent stressful events, coping strategies, alcohol and cannabis use, daily activities, exercise, sleep and eating patterns. These areas of assessment and feedback were chosen based upon earlier study development,9 consultations with adolescent physicians, and adolescent screening research.11 At the 2–4 week follow up medical review, participants reviewed the summary report with their doctor via a secure intranet site or printed copy. Figure 1 displays an example summary report.
After this review, participants completed the DASS10 (referencing the monitoring period) and a feedback questionnaire regarding their experience using the mobiletype program. This included closed ended questions and forced choice scales to assess:
- truthfulness in answering the mobiletype entries
- the experience of daily monitoring
- whether the mobiletype program adequately captured their mood, stress, and coping during the day
- the experience of reviewing the monitoring information with their doctor, in particular, how they felt, the accuracy and usefulness of the feedback, and whether the feedback resulted in their doctor understanding them better.
Paediatricians completed a feedback form for each participant that assessed, via closed ended questions:
- how well they thought the diary captured the patient’s moods, stresses, coping strategies, daily activities, exercise, eating patterns, and alcohol and cannabis use
- how helpful this information was
- to what extent the data assisted them in understanding their patient’s current functioning.
Figure 1. An example of the summary report page of the mobiletype website
Qualitative interviews
We conducted a structured interview with each paediatrician (n=6) to understand their overall experience using the mobiletype program. The interview focused on the utility and usability of the program and the way in which the data from the mobiletype program contributed to detection and management of mental health problems.
Analyses
We conducted a formative evaluation12 drawing together the quantitative data from participants and paediatricians regarding their experience using the mobiletype program and the qualitative data from the structured interviews. The qualitative data were analysed by thematic analysis13 with the aim of analysing patterns of usability and utility of the mobiletype program and detection and management of mental health problems to identify explicit patterns in the dataset. Recurrent themes throughout the transcripts were identified and recorded.
Results
Participants included 47 adolescents (40 female) aged 14–19 years (M=15.59, SD=1.12). Unfortunately, no record was kept of adolescents who declined to participate, hence a formal response rate is not available.
Thirty-seven (79%) participants returned for their medical review, 32 (86%) reviewed the data with their doctor, and 35 (95%) completed feedback forms. Paediatricians completed one feedback report per participant who they reviewed the data with (ie. 32 reports were completed). Participants monitored their mental health symptoms with the mobiletype program for an average of 22.34 days (SD=8.51) (n=46). On average, participants completed 91% of the mobiletype entries every day in week 1 (n=47), 71% in week 2 (n=46), 69% in week 3 (n=28) and 58% in week 4 (n=17).
Feedback review
The paediatricians reviewed the summary report, and the mood and stress and coping data most frequently (Table 1). Most paediatricians found the data to be somewhat to very helpful.
There were very few reports of alcohol or cannabis consumption, which may explain why the alcohol/ cannabis data was not rated as helpful.
Table 1. Patterns of use and valid ratings of helpfulness by paediatricians postreviewing data from the mobiletype program (n=32)
| Sections of report reviewed with each patient % (n) | How helpful was the information on the report? Valid % (n) |
Not at all helpful | Somewhat helpful | Very helpful |
Summary report |
97 (31) |
8 (2) |
62 (16) |
30 (8) |
Mood |
91 (29) |
13 (3) |
50 (12) |
37 (9) |
Stress and coping |
81 (26) |
19 (4) |
38 (8) |
43 (9) |
Exercise |
81 (25) |
25 (6) |
46 (10) |
29 (7) |
Eating |
84 (26) |
16 (4) |
60 (14) |
24 (6) |
Daily activities |
71 (22) |
21 (4) |
63 (11) |
37 (3) |
Alcohol and cannabis use |
59 (19) |
33 (6) |
39 (6) |
28 (5) |
Note: n refers to number of patients reported on |
Mobiletype experience
The paediatricians reported that in 89% of cases (31/35) the mobiletype program assisted them to better understand the patient’s functioning, and believed for 80% of the participants (28/35), the mobiletype program assisted their patients to understand themselves better. Overall, paediatricians reported that the patients were positive about the experience of sharing the information.
The majority of participants agreed the mobiletype program captured their current situation, feeling and thoughts well (Table 2).
Of the participants who reviewed their data, 19 out of 25 participants (76%) reported positive feelings (grateful, pleased, surprised or fine) about reviewing their monitoring information with their doctor. Four (16%) indicated negative feelings (embarrassed, annoyed, angry, scared or ashamed), two (8%) reported indifference and one (4%) reported not caring. Of respondents, 95% (21/22) reported the feedback information reflected their actual experiences, was accurate 95% (20/21), was helpful to them 71% (17/24), and aided their doctor to understood them better 82% (18/22). Analysis of paired DASS scores indicated no significant reduction in patients’ depression, anxiety and stress scores at follow up.
Table 2. Participants’ perceptions of the face validity of the mobiletype program (n=35)
| Poor to average % (n) | Good to excellent % (n) |
How well did the questions in the mobiletype program capture your: |
|
|
|
17 (6) |
83 (29) |
|
14 (5) |
86 (30) |
|
43 (15) |
43 (15) |
Qualitative interviews
Three main themes were identified from analysis of the structured interviews with the paediatricians.
The first theme was of saving time by using the summary data to capture information that was hard to gather from young people (eg. mood, stress, or drug and alcohol use). Comments such as it ‘shortened assessment time considerably’ or the program was ‘a great time saver, you could ask a kid five different questions in a consult to work out they had a bad day yesterday, but the report had it right there.’ [Paediatrician 1]
Although the drug data was not generally well utilised it was still valued: ‘it was great to have the drug and alcohol data already there; that can be hard to get.’ [Paediatrician 2]
The second theme was around focusing the consultation to particular areas of concern for each patient. At its broadest, the ‘mobiletype was a vehicle to talk about various issues.’ [Paediatrician 3]
Another paediatrician commented that ‘it allowed me to have more specific discussions about particular issues and instances, and I could pinpoint these to a day.’ [Paediatrician 1]
Furthermore ‘reviewing the data allowed the patient to objectify their experience and talk about it more easily without having to cover unnecessary ground as the data was all there.’ [Paediatrician 3]
The third theme was around establishing rapport. Using the program and feedback data was a useful way to communicate about difficult areas, especially with new patients:
‘It was good for establishing rapport, especially with new patients;’ [Paediatrician 4] it was ‘helpful with new patients who find it hard to talk;’ [Paediatrician 5] ‘a great thing to do for engagement;’ [Paediatrician 5] and ‘it worked for kids who struggle to tell you things in words, so they put it all into the phone and it was there for us to talk about.’ [Paediatrician 2]
Discussion
Our findings provide initial support for the feasibility and usability of mobiletype as a mental health assessment and monitoring tool for clinicians working with adolescents. The results suggest that adolescents will monitor their daily moods, stresses and other mental health factors with a mobile phone program in order to share this information with their doctor. Paediatricians were able to use the monitoring data in their clinical practice. The themes identified from qualitative interviews with paediatricians suggested that mobiletype saved time in capturing sensitive information about mental health. Both adolescents and paediatricians affirmed that mobiletype accurately captured adolescents’ experiences of mood, stresses and other mental health related symptoms. One participant reported: ‘I liked the information that I got at the end. I didn’t expect it to be as accurate as it was.’ Finally, both adolescents and paediatricians found that the program assisted the doctor to understand their patient better.
Not all sections of the reports were reviewed by the paediatricians and participants, and some sections were rated as more helpful by the paediatricians than others. In the structured interviews the paediatricians suggested that the sections rated as unhelpful were rated in that way because that area of functioning was not of interest or concern for a particular patient. This is likely to reflect the variety of young people and their health concerns. Although overall rated as accurate, ‘thoughts’ appeared to be far less accurate than other areas of reporting (43% in the poor to average category). Future amendments to the program should focus on improving the thoughts capture and reporting.
To the authors’ knowledge, this is the first study evaluating a self monitoring program in the healthcare setting and has some limitations. First, an adolescent clinic was targeted to recruit as many young people as possible in a short timeframe and may not be generalisable to primary care. Further studies will assess the usability of the mobiletype program in primary care. Second, the sample was small and heterogeneous in their presentation with an overrepresentation (85%) of female patients. Third, a specific response rate cannot be reported, nor can we provide details about differences between respondents and nonrespondents. Finally, although the qualitative interviews provided similar responses and the three key themes were common to all interviews, we cannot be sure that we reached thematic saturation and further exploration is important.
While the specialised nature of this setting limits the generalisability of our findings, the major findings – of saving clinicians’ time in performing risk assessments and providing detailed information for clinicians managing patients with emotional distress – suggest that the program could be of even greater assistance in primary care settings, where time is an issue and where adolescent health is not always the major focus.
These findings suggest that mobiletype can support time pressured clinicians in screening and managing adolescent mental health, by providing comprehensive mental health data gathered over time in a youth friendly, time saving manner, thereby increasing doctor familiarity with patients and their specific difficulties. The development and trial of the mobiletype program is timely, as new and innovative approaches are needed to support adolescent healthcare, with the integration of ‘e-health’ tools in primary healthcare a top priority.14
Implications for general practice
- Mobile phone applications have the potential to assist clinicians with assessing and detecting mental health problems in adolescence.
- Results from the present study found support for the usability and feasibility of employing a mobile phone application in clinical practice. For example, adolescent paediatricians reported that for 89% of cases, the mobiletype program assisted them to understand their patient’s functioning better and 82% of participants felt that their doctor understood them better due to the mobiletype program.
- Qualitative responses from involved clinicians indicate the program saved time, pinpointed problem areas and helped established rapport with adolescents.
Conflict of interest: none declared.
Acknowledgements
This study was supported by research grants from the beyondblue Victorian Centre of Excellence in Depression and Related Disorders and the Jack Brockhoff Foundation. Sophie Reid was supported by a Windermere Foundation Health Fellowship. The Telstra Foundation provided seeding funding to begin the development of the mobiletype program as a clinical tool to implement in this study. Study mobile phones were donated by ZTE and Telstra Corporation donated the start up SIM cards and provided telecommunications support. Alexander Crooke of Murdoch Childrens Research Institute provided guidance in writing the qualitative section of this paper. The study sponsors were not involved in the study design, collection, analysis and interpretation of data, the writing of the report, or the decision to submit the manuscript for publication.
References