Iterative Usability Testing: Patient-Provider Chat

Background

MyChart Bedside, an application that allows patients to access their hospital stay information, has a messaging feature that patients use to communicate with their care teams. Providers view and respond to patient messages in a Bedside Chat sidebar in their Epic Electronic Healthcare System.

Problem

Providers found it difficult to access Bedside Chat – a tool for chatting with patients -- especially when using a new dashboard was implemented. This dashboard meant that providers no longer frequented the page from which they would see and respond to patient messages.

Bedside decided to leverage an already existing chat feature that providers used – Secure Chat – to make it easier for providers to find, access, and respond to patient messages. However, organizations and providers alike feared that providers would accidentally send patients messages that they had intended to send to other providers. In a worst-case-scenario this could result in costly lawsuits.

We needed to make it easy for providers to understand, at all times, where their message was going. We also had to safeguard patient confidentiality and ensure that providers were aware when patient proxies, such as parents or caretakers, would receive messages. At the same time, we wanted the chat feature to be effective, efficient, and a joy to use.


Research Approach

Usability Study + Interview #1

● Workflows in which participants were asked to identify conversations including patients, patients in messages, clinicians in messages
● Interview questions about expected behavior, such as what should happen to conversations when patients are discharged and if patients should see read receipts
● Conducted during the design phase
● High-fidelity mockups
● 22 participants
● Completion rates, error types and amount, ease of use analyzed
Usability Study #2

● Workflows in which participants were asked to start or respond to conversations from patients or providers, determine participants and add participants to conversations.
● Live development tested
● 14 participants
● Completion rates, error types and amount, ease of use, satisfaction (with SUS) analyzed
Usability Study #3

● Workflows in which participants were asked to start or respond to conversations from patients or providers, determine participants and add participants to conversations.
● Live development tested
● Completion rates, error types and amount, ease of use, satisfaction (with SUS) analyzed


Findings and Solutions

Usability Test + Interview #1
22% of participants failed to distinguish conversation with patients in the conversation list. At the same time, the yellow color and multitude of indicators within conversations proved unappealing to users. Participants were also concerned about showing patients read receipts even stating that they would be dis-incentivized from viewing messages if patients could see that they viewed it.

Participants were divided about what should occur to messages after discharge. Some believed they should be purged, other that they would be saved in the patient chart. At the same time, organizations had to decide whether these messages constituted part of the patient’s Legal Medical Record.

After this study, the conversation list tabs were reorganized. Instead of having 3 tabs – one with patient and provider conversations, one with only provider conversations, and the last with only patient conversations – we removed the combined view tab with patient and provider conversations. Instead we would have one tab with only patient messages and another with only provider messages.

The yellow coding of patient messages and conversations were pared back. We decided to rely on other indicators, such as patient chips in conversation lists, headers, and participant lists. The color would now match the page theming.

Based on conversations with users, we decided not to show patients read receipts and to remove messages from the chat when patients are discharged. We would, however, show message content in the patient chart using print groups which could be configured by organizations.




Usability Study #2

After switching to 2 distinct tabs which separated patient and provider conversions, 100% of users were able to differentiate patient and provider conversations in the conversation list.

Issues were encountered, however, when participants tried to determine participants in a conversation and to add participants to the conversation. These tasks could be accomplished from a conversation details pane on the right-hand side of the screen. In the pane, a patient-summary tab was shown by default, to respect the default in provider messages. Participants had to click another tab to see conversation details. 38% of users failed to add a new participant to the list and, from those who completed the task, 62% experienced difficulty finding conversation details.

Additionally, 15% of users failed to send a new provider message and, of those who completed the task, 55% struggled to complete it. We realized that part of the problem here was due to hierarchy. Providers who used the chat feature to talk to other staff could set availability, forward messages, and create new provider messages. These buttons displayed above the participant tabs in the conversation list, however, they did not apply to patient conversations. Providers could not forward patient messages or initiate conversations with patients, and availability settings would not appear to patients.

To surface conversation details, we made the conversation details tab in the right-hand pane the default displayed pane. We also changed the conversation list hierarchy, moving the patient and provider tabs up. Settings and actions only applicable to provider conversations would now fall under the provider tab. On the patient tab, we also included a dismissible banner which stated that creating new messages and certain settings only applied to provider messages.

Usability Study #3

This study validated our designs and development showing that participants could easily distinguish between patient and provider conversations, determine and add participants to conversations, and complete major workflows such as creating new or replying to messages.


Outcomes

The functionality to communicate with patients through Secure Chat is slated for a piloted release in November 2021.