AI-enabled Consumer-facing Symptom Checkers

August, 2022

Photo credit: https://www.georgeclinical.com/resources/research/digital-health-beneficial-clinical-trials

Overview

An AI-based symptom checker is a type of healthcare app that solicits symptom information from users and provides medical suggestions and possible diagnoses.

This project aims to understand the socio-technical implications of AI-enabled symptom checkers. This project followed the user-centered design principles in the product development cycle by utilizing mixed research methods, such as semi-structured interviews and an online experiment.

My role

Led and conducted the whole project in all research stages, such as research plan creation, data collection, participants recruitment, and data analysis.

Generative Study 1

Research question:

How do users perceive the effectiveness of chatbot-based symptom checker applications?


Time:

1 month

Methods:

(1) App feature analysis: to shed light on the functions and features of the chatbot-based symptom checker apps


(2)
App review analysis and semi-structured interviews: to acquire in-depth insights as to how users perceive chatbot-based symptom checker apps


Findings:

The existing symptom checker apps lack the functions to support the whole diagnostic process of an offline medical visit. Users perceive the current apps lack support for a comprehensive medical history, flexible symptom input, user-friendly conversation design (e.g.,comprehensible questions and human-like language style), and diverse diseases and user groups

Generative Study 2

Research question:

Based on the preliminary findings of study 1, we conducted more interviews focusing on "how do users perceive and understand the reliability (authority) of symptom checkers?"


Time:

1 month

Method:

30 Semi-structured interviews


Findings:

Users assess the authority of symptom checkers using various factors: including interaction design of symptom checkers (e.g., explanations might enhance users' trust in symptom checkers), associations with established medical authorities like hospitals, and comparisons with other health technologies

Evaluative Study 1


Research question:

Based on the qualitative findings, we noticed the importance of explanations, so we wondered "how could explanations be used to promote the diagnostic transparency of online symptom checkers?"


Time:

2 months


Method:

A lab-controlled experiment:

Designed a prototype with 3 different explanation styles based on our interview findings to explore what kind of and whether explanations can improve user experience

Findings:

Providing explanations can improve diagnostic quality, medical decision-making, user trust in diagnosis, diagnostic transparency, and health awareness

Evaluative Study 2

Research question:

Still based on qualitative findings, we noticed people desired for user-friendly conversation design of symptom checkers, so we proposed the question "How should we design the conversation of symptom checkers and how do different conversational styles influence user experience with symptom checkers?"


Time:

3 months

Method:


A lab-controlled experiment with follow-up interviews:


Designed and implemented a chatbot-based symptom checker with different conversational styles to explore how these styles affect user experience