Evaluation of an AI-Based Voice Biomarker Tool to Detect Signals Consistent With Moderate to Severe Depression

Alexa Mazur; Harrison Costantino; Prentice Tom; Michael P. Wilson; Ronald G. Thompson

doi:10.1370/afm.240091

Article Figures & Data

Figures

Tables
Additional Files

Download figure
Open in new tab
Download powerpoint
Figure 1.
Participant Exclusion and Audio Preprocessing Criteria Used to Create Training and Validation Sets to Train and Tune the Model and Evaluate its Performance
Note: Eligible participants for inclusion in the analysis data sets were adults aged ≥18 years living in the United States or Canada who provided a voice sample in English containing at least 25 seconds of speech content meeting audio quality parameters. The training set and validation set were split to evenly distribute samples on the basis of participant characteristics and audio length.

View popup

Table 1.

Participant Demographic Characteristics

Characteristic	Training	Validation
Age, y
Average (SD)	37.3 (14.3)	37.3 (14.2)
Median	34.0	34.0
Mode	22.0	25.0
Range	18-93	18-86
Gender, %
Female	69.5	69.4
Male	27.3	27.9
Not specified	2.4	2.1
Other	0.9	0.7
Race/ethnicity, %
Asian or Pacific Islander	15.9	16.2
Black or African American	9.4	9.4
Hispanic or Latine	7.5	7.7
Native American or American Indian	1.2	1.3
Not specified	1.5	1.8
Other or mixed race	5.9	5.8
White	58.5	57.8
Audio duration, s
Average (SD)	55.1 (10.1)	55.0 (10.1)
Median	57.9	57.9
Mode	58.6	58.5
Range	25.0-74.9	25.0-74.9
PHQ-9 score
Average (SD)	9.8 (6.7)	9.7 (6.7)
Median	9.0	9.0
Mode	9.0	0
Range	0-27	0-27

PHQ-9 = Patient Health Questionnaire-9

View popup
Table 2.
Model Performance
Metric Value (95% CI)
Sensitivity 71.3 (69.0-73.5)
Specificity 73.5 (71.5-75.5)
PPV 69.3 (67.1-71.5)
NPV 75.3 (73.3-77.2)
NPV = negative predictive value; PPV = positive predictive value.

View popup

Table 3.

Subpopulation Performance

Metric	Value (95% CI)
Sensitivity
All	71.3 (69.0-73.5)
Gender
Female	74.0 (71.4-76.5)
Male	59.3 (54.0-64.4)
Age, y
<60	71.9 (69.5-74.2)
≥60	63.4 (54.3-71.9)
Race/ethnicity
Asian or Pacific Islander	67.4 (60.7-73.7)
Black or African American	72.4 (64.0-79.8)
Hispanic or Latine	80.3 (72.6-86.6)
White	70.7 (67.7-73.5)
Specificity
All	73.5 (71.5-75.5)
Gender
Female	68.9 (66.2-71.4)
Male	83.9 (80.8-86.7)
Age, y
<60	71.8 (69.6-73.9)
≥60	86.8 (81.6-91.0)
Race/ethnicity
Asian or Pacific Islander	77.5 (72.8-81.8)
Black or African American	75.9 (69.3-81.7)
Hispanic or Latine	68.6 (60.1-76.3)
White	72.8 (70.0-75.4)

Additional Files

Figures
Tables

PLAIN-LANGUAGE SUMMARY

Original Research

AI-Based Voice Biomarker Tool Shows Promise in Detecting Moderate to Severe Depression

Background and Goal:Depression is a leading cause of disability, impacting an estimated 18 million Americans each year, with a lifetime prevalence of major depression approaching 30%. Despite recommendations for universal screening, depression screening rarely occurs in the outpatient setting with some estimates placing screening rates at less than 4% of primary care encounters. This study evaluated an AI-based machine learning biomarker tool that uses speech patterns to detect moderate to severe depression, aiming to improve access to screening in primary care settings.
Study Approach: The study analyzed over 14,000 voice samples from U.S. and Canadian adults. Participants answered the question, “How was your day?” with at least 25 seconds of free-form speech. The tool analyzed vocal biomarkers associated with depression, including speech cadence, hesitations, pauses, and other acoustic features. These were compared to results from the Patient Health Questionnaire-9 (PHQ-9), a standard depression screening tool. A PHQ-9 score of 10 or higher indicated moderate to severe depression. The AI tool provided three outputs: Signs of Depression Detected, Signs of Depression Not Detected, and Further Evaluation Recommended (for uncertain cases).
Main Results:The dataset used to train the AI model consisted of 10,442 samples, while an additional 4,456 samples were used in a validation set to assess its accuracy.

The tool demonstrated a sensitivity of 71%, meaning it correctly identified depression in 71% of people who had it.

Specificity was 74%, indicating that the tool correctly ruled out depression in 74% of people who did not have it.

In about 20% of cases, the tool flagged results as uncertain, recommending further evaluation by a clinician.

Why It Matters: While not a replacement for formal clinical interviews or assessments by qualified clinicians, the study findings suggest that machine learning technology could serve as a complementary decision-support tool. These findings are preliminary, and more work is needed to validate the tool and explore its integration into primary care workflows. This study represents a promising avenue for using physiologic voice biomarkers to assist clinicians in identifying and addressing depression, with future research needed to refine the technology and assess its broader applicability.

Evaluation of an AI-Based Voice Biomarker Tool to Detect Signals Consistent With Moderate to Severe Depression

Alexa Mazur, BA, et al

Kintsugi Mindful Wellness, Inc, San Francisco, California

Visual Abstract:
SUPPLEMENTAL MATERIALS IN PDF FILE BELOW

Mazur_Supplemental_Materials.pdf -
PDF FILE
VISUAL ABSTRACT IN PDF FILE BELOW

Mazur_Visual_Abstract.pdf -
PDF file