Web Speech API

for Voice Data Entry

Cameron Jacoby
RailsConf | April 25, 2017

Hi, I'm Cameron

I build expert-use software for Stitch Fix
in Ruby on Rails.

Stitch Fix Overview

Lifecycle of an Item

What We'll Cover

  • Data Entry by Warehouse Associates
  • Getting Started with the Web Speech API
  • Voice Dictation Challenges
  • Is Voice the Right Solution for You?

Data Entry by Warehouse Associates

Points of Measure

Collecting Measurements

Collecting Measurements

Collecting Measurements

User Research

Measuring in Pairs

Voice Usability Study

Voice Usability Study

  • Would voice entry affect the overall time to measure a garment?
  • Would the noisy environment pose accuracy issues?
  • How would warehouse associates feel about voice entry?

Voice Usability Study - Results

Keyboard Voice
Participant 1 9:53 6:43
Participant 2 5:25 4:47

Times based on a men's long-sleeve woven shirt with 16 points of measure.
Averaged 9-10 minutes using Google sheets.

Voice Usability Study - Results

Investing in high-quality headsets mitigated accuracy issues from the noisy environment.

Voice Usability Study - Results

"Better for my back"

Voice Usability Study - Results

"I liked that I never had to remove my hands
from the tape"

Getting Started
with the Web Speech API

Voice Dictation Challenges

Contextual Formatting

results[0] results[1] results[2] results[3]
two two and a half two and a half ice creams two and a half ice creams
two two and a half 2 1/2 ---

Dictation Errors

Voice Input Result Expected
thirty-five eighths 35/8 30 5/8
four quarters 4/4 4 1/4

Reliability

Not 100% reliable. Isolated and tested:

✓ Headset

✓ Laptop internal mic

✓ Up-to-date browser

Measurements Out of Range

Is Voice the Right Solution for You?

Voice Considerations

  • Browser control
  • Structured data
  • Flexible user base
  • Fallback plan

Key Takeaways

UX and engineering collaboration allowed us to:

  • Empathetically build expert-use software
  • Quickly prototype, iterate, and solve problems

Acknowledgments

  • Sarah Poon, Senior UX Designer
  • Leah Becker, Associate Technical Designer
  • Allison Bohannon, Operations Manager
  • Christina Bonelli, Technical Design Manager
  • Kaylen Fink, Operations Manager
  • Bobby Lanchester, Director of Technical Design
  • Jonathan Wallace, Principal Engineer
  • Zhou Yu, Data Scientist

Thank You

Twitter: @cejtweets
Stitch Fix: multithreaded.stitchfix.com
Web Speech API: developer.mozilla.org/docs/Web/API/Web_Speech_API