Audio Text, Project

Project Description

Motivation

The desire is to be able to work with tightly integrated text and audio for applications such as:

• Real-time emergency call center, team response.
• "Living-text" notes of Audio transcripts (e.g. meetings, presentations).
• Real-time negotiation team support.
• Musical/Lyrical transcripting and browsing.

Text widgets are tightly integrated with audio stream through associating an audio position with each textual character.

A User Story

A 911 emergency call center operator recieves an urgent call; typing notes as the caller speaks. Other 911 call center agents are alerted to assist operator and are able to see operator's notes in real-time. The assisting agents click on specific notes of interest; reviewing audio, adjusting notes, and prompting operator.

Implementation

A proof-of-concept performed through developing real-time, two-way widgets for defining and using synchronized audio and text.

• Audio positions are associated with each textual character.
• Text widget modes: edit, playback
• Two-way text:
  • Click to change audio position.
  • Dynamicly highlight text upon playback.
• Two-way slider: actively set and playback audio position.

Notable technical challenges:

• Develop strict and minimal line-oriented data interfaces to assure synchonization between associated data (text and position).
• Normalize behavior of windows text edit widget (implementing the strict/minimal interfaces).
• Develop intuitive audio assocation with various text editing cases (e.g. editing multiple audio/text positions during paused audio).

Coding was done with Embarcadero, Delphi (Object Pascal).

Illustrations