How AI Speech to Text Works

1. Acoustic Signal Processing: From Sound Pressure to Spectrograms

Human speech produces vibrational air pressure waves captured by microphone diaphragms as continuous analog signals. To digitize this sound for neural computation, the browser samples the audio stream at a standard rate of 16,000 to 44,100 samples per second (16kHz – 44.1kHz).

Because raw time-domain waveforms convey amplitude fluctuations rather than distinct frequency pitches, the audio is sliced into overlapping 20-to-25 millisecond time windows. A Short-Time Fourier Transform (STFT) is computed for each frame to calculate its frequency spectrum. These frequencies are subsequently warped onto the Mel Scale—a nonlinear scale that mimics the human inner ear's cochlear sensitivity to pitch.

2. Neural Feature Extraction: Convolutional Encoders

The resulting 80-channel log-mel spectrogram serves as an "image" of the speech signal. The first stage of modern neural ASR models (such as the OpenAI Whisper and Conformer architectures) passes this spectrogram through 1D and 2D convolutional neural network (CNN) layers.

These convolutions downsample temporal redundancy while extracting local acoustic invariance—recognizing voice phonemes regardless of pitch shifts, slight microphone distortions, or background hum.

3. Contextual Decoding: Multi-Head Self-Attention Transformers

Unlike older Recurrent Neural Networks (RNNs) or LSTMs that suffered from vanishing gradients across long audio clips, modern Transformer encoders apply Multi-Head Self-Attention. Every audio frame simultaneously attends to every other frame in a 30-second context window.

This enables the model to:

• Disambiguate Homophones: Differentiate "write", "right", and "rite" by evaluating surrounding sentence semantics.
• Filter Background Noise: Dynamically suppress air conditioning hum, typing clicks, and cafe chatter.
• Handle Varied Dialects: Recognize distinct regional vowel shifts without losing phonetic alignment.

4. Language Modeling and Auto-Punctuation

The autoregressive decoder outputs a sequence of Byte-Pair Encoded (BPE) subword tokens. An integrated language model predicts token probability distributions, ensuring grammatically coherent output. Finally, punctuation prediction heads insert periods, commas, and question marks based on acoustic cadence pauses.

5. Why Client-Side Browser Execution Matters

Traditional speech recognition required streaming raw audio over the internet to remote corporate servers, creating three significant drawbacks:

1. Privacy Vulnerabilities: Confidential corporate meetings or medical notes were stored on third-party cloud databases.
2. Network Latency: Round-trip packet latency caused perceptible lag in voice typing.
3. Usage Costs: Cloud APIs charge per audio minute, forcing developers to implement restrictive usage quotas.

Our tool leverages native Web Speech API and Web Audio engines compiled to run locally inside your browser sandbox. Your audio stays 100% on your machine with zero latency and absolute privacy.