Why isn't speech recognition software more accurate? originall...
Why isn’t speech recognition software more accurate? This is an excellent question to start off an automatic speech recognition (ASR) interview. I would slightly rephrase the question as “Why is speech recognition hard?”
The reasons are plenty and here is my take on the topic:
An ASR is just like any other machine learning (ML) problem, where the objective is to classify a sound wave into one of the basic units of speech (also called a “class” in ML terminology), such as a word. The problem with human speech is the huge amount of variation that occurs while pronouncing a word. For example, below are two recordings of the word “Yes” spoken by the same person (wave source: AN4 dataset [1]). It can easily be seen that the signals differ and the same can be verified by analyzing it in frequency or time-frequency domain. Comparison of two different recording of the word “Yes” in the time domain.
There are several reasons for this variation, namely stress on the vocal chords, environmental conditions, and microphone conditions, to mention a few. To capture this variation, ML algorithms such as the hidden Markov model (HMM)[2] along with Gaussian mixture models are used. More recently, deep neural networks (DNN) have been shown to perform better.
One way to do ASR is to train ML models for each word. During the training phase, the speech signal is broken down into a set of features (such as Mel frequency cepstral coefficients, or MFCC for short) which are then used to build the model. These models are called acoustic models (AM). When a speech signal has to be “recognized” (testing phase), features are again extracted, and are compared against each word model. The signal is assigned to represent the word, which has the highest probability value. This way of doing ASR works pretty well for small vocabularies. When the number of words increases, we end up comparing with a very large set of models, which is computationally not feasible. There is another problem of finding enough data to train these models. The word model fails for large vocabulary continuous speech recognition tasks due to the high complexity involved in decoding as well the need for the high amounts of training data.
To overcome this problem, we divide words into smaller units called phones. In the English language (and many Indian languages), there are approximately fifty phones that can be combined to make up any word. For example the word “Hello” can be broken in to “HH, AH, L, OW”. You can look up the CMU pronunciation dictionary [6] for phonetic expansion of English words.
