language for our communication is essential – the more devastating it is when people lose their ability to speak through injury or illness. However, the modern technology and, above all, direct interfaces between brain and Computer to create new possibilities, the brain signals to read and interpret. This can also be used for the so-called Brain-to-Text systems. Because when we hear words or speak, produces this characteristic pattern of activity in the brain. Capable of learning computer systems can recognize these patterns and thus Signal-to-word mapping. In fact, researchers have succeeded to detect such systems in spoken syllables and words solely on the basis of the accompanying brain signals and partially understand spoken language convert. However, the vocabulary was limited to such Attempts, so far, mostly on less than 100 words, and the error rate when Recognizing was approximately 25 percent, is relatively high.
Two coupled networks as a “Translator”
Now, Joseph Makin of the University of California, San Francisco, and his colleagues have developed a System that achieved a significantly higher accuracy with relatively little Training. In their Experiment, four subjects took part, of which a network was implanted electrodes into the cerebral cortex. Initially, these electrodes were used to localize the foci of their epileptic seizures, but they offered Makin and his Team, the Chance of language-related brain signals in high-resolution to derive. The experiment began when the participants were shown simple English sentences, reading out loud, you on a Monitor. “The sentences were nine words in length, and resulted in a total of a vocabulary of 250 words,” say the researchers. In Parallel, they recorded the resulting brain signals.
This combination of brain signals and the associated acoustic speech recordings took Makin and his Team and then to train a System consisting of two adaptive neural networks. The first network, the so-called Encoder, is used as a kind of Filter that the recorded brain signals according to recurring Patterns, addiction Patterns, which could be related to the spoken words in the context. By repeated comparison with the records of this language System improved in the course of the training, his marksmanship. The second System, the decoder uses the data of its predecessor to generate the adjusted signals back to the words. “This neural network is trained at each step, either a suitable word or to but the stop signal for the end of the sentence,” explain Makin and his colleagues.
– word error rate of under five percent
The experiments showed that the coupled AI-systems is reached already after a few Training passes, a relatively high precision. “Even if a minimum of 15 reps of a set were for the Training is available, could the word error of 25 percent in rates to be lowered – this is the upper limit for an acceptable voice transcription,” say the researchers. The subjects had the single sets more frequently than 15 Times repeatedly, increased the accuracy significantly: The systems achieved a word error rate of only three percent. “Error rates of five percent is already deemed a professional level,” says Makin and his Team. In a supplemental Test, they found that the success in training of the AI could even be from one subjects to another. The Encoder network was trained on a patient, it fell to him then, and it is much easier to detect the characteristic brain signals of a second patient, the Training took a lot shorter. According to the researchers, the System could be optimized, therefore, to the extent that you train it before use on a patient in a kind of generalized intensive language model.
According to the view of Makin and his Team, such an AI could contribute-based decoder systems in the future, patients a Computer to translate language ability, play by their brain signals directly into language. As the researchers emphasize the features of your Experiment, although a significantly reduced vocabulary of only 250 words. But enough for the AI systems, even voice and Brain-wave-recordings of only 30 minutes in length. “Our results suggest that an increase in the amount of data on these 30 minutes would allow for an expansion of vocabulary and a greater flexibility of the sentence structure,” say the researchers. “In addition, a couple of hundred words for a patient who is otherwise unable to speak at all, could be very helpful.”
source: Joseph Makin (University of California, San Francisco) et al., Nature Neuroscience, doi: 10.1038/s41593-020-0608-8
*The contribution of “AI System makes the brain signals to language” is published by Wissenschaft.de. Contact with the executives here.
Wissenschaft.de
