创新背景

每年，成千上万的人因中风、意外事故或疾病而丧失说话能力。这项研究成果是与临床研究试验的第一位参与者合作开发的。这项研究建立在加州大学旧金山分校神经外科医生爱德华·张(EdwardChang)十多年的努力基础上，开发出一种技术，让瘫痪患者即使无法自己说话，也能进行交流。

创新过程

美国加州大学旧金山分校的研究人员在《新英格兰医学杂志》上发表论文称，他们开发出了一种“语言神经假肢”，可以将控制声道的大脑信号直接转换成屏幕上的文字，从而帮助因严重瘫痪而失语的病人用句子进行交流。
这是首次从失语患者的大脑活动中直接解码出完整的单词，显示了利用大脑的自然语言机制来恢复交流的强大前景。新研究利用了语言自然流畅的特性，直接翻译控制发声系统肌肉的信号，保证了更快速、省力的交流。Chang说：“我们能够以每分钟150~200个单词的速度交流信息，而打字、书写和控制光标的拼写方式不仅慢得多，也更加费力。”

将脑信号转化为言语

以前，神经修复术领域的工作重点是通过基于拼写的方法，逐个打印文字来恢复沟通。张的研究在一个关键的方面与这些努力不同：他的团队正在转换信号，以控制说话系统的肌肉，而不是移动手臂或手的信号，以便打字。张说，这种方法深入到自然和流畅的说话方面，并承诺更快速和有机的沟通。
在UCSF癫痫中心接受神经外科手术的患者们通过在大脑表面放置电极阵列来确定癫痫发作的起源，促进了张的这一目标的进展。这些病人都有正常的语言能力，他们自愿接受大脑记录的语音相关活动的分析。这些病人志愿者的早期成功为目前瘫痪患者的试验铺平了道路。
在此之前，Chang和他在UCSFWeill神经科学研究所的同事绘制了与产生每个辅音和元音的声道运动相关的皮层活动模式。为了将这些发现转化为全词的语音识别，张实验室的博士后工程师、新研究的主要作者之一大卫·摩西博士，开发了对这些模式和统计语言模型进行实时解码的新方法，以提高准确性。
Chang分析了具有正常语言能力的志愿者大脑中与语言相关的活动。随后，研究人员绘制了与声道运动相关的皮质活动模式。为了将这些发现转化为完整单词的语音识别，论文第一作者David Moses博士开发了实时解码这些模式的新方法，并结合统计语言模型提高了准确性。
然而，这些还不够，科学家们此时并不能保证该技术在声带瘫痪病人的身上也能起作用。“我们的模型需要学习复杂的大脑活动模式和有意义语言之间的映射，当患者不能说话时，这就构成了一个重大挑战。”Moses说。此外，对于那些多年无法活动发声肌肉的人而言，控制声道的大脑信号是否仍然完整也未可知。

前50个字

为了研究这种技术在瘫痪患者中的潜力，张与同事Karunesh Ganguly，医学博士，神经学副教授合作，发起了一项名为“Bravo”的研究(手臂和声音的脑-计算机接口恢复)。试验的第一位参与者是一位30多岁的男子，他在15年前遭受了一次破坏性的脑干中风，严重损害了他的大脑与声道和四肢之间的联系。自从受伤以来，他的头部、颈部和四肢的活动都非常有限，通过使用棒球帽上的指针在屏幕上戳字母来进行交流。
为了进行这项研究，Chang在BRAVO 1的语音运动皮层上植入了一个高密度的电极阵列。在参与者完全康复后，他的团队记录了这个大脑区域在48次会议和几个月内的22个小时的神经活动。在每一次会议中，BRAVO 1试图说出50个词汇中的每一个单词多次，而电极记录的大脑信号从他的言语皮层。

将尝试性言语翻译成文本

为了将记录下来的神经活动模式转化为特定的预期词，本研究的另外两位主要作者肖恩·梅茨格(Sean Metzger)、MS和杰西·刘(Jessie Liu)，这两位在常实验室的生物工程博士生，都使用了自定义神经网络模型，这是人工智能的形式。当参与者试图说话时，这些网络区分大脑活动中的微妙模式，以检测语言尝试，并识别他想说的话。
为了测试他们的方法，研究小组首先向BRAVO 1展示了由50个词汇组成的短句，并让他试着说几遍。当他尝试的时候，这些单词从他的大脑活动中一个接一个地在屏幕上被解码出来。
然后，团队转向向他提出诸如“你今天过得怎么样”这样的问题。“你想喝点水吗？”和以前一样，BRAVO 1的尝试演讲出现在屏幕上。“我很好，”和“不，我不渴。”
研究小组发现，该系统能够以每分钟18字的速度解码大脑活动中的单词，准确率高达93%(中位数75%)。摩西应用的一种语言模型实现了“自动校正”功能，类似于消费者短信和语音识别软件所使用的功能，这是成功的原因之一。
摩西把早期的审判结果描述为一种原则的证明。他说：“看到各种有意义的句子的准确解码，我们感到非常兴奋。”“我们已经证明，以这种方式促进交流实际上是可能的，而且它在会话设置中也有使用的潜力。”

创新价值

“语言神经假肢”让瘫痪患者即使无法自己说话，也能进行交流，对于一个无法自然交流的人来说，这是一个重要的技术里程碑，它展示了这种方法为严重瘫痪和语言丧失的人提供声音的潜力。

创新关键点

通过手术将高密度电极阵列植入语言运动皮层，并记录了该区域22小时的神经活动，结合新技术的模型能够快速解码大脑活动的单词，准确率高达93%，这得益于其自动更正功能。

创新主体

加利福尼亚大学旧金山分校（University of California, San Francisco），简称UCSF，位于美国加利福尼亚州旧金山， 是美国加利福尼亚大学系统的第二所公立大学，是世界著名的生命科学及医学研究教学中心。
加利福尼亚大学旧金山分校的前身是1864年于旧金山建立的托兰医学院（Toland Medical College），1873年加入加利福尼亚大学，成为加利福尼亚大学伯克利分校附属的医学院，在伯克利、旧金山等地均有设点。1952年加州大学开始体制改革后，加州大学旧金山分校逐渐成为一所独立的公立大学，学校设施均迁入旧金山。
加州大学旧金山分校是加州大学系统中唯一的只专注于健康和生命科学的大学，也是加州大学系统中唯一的只进行严格的研究生教育的大学，以医学和生命科学而闻名。截止至2020年10月，共有10位加州大学旧金山分校的教授或研究人员曾获得诺贝尔奖。2018-19年度，在软科世界大学学术排名中，加州大学旧金山分校临床医学位列世界第2、生命科学位列世界第3；在US News全美最佳医院排名中，UCSF附属医院（UCSF Medical Center）位列全美第6、美国西部第1。 2019-20年，该校位列软科世界大学学术排名（ARWU）世界第20。
加州大学旧金山分校设有牙医学院、医学院、护理学院、药学院和研究生院；热门专业有医学、牙医学、护理学和生物化学等专业。

"Speech Neuroprosthesis" : Giving speech back to the paralysed

Writing in the New England Journal of Medicine, researchers at the University of California, San Francisco say they have developed a "speech neuroprosthesis" that converts brain signals controlling the vocal tract directly into words on a screen to help patients who are severely paralyzed and unable to speak communicate in sentences.
This is the first time complete words have been decoded directly from the brain activity of aphasia patients, showing strong promise for harnessing the brain's natural language mechanisms to restore communication. The new research takes advantage of the natural flow of language to translate directly the signals that control the muscles of the vocal system, allowing faster and less laborious communication. "We can communicate at 150 to 200 words per minute," Chang says. "Typing, writing and controlling cursor spelling are much slower and more laborious."

Turn brain signals into speech

Previously, work in the field of neuroprosthetics focused on restoring communication by printing words one by one, based on spelling. Zhang's research differs from these efforts in one key respect: His team is converting signals to control the muscles of the speech system, rather than signals to move an arm or hand in order to type. Zhang says the approach taps into natural and fluent speech and promises faster and organic communication.
Patients undergoing neurosurgery at UCSF Epilepsy Center have advanced Zhang's goal by placing an array of electrodes on the surface of the brain to determine the origin of seizures. The patients, all of whom had normal speech abilities, volunteered for an analysis of voice-related activity recorded by their brains. The early success of these patient volunteers paved the way for current trials in paralysed patients.
Previously, Chang and his colleagues at the UCSFWeill Neuroscience Institute mapped patterns of cortical activity associated with the vocal tract movement that produces each consonant and vowel sound. To translate these findings into whole-word speech recognition, David Moses, PhD, a postdoctoral engineer in Zhang's lab and one of the lead authors of the new study, developed new methods for decoding these patterns and statistical language models in real time to improve accuracy.
Chang analyzed language-related activity in the brains of volunteers with normal language abilities. The researchers then mapped cortical activity patterns associated with vocal tract movement. To translate these findings into speech recognition for whole words, David Moses, PhD, the paper's first author, developed new methods to decode these patterns in real time, combined with statistical language models to improve accuracy.
However, this is not enough, and scientists are not yet sure that the technique will work in patients with paralyzed vocal cords. "Our model requires learning the mapping between complex patterns of brain activity and meaningful language, which poses a major challenge when patients are unable to speak." Moses said. It is also unclear whether the brain signals that control the vocal tract remain intact in people who have been unable to use their vocal muscles for years.

The first 50 words

To investigate the potential of this technique in paralyzed patients, Zhang teamed up with colleague Karunesh Ganguly, MD, associate professor of neurology, to launch a study called Bravo (Brain-computer Interface Recovery of Arm and Voice). The first participant in the trial was a man in his 30s who had suffered a devastating brainstem stroke 15 years earlier, severely damaging his brain's connections to the vocal tract and limbs. Since his injury, he has had limited movement in his head, neck and limbs, communicating by using a pointer on a baseball cap to poke letters on a screen.
To conduct the study, Chang implanted a high-density electrode array in BRAVO 1's phonomotor cortex. After the participants fully recovered, his team recorded neural activity in this brain region over 48 sessions and 22 hours over several months. During each session, BRAVO 1 tried to say each of the 50 words multiple times, while electrodes recorded brain signals from his speech cortex.

Translate tentative words into text

To translate recorded patterns of neural activity into specific expected words, the study's two other lead authors, Sean Metzger, MS, and Jessie Liu, two bioengineering doctoral students in the lab, all used custom neural network models, a form of artificial intelligence. These networks distinguish between subtle patterns in brain activity when a participant is trying to speak to detect verbal attempts and identify what he is trying to say.
To test their method, the team first showed BRAVO 1 short sentences of 50 words and asked him to try saying them a few times. As he tried, the words were decoded on the screen, one by one, from his brain activity.
Then the team turned to him with questions like, "How was your day?" "Would you like some water?" As before, BRAVO 1's trial speech appears onscreen. "I'm fine," and "No, I'm not thirsty."
The team found that the system was able to decode words in brain activity at a rate of 18 words per minute, with an accuracy of 93 percent (median 75 percent). One reason for its success is that a language model Mr Moses applied implemented "auto-correction" features similar to those used by consumer messaging and speech recognition software.
Moses described the early judgment as a proof of principle. "We are very excited to see the accurate decoding of all kinds of meaningful sentences," he said. "We have shown that it is actually possible to facilitate communication in this way, and it has the potential to be used in session Settings as well."