创新背景

当研究人员想让机器人探索深海，尤其是以成群结队的形态时，在2万英尺外的海面上用操纵杆控制它们几乎是不可能的。研究人员也无法给它们提供它们导航所需的当地洋流的数据，因为无法从海面上探测到它们。相反，在某种程度上，研究人员需要海洋无人机能够自行决定如何移动。

创新过程

人工智能的性能通过计算机模拟进行了测试，但背后的团队还开发了一个手掌大小的小型机器人，它在一个微型计算机芯片上运行算法，可以为地球和其他星球上的海上无人机提供动力。他们的目标是创建一个自主系统来监测地球上的海洋状况，例如，将算法与他们之前开发的假肢结合起来，帮助水母在指令下游得更快。运行该算法的全机械机器人甚至可以探索其他星球的海洋，如土卫二或木卫二。

在任何一种情况下，无人机都需要能够自己决定去哪里以及到达那里的最有效方式。要做到这一点，可能只有它们自己能收集到关于它们目前正在经历的水流的有用的数据。

为了应对这一挑战，研究人员转向强化学习(RL)网络。与传统的神经网络相比，强化学习网络不是在静态数据集上进行训练，而是在收集经验时尽可能快地进行训练。这个方案允许它们在更小的计算机上运行。为了这个项目的目的，团队编写了可以在teensy上安装和运行的软件，teensy是一个2.4 × 0.7英寸的微控制器，任何人都可以在亚马逊上花不到30美元买到，而且只需要大约半瓦的电力。

通过计算机模拟，当水流经过水中的障碍物时，会产生几个向相反方向移动的漩涡，该团队教会人工智能以这样一种方式导航，即利用漩涡尾流的低速区域，以最小的电力消耗滑行到目标位置。为了帮助它的导航，模拟游泳者只能接触到它所处位置的水流信息，但它很快就学会了如何利用漩涡向预期目标滑行。在物理机器人中，人工智能同样只能访问从机载陀螺仪和加速计收集到的信息，这两者都是相对较小和低成本的机器人平台传感器。

这种导航方式类似于鹰和鹰乘坐空气中的热气流，从气流中提取能量，以最小的能量消耗机动到所需的位置。研究人员发现他们的强化学习算法可以学习导航策略，甚至比海洋中真正的鱼所使用的导航策略更有效。

这项技术仍处于初级阶段：目前，该团队希望测试AI在海洋任务中可能遇到的每种不同类型的流动干扰。例如通过旋转漩涡与流动潮流以评估其在野外的有效性。然而，通过在强化学习策略中融入他们的海洋流动物理知识，研究人员的目标是克服这一限制。目前的研究证明了RL网络在应对这一挑战方面的潜在有效性——特别是因为它们可以在如此小的设备上运行。为了在现场进行试验，该团队将Teensy放在定制无人机上，称为“CARL-Bot”（加州理工学院自主强化学习机器人）。CARL-Bot将被扔进加州理工学院校园里一个新建成的两层楼高的水箱里，学习如何驾驭洋流。

创新关键点

与传统的神经网络相比，强化学习网络不是在静态数据集上进行训练，而是在收集经验时尽可能快地进行训练，因此它们可以在更小的计算机上运行。

创新价值

该项研究开发的新型AI与小型机器人可利用洋流进行自主导航并收集信息，帮助人们了解海洋流动的物理知识。

New AI allows drones to navigate autonomously using ocean currents

The AI's performance was tested using computer simulations, but the team behind it also developed a small palm-sized robot that runs algorithms on a tiny computer chip that could power maritime drones on Earth and other planets. Their goal is to create an autonomous system to monitor ocean conditions on Earth, for example, by combining algorithms with prosthetics they previously developed to help jellyfish travel faster downstream on commands. An all-mechanical robot running the algorithm could even explore the oceans of other worlds, such as Enceladus or Europa.

In either case, drones need to be able to decide for themselves where to go and the most efficient way to get there. To do this, they may be the only ones who can gather useful data about the currents they are currently experiencing.

To address this challenge, researchers have turned to reinforcement learning (RL) networks. In contrast to traditional neural networks, reinforcement learning networks are not trained on static datasets, but are trained as fast as possible while collecting experience. This scheme allows them to run on smaller computers. For the purposes of the project, the team wrote software that could be installed and run on teensy, a 2.4-by-0.7-inch microcontroller that anyone can buy on Amazon for less than $30 and that requires only about half a watt of power.

Using computer simulations that create several eddies moving in opposite directions as water passes over obstacles in the water, the team taught the AI to navigate in such a way that it uses the low-speed area of the vortex wake to glide to a target location with minimal power consumption. To aid its navigation, the simulated swimmer only has access to information about the current at its location, but it quickly learns how to use the whirlpool to glide toward its intended target. In physical robots, AI similarly only has access to information gathered from onboard gyroscopes and accelerometers, both of which are relatively small and low-cost robotic platform sensors.

This way of navigation is similar to that of hawks and eagles riding thermals in the air to extract energy from the airflow and maneuver to the desired location with minimal energy expenditure. The researchers found that their reinforcement learning algorithm could learn navigation strategies even more effectively than those used by real fish in the ocean.

The technology is still in its infancy: For now, the team wants to test every different type of flow interference an AI might encounter during an ocean mission. Such as rotating vortices and flowing currents to assess their effectiveness in the field. However, by incorporating their knowledge of ocean flow physics into reinforcement learning strategies, the researchers aim to overcome this limitation. The current study demonstrates the potential effectiveness of RL networks in meeting this challenge - especially since they can run on such small devices. To test it in the field, the team put Teensy on a custom drone called a "CARL-Bot" (Caltech Autonomous Reinforcement Learning Robot). CARL-Bot will be dropped into a newly built two-story tank on the Caltech campus to learn how to navigate ocean currents.