Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Remotely listening through robot ears

User story

As a remote robot teleoperator, I can hear what the robot is hearing.

This use case improves remote robot teleoperator's situation awareness around the robot's local environment. This is particularly helpful for remotely teleoperating robots that do not have omnidirectional vision.

In terms of human attention, the ears tell the eyes where to look, especially when you are driving around a robot that might drop things, might run into environmental obstacles (e.g., walls, doors, table, chairs), and/or is operating around people or animals.

Start capturing sound from robot microphones

On robot:

Start capturing audio data using the following command

roslaunch audio_capture capture.launch

This will produce an /audio topic. Check for its existence using:

rostopic list

Check to see if it is streaming any data:

rostopic hz /audio

If it is working properly, then you will see average rates for the data streaming from the mics into the /audio data stream.

Do not kill this node, and continue to the next section.

If it is not working properly, then you will not see any rates for the data streaming from the mics. If this happens, then check the robot's audio settings to make sure that the robot is listening through the right microphones and that it is using the correct audio mixer.

Connect local computer to robot

On local computer:

Ping the robot to identify its IP address (where prX is a placeholder variable for the name of your robot).

ping prX

Use the robot's IP address to set your ROS_MASTER_URI to point your local computer to the robot (where xx.x.x.xxx is a placeholder variable for the IP address of your robot).

export ROS_MASTER_URI=http://xx.x.x.xxx:11311

You can test your connection by checking the audio data streaming on the robot.

rostopic hz /audio

If it is working properly, then you will see same average rates for the data streaming from the mics into the /audio data stream that you saw in the previous step.

Start listening through robot microphones from local computer

Now that sound is being captured from the robot's microphones and your local computer is connected to the robot, you can listen through the robot's microphones from your local computer.

On local computer:

Plug headphones into your computer (or listen through computer loudspeakers). Test your sound by playing some music from your local machine or streaming music from a web site.

Open a new terminal, and change directory to the audio_common sources.

cd audio_common
export ROS_PACKAGE_PATH=${PWD}:$ROS_PACKAGE_PATH

Now launch the audio_play node.

roslaunch audio_play play.launch do_timestamp:=false

This will start a new node that listens to the /audio topic, and plays the incoming messages.

At this point, you should be able to hear what the robot's microphones are hearing. To test this, ask someone to make noises around the robot while you listen through your local computer's speakers or headphones.

Stop listening

To stop listening, you simply need to cancel the audio_common processes running on the robot and on your local computer.

On robot:

In the terminal that is running the audio_capture node, type Ctrl-C.

On local computer:

In the terminal that is running the audio_play node, type Ctrl-C.

If it worked properly, then you should no longer hear any sound coming through your local computer. You can check to see if the microphones have stopped publishing by checking the /audio rostopic.

rostopic list | grep audio

If the microphones have stopped capturing sound, then it will not return an /audio topic.