This page describes the ROS bag format, which is a logging format for storing ROS messages in files. Files using this format are called bags, and have the file extension .bag. Bags are recorded, played back, and generally manipulated by tools in the rosbag package, including rosrecord, rosplay, and rosrebag.
Contents
Versioning
The ROS bag format has changed over time. This document describes all formats that have ever been officially supported.
There is no guarantee of compatibility between different versions. Newly created bags are always recorded at the current version. While we make every effort to support playback of older bags, in some cases user intervention may be required to update an old bag for playback.
All bags begin with a newline-terminated string of the form:
#ROSRECORD VX.Y
Where X and Y are the major and minor version numbers, e.g. for version 1.1, the first line of the bag reads:
#ROSRECORD V1.1
After this line, the details vary by version.
Version 1.2
Usage
The 1.2 bag format is used by ROS releases in the 0.5.x line. The primary differences compared to 1.1 are that 1.2 organizes the message metadata in a more extensible manner, and stores the full text of each message definition, instead of just the type and md5sum. This information was added to improve bag longevity by allowing for tools that manipulate a bag after the message types stored within have been changed, moved, or deleted.
Format
Following the version line, records are written, one after the other:
#ROSRECORD V1.2 <record 1><record 2>....<record N>
Each record has the following format:
<header_len><header><data_len><data>
Name |
Description |
Format |
Length |
header_len |
length, in bytes, of the header to follow |
little-endian integer |
4 bytes |
header |
metadata for the record |
(see below) |
header_len bytes |
data_len |
length, in bytes, of the data to follow |
little-endian integer |
4 bytes |
data |
record specific data |
(see below) |
data_len bytes |
Header format
Each record header (the header field that precedes the data) contains a sequence of name=value fields, formatted as follows:
<field1_len><field1_name>=<field1_value><field2_len><field2_name>=<field2_value>...<fieldN_len><fieldN_name>=<fieldN_value>
Name |
Description |
Format |
Length |
fieldX_len |
length, in bytes, of the name and value to follow |
little-endian integer |
4 bytes |
fieldX_name |
name of the field |
character string |
variable; terminated by '=' |
fieldX_value |
value of the field |
field-specific |
variable |
The total length of fieldX_name=fieldX_value, including the '=' character, is fieldX_len bytes. The total length of this header sequence (including the names, '=' characters, value lengths, and values) is header_len bytes.
Header fields may appear in any order. Field names can contain any printable ASCII character (0x20 - 0x7e), except = (0x3d). Field values can contain any data (including binary data with embedded nulls, newlines, etc.).
Every header is required to include the following field:
Name |
Description |
Format |
Length |
op |
what kind of header this is (see below) |
unsigned byte |
1 byte |
The op field is used to distinguish between different types of headers:
0x01 : Message definition. Includes full text of message definition and is not followed by actual message data (the format is unchanged, so data_len is set to 0).
- 0x02 : Message data. Does not include the full definition, and is followed by actual message data (which can be zero-length). In a well-formed bag file, the message definition for a given topic will appear exactly once, prior to any message data for that topic.
ROS 0.11 introduced two new header types:
0x03 : Bag header (optional). Stores information about the entire file, such as the offset to the first index data record. It does not include any data.
0x04 : Index data (optional). Stores a versioned index of messages in the bag file.
The following fields are guaranteed to appear in the message headers (op=0x01,0x02):
Name |
Description |
Format |
Length |
topic |
topic on which the message arrived |
character string |
variable |
md5 |
md5sum for the message type |
character string |
variable |
type |
message type |
character string |
variable |
The following fields are guaranteed to appear in a message definition header (op=0x01):
Name |
Description |
Format |
Length |
def |
full text of message definition |
character string, produced by gendeps |
variable |
The following fields are guaranteed to appear in a message data header (op=0x02):
Name |
Description |
Format |
Length |
sec |
time at which the message was received, integral seconds part |
little-endian integer |
4 bytes |
nsec |
time at which the message was received, integral nanoseconds part |
little-endian integer |
4 bytes |
The data in these records is the serialized message data in the ROS serialization format.
The following field is guaranteed to appear in the bag header (op=0x03):
Name |
Description |
Format |
Length |
index_pos |
offset of first index data record |
little-endian long integer |
8 bytes |
The bag header record is padded out by filling data with ASCII space characters (0x20) so that additional information can be added after the bag file is recorded. Currently, this padding is such that the total record is 4096 bytes long.
The following field is guaranteed to appear in an index data header (op=0x04):
Name |
Description |
Format |
Length |
ver |
index data version |
little-endian integer |
4 bytes |
Index Data
Version 0 of the bag index stores one index data record per topic.
The following fields are guaranteed to appear in these headers:
Name |
Description |
Format |
Length |
topic |
topic on which the messages arrived |
character string |
variable |
type |
message type |
character string |
variable |
count |
number of messages on topic in the bag file |
little-endian integer |
4 bytes |
The data in each of these records consists of count repeating occurrences of timestamps and offsets:
Name |
Description |
Format |
Length |
sec |
time at which the message was received, integral seconds part |
little-endian integer |
4 bytes |
nsec |
time at which the message was received, integral nanoseconds part |
little-endian integer |
4 bytes |
pos |
message record offset |
little-endian long integer |
8 bytes |
The total data_len is count * 16 bytes.
The first pos entry points to the message definition record if the earliest message for type in this bag file occurs on this topic. Otherwise, the first pos entry points to a message data record.
Version 1.1
Usage
The 1.1 bag format is the first format that was officially released. It is used by ROS releases in the 0.4.x line. It appears to have been in use in the repository since at least r2592.
Format
Following the version line, messages are written, one after the other: #ROSRECORD V1.1<message 1><message 2>....<message N>}}}
Each message has the following format:
<topic> <md5sum> <type> <time_sec><time_nsec><length><data>
Note that the first 3 fields are newline-terminated. The last 4 fields, including data, are not newline-terminated.
Name |
Description |
Format |
Length |
topic |
topic on which the message arrived |
character string |
variable length, newline-terminated |
md5sum |
md5sum for the message type |
character string |
variable length (should be 32 bytes), newline-terminated |
type |
message type |
character string |
variable length, newline-terminated |
time_sec |
time at which the message was received, integral seconds part |
little-endian integer |
4 bytes |
time_nsec |
time at which the message was received, integral nanoseconds part |
little-endian integer |
4 bytes |
length |
length, in bytes, of the data to follow |
little-endian integer |
4 bytes |
data |
serialized message data |
ROS serialization format |
length bytes |
Example
A 1.1 bag file might start like this (showing the version string and two messages):
#ROSRECORD V1.1 /mechanism_state 2fc442e85b325b48a5c28bfff1aec3e8 robot_msgs/MechanismState <binary data here, which can include any bytes, including nulls and newlines>/tf_message bf97565a3af47c603375c0814456d7db tf/tfMessage <binary data here, which can include any bytes, including nulls and newlines>
Version 1.0
The 1.0 bag format was never officially released.
TODO: describe the format, just in case people have very old bags lying around.