Revision 30 as of 2010-01-14 20:18:24

Clear message

This page describes the ROS bag format, which is a logging format for storing ROS messages in files. Files using this format are called bags, and have the file extension .bag. Bags are recorded, played back, and generally manipulated by tools in the rosbag package, including rosrecord, rosplay, and rosrebag.

Versioning

The ROS bag format has changed over time. This document describes all formats that have ever been officially supported.

There is no guarantee of compatibility between different versions. Newly created bags are always recorded at the current version. While we make every effort to support playback of older bags, in some cases user intervention may be required to update an old bag for playback.

All bags begin with a newline-terminated string of the form:

#ROSRECORD VX.Y

Where X and Y are the major and minor version numbers, e.g. for version 1.1, the first line of the bag reads:

#ROSRECORD V1.1

After this line, the details vary by version.

Version 1.2

Usage

The 1.2 bag format is used by ROS releases in the 0.5.x line. The primary differences compared to 1.1 are that 1.2 organizes the message metadata in a more extensible manner, and stores the full text of each message definition, instead of just the type and md5sum. This information was added to improve bag longevity by allowing for tools that manipulate a bag after the message types stored within have been changed, moved, or deleted.

Format

Following the version line, records are written, one after the other:

#ROSRECORD V1.2
<record 1><record 2>....<record N>

Each record has the following format:

<header_len><header><data_len><data>

Name

Description

Format

Length

header_len

length, in bytes, of the header to follow

little-endian integer

4 bytes

header

metadata for the record

(see below)

header_len bytes

data_len

length, in bytes, of the data to follow

little-endian integer

4 bytes

data

record specific data

(see below)

data_len bytes

Header format

Each record header (the header field that precedes the data) contains a sequence of name=value fields, formatted as follows:

<field1_len><field1_name>=<field1_value><field2_len><field2_name>=<field2_value>...<fieldN_len><fieldN_name>=<fieldN_value>

Name

Description

Format

Length

fieldX_len

length, in bytes, of the name and value to follow

little-endian integer

4 bytes

fieldX_name

name of the field

character string

variable; terminated by '='

fieldX_value

value of the field

field-specific

variable

The total length of fieldX_name=fieldX_value, including the '=' character, is fieldX_len bytes. The total length of this header sequence (including the names, '=' characters, value lengths, and values) is header_len bytes.

Header fields may appear in any order. Field names can contain any printable ASCII character (0x20 - 0x7e), except = (0x3d). Field values can contain any data (including binary data with embedded nulls, newlines, etc.).

Every header is required to include the following field:

Name

Description

Format

Length

op

what kind of header this is (see below)

unsigned byte

1 byte

The op field is used to distinguish between different types of headers:

  • 0x01 : Message definition. Includes full text of message definition and is not followed by actual message data (the format is unchanged, so data_len is set to 0).

  • 0x02 : Message data. Does not include the full definition, and is followed by actual message data (which can be zero-length). In a well-formed bag file, the message definition for a given topic will appear exactly once, prior to any message data for that topic.

ROS 0.11 introduced two new header types:

  • 0x03 : Bag header (optional). Stores information about the entire file, such as the offset to the first index data record. It does not include any data.

  • 0x04 : Index data (optional). Stores a versioned index of messages in the bag file.

The following fields are guaranteed to appear in the message headers (op=0x01,0x02):

Name

Description

Format

Length

topic

topic on which the message arrived

character string

variable

md5

md5sum for the message type

character string

variable

type

message type

character string

variable

The following fields are guaranteed to appear in a message definition header (op=0x01):

Name

Description

Format

Length

def

full text of message definition

character string, produced by gendeps

variable

The following fields are guaranteed to appear in a message data header (op=0x02):

Name

Description

Format

Length

sec

time at which the message was received, integral seconds part

little-endian integer

4 bytes

nsec

time at which the message was received, integral nanoseconds part

little-endian integer

4 bytes

The data in these records is the serialized message data in the ROS serialization format.

The following field is guaranteed to appear in the bag header (op=0x03):

Name

Description

Format

Length

index_pos

offset of first index data record

little-endian long integer

8 bytes

The bag header record is padded out by filling data with ASCII space characters (0x20) so that additional information can be added after the bag file is recorded. Currently, this padding is such that the total record is 4096 bytes long.

The following field is guaranteed to appear in an index data header (op=0x04):

Name

Description

Format

Length

ver

index data version

little-endian integer

4 bytes

Index Data

Version 0 of the bag index stores one index data record per topic.

The following fields are guaranteed to appear in these headers:

Name

Description

Format

Length

topic

topic on which the messages arrived

character string

variable

type

message type

character string

variable

count

number of messages on topic in the bag file

little-endian integer

4 bytes

The data in each of these records consists of count repeating occurrences of timestamps and offsets:

Name

Description

Format

Length

sec

time at which the message was received, integral seconds part

little-endian integer

4 bytes

nsec

time at which the message was received, integral nanoseconds part

little-endian integer

4 bytes

pos

message record offset

little-endian long integer

8 bytes

The total data_len is count * 16 bytes.

The first pos entry points to the message definition record if the earliest message for type in this bag file occurs on this topic. Otherwise, the first pos entry points to a message data record.

Version 1.1

Usage

The 1.1 bag format is the first format that was officially released. It is used by ROS releases in the 0.4.x line. It appears to have been in use in the repository since at least r2592.

Format

Following the version line, messages are written, one after the other: #ROSRECORD V1.1<message 1><message 2>....<message N>}}}

Each message has the following format:

<topic>
<md5sum>
<type>
<time_sec><time_nsec><length><data>

Note that the first 3 fields are newline-terminated. The last 4 fields, including data, are not newline-terminated.

Name

Description

Format

Length

topic

topic on which the message arrived

character string

variable length, newline-terminated

md5sum

md5sum for the message type

character string

variable length (should be 32 bytes), newline-terminated

type

message type

character string

variable length, newline-terminated

time_sec

time at which the message was received, integral seconds part

little-endian integer

4 bytes

time_nsec

time at which the message was received, integral nanoseconds part

little-endian integer

4 bytes

length

length, in bytes, of the data to follow

little-endian integer

4 bytes

data

serialized message data

ROS serialization format

length bytes

Example

A 1.1 bag file might start like this (showing the version string and two messages):

#ROSRECORD V1.1
/mechanism_state
2fc442e85b325b48a5c28bfff1aec3e8
robot_msgs/MechanismState
<binary data here, which can include any bytes, including nulls and newlines>/tf_message
bf97565a3af47c603375c0814456d7db
tf/tfMessage
<binary data here, which can include any bytes, including nulls and newlines>

Version 1.0

The 1.0 bag format was never officially released.

TODO: describe the format, just in case people have very old bags lying around.