Skip to content

Architecture

The B-Human architecture1 is based on the framework of the GermanTeam 20072, adapted to the NAO. This chapter summarizes the major features of the architecture: binding, threads, modules and representations, communication, and debugging support.

Binding

On the NAO, the B-Human software consists of a single binary file and a number of configuration files, all located in /home/nao/Config. When the program is started, it first contacts LoLA to retrieve the serial number of the body that is then mapped to a body name using the file Config/Robots/robots.cfg. The Unix hostname is used as the robot's head name. Then, the connection to LoLA is closed again and the program forks itself to start the main robot control program. After that has terminated, another connection to LoLA is established to sit down the robot if necessary, switch off all joints, and display the exit status of the main program in the eyes (blue: terminated normally, red: crashed).

In the main robot control program, the module NaoProvider exchanges data with LoLA following the protocol defined by SoftBank Robotics for RoboCup teams. For efficiency, only the first packet received is actually parsed, determining the addresses of all relevant data in that packet and using these addresses for later packets rather than parsing them again. Similarly, a template for packets sent to LoLA is only created once and the actual data is patched in in each frame. The NaoProvider uses blocking communication, i.e. the whole thread Motion (cf. this section) waits until a packet from LoLA is received before it continues.

Threads

Most robot control programs use concurrent processes and threads. Of those, we only use threads in order to have a single shared memory context. The number of parallel threads is best dictated by external requirements coming from the robot itself or its operating system. The NAO provides images from each camera at a frequency of 30  Hz and accepts new joint angle commands at 83 Hz. For handling the camera images, there would actually have been two options: either to have two threads each of which processes the images of one of the two cameras and a third one that collects the results of the image processing and executes world modeling and behavior control, or to have a single thread that alternately processes the images of both cameras and also performs all further steps. We are currently using the first approach in order to better exploit the multi-core hardware of the robot. This makes it possible to process both images in parallel and leads to significantly more available computation time per image. These threads run at 30 Hz and handle one camera image each. They both trigger a third thread, which processes the results as described above. This one runs with 60 Hz in parallel to the image processing. In addition, there is a thread that runs at the motion frame rate of the NAO, i.e. at 83 Hz and one that records data from NAO's microphones and processes them. Another thread performs the TCP communication with a host PC for the purpose of debugging.

The threads used on the NAO This results in the six threads Upper, Lower, Cognition, Motion, Audio, and Debug actually used in the B-Human system. The perception threads Upper and Lower receive camera images from Video for Linux. In addition, they receive data from the thread Cognition about the world model as well as sensor data from the thread Motion. They processes the images and send the detection results to the thread Cognition. This thread actually uses this information together with sensor data from the thread Motion for world modeling and behavior control and sends high-level motion commands to the thread Motion. This one actually executes these commands by generating the target angles for the 25 joints of the NAO. It sends these target angles through the NaoProvider to the NAO's interface LoLA, and it receives sensor readings such as the actual joint angles, body acceleration and gyro measurements, etc. In addition, Motion reports about the motion of the robot, e.g. by providing the results of dead reckoning. The thread Audio performs the whistle detection if the current game state requires it and reports the results back to Cognition. The thread Debug communicates with the host PC. It distributes the data received from it to the other threads and it collects the data provided by them and forwards it back to the host machine. It is inactive during actual games.

Modules and Representations

A robot control program usually consists of several modules, each performing a certain task, e.g. image processing, self-localization, or walking. Modules require a certain input and produce a certain output (so-called representations). Therefore, they have to be executed in a specific order to make the whole system work. The module framework introduced by the GermanTeam2 simplifies the definition of the interfaces of modules and automatically determines the sequence in which the modules must be executed. It consists of the blackboard, the module definition, and a visualization component (cf. this section).

Blackboard

The blackboard3 is the central storage for information, i.e. for the representations. Each thread is associated with its own instance of the blackboard. Representations are transmitted through inter-thread communication if a module in one thread requires a representation that is provided by a module in another thread. The blackboard itself is a map that associates names of representations to reference-counted instances of these representations. It only contains entries for the representations that at least one of the modules in associated thread actually requires or provides.

Module Definition

The definition of a module consists of three parts: the module interface, its actual implementation, and a statement that allows instantiating the module. Here is an example:

MODULE(SimpleBallLocator,
{,
  REQUIRES(BallPercept),
  REQUIRES(FrameInfo),
  PROVIDES(BallModel),
  DEFINES_PARAMETERS(
  {,
    (Vector2f)(5.f, 0.f) offset,
    (float)(1.1f) scale,
  }),
});

class SimpleBallLocator : public SimpleBallLocatorBase
{
  void update(BallModel& ballModel)
  {
    if(theBallPercept.wasSeen)
    {
      ballModel.position = theBallPercept.position * scale + offset;
      ballModel.wasLastSeen = theFrameInfo.time;
    }
  }
}

MAKE_MODULE(SimpleBallLocator, modeling);

The module interface defines the name of the module (e.g. SimpleBallLocator), the representations that are required to perform its task, the representations provided by the module, and the parameters of the module, the values of which can either be defined in place or loaded from a file. The interface basically creates a base class for the actual module following the naming scheme <ModuleName>Base. The actual implementation of the module is a class that is derived from that base class. The following statements are available in a module definition:

  • REQUIRES declares that this module has read-only access to this representation in the blackboard (and only to this) via the<representationName>. As is described in this section, modules can expect that all their required representations have been updated before any of their provider methods is called. If a required representation is provided by multiple threads, a so-called alias must be used to specify which representation it uses. An alias is a representation that has a thread name prefix before the actual name, e.g. <thread><representation>. The representation must be derived from the actual representation without adding members to it.
  • USES declares that this module accesses a representation, which does not necessarily have to be updated before the module is run. This should only be used to resolve conflicts in the configuration. Note that USES is not considered when exchanging data between threads.
  • PROVIDES declares that this module will update this representation. It must define an update method for each representation that is provided. For each representation the execution time can be determined (cf. this section) and it can be sent to a host PC or even altered by it.
  • PROVIDES_WITHOUT_MODIFY does exactly the same as PROVIDES, except that the representation can not be inspected or altered by the host PC.
  • DEFINES_PARAMETERS and LOADS_PARAMETERS allow the modifiable parameterization of modules, as described in this section. It is recommended to use this for all parameters.

Finally, the MAKE_MODULE statement allows the module to be instantiated. Its second parameter defines a category that is used for a more structured visualization of the module configuration (cf. this section). The optional third parameter can override the static function that delivers the list of required and provided representations to the system (cf. this section). This can be used to add requirements to a module that can not be part of the module definition, as is needed by the behavior framework. While the module interface is usually part of the header file, the MAKE_MODULE statement has to be part of the implementation file.

MODULE is a macro that gets all the information about the module as parameters, i.e. they are all separated by commas. The macro ignores its second and its last parameter, because by convention, these are used for opening and closing curly brackets. These let some source code formatting tools indent the definitions as a block. Currently, MODULE is limited to up to 90 definitions between the curly brackets. When the macro is expanded, it creates a lot of hidden functionality. Each entry that references a representation makes sure that it is created in the blackboard when the module is constructed and freed when the module is destroyed. The information that a module has certain requirements and provides certain representations is not only used to generate a base class for that module, but is also available for sorting the providers, and can be requested by a host PC. On a host PC, the information can be used to change the configuration and for visualization (cf. this section). If a MessageID id<representation> exists, the representation can also be logged (cf. this section).

If a representation provided defines a parameterless method draw, that method will be called after the representation was updated. The method is intended to visualize the representation using the techniques described in this section. If the representation defines a parameterless method verify, that method will be called in Debug and Develop builds after the representation was updated as well. A verify method should contain ASSERTs that check whether the contents of the representation are plausible. Both methods are only called if they are defined in the representation itself and not if they are inherited from its base class.

Configuring Providers and Threads

Since modules can provide more than a single representation, the configuration has to be performed on the level of providers. For each representation, it can be selected which module provides it or that it is not provided at all. Normally, the configuration is read from the file Config/Scenarios/<scenario>/threads.cfg during the start-up of the robot control program, but it can also be changed interactively when the robot has a debug connection to a host PC using the command mr (cf. this section). This file defines which threads exist at all and which providers run in them. This allows for completely different configurations in different scenarios.

The configuration does not specify the sequence in which the providers are executed. This sequence is automatically determined at runtime based on the rule that all representations required by a provider must already have been provided by other providers before, i.e. those providers have to be executed earlier. This is calculated in the thread Debug. Only valid configurations are then sent to all other threads.

In some situations, it is required that a certain representation is provided by a module before any other representation is provided by the same module, e.g., when the main task of the module is performed in the update method of that representation, and the other update methods rely on results computed in the first one. Such a case can be implemented by both requiring and providing a representation in the same module.

Default Representations

During the development of the robot control software, it is sometimes desirable to simply deactivate a certain provider or module. As mentioned above, it can always be decided not to provide a certain representation, i.e. all providers generating the representation are switched off. However, not providing a representation typically makes the set of providers inconsistent, because other providers rely on that representation, so they would have to be deactivated as well. This has a cascading effect. In many situations, it would be better to be able to deactivate a provider without any effect on the dependencies between the modules. In this case it is possible to enter representations to a separate list in the thread configuration file. It will never change any of the representations – so they basically remain in their initial state – but it will make sure that they exist, and thereby, all dependencies can be resolved. However, in terms of functionality, a configuration using that list is not complete.

Parameterizing Modules

Modules usually need some parameters to function properly. Those parameters can also be defined in the module's interface description. Parameters behave like protected class members and can be accessed in the same way. Additionally, they can be manipulated from the console using the commands get parameters:<ModuleName> or vd parameters:<ModuleName> (cf. this section).

There are two different parameter initialization methods. In the hard-coded approach, the initialization values are specified as part of the module definition. They are defined using the DEFINES_PARAMETERS macro. This macro is intended for parameters that may change during development but will never change again afterwards. In contrast, loadable parameters are initialized to values that are loaded from a configuration file upon module creation, i.e. the initialization values are not specified in the source file. These parameters are defined using the LOADS_PARAMETERS macro. By default, parameters are loaded from a file with the same base name as the module, but starting with a lowercase letter4 and the extension .cfg. For instance if a module is named SimpleBallLocator, its configuration file is simpleBallLocator.cfg. This file can be placed anywhere in the usual configuration file search path (cf. this section). It is also possible to assign a custom name to a module's configuration file by passing the name as a parameter to the constructor of the module's base class.

Only either DEFINES_PARAMETERS or LOADS_PARAMETERS can be used in a module definition. They can both only be used once. Their syntax follows the definition of generated streamable classes (cf. this section). Parameters may have any data type as long as it is streamable (cf. this section).

Serialization

In most applications, it is necessary that data can be serialized, i.e. transformed into a sequence of bytes. While this is straightforward for data structures that already consist of a single block of memory, it is a more complex task for dynamic structures, e.g. lists, trees, or graphs. Our implementation for streaming data follows the ideas introduced by the C++ iostreams library, i.e., the operators << and >> are used to implement the process of serialization. In contrast to the iostreams library, our implementation guarantees that data is streamed in a way that it can be read back without any special handling, even when streaming into and from text files, i.e. the user of a stream does not need to know about the representation that is used for the serialized data (cf. this section).

On top of the basic streaming class hierarchy, it is also possible to derive classes from class Streamable and implement the mandatory methods read(In&) and write(Out&) const. In addition, the basic concept of streaming data was extended by a mechanism to register information on the structure of serializable datatypes. This information is used to translate between the data in binary form and a human-readable format that reflects the hierarchy of a data structure, its variables, and their actual values.

As a third layer of serialization, two macros allow defining classes that automatically implement the methods read(In&) and write(Out&) const as well as datatype registration.

Streams

The foundation of B-Human's implementation of serialization is a hierarchy of streams. As a convention, all classes that write data into a stream have a name starting with Out, while classes that read data from a stream start with In. In fact, all writing classes are derived from the class Out, and all reading classes are derivations of the class In. All classes support reading or writing basic datatypes with the exceptions of long and long long (both signed and unsigned). They also provide the ability to read or write raw binary data.

All streaming classes derived from In and Out are composed of two components: One for reading/writing the data from/to a physical medium and one for formatting the data from/to a specific format. Classes writing to physical media derive from PhysicalOutStream, classes for reading derive from PhysicalInStream. Classes for formatted writing of data derive from StreamWriter, classes for reading derive from StreamReader. The composition is done by the OutStream and InStream class templates.

A special case are the OutMap and the InMap streams. They only work together with classes that are derived from the class Streamable, because they use the structural information that is gathered in the read and write methods. They are both directly derived from Out and In, respectively.

Currently, the following classes are implemented:

  • PhysicalOutStream: Abstract class
    • OutFile: Writing into files
    • OutMemory: Writing into memory
    • OutMemoryForText: Writing into memory, ensuring that it is null-terminated
    • OutMessageQueue: Writing into a MessageQueue
  • StreamWriter: Abstract class
    • OutBinary: Formats data binary
    • OutText: Formats data as text
    • OutTextRaw: Formats data as raw text (same output as cout)
  • Out: Abstract class
    • OutStream<PhysicalOutStream, StreamWriter>: Abstract template class
      • OutBinaryFile: Writing into binary files
      • OutBinaryMemory: Writing binary into memory
      • OutBinaryMessage: Writing binary into a MessageQueue
      • OutTextFile: Writing into text files
      • OutTextMemory: Writing into memory as text
      • OutTextMessage: Writing into a MessageQueue as text
      • OutTextRawFile: Writing into raw text files
      • OutTextRawMemory: Writing into memory as raw text
      • OutTextRawMessage: Writing into a MessageQueue as raw text
    • OutMap: Writing into a stream in configuration map format (cf. this section). This only works together with serialization (cf. this section), i.e. a streamable object has to be written. This class cannot be used directly.
      • OutMapFile: Writing into a file in configuration map format
      • OutMapMemory: Writing into a memory area in configuration map format
  • PhysicalInStream: Abstract class
    • InFile: Reading from files
    • InMemory: Reading from memory
    • InMessageQueue: Reading from a MessageQueue
  • StreamReader: Abstract class
    • InBinary: Binary reading
    • InConfig: Reading configuration file data from streams
    • InText: Reading data as text
  • In: Abstract class
    • InStream<PhysicalInStream, StreamReader>: Abstract class template
      • InBinaryFile: Reading from binary files
      • InTextFile: Reading from text files
      • InConfigFile: Reading from configuration files
      • InBinaryMemory: Reading binary data from memory
      • InTextMemory: Reading text data from memory
      • InConfigMemory: Reading config-file-style text data from memory
      • InBinaryMessage: Reading binary data from a MessageQueue
      • InTextMessage: Reading text data from a MessageQueue
    • InMap: Reading from a stream in configuration map format (cf. this section). This only works together with serialization (cf. this section), i.e. a streamable object has to be read. This class cannot be used directly.
      • InMapFile: Reading from a file in configuration map format
      • InMapMemory: Reading from a memory area in configuration map format

Streaming Data

To write data into a stream, Src/Libs/Streaming/OutStreams.h must be included, a stream must be constructed, and the data must be written into the stream. For example, to write data into a text file, the following code would be appropriate:

#include "Streaming/OutStreams.h"
// ...
OutTextFile stream("MyFile.txt");
stream << 1 << 3.14 << "Hello Dolly" << endl << 42;

The file will be written into the configuration directory, e.g. Config/MyFile.txt on the PC. It will look like this:

 1 3.14 "Hello Dolly"
 42

As spaces are used to separate entries in text files, the string "Hello Dolly" is enclosed in double quotes. The data can be read back using the following code:

#include "Streaming/InStreams.h"
// ...
InTextFile stream("MyFile.txt");
int a, d;
double b;
std::string c;
stream >> a >> b >> c >> d;

It is not necessary to read the symbol endl here, although it would also work, i.e. it would be ignored.

For writing to text streams without the separation of entries and the addition of double quotes, OutTextRawFile can be used instead of OutTextFile. It formats the data such as known from the ANSI C++ cout stream. The example above is formatted as following:

13.14Hello Dolly
42

To make streaming independent of the kind of the stream used, it could be encapsulated in functions. In this case, only the abstract base classes In and Out should be used to pass streams as parameters, because this makes the functions usable independently from the type of the streams:

#include "Streaming/InOut.h"

void write(Out& stream)
{
  stream << 1 << 3.14 << "Hello Dolly" << endl << 42;
}

void read(In& stream)
{
  int a, d;
  double b;
  std::string c;
  stream >> a >> b >> c >> d;
}
// ...
OutTextFile stream("MyFile.txt");
write(stream);
// ...
InTextFile stream("MyFile.txt");
read(stream);

Generating Streamable Classes

Values of basic types are not the only kind of data that can be streamed. Instances of classes can be streamed as well. However, in contrast to basic data types, the streaming framework cannot know how to stream an object that is composed of several member variables. Therefore, additional information must be provided that allows serializing and de-serializing the instances of a class and even transmitting and querying its specification. The underlying details are given in this section. Here, we only describe how two macros (defined in Src/Libs/Streaming/AutoStreamable.h) generate this extra information for a streamable struct5 and optionally also initialize its member variables. The first is:

STREAMABLE(<class>,
{ <header>,
  <comma-separated-declarations>,
});

The second is very similar:

STREAMABLE_WITH_BASE(<class>, <base>, ...

The parameters have the following meaning:

  • class: The name of the struct to be declared.
  • base: Its base class. It must be streamable itself.
  • header: Everything that can be part of a class body except for the member variables that should be streamable. Please note that this part must not contain commas that are not surrounded by parentheses, because C++ would consider it to be more than a single macro parameter otherwise. A workaround is to use the macro COMMA instead of an actual comma. However, the use of that macro should be avoided if possible, e.g. by defining constructors with comma-separated initializer lists outside of the STREAMABLE's body.
  • comma-separated-declarations: Declarations of the streamable member variables6 in two possible forms:

    (<type>) <var>
    (<type>)(<init>) <var>
    
    • type: The type of the member variable that is declared.
    • var: The name of the member variable that is declared.
    • init: The initial value of the member variable, or if an object is declared, the parameter(s) passed to its constructor.

Please note that all these parts, including each declaration of a streamable member variable, are separated by commas, since they are parameters of a macro. Here is an example:

STREAMABLE(Example,
{
  ENUM(ABC,
  {,
    a,
    b,
    c,
  });

  Example()
  {
    std::memset(array, 0, sizeof(array));
  },

  (int) anInt,
  (float)(3.14f) pi,
  (int[4]) array,
  (Vector2f)(1.f, 2.f) aVector,
  (ABC) aLetter,
  (MotionRequest::Motion)(MotionRequest::stand) motionId,
});

In this example, all member variables except for anInt and aLetter would be initialized when an instance of the class is created.

Instances of Example can now be streamed like any basic data type. Since it is sometimes necessary to do some post-processing after an object was read from a stream, e.g. to recompute member variables that were not streamed, a method onRead() can be defined within the STREAMABLE, which is called after all streamed member variables were read.

Type Registry

The class TypeRegistry stores the specification of all streamable data types. It is automatically filled at the start of the program and is not changed afterwards. It has mainly three purposes:

  • Sending the type specifications to a remote PC. Thereby, the PC can exchange and visualize data, even if it is not running the same version of the B-Human software.
  • Providing the specification of all types for logging. This allows to replay log files even if some of the type specifications have changed since the file was recorded.
  • Mapping between the values and names of enumeration constants when streaming data as text.

The specification of a type is recorded through a static or global function. For instance, the STREAMABLE used as an example in this section generated a function like this:

static void reg()
{
  REG_CLASS(Example);
  REG(int, anInt);
  REG(float, pi);
  REG(int[4], array);
  REG(Vector2f, aVector);
  REG(ABC, aLetter);
  REG(MotionRequest::Motion, motionId);
}

If the class that is registered has a streamable base class, it must be registered with another macro, because otherwise the type registry cannot know about this relation:

REG_CLASS_WITH_BASE(Example, SomeBaseClass);

To execute the function at the start of the program, it must be published:

PUBLISH(reg);

PUBLISH must be placed in a position of the code that is guaranteed to be linked. It publishes the registration function through its pure existence7, not by being run through the normal flow of execution. In most cases, PUBLISH can just be placed in the registration function itself. However, for templates, it must be written into a function that is actually referenced from somewhere, otherwise it will never be executed, because it is not part of the executable. The same is true for code that is linked in from a static library.

The TypeRegistry is optimized for performance. It mostly stores addresses of names rather than the whole strings. Type names are kept in the mangled form the compiler generated. This means the data is only valid for the current execution of the program. To store the information into a file or to transmit it to another computer, an instance of the class TypeInfo must be filled first, which can then be streamed. TypeInfo can represent specification data that is independent from the current executable, e.g., it can contain the type specifications read from an older log file. All type names are de-mangled in a platform-independent way.

Streamable Classes

A class is made streamable by deriving it from the class Streamable, implementing the abstract methods read(In& stream) and write(Out& stream) const, and adding a method that registers its type (cf. this section). For data types derived from Streamable, streaming operators are provided, meaning they may be used as any other data type with standard streaming operators implemented. The following two macros can be used to specify the data to stream in the methods read and write. Both macros expect that the parameter of both methods is actually called stream. The use of these macros must be identical in both methods:

  • STREAM_BASE(<class>) streams the base class. If present, it must be the first STREAM statement in the read and write methods.

  • STREAM(<member>) streams a member variable.

For instance, the STREAMABLE used as an example in this section generated a function like this:

void read(In& stream) override
{
  STREAM(anInt);
  STREAM(pi);
  STREAM(array);
  STREAM(aVector);
  STREAM(aLetter);
  STREAM(motionId);
}

void write(Out& stream) const override
{
  STREAM(anInt);
  STREAM(pi);
  STREAM(array);
  STREAM(aVector);
  STREAM(aLetter);
  STREAM(motionId);
}

In addition to these two macros there is also a version to be used in streaming operators, i.e. in operator<< and operator>>. These macros do not expect that the stream has a pre-defined name. Instead, the stream is passed as their first parameter:

template<typename T> Out& operator<<(Out& out, const Range<T>& range)
{
  STREAM_EXT(out, range.min);
  STREAM_EXT(out, range.max);
  return out;
}

Configuration Maps

Configuration maps introduce the ability to handle serialized data from files in an arbitrary order. The sequence of entries in the file does not have to match the order of the member variables in the C++ data structure that is filled with them. In contrast to most streams presented in this section, configuration maps do not contain a serialization of a data structure, but rather a hierarchical representation.

Since configuration maps can be read from and be written to files, there is a special syntax for such files. A file consists of an arbitrary number of pairs of keys and values, separated by an equality sign, and completed by a semicolon. Values can be lists (encased by square brackets), structured values (encased by curly brackets), or plain values. If a plain value does not contain any whitespaces, periods, semicolons, commas, or equality signs, it can be written without quotation marks, otherwise it has to be encased in double quotes. Configuration map files have to follow this grammar:

map     ::= record
record  ::= field ';' { field ';' }
field   ::= literal '=' ( literal | '{' record '}' | array )
array   ::= '[' element { ',' element } [ ',' ] ']'
element ::= literal | '{' record '}'
literal ::= '"' { anychar1 } '"' | { anychar2 }

anychar1 must escape doublequotes and the backslash with a backslash. anychar2 cannot contain whitespace and other characters used by the grammar. However, when such a configuration map is read, each literal must be a valid literal for the datatype of the variable it is read into. As in C++, comments can be denoted either by // for a single line or by /* ... */ for multiple lines. Here is an example:

// A record
defensiveGoaliePose = {
  rotation = 0;
  translation = {x = -4300; y = 0;};
};

/* An array of
 * three records
 */
kickoffTargets = [
  {x = 2100; y = 0;},
  {x = 1500; y = -1350;},
  {x = 1500; y = 1350;}
];

// Some individual values
outOfCenterCircleTolerance = 150.0;
ballCounterThreshold = 10;

Configuration maps can only be read or written through the streams derived from OutMap and InMap. Accordingly, they require an object of a streamable class to either parse the data in map format or to produce it. Here is an example of code that reads the file shown above:

STREAMABLE(KickOffInfo,
{,
  (Pose2f) defensiveGoaliePose,
  (std::vector<Vector2f>) kickoffTargets,
  (float) outOfCenterCircleTolerance,
  (int) ballCounterThreshold,
});

InMapFile stream("kickOffInfo.cfg");
KickOffInfo info;
if(stream.exists())
  stream >> info;

Enumerations

To support streaming, enumeration types should be defined using the macro ENUM defined in Src/Libs/Streaming/Enum.h rather than using the C++ enum keyword directly. The macro's first parameter is the name of the enumeration type. The second and the last parameter are reserved for curly brackets and are ignored. All other parameters are the elements of the enumeration type defined. It is not allowed to assign specific integer values to the elements of the enumeration type, with one exception: It is allowed to initialize an element with the symbolic value of the element that has immediately been defined before (see example below). The macro automatically registers the enumeration type and its elements (except for elements used to initialize other elements) in the TypeRegistry (cf. this section).

The macro also automatically defines a constant numOf<Typename>s, which reflects the number of elements in the enumeration type. Since the name of that constant has an added s at the end, enumeration type names should be singular. If the enumeration type name already ends with an s, it might be a good idea to define a constant outside the enumeration type that can be used instead, e.g. static constexpr unsigned char numOfClasses = numOfClasss for an enumeration type with the name Class.

The following example defines an enumeration type Letter with the "real" enumeration elements a, b, c, and d, a user-defined helper constant numOfLettersBeforeC, and an automatically defined helper constant numOfLetters. The numerical values of these elements are a = 0, b = 1, c = 2, d = 3, numOfLettersBeforeC = 2, numOfLetters = 4.

ENUM(Letter,
{,
  a,
  b,
  numOfLettersBeforeC,
  c = numOfLettersBeforeC,
  d,
});

Iterating over Enumerations

It is often necessary to enumerate all constants defined in an enumeration type. This can easily be done using the FOREACH_ENUM macro:

FOREACH_ENUM(Letter, letter, numOfLettersBeforeC)
{
  // do something with "letter", which is of type "Letter"
}

The third parameter is optional. If specified, it defines a different upper limit than the end of the enumeration.

Enumerations as Array Indices

In the B-Human code, enumerations are often used as indices for arrays, because this gives entries a name, but still allows to iterate over these entries. However, in configuration files and in the UI, it is hard to find specific entries in arrays, in particular if they have a larger number of elements. For instance, the arrays of all joint angles have 26 elements, making it hard to identify the angle of a specific joint. Therefore, a special macro (defined in Libs/Streaming/EnumIndexedArray.h) that is backed by a template class allows to define an array indexed by an enumeration type that solves this problem by streaming such an array as if it were a structure with a member variable for each of its elements named after the respective enumeration constant. Technically, such an array is derived from std::array and simply defines the methods read and write (cf. this section). For example, an array of all joint angles could be defined by using the enumeration Joints::Joint as index:

ENUM_INDEXED_ARRAY(Angle, Joints::Joint) jointAngles;

jointAngles still behaves like an array, i.e. it is derived from std::array<Angle, Joints::numOfJoints>, but when, e.g., written to a file using OutMapFile (cf. this section), it would appear differently:

headYaw = 0deg;
headPitch = 0deg;
lShoulderPitch = 0deg;
...

Functions

Not all data in a representation can reasonably be computed before it is used. This would lead to a certain overhead, because the data has to be computed in advance without knowing whether it will actually be required in the current situation. In addition, there is a lot of data that cannot be computed in advance, because its computation depends on external values. For instance, a path planner must know the target to which it has to plan a path before it can be executed. However, in many situations a path planner is not needed at all, because the motion is generated reactively.

Functions in representations allow modules that require the representation to execute code of the module that provided that representation at any time and they allow to pass parameters to that implementation. These modules can be switched to other implementations as all other B-Human modules can, giving a greater flexibility when improving the functionality of the system.

These functions are based on std::function from the C++ runtime library and the assigned implementations are usually lambda expressions. However, to make them more compliant with the general B-Human architecture, they differ from the normal standard implementation in their behavior if no value was assigned to them (their default behavior). Instead of throwing an exception as std::function would do when called, they simply do nothing. If they have a return value, they return the default value of the type returned or Zero() in case of Eigen types. However, there are some specialties to functions in representations:

  • Functions cannot be streamed in any way, although they must be part of a class that is derived from the class Streamable. This also means that a module cannot call a function of a representation the provider of which is executed in another thread. However, representations containing functions can be streamed, but the functions cannot be part of the data that is streamed, i.e. they must be ignored during streaming.
  • Representations containing functions are treated in a special way. They are always reset when their provider is switched. The reason is that the function object would otherwise still contain a reference to the module that originally provided the implementation, but the module does not exist anymore. In order to make this work with classes derived from classes containing functions, but not declaring functions themselves, the derived class has to contain the macro BASE_HAS_FUNCTION in its body.

For instance, the path planner provides its functionality through the representation PathPlanner:

STREAMABLE(PathPlanner,
{
  FUNCTION(MotionRequest(const Pose2f& target, const Pose2f& speed,
           bool excludePenaltyArea)) plan,
});

It assigns the function in its method update:

pathPlanner.plan = [this](const Pose2f& target, const Pose2f& speed,
                          bool excludePenaltyArea) -> MotionRequest
{ // ...
};

In the behavior control, the path planner is executed by this call:

theMotionRequest = thePathPlanner.plan(target, Pose2f(speed, speed, speed), avoidOwnPenaltyArea);

Communication

Three kinds of communication are implemented in the B-Human framework: inter-thread communication, debug communication, and team communication.

Inter-thread Communication

The representations sent back and forth between the threads (so-called shared representations) are automatically calculated by the ModuleGraphCreator based on the representations required by modules loaded in the respective thread but not provided by modules in the same thread. The directions in which they are sent are also automatically determined by the ModuleGraphCreator.

All inter-thread communication is triple-buffered. Thus, threads never block each other, because they never access the same memory blocks at the same time. In addition, a receiving thread always gets the most current version of a packet sent by another thread.

Message Queues

The debug communication, parts of the team communication, and logging are all based on the same technology: message queues. The class MessageQueue allows storing and transmitting a sequence of messages. Each message has a type (defined in Src/Libs/Streaming/MessageIDs.h) and a content. Each queue has a maximum size, which is defined in advance. On the robot, the amount of memory required is pre-allocated to avoid allocations during runtime. On the PC, the memory is allocated on demand, because several sets of robot threads can be instantiated at the same time, and the maximum size of the queues is rarely needed.

Since almost all data types are streamable (cf. this section), it is easy to store them in message queues. The class MessageQueue provides different write streams for different formats: messages that are stored through out.bin are formatted binary. The stream out.text formats data as text and out.textRaw as raw text. After all data of a message was streamed into a queue, the message must be finished with out.finishMessage(MessageID), giving it a message id, i.e. a type.

MessageQueue m;
m.setSize(1000); // can be omitted on PC
m.out.text << "Hello world!";
m.out.finishMessage(idText);

To declare a new message type, an id for the message must be added to the enumeration type MessageID in Src/Libs/Streaming/MessageIDs.h. The enumeration type has two sections: the first for representations that should be recorded in log files, and the second for infrastructure messages.

Messages are read from a queue through a message handler that is passed to the queue's method handleAllMessages(MessageHandler&). Such a handler must implement the method handleMessage(InMessage&) that is called for each message in the queue. It must be implemented in a way as the following example shows:

class MyClass : public MessageHandler
{
protected:
  bool handleMessage(InMessage& message)
  {
    switch(message.getMessageID())
    {
      default:
        return false;

      case idText:
      {
        std::string text;
        message.text >> text;
        return true;
      }
    }
  }
};

The handler has to return whether it handled the message or not. Messages are read from a MessageQueue via streams. Thereto, message.bin provides a binary stream, message.text a text stream, and message.config a text stream that skips comments.

Debug Communication

For debugging purposes, there is a communication infrastructure between the threads and the PC. This is accomplished by message queues. Each thread has two of them: theDebugSender and theDebugReceiver. The macro OUTPUT(<id>, <format>, <sequence>) defined in Src/Libs/Streaming/Output.h simplifies writing data to the outgoing debug message queue. id is a valid message id, format is text, bin, or textRaw, and sequence is a streamable expression, i.e. an expression that contains streamable objects, which – if more than one – are separated by the streaming operator <<.

OUTPUT(idText, text, "Could not load file " << filename << " from " << path);
OUTPUT(idCameraImage, bin, CameraImage());

For receiving debugging information from the PC, each thread also has a message handler, i.e. it implements the method handleMessage to distribute the data received.

The thread Debug manages the communication of the robot control program with the tools on the PC. For each of the other threads, it has a sender and a receiver for their debug message queues (cf. this section). Messages that arrive via Wi-Fi or Ethernet from the PC are stored in debugIn. The method Debug::handleMessage(InMessage&) distributes all messages in debugIn to the other threads. The messages received from Upper, Lower, Cognition and Motion are stored in debugOut. When a Wi-Fi or Ethernet connection is established, they are sent to the PC via TCP/IP.

The debug communication and the thread Debug are not available on the actual NAO when the software deployed was compiled in the configuration Release.

Team Communication

The purpose of the team communication is to send messages to the other robots in the team. These messages are always broadcasted via UDP, so all teammates can receive them. Sending and receiving team messages is done in the thread Cognition using the representations TeamData for handling received messages and BHumanMessageOutputGenerator for generating messages to send. These representations are both generated and filled with their functionality by the module TeamMessageHandler.

The format of team messages is given by the SPLStandardMessage, which consists of a standardized and a non-standard part. The custom data part is subdivided into two sections: The first one is the BHumanStandardMessage, which contains many representations in a compressed version, e.g. timestamps are quantized and made relative to the timestamp when the message is sent to use fewer bits and some floating point numbers are quantized, too. The other section is the BHumanArbitraryMessage which embeds a message queue that is used especially for variable-length representations that may not fit into the message at all.

In order to be transmitted by the TeamMessageHandler, a representation has to extend the struct BHumanMessageParticle and provide implementations for reading and writing its data to or from a message. If a representation is only supposed to be included in the message queue and does not fill in any standardized fields of the SPLStandardMessage, it can also just extend the struct PureBHumanArbitraryMessageParticle. The representation has to be added in three locations: the handleMessage method of the Teammate struct and the generateMessage and parseMessageIntoBMate methods of the TeamMessageHandler.

According to the rules, team communication packets are only broadcasted once every second. The BHumanMessageOutputGenerator contains a flag set by the TeamMessageHandler that states whether a team communication packet will be sent out in the current frame or not. The TeamMessageHandler also implements the network time protocol (NTP) and translates time stamps contained in the messages it receives into the local time of the robot.

Debugging Support

Debugging mechanisms are an integral part of the B-Human framework. They are all based on the debug message queues already described in this section. These mechanisms are available in all project configurations except for Release on the actual NAO.

Debug Requests

Debug requests are used to enable and disable parts of the source code. They can be seen as runtime switches for debugging.

The debug requests can be used to trigger certain debug messages to be sent as well as to switch on certain parts of algorithms. They can be sent using the SimRobot software when connected to a NAO (cf. command dr in this section). The following macros ease the use of the mechanism as well as hide its implementation details:

  • DEBUG_RESPONSE(<id>) executes the following statement or block if the debug request with the name id is enabled.
  • DEBUG_RESPONSE_ONCE(<id>) executes the following statement or block once when the debug request with the name id is enabled.
  • DEBUG_RESPONSE_NOT(<id>) executes the following statement or block if the debug request with the name id is not enabled.

These macros can be used anywhere in the source code, allowing for easy debugging. For example:

DEBUG_RESPONSE("test") test();

This statement calls the method test() if the debug request with the identifier "test" is enabled. Debug requests are commonly used to send messages on request as the following example shows:

DEBUG_RESPONSE("sayHello") OUTPUT(idText, text, "Hello");

This statement sends the text "Hello" if the debug request with the name "sayHello" is activated. Please note that only those debug requests are usable that are in the current path of execution. This means that only debug requests in those modules can be activated that are currently executed. To determine which debug requests are currently available, a method called polling is employed. It asks each debug response to report the name of the debug request that would activate it. This information is collected and sent to the PC (cf. command poll in this section).

Debug Images

Debug images are used for low level visualization of image processing debug data. They can either be displayed as background image of an image view (cf. this section) or in a color space view (cf. this section). Each debug image consists of pixels in one of the formats RGB, BGRA, YUYV, YUV, Grayscale, Colored, Hue, Binary, and Edge2, which are defined in Src/Libs/ImageProcessing/PixelTypes.h. In the RGB, RGBA, BGRA, YUYV, and YUV formats, pixels are made up of four unsigned byte values, each representing one of the color channels of the format. The YUYV format is special as one word of the debug image describes two image pixels by specifying two luminance values, but only one value per U and V channel. Thus, it resembles the YUV422 format of the images supplied by the NAO's cameras. Debug images in the Grayscale format only contain a single channel of unsigned byte values describing the luminance of the associated pixel. The Colored, Hue, and Binary formats consist of only one channel, too. Values in the Colored channel are entries of the PixelTypes::Color enumeration, which contains identifiers for the color classes used for image processing. The Edge2 format has one channel each for the intensity in x and y direction.

Debug images are supposed to be declared as instances of the template class Image, instantiated with one of the pixel formats named above, or the CameraImage class, which is only used for images provided by the NAO's cameras. The following macros are used to transfer debug images to a connected PC.

  • SEND_DEBUG_IMAGE(<id>, <image>, [<method>]) sends the debug image to the PC. The identifier given to this macro is the name by which the image can be requested. The optional parameter is a pixel type that allows to apply a special drawing method for the image.
  • COMPLEX_IMAGE(<id>) only executes the following statement if the creation of a certain debug image is requested. For debug images that require complex instructions to paint, it can significantly improve the performance to encapsulate the drawing instructions in this macro (and maybe additionally in a separate method).

These macros can be used anywhere in the source code, allowing for easy creation of debug images. For example:

class Test
{
private:
  Image<GrayscaledPixel> testImage;

public:
  void doSomething()
  {
    // [...]
    COMPLEX_IMAGE("test") draw();
    // [...]
  }

  void draw()
  {
    testImage.setResolution(640, 480);
    memset(testImage[0], 0x7F, testImage.width * testImage.height);
    SEND_DEBUG_IMAGE("test", testImage);
  }
};

The example calls the draw() method if the "test" image was requested, which then initializes a grayscale debug image, paints it gray, and sends it to the PC.

Debug Drawings

Debug drawings provide a virtual 2-D drawing canvas and a number of drawing primitives, as well as mechanisms for requesting, sending, and drawing these primitives to the screen of the PC. In contrast to debug images, which are raster-based, debug drawings are vector-based, i.e., they store drawing instructions instead of a rasterized image. Each drawing has an identifier and an associated type that enables the application on the PC to render the drawing to the right kind of drawing canvas. In the B-Human system, two standard drawing canvases are provided, called "drawingOnImage" and "drawingOnField". This refers to the two standard applications of debug drawings, namely drawing in the system of coordinates of an image and drawing in the system of coordinates of the field. Hence, all debug drawings of type "drawingOnImage" can be displayed in an image view (cf. this section) and all drawings of type "drawingOnField" can be rendered into a field view (cf. this section).

The creation of debug drawings is encapsulated in a number of macros in Src/Libs/Debugging/DebugDrawings.h. Most of the drawing macros have parameters such as pen style, fill style, or color. Available pen styles (solidPen, dashedPen, dottedPen, and noPen) and fill styles (solidBrush and noBrush) are part of the namespace Drawings. Colors can be specified as ColorRGBA. The class also contains a number of predefined colors such as ColorRGBA::red. A few examples for drawing macros are:

  • DECLARE_DEBUG_DRAWING(<id>, <type>) declares a debug drawing with the specified id and type.
  • COMPLEX_DRAWING(<id>) only executes the following statement or block if the creation of a certain debug drawing is requested. This can significantly improve the performance when a debug drawing is not requested, because for each drawing instruction it has to be tested whether it is currently required or not. By encapsulating them in this macro (and maybe in addition in a separate method), only a single test is required. However, the macro DECLARE_DEBUG_DRAWING must be placed outside of COMPLEX_DRAWING.
  • DEBUG_DRAWING(<id>, <type>) is a combination of DECLARE_DEBUG_DRAWING and COMPLEX_DRAWING. It declares a debug drawing with the specified id and type. It also executes the following statement or block if the debug drawing is requested.
  • CIRCLE(<id>, <x>, <y>, <radius>, <penWidth>, <penStyle>, <penColor>, <fillStyle>, <fillColor>) draws a circle with the specified radius, pen width, pen style, pen color, fill style, and fill color at the coordinates (x, y) on the virtual drawing canvas.
  • LINE(<id>, <x1>, <y1>, <x2>, <y2>, <penWidth>, <penStyle>, <penColor>) draws a line with the pen color, width, and style from the point (x1, y1) to the point (x2, y2) on the virtual drawing canvas.
  • DOT(<id>, <x>, <y>, <penColor>, <fillColor>) draws a dot with the pen color and fill color at the coordinates (x,y) on the virtual drawing canvas. There also exist two macros MID_DOT and LARGE_DOT with the same parameters that draw dots of larger size.
  • DRAWTEXT(<id>, <x>, <y>, <fontSize>, <color>, <text>) writes a text with a font size in a color to a virtual drawing canvas. The left end of the baseline of the text will be at coordinates (x, y).
  • TIP(<id>, <x>, <y>, <radius>, <text>) adds a tool tip to the drawing that will pop up when the mouse cursor is closer to the coordinates (x, y) than the given radius.
  • ORIGIN(<id>, <x>, <y>, <angle>) changes the system of coordinates. The new origin will be at (x, y) and the system of coordinates will be rotated by angle (given in radians). All further drawing instructions, even in other debug drawings that are rendered afterwards in the same view, will be relative to the new system of coordinates, until the next origin is set. The origin itself is always absolute, i.e. a new origin is not relative to the previous one.
  • THREAD(<id>, <thread>) defines as part of which thread a drawing should be managed. The default is the thread that contains the drawing. However, if a drawing in a thread belongs to data that was received from another thread, it should be linked to that thread by using this macro. This is particularly true for image drawings that always have to be linked to one of the two image processing threads, i.e. Upper and Lower, but it can also be necessary for field drawings.

These macros can be used wherever statements are allowed in the source code. For example:

DECLARE_DEBUG_DRAWING("test", "drawingOnField");
CIRCLE("test", 0, 0, 1000, 10, Drawings::solidPen, ColorRGBA::blue,
       Drawings::solidBrush, ColorRGBA(0, 0, 255, 128));

This example initializes a drawing called test of type drawingOnField that draws a blue circle with a solid border and a semi-transparent inner area.

3-D Debug Drawings

In addition to the aforementioned two-dimensional debug drawings, there is a second set of macros in Src/Libs/Debugging/DebugDrawings3D.h, which provides the ability to create three-dimensional debug drawings.

3-D debug drawings can be declared with the macro DECLARE_DEBUG_DRAWING3D(<id>, <type>). The id can then be used to add three dimensional shapes to this drawing. type defines the coordinate system in which the drawing is displayed. It can be set to "field", "robot", "camera", or any named part of the robot model in the scene description. Note that drawings directly attached to hinges will be drawn relative to the base of the hinge, not relative to the moving part. Drawings of the type "field" are drawn relative to the center of the field, whereas drawings of the type "robot" are drawn relative to the origin of the robot, i.e. the middle between the two hip joints. It is often used as reference frame for 3-D debug drawings in the current code. The type "camera" is an alias for the camera of the perception thread from which the drawing is generated.

The parameters of macros adding shapes to a 3-D debug drawing start with the id of the drawing this shape will be added to, followed, e.g., by the coordinates defining a set of reference points (such as corners of a rectangle), and finally the drawing color. Some shapes also have other parameters such as the thickness of a line. Here are a few examples for shapes that can be used in 3-D debug drawings:

  • LINE3D(<id>, <fromX>, <fromY>, <fromZ>, <toX>, <toY>, <toZ>, <size>, <color> ) draws a line between the given points.
  • QUAD3D(<id>, <corner1>, <corner2>, <corner3>, <corner4>, <color>) draws a quadrangle with its four corner points given as 3-D vectors and specified color.
  • SPHERE3D(<id>, <x>, <y>, <z>, <radius>, <color>) draws a sphere with specified radius and color at the coordinates (x, y, z).
  • COORDINATES3D(<id>, <length>, <width>) draws the axes of the coordinate system with specified length and width into positive direction.
  • COMPLEX_DRAWING3D(<id>) only executes the following statement or block if the creation of the debug drawing is requested (similar to COMPLEX_DRAWING(<id>) for 2-D drawings).
  • DEBUG_DRAWING3D(<id>, <type>) is a combination of DECLARE_DEBUG_DRAWING3D and COMPLEX_DRAWING3D. It declares a debug drawing with the specified id and type. It also executes the following statement or block if the debug drawing is requested.

The header file furthermore defines some macros to scale, rotate, and translate an entire 3-D debug drawing:

  • SCALE3D(<id>, <x>, <y>, <z>) scales all drawing elements by given factors for x, y, and z axis.
  • ROTATE3D(<id>, <x>, <y>, <z>) rotates the drawing counterclockwise around the three axes by given radians.
  • TRANSLATE3D(<id>, <x>, <y>, <z>) translates the drawing according to the given coordinates.

An example for 3-D debug drawings (analogous to the example for regular 2-D debug drawings):

DECLARE_DEBUG_DRAWING3D("test3D", "field");
SPHERE3D("test3D", 0, 0, 250, 75, ColorRGBA::blue);

This example initializes a 3-D debug drawing called "test3D", which draws a blue sphere. Because the drawing is of type "field" and the origin of the field coordinate system is located in the center of the field, the sphere's center will appear 250 mm above the center point.

In order to add a drawing to the scene, a debug request (cf. this section) for a 3-D debug drawing with the ID of the desired drawing prefixed by debugDrawing3d: must be sent. For example, to see the drawing generated by the code above, the command would be dr debugDrawing3d:test3D. The rendering of debug drawings can be configured for individual scene views by right-clicking on the view and selecting the desired shading and occlusion mode in the "Drawings Rendering" submenu.

Plots

The macro PLOT(<id>, <number>) allows plotting data over time. The plot view (cf. this section) will keep a history of predefined size of the values sent by the macro PLOT and plot them in different colors. Hence, the previous development of certain values can be observed as a time series. Each plot has an identifier that is used to distinguish the different plots from each other. A plot view can be created with the console commands vp and vpd (cf. this section).

For example, the following code statement plots the measurements of the gyro for the pitch axis in degrees. It should be placed in a part of the code that is executed regularly, e.g. inside the update method of a module.

PLOT("gyroY", theInertialSensorData.gyro.y().toDegrees());

The macro DECLARE_PLOT(<id>) allows using the PLOT(<id>, <number>) macro within a part of code that is not regularly executed as long as the DECLARE_PLOT(<id>) macro is executed regularly.

Modify

The macro MODIFY(<id>, <object>) allows inspecting and modifying data on the actual robot during runtime. Every streamable data type (cf. this section) can be manipulated and read, because its inner structure was gathered in the TypeRegistry (cf. this section). This allows generic manipulation of runtime data using the console commands get and set (cf. this section). The first parameter of MODIFY specifies the identifier that is used to refer to the object from the PC, the second parameter is the object to be manipulated itself. When an object is modified using the console command set, it will be overridden each time the MODIFY macro is executed.

int i = 3;
MODIFY("i", i);
MotionRequest m;
MODIFY("representation:MotionRequest", m);

The macro PROVIDES of the module framework (cf. this section) includes the MODIFY macro for the representation provided. For instance, if a representation Foo is provided by PROVIDES(Foo), it is modifiable under the name representation:Foo. If a representation provided should not be modifiable, e.g., because its serialization does not register all member variables, it must be provided using PROVIDES_WITHOUT_MODIFY.

Stopwatches

Stopwatches allow the measurement of the execution time of parts of the code. The macro STOPWATCH(<id>) (declared in Src/Libs/Debugging/Stopwatch.h) measures the runtime of the statement or block that follows. id is a string used to identify the time measurement. To activate the time measurement of all stopwatches, the debug request dr timing has to be sent. The measured times can be seen in a timing view (cf. this section). By default, a stopwatch is already defined for each representation that is currently provided, and for the overall execution of all modules in each thread.

An example to measure the runtime of a method called myCode:

STOPWATCH("myCode") myCode();

Using the STOPWATCH macro also adds a plot with the stopwatch identifier prefixed by stopwatch:.

Logging

The B-Human framework offers sophisticated logging functionality that can be used to log the values of selected representations while the robot is playing. There are two different ways of logging data: online logging and remote logging.

Online Logging

The online logging feature can be used to log data directly on the robot during regular games. It is implemented as part of the robot control program and is designed to log representations in real-time. Log files are written into the directory /home/nao/logs or – if available – into /media/usb/logs, i.e. a USB flash drive on which the directory called logs will be created automatically. To use a USB flash drive for logging, it must already have been plugged in when the B-Human software is started.

Online logging starts as soon as the robot enters the ready state and stops upon entering the finished state. The name of the log file consists of the names of the head and body of the robot as well as its player number, the scenario, and the location. If connected to the GameController, the name of the opponent team and the half are also added to the log file name. Otherwise, "Testing" is used instead. If a log file with the given name already exists, a number is added that is incremented for each duplicate.

To retain the real-time properties of the threads, the writing of the file is done in a separate thread without real-time scheduling. This thread uses the remaining processor time that is not used by one of the real-time parts of the system. Communication with this thread is realized using a very large ring buffer (usually around 2.4 GB). Each element of the ring buffer represents one frame of data. If a buffer is full, the current frame cannot be logged.

Due to the limited buffer size, the logging of very large representations such as the CameraImage is not possible. It would cause a buffer overflow within less than two minutes rendering the resulting log file unusable. However, without images a log file is nearly useless, therefore loggable JPEG images are generated and used instead.

In addition to the logged representations, online log files also contain timing data, which can be seen using the timing view (cf. this section).

Configuring the Online Logger

The online logger can be configured by changing the following values in the file Config/Scenarios/<scenario>/logger.cfg:

  • enabled: The logger is enabled if this is true. Note that it is not possible to enable the logger inside the simulator.
  • path: Log files will be stored in this path on the NAO.
  • numOfBuffers: The number of buffers allocated.
  • sizeOfBuffer: The size of each buffer in bytes. Note that numOfBuffers × sizeOfBuffer is always allocated if the logger is enabled.
  • writePriority: Priority of the logging thread. Priorities greater than zero use the real-time scheduler, zero uses the normal scheduler, and negative values (-1 and -2) use idle priorities.
  • minFreeDriveSpace: The minimum amount of disk space that should always be left on the target device in MB. If the free space falls below this value the logger will cease operation.
  • representationsPerThread: List of all representations that should be logged per thread.

Remote Logging

Online logging provides the maximum possible amount of debugging information that can be collected without breaking the real-time capability. However in some situations one is interested in high precision data (i.e. full resolution images) and does not care about real-time. The remote logging feature provides this kind of log files. It utilizes the debugging connection (cf. this section) to a remote computer and logs all requested representations on that computer. This way the heavy lifting is outsourced to a computer with much more processing power and a bigger hard drive. However sending large representations over the network severely impacts the NAO's performance and can result in a loss of the real-time capability.

To reduce the network load, it is usually a good idea to limit the number of representations to the ones that are really needed for the task at hand. The follow listing shows the commands that need to be entered into SimRobot to record a minimal vision log. Alternatively, call Includes/VisionLog can be executed.

dr off
dr representation:BodyContour
dr representation:CameraImage
dr representation:CameraInfo
dr representation:CameraMatrix
dr representation:ImageCoordinateSystem
dr representation:OdometryData
log start
log stop
log save <filename>

Log File Format

In general, log files consist of serialized message queues (cf. this section). Each log file consists of up to three chunks. Each chunk is prefixed by a single byte defining its type. The enum LogFileFormat in Src/Libs/Framework/LoggingTools.h defines these chunk identifiers in the namespace LoggingTools that have to appear in the given sequence in the log file if they appear at all:

  • logFileMessageIDs: This chunk contains a string representation of the MessageIDs (cf. this section) stored in this log file. It is used to convert the MessageIDs from the log file to the ones defined in the version of SimRobot (cf. this chaper) that is replaying the log file. Thereby, log files still remain usable after the enumeration MessageID was changed.
  • logFileTypeInfo: This chunk contains the specification of all datatypes used in the log file. It is used to convert the logged data to the specifications that are defined in the version of SimRobot that is replaying the log file. If the specification changed, messages will appear in SimRobot's console about the representations that are converted. Please note that a conversion is only possible for representations the specification of which is fully registered. This is not the case for representations that use read and write methods to serialize their data, e.g. CameraImage, JPEGImage, and Thumbnail. Therefore, such representations cannot be converted and will likely crash SimRobot when trying to replay log files containing them after they were changed.
  • logFileUncompressed: Uncompressed log data is stored in a single MessageQueue. Its serialized form starts with an eight byte header containing two values. These are the size used by the log followed by the number of messages in the queue. Those values can also be set to -1 indicating that the size and number of messages is unknown. In this case the amount will be counted when reading the log file. The header is followed by several frames.A frame consists of multiple log messages enclosed by an idFrameBegin and idFrameFinished message. Every log message consists of its id, its size and its payload. This kind of chunk is created by remote logging.
  • logFileCompressed: This chunk contains a sequence of compressed MessageQueues. Each queue contains a single frame worth of log data and is compressed using Google's Snappy8 compression. This chunk can still be read, but it is not created anymore, because the log files mainly consist of JPEG images that cannot be compressed any further.

The format of a chunk of the type logFileUncompressed

The figure is based on the one by Trocha, 20109. The first two chunks are optional. Each log file must contain one of the latter two chunks, which is also the last chunk in the file.

Replaying Log Files

Log files are replayed using the simulator. A special module, the LogDataProvider, automatically provides all representations that are present in the log file. All other representations are provided by their usual providers, which are simulated. This way log file data can be used to test and evolve existing modules without access to an actual robot. If the name of a log file follows the naming convention used by the online logger, the head name, body name, player number, scenario, and location will be set to the ones given in the log file name before the log file is replayed. Thereby, modules will read the matching versions of their configuration files. The SimRobot scene ReplayRobot.ros2 can be used to load the log file. In addition to loading the log file this scene also provides several keyboard shortcuts for navigation inside the log file. However, the most convenient way to control log file playback is the log player view (cf. this section). If you want to replay several log files at the same time, simply create a file Config/Scenes/Includes/replay.con and add several sl statements to it. If you want to replay a folder or an entire folder structure use the sml command (cf. this section). The data of each log file will be fed into a separate instance of the B-Human code.

Annotations

To further enhance the usage of log files, there is the possibility for our modules to annotate individual frames of a log file with important information. This is, for example, information about a change of game state, the execution of a kick, or other information that may help us to debug our code. Thereby, when replaying the log file, we may consult a list of those annotations to see whether specific events actually did happen during the game. In addition, if an annotation was recorded, we are able to directly jump to the corresponding frame of the log file to review the emergence and effects of the event without having to search through the whole log file.

This feature is accessed via the ANNOTATION-Macro. An example is given below:

#include "Debugging/Annotation.h"
...
ANNOTATION("GroundContactDetector", "Lost GroundContact");

It is advised to be careful to not send an annotation in each frame because this will clutter the log file. When using annotations inside the behavior, the skill Annotation should be used to make sure annotations are not sent multiple times.

The annotations recorded can be displayed while replaying the log file in the annotation view (cf. this section).


  1. Thomas Röfer and Tim Laue. On B-Human's code releases in the Standard Platform League – software architecture and impact. In Sven Behnke, Manuela Veloso, Arnoud Visser, and Rong Xiong, editors, RoboCup 2013: Robot World Cup XVII, volume 8371 of Lecture Notes in Artificial Intelligence, pages 648–656. Springer, 2014. 

  2. Thomas Röfer, Jörg Brose, Daniel Göhring, Matthias Jüngel, Tim Laue, and Max Risler. GermanTeam 2007. In Ubbo Visser, Fernando Ribeiro, Takeshi Ohashi, and Frank Dellaert, editors, RoboCup 2007: Robot Soccer World Cup XI Preproceedings, Atlanta, GA, USA, 2007. RoboCup Federation. 

  3. V. Jagannathan, Rajendra Dodhiawala, and Lawrence S. Baum, editors. Blackboard Architectures and Applications. Academic Press, Boston, 1989. 

  4. Actually, if a module name begins with more than one uppercase letter, all initial uppercase letters but the last one are transformed to lowercase, e.g. the module LEDHandler would read the file ledHandler.cfg if it would read its parameters from a file. 

  5. Meaning, the default access for member variables is public

  6. Currently, the macros support up to 119 entries. 

  7. Technically, a template is instantiated that adds itself to a linked list of all registration functions at the start of the program. 

  8. Google. Snappy – a fast compressor/decompressor. Online: https://google.github.io/snappy/, September 2019. 

  9. Max Trocha. Werkzeug zur taktischen Auswertung von Spielsituationen. Bachelor's thesis, University of Bremen, 2010.