spinnaker_synchronized_camera_driver

ROS2 driver for synchronized flir cameras using the Spinnaker SDK

README

Spinnaker ROS2 Synchronized Camera Driver

This package provides a driver specifically for hardware synchronized cameras made by Teledyne/FLIR that work with the Spinnaker SDK. The main difference to running several instances of the unsynchronized camera driver is that this driver will assign identical header time stamps to frames that have been generated by the same synchronization pulse.

NOTE: This driver is not written or supported by FLIR.

Tested Configurations

The following cameras have been used with this driver:

  • Blackfly S (USB3)

  • Blackfly (GigE)

  • ROS2 Galactic under Ubuntu 20.04 LTS (no longer actively tested)

  • ROS2 Humble/Iron/Rolling under Ubuntu 22.04 LTS

  • Spinnaker 3.1.0.79 (other versions may work as well but this is what the continuous integration builds are using)

How to install

The installation is analogous to the one for the unsynchronized spinnaker camera driver.

How to use

It is recommended to first get the cameras working with SpinView, then with the single-camera driver individually before proceeding with the synchronized setup.

The synchronized camera server works by instantiating a number of individual camera servers and exposure controllers. Each camera server is fully configurable with all parameters available as listed in the spinnaker camera driver. Likewise each exposure controller has its own parameter set.

There are two types of controllers: master controllers and followers. The master controllers regulate the brightness of the camera they are controlling, while follower controllers set the exposure parameters of the camera they are controlling based on the exposure parameters of the master controller they follow.

Since there are many parameters involved the setup can be tricky. It is recommended to start from the follower_example.launch.py example when using a stereo camera for e.g. VIO, or the master_example.launch.py when running cameras with individual exposure control.

Published:

  • ~/<camera_name>/image_raw: the synchronized camera image

  • ~/<camera_name>/camera_info: the synchronized camera calibration messages

  • ~/<camera_name>/meta: synchronized meta data like exposure time and gain

  • cameras (list of strings): names of the cameras. Default: empty list.

  • exposure_controllers (list of strings): names of the exposure controllers. List must be of same length as the list of camera names.

For a list of all parameters, see the spinnaker camera driver. The parameters are exposed under <camera_name>.parameter.

  • Master exposure controller:

    • brightness target (int): average image brightness to accomplish. Value range is [0..255]. Default: 120.

    • brightness_tolerance (int): how much actual brightness can deviate for brightness_target before the control parameters are updated. Default: 5.

    • exposure_parameter (string): Name of the ROS parameter under which exposure time is accessible for the single camera driver. This must match the ros parameter name associated with the Spinnaker node that controls exposure time, which is set in the camera .yaml config file. Default: exposure_time.

    • gain_parameter (string): Name of the ROS parameter controlling camera gain. See camera .yaml config file. Default: gain.

    • gain_priority (boolean): Gain priority means: If image is too bright, first try reducing the gain, and only update exposure time if gain is zero. If image is too dark, first try increasing the exposure time, and only increase gain if maximum exposure time has been reached. Time priority means: if image is too bright, first try reducing the exposure time, and only when min_exposure_time has been reached, reduce the gain. If image is too dark and gain is below max_gain, increase the gain, otherwise increase the exposure time. Default: false (i.e. use Time priority).

    • max_exposure_time (int): Maximum exposure time (in microseconds). Default: 1000.

    • max_frames_skip (int): It sometimes takes a few frames before a change commanded to exposure time or gain will actually be executed by the camera, and the frame meta data will indicate the change. After commanding a change of exposure parameters, the exposure controller will wait for at most max_frames_skip before it gives up and sends a new command to change exposure parameters (gain or time). Default: 10.

    • max_gain (float): Maximum gain (in db). Default: 10.0

    • min_exposure_time (int): Minimum exposure time (in microseconds). This is not a hard limit, but has the following function: if the brightness needs to be reduced and the exposure time would fall below min_exposure_time, then the gain (if greater than zero) is reduced first. Only if the gain is zero and the image is still too bright will the exposure time be reduced below min_exposure_time. Default: 1us

  • Follower exposure controller:

    • exposure_parameter (string): see Master exposure controller.

    • gain_parameter (string): see Master exposure controller.

    • master (string): the name of the master controller to use

    • max_frames_skip (int): see Master exposure controller.

The driver comes with several example launch files that need to be modified for your purposes (update serial numbers etc). The follower_example.launch.py can be used as template for stereo cameras, the master_example.launch.py for situations where each camera should run their own exposure control.

ros2 launch spinnaker_synchronized_camera_driver follower_example.launch.py

Carefully examine the camera parameters that are set in the launch file, in particular the following ones:

  • compute_brightness must be true for any camera governed by a master controller.

  • exposure_auto must be off (disable the individual camera controller).

  • chunk_mode_active must be true, and chunk exposure time, gain and frame id must be enabled

For the special case where one camera (primary) triggers the sync pulse of a secondary camera, see the example launch file primary_secondary.launch.py. If you don’t use Blackfly S cameras you will have to adjust the digital I/O parameters.

Known issues

See the caveats for the spinnaker camera driver.

How to contribute

Bug fixes and config files for new cameras are greatly appreciated. Before submitting a pull request, run this to see if your commit passes some basic lint tests:

colcon test --packages-select spinnaker_synchronized_camera_driver && colcon test-result --verbose

License

This software is issued under the Apache License Version 2.0.