Catkin Package

Ref : http://wiki.ros.org/ROS/Tutorials/catkin/CreatingPackage

We are focusing here only in python packages. We will use the “catkin way of doing things” described here, as a reference when discussing improvements brought with catkin_pip.

Package Structure

This is a generic example inspired by the pure catkin package from the Pyros dependencies : pyros-test

Files hierarchy

The source file hierarchy is:

.
├── CHANGELOG.rst              # A changelog, usually generated from git commit with catkin_generate_changelog
├── CMakeLists.txt             # A CMakeLists, using Catkin macros
├── doc
│   ├── changelog_link.rst     # Documentation link to existing changelog
│   ├── conf.py                # Sphinx doc configuration
│   ├── mydocs.rst             # Documentation for Sphinx
│   └── readme_link.rst        # Documentation link to existing readme
├── nodes
│   └── node1.py               # python script launching a ROS node
├── scripts
│   └── script1.py             # python utility script
├── package.xml                # package.xml required by catkin
├── README.rst
├── setup.py                   # setup.py using distutils and catkin_pkg extension
├── src
│   └── mypkg                  # mypkg python package
│       ├── module1.py
│       └── __init__.py
├── msg
│   └── mymessage.msg          # ROS message definition
└── srv
    └── myservice.srv          # ROS service definition

  • package.xml looks like

    <?xml version="1.0"?>
<package format="2">
  <name>mypkg</name>
  <version>0.0.4</version>
  <description>
    MyPkg description
  </description>

  <license>BSD</license>

  <url type="repository">https://github.com/author/mypkg</url>
  <url type="bugtracker">https://github.com/author/mypkg/issues</url>

  <author email="author@gmail.com">Author</author>
  <maintainer email="maintainer@gmail.com">Maintainer</maintainer>

  <buildtool_depend version_gte="0.6.18">catkin</buildtool_depend>

  <depend version_gte="1.11.19">rospy</depend>
  <depend version_gte="0.5.10">std_msgs</depend>

  <build_depend version_gte="0.2.10">message_generation</build_depend>
  <build_depend version_gte="0.10.0">roslint</build_depend>

  <exec_depend version_gte="0.4.12">message_runtime</exec_depend>

  <!-- these dependencies are only for testing -->
  <test_depend version_gte="1.11.19">rostest</test_depend>
  <test_depend version_gte="1.11.12">rosunit</test_depend>

  <!-- documentation dependencies -->
  <doc_depend version_gte="0.2.10">python-catkin-pkg</doc_depend>

</package>

  • CMakeLists.txt looks like

    cmake_minimum_required(VERSION 2.8.3)
project(mypkg)

# Minimal Python module setup
find_package(catkin REQUIRED COMPONENTS
    roslint
    rospy
    std_msgs
    message_generation
)

catkin_python_setup()

add_message_files(DIRECTORY msg
    FILES
    mymessage.msg
)
add_service_files(DIRECTORY srv
    FILES
    myservice.srv
)
generate_messages(DEPENDENCIES std_msgs)

catkin_package( CATKIN_DEPENDS message_runtime std_msgs)

install(
    PROGRAMS
        nodes/node1.py
    DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
    )

install(
    PROGRAMS
        scripts/script1.py
    DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
    )

# Lint Python modules
file(GLOB_RECURSE ${PROJECT_NAME}_PY_SRCS
     RELATIVE ${PROJECT_SOURCE_DIR} src/pyros_test/*.py)
roslint_python(${${PROJECT_NAME}_PY_SRCS})

  • setup.py looks like:

    from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

# ROS PACKAGING
# using distutils : https://docs.python.org/2/distutils
# fetch values from package.xml
setup_args = generate_distutils_setup(
    packages=[
        'mypkg',
    ],
    package_dir={
        'mypkg': 'src/mypkg',
    }
)
setup(**setup_args)

Dependencies

Dependencies are expressed via rosdep keys in the package.xml file.

Note these rosdep keys can refer to pip packages (check the rosdistro repository, you will find pip keys) but AFAIK:

  • there are no guarantees that these pip packages will play nice along the rest of your ROS distro.
  • there are little information on how rosdep handle pip dependencies (version requirements ?)
  • there is no clear visibility on rosdep pip support in the long term.

If you think I am mistaken please open an issue on catkin_pip repository, and share the information you have regarding these topics.

Package Development Workflow

This is a description of the generic ROS catkin workflow to retrieve, develop, build, test and release a catkin-based package. We will use pyros-test project as an example. TODO : travis check these with doctest + running these in isolation in container

  • Retrieve the project:

    $ mkdir -p catkin_ws/src
$ cd catkin_ws/src/
$ wstool init
Writing /home/alexv/doctest/catkin_ws/src/.rosinstall

update complete.

$ wstool set pyros-test https://github.com/asmodehn/pyros-test.git --git

     Add new elements:
  pyros-test        git  https://github.com/asmodehn/pyros-test.git

Continue: (y)es, (n)o: y
Overwriting /home/alexv/doctest/catkin_ws/src/.rosinstall
Config changed, remember to run 'wstool update pyros-test' to update the folder from git

$ wstool update pyros-test
[pyros-test] Fetching https://github.com/asmodehn/pyros-test.git (version None) to /home/alexv/doctest/catkin_ws/src/pyros-test
Cloning into '/home/alexv/doctest/catkin_ws/src/pyros-test'...
remote: Counting objects: 87, done.
remote: Total 87 (delta 0), reused 0 (delta 0), pack-reused 87
Unpacking objects: 100% (87/87), done.
Checking connectivity... done.
[pyros-test] Done.

  • Source your ROS environment:

    $ source /opt/ros/indigo/setup.bash

  • Build with catkin_make:

    $ catkin_make
Base path: /home/alexv/doctest/catkin_ws
Source space: /home/alexv/doctest/catkin_ws/src
Build space: /home/alexv/doctest/catkin_ws/build
Devel space: /home/alexv/doctest/catkin_ws/devel
Install space: /home/alexv/doctest/catkin_ws/install
Creating symlink "/home/alexv/doctest/catkin_ws/src/CMakeLists.txt" pointing to "/opt/ros/indigo/share/catkin/cmake/toplevel.cmake"
####
#### Running command: "cmake /home/alexv/doctest/catkin_ws/src -DCATKIN_DEVEL_PREFIX=/home/alexv/doctest/catkin_ws/devel -DCMAKE_INSTALL_PREFIX=/home/alexv/doctest/catkin_ws/install -G Unix Makefiles" in "/home/alexv/doctest/catkin_ws/build"
####
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Using CATKIN_DEVEL_PREFIX: /home/alexv/doctest/catkin_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/indigo
-- This workspace overlays: /opt/ros/indigo
-- Found PythonInterp: /usr/bin/python (found version "2.7.6")
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/alexv/doctest/catkin_ws/build/test_results
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.6.18
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 1 packages in topological order:
-- ~~  - pyros_test
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'pyros_test'
-- ==> add_subdirectory(pyros-test)
-- Using these message generators: gencpp;genlisp;genpy
-- pyros_test: 0 messages, 1 services
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alexv/doctest/catkin_ws/build
####
#### Running command: "make -j8 -l8" in "/home/alexv/doctest/catkin_ws/build"
####
Scanning dependencies of target std_msgs_generate_messages_cpp
Scanning dependencies of target std_msgs_generate_messages_py
Scanning dependencies of target _pyros_test_generate_messages_check_deps_StringEchoService
Scanning dependencies of target std_msgs_generate_messages_lisp
[  0%] [  0%] Built target std_msgs_generate_messages_cpp
[  0%] Built target std_msgs_generate_messages_py
Built target std_msgs_generate_messages_lisp
[  0%] Built target _pyros_test_generate_messages_check_deps_StringEchoService
Scanning dependencies of target pyros_test_generate_messages_cpp
Scanning dependencies of target pyros_test_generate_messages_lisp
Scanning dependencies of target pyros_test_generate_messages_py
[ 75%] [ 75%] [ 75%] Generating Lisp code from pyros_test/StringEchoService.srv
Generating C++ code from pyros_test/StringEchoService.srv
Generating Python code from SRV pyros_test/StringEchoService
[100%] Generating Python srv __init__.py for pyros_test
[100%] Built target pyros_test_generate_messages_lisp
[100%] Built target pyros_test_generate_messages_py
[100%] Built target pyros_test_generate_messages_cpp
Scanning dependencies of target pyros_test_generate_messages
[100%] Built target pyros_test_generate_messages

  • Source devel space:

    $ source devel/setup.bash

  • Run tests with catkin_make test:

    $ catkin_make test
Base path: /home/alexv/doctest/catkin_ws
Source space: /home/alexv/doctest/catkin_ws/src
Build space: /home/alexv/doctest/catkin_ws/build
Devel space: /home/alexv/doctest/catkin_ws/devel
Install space: /home/alexv/doctest/catkin_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/alexv/doctest/catkin_ws/build"
####
####
#### Running command: "make test -j8 -l8" in "/home/alexv/doctest/catkin_ws/build"
####
Running tests...
Test project /home/alexv/doctest/catkin_ws/build
No tests were found!!!

  • Debug tests with catkin_make run_tests:

    $ catkin_make run_tests
Base path: /home/alexv/doctest/catkin_ws
Source space: /home/alexv/doctest/catkin_ws/src
Build space: /home/alexv/doctest/catkin_ws/build
Devel space: /home/alexv/doctest/catkin_ws/devel
Install space: /home/alexv/doctest/catkin_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/alexv/doctest/catkin_ws/build"
####
####
#### Running command: "make run_tests -j8 -l8" in "/home/alexv/doctest/catkin_ws/build"
####
Scanning dependencies of target run_tests
Built target run_tests

From a different shell to not have your environment polluted with devel space:

  • Source your ROS environment:

    $ source /opt/ros/indigo/setup.bash

  • Install with catkin_make install:

    $ catkin_make install
Base path: /home/alexv/doctest/catkin_ws
Source space: /home/alexv/doctest/catkin_ws/src
Build space: /home/alexv/doctest/catkin_ws/build
Devel space: /home/alexv/doctest/catkin_ws/devel
Install space: /home/alexv/doctest/catkin_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/alexv/doctest/catkin_ws/build"
####
####
#### Running command: "make install -j8 -l8" in "/home/alexv/doctest/catkin_ws/build"
####
[  0%] [  0%] [  0%] Built target std_msgs_generate_messages_lisp
Built target std_msgs_generate_messages_py
Built target std_msgs_generate_messages_cpp
[  0%] Built target _pyros_test_generate_messages_check_deps_StringEchoService
[100%] [100%] [100%] Built target pyros_test_generate_messages_lisp
Built target pyros_test_generate_messages_cpp
Built target pyros_test_generate_messages_py
[100%] Built target pyros_test_generate_messages
Install the project...
-- Install configuration: ""
-- Installing: /home/alexv/doctest/catkin_ws/install/_setup_util.py
-- Installing: /home/alexv/doctest/catkin_ws/install/env.sh
-- Installing: /home/alexv/doctest/catkin_ws/install/setup.bash
-- Installing: /home/alexv/doctest/catkin_ws/install/setup.sh
-- Installing: /home/alexv/doctest/catkin_ws/install/setup.zsh
-- Installing: /home/alexv/doctest/catkin_ws/install/.rosinstall
+ cd /home/alexv/doctest/catkin_ws/src/pyros-test
+ mkdir -p /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages
+ /usr/bin/env PYTHONPATH=/home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages:/home/alexv/doctest/catkin_ws/build/lib/python2.7/dist-packages:/opt/ros/indigo/lib/python2.7/dist-packages CATKIN_BINARY_DIR=/home/alexv/doctest/catkin_ws/build /usr/bin/python /home/alexv/doctest/catkin_ws/src/pyros-test/setup.py build --build-base /home/alexv/doctest/catkin_ws/build/pyros-test install --install-layout=deb --prefix=/home/alexv/doctest/catkin_ws/install --install-scripts=/home/alexv/doctest/catkin_ws/install/bin
running build
running build_py
creating /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7
creating /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7/pyros_test
copying src/pyros_test/echo_node.py -> /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7/pyros_test
copying src/pyros_test/__init__.py -> /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7/pyros_test
running install
running install_lib
creating /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test
copying /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7/pyros_test/echo_node.py -> /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test
copying /home/alexv/doctest/catkin_ws/build/pyros-test/lib.linux-x86_64-2.7/pyros_test/__init__.py -> /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test
byte-compiling /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/echo_node.py to echo_node.pyc
byte-compiling /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/__init__.py to __init__.pyc
running install_egg_info
Writing /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test-0.0.4.egg-info
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/srv/StringEchoService.srv
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/cmake/pyros_test-msg-paths.cmake
-- Installing: /home/alexv/doctest/catkin_ws/install/include/pyros_test
-- Installing: /home/alexv/doctest/catkin_ws/install/include/pyros_test/StringEchoServiceResponse.h
-- Installing: /home/alexv/doctest/catkin_ws/install/include/pyros_test/StringEchoServiceRequest.h
-- Installing: /home/alexv/doctest/catkin_ws/install/include/pyros_test/StringEchoService.h
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test/srv
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test/srv/_package.lisp
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test/srv/pyros_test-srv.asd
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test/srv/StringEchoService.lisp
-- Installing: /home/alexv/doctest/catkin_ws/install/share/common-lisp/ros/pyros_test/srv/_package_StringEchoService.lisp
Listing /home/alexv/doctest/catkin_ws/devel/lib/python2.7/dist-packages/pyros_test ...
Compiling /home/alexv/doctest/catkin_ws/devel/lib/python2.7/dist-packages/pyros_test/__init__.py ...
Listing /home/alexv/doctest/catkin_ws/devel/lib/python2.7/dist-packages/pyros_test/srv ...
Compiling /home/alexv/doctest/catkin_ws/devel/lib/python2.7/dist-packages/pyros_test/srv/_StringEchoService.py ...
Compiling /home/alexv/doctest/catkin_ws/devel/lib/python2.7/dist-packages/pyros_test/srv/__init__.py ...
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv/_StringEchoService.pyc
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv/_StringEchoService.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv/__init__.pyc
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/python2.7/dist-packages/pyros_test/srv/__init__.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pkgconfig/pyros_test.pc
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/cmake/pyros_test-msg-extras.cmake
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/cmake/pyros_testConfig.cmake
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/cmake/pyros_testConfig-version.cmake
-- Installing: /home/alexv/doctest/catkin_ws/install/share/pyros_test/package.xml
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/echo.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/emptyService.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/slowService.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/triggerService.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/common.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/string_pub_node.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/string_pubnot_node.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/string_slow_node.py
-- Installing: /home/alexv/doctest/catkin_ws/install/lib/pyros_test/string_sub_node.py

  • Source the install space:

    $ source install/setup.bash

  • Run tests as a final user would:

    **TODO**

Release

  • Change to the project directory you want to release:

    $ cd src/pyros-test/

  • Generate the changelog:

    $ catkin_generate_changelog
Found packages: pyros_test
Querying commit information since latest tag...
Updating forthcoming section of changelog files...
- updating './CHANGELOG.rst'
Done.
Please review the extracted commit messages and consolidate the changelog entries before committing the files!

  • Prepare the release:

    $ catkin_prepare_release

  • bloom-release –rosdistro indigo –track indigo pyros_test

If this is not accurate, if I missed a step, or if there is a better way to do the same, please open a PullRequest or an issue on the catkin_pip repository with the related information so this doc can be corrected

TODO : review this with the new catkin tools coming up (ie catkin build)

Continuous Testing Workflow

Because no software works until it has been tested, you should configure travis on your repository to run test with each commit and pull request.

Catkin testing can be done with a simple .travis.yml file and a small shell script.