launch_pytest

A package to create tests which involve launch files and multiple processes.

README

launch_pytest

This is a framework for launch integration testing. For example:

The exit codes of all processes are available to the tests.
Tests can check that all processes shut down normally, or with specific exit codes.
Tests can fail when a process dies unexpectedly.
The stdout and stderr of all processes are available to the tests.
The command-line used to launch the processes are available to the tests.
Some tests run concurrently with the launch and can interact with the running processes.

Differences with launch_testing

launch_testing is an standalone testing tool, which lacks many features:

It’s impossible to filter test cases by name and run only some.
It’s impossible to mark a test as skipped or xfail.
The error reporting of the tool was custom, and the output wasn’t as nice as the output generated by other testing frameworks such as unittest and pytest.

launch_pytest is a really simple pytest plugin leveraging pytest fixtures to manage a launch service lifetime easily.

Quick start example

Start with the pytest_hello_world.py example.

Run the example by doing:

python3 -m pytest test/launch_pytest/examples/pytest_hello_world.py

The launch_pytest plugin will launch the nodes found in the launch_description fixture, run the tests from the test_read_stdout() class, shut down the launched nodes, and then run the statements after the yield statement in test_read_stdout().

launch_pytest fixtures

@launch_pytest.fixture
def launch_description(hello_world_proc):
    """Launch a simple process to print 'hello_world'."""
    return launch.LaunchDescription([
        hello_world_proc,
        # Tell launch when to start the test
        # If no ReadyToTest action is added, one will be appended automatically.
        launch_pytest.actions.ReadyToTest()
    ])

A @launch_pytest.fixture function should return a launch.LaunchDescription object, or a sequence of objects whose first item is a launch.LaunchDescription. This launch description will be used in all tests with a mark @pytest.mark.launch(fixture=<your_fixture_name>), in this case <your_fixture_name>=launch_description.

The launch description can include a ReadyToTest action to signal to the test framework that it’s safe to start the active tests. If one isn’t included, a ReadyToTest action will be appended at the end.

launch_pytest fixtures can have module, class or function scope. The default is function. For example:

@launch_pytest.fixture(scope=my_scope)
def my_fixture():
    return LaunchDescription(...)

@pytest.mark.launch(fixture=my_fixture)
def test_case_1():
    pass

@pytest.mark.launch(fixture=my_fixture)
def test_case_2():
    pass

If my_scope=function, the following happens:

A launch service using the LaunchDescription returned by my_fixture() is started.
test_case_1() is run.
The launch service is shutdown.
Another launch service using the LaunchDescription returned by my_fixture() is started, my_fixture() is called again.
test_case_2() is run.
The launch service is shutdown.

Whereas when my_scope=module, test case_2() will run immediately after test case_1(), concurrently with the same launch service.

It’s not recommended to mix fixtures with module scope with fixtures of class/function scope in the same file. It’s not recommended to use fixtures with scope larger than module. A test shouldn’t depend on more than one launch_pytest fixture. Neither of the three things above automatically generates an error in the current launch_pytest implementation, but future versions might.

Active Tests and shutdown tests

Test cases marked with @pytest.mark.launch will be run concurrently with the launch service or after launch shutdown, depending on the object being marked and the mark arguments.

functions: Functions marked with @pytest.mark.launch will run concurrently with the launch service, except when shutdown=True is passed as an argument to the decorator.

@pytest.mark.launch(fixture=my_ld_fixture)
def normal_test_case():
    pass

@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True)
def shutdown_test_case():
    pass

coroutine functions: The same rules as normal functions apply. Coroutines will be run in the same event loop as the launch description, whereas normal functions run concurrently in another thread.

@pytest.mark.launch(fixture=my_ld_fixture)
async def normal_test_case():
    pass

@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True)
async def shutdown_test_case():
    pass

generators: The first time the generator is called it runs concurrently with the launch service. The generator will be resumed after the launch service is shutdown. i.e. This allows to write a test that has a step that runs concurrently with the service and one that runs after shutdown easily. The yielded value is ignored. If the generator doesn’t stop iteration after being resumed for a second time, the test will fail. Passing a shutdown argument to the decorator is not allowed in this case.

@pytest.mark.launch(fixture=my_ld_fixture)
def normal_test_case():
    assert True
    yield
    assert True

async generators: The same rules as for generators apply here as well. The only difference between the two is that async generator will run in the same event loop as the launch service, whereas a generator will run concurrently in another thread.

@pytest.mark.launch(fixture=my_ld_fixture)
async def normal_test_case():
    assert True
    yield
    assert True

Fixtures

The launch_pytest plugin will provide the following fixtures.

launch_service: The launch service being used to run the tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.
launch_context: The launch context being used to run the tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.
event_loop: The event loop being used to run the launch service and to run async tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.