osrf_pycommon.process_utils.async_execute_process module

class osrf_pycommon.process_utils.async_execute_process.AsyncSubprocessProtocol(stdin=None, stdout=None, stderr=None)

Bases: SubprocessProtocol

Protocol to subclass to get events from async_execute_process().

When subclassing this Protocol class, you should override these functions:

def on_stdout_received(self, data):
    # ...

def on_stderr_received(self, data):
    # ...

def on_process_exited(self, returncode):
    # ...

By default these functions just print the data received from stdout and stderr and does nothing when the process exits.

Data received by the on_stdout_received and on_stderr_received functions is always in bytes. Therefore, it may be necessary to call .decode() on the data before printing to the screen.

Additionally, the data received will not be stripped of new lines, so take that into consideration when printing the result.

You can also override these less commonly used functions:

def on_stdout_open(self):
    # ...

def on_stdout_close(self, exc):
    # ...

def on_stderr_open(self):
    # ...

def on_stderr_close(self, exc):
    # ...

These functions are called when stdout/stderr are opened and closed, and can be useful when using pty’s for example. The exc parameter of the *_close functions is None unless there was an exception.

In addition to the overridable functions this class has a few useful public attributes. The stdin attribute is a reference to the PipeProto which follows the asyncio.WriteTransport interface. The stdout and stderr attributes also reference their PipeProto. The complete attribute is a asyncio.Future which is set to complete when the process exits and its result is the return code.

The complete attribute can be used like this:

import asyncio
from osrf_pycommon.process_utils import async_execute_process
from osrf_pycommon.process_utils import AsyncSubprocessProtocol
from osrf_pycommon.process_utils import get_loop


async def setup():
    transport, protocol = await async_execute_process(
        AsyncSubprocessProtocol, ['ls', '-G', '/usr'])
    retcode = await protocol.complete
    print("Exited with", retcode)

# This will block until the protocol.complete Future is done.
get_loop().run_until_complete(setup())
get_loop().close()

connection_made(transport)

Called when a connection is made.

The argument is the transport representing the pipe connection. To receive data, wait for data_received() calls. When the connection is closed, connection_lost() is called.

on_process_exited(returncode)

pipe_data_received(fd, data)

Called when the subprocess writes data into stdout/stderr pipe.

fd is int file descriptor. data is bytes object.

process_exited(): Called when subprocess has exited.

async osrf_pycommon.process_utils.async_execute_process.async_execute_process(protocol_class, cmd=None, cwd=None, env=None, shell=False, emulate_tty=False, stderr_to_stdout=True)

Coroutine to execute a subprocess and yield the output back asynchronously.

This function is meant to be used with the Python asyncio module, which is available in Python 3.5 or greater.

Here is an example of how to use this function:

import asyncio
from osrf_pycommon.process_utils import async_execute_process
from osrf_pycommon.process_utils import AsyncSubprocessProtocol
from osrf_pycommon.process_utils import get_loop


async def setup():
    transport, protocol = await async_execute_process(
        AsyncSubprocessProtocol, ['ls', '/usr'])
    returncode = await protocol.complete
    return returncode

retcode = get_loop().run_until_complete(setup())
get_loop().close()

Tthe first argument is the default AsyncSubprocessProtocol protocol class, which simply prints output from stdout to stdout and output from stderr to stderr.

If you want to capture and do something with the output or write to the stdin, then you need to subclass from the AsyncSubprocessProtocol class, and override the on_stdout_received, on_stderr_received, and on_process_exited functions.

See the documentation for the AsyncSubprocessProtocol class for more details, but here is an example which uses asyncio from Python 3.5:

import asyncio
from osrf_pycommon.process_utils import async_execute_process
from osrf_pycommon.process_utils import AsyncSubprocessProtocol
from osrf_pycommon.process_utils import get_loop


class MyProtocol(AsyncSubprocessProtocol):
    def __init__(self, file_name, **kwargs):
        self.fh = open(file_name, 'w')
        AsyncSubprocessProtocol.__init__(self, **kwargs)

    def on_stdout_received(self, data):
        # Data has line endings intact, but is bytes in Python 3
        self.fh.write(data.decode('utf-8'))

    def on_stderr_received(self, data):
        self.fh.write(data.decode('utf-8'))

    def on_process_exited(self, returncode):
        self.fh.write("Exited with return code: {0}".format(returncode))
        self.fh.close()


async def log_command_to_file(cmd, file_name):

    def create_protocol(**kwargs):
        return MyProtocol(file_name, **kwargs)

    transport, protocol = await async_execute_process(
        create_protocol, cmd)
    returncode = await protocol.complete
    return returncode

get_loop().run_until_complete(
    log_command_to_file(['ls', '/'], '/tmp/out.txt'))
get_loop().close()

See the subprocess.Popen class for more details on some of the parameters to this function like cwd, env, and shell.

See the osrf_pycommon.process_utils.execute_process() function for more details on the emulate_tty parameter.

Parameters:

protocol_class (AsyncSubprocessProtocol or a subclass) – Protocol class which handles subprocess callbacks
cmd (list) – list of arguments where the executable is the first item
cwd (str) – directory in which to run the command
env (dict) – a dictionary of environment variable names to values
shell (bool) – if True, the cmd variable is interpreted by a the shell
emulate_tty (bool) – if True, pty’s are passed to the subprocess for stdout and stderr, see osrf_pycommon.process_utils.execute_process().
stderr_to_stdout (bool) – if True, stderr is directed to stdout, so they are not captured separately.

osrf_pycommon.process_utils.async_execute_process.get_loop()

This function will return the proper event loop for the subprocess async calls.

On Unix this just returns asyncio.get_event_loop(), but on Windows it will set and return a asyncio.ProactorEventLoop instead.