Inheritance diagram for grepros.plugins.auto.postgres.PostgresSink:

[legend]

Public Member Functions
def	__init__ (self, args=None, **kwargs)

def	autodetect (cls, target)

def	validate (self)

Public Member Functions inherited from grepros.plugins.auto.dbbase.BaseDataSink
def	close (self)

def	close_output (self)

def	emit (self, topic, msg, stamp=None, match=None, index=None)

Public Member Functions inherited from grepros.outputs.Sink
def	__enter__ (self)

def	__exit__ (self, exc_type, exc_value, traceback)

def	bind (self, source)

def	emit_meta (self)

def	flush (self)

def	is_highlighting (self)

def	thread_excepthook (self, text, exc)

Public Attributes
	valid

Public Attributes inherited from grepros.plugins.auto.dbbase.BaseDataSink
	COMMIT_INTERVAL

	db
	Database connection. More...

Public Attributes inherited from grepros.outputs.Sink
	args

	source
	inputs.Source instance bound to this sink More...

	valid
	Result of validate() More...

Static Public Attributes
string	ENGINE = "Postgres"
	Database engine name. More...

int	ID_SEQUENCE_STEP = 100
	Sequence length per table to reserve for inserted message IDs. More...

list	MESSAGE_TYPE_BASECOLS
	Default columns for pkg/MsgType tables. More...

list	MESSAGE_TYPE_NESTCOLS
	Additional default columns for pkg/MsgType tables with nesting output. More...

list	MESSAGE_TYPE_TOPICCOLS
	Default topic-related columns for pkg/MsgType tables, as [(name, SQL type)]. More...

string	SELECT_TYPE_COLUMNS
	SQL statement for selecting metainfo on pkg/MsgType table columns. More...

Static Public Attributes inherited from grepros.plugins.auto.dbbase.BaseDataSink
int	COMMIT_INTERVAL = 1000
	Number of emits between commits; 0 is autocommit. More...

	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, VERBOSE=False)
	Constructor argument defaults. More...

	ENGINE = None
	Database engine name, overridden in subclasses. More...

list	MESSAGE_TYPE_BASECOLS
	Default columns for pkg/MsgType tables. More...

list	MESSAGE_TYPE_NESTCOLS
	Additional default columns for pkg/MsgType tables with nesting output. More...

list	MESSAGE_TYPE_TOPICCOLS
	Default topic-related columns for pkg/MsgType tables. More...

Static Public Attributes inherited from grepros.outputs.Sink
	DEFAULT_ARGS = dict(META=False)
	Constructor argument defaults. More...

tuple	FILE_EXTENSIONS = ()
	Auto-detection file extensions for subclasses, as (".ext", ) More...

Static Public Attributes inherited from grepros.plugins.auto.sqlbase.SqlMixin
	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, MATCH_WRAPPER=None, VERBOSE=False)
	Constructor argument defaults. More...

string	DEFAULT_DIALECT = "sqlite"
	Default SQL dialect used if dialect not specified. More...

dictionary	DIALECTS
	Supported SQL dialects and options. More...

list	KEYWORDS
	Words that need quoting if in name context, like table name. More...

Private Member Functions
def	_connect (self)

def	_execute_insert (self, sql, args)

def	_executemany (self, sql, argses)

def	_executescript (self, sql)

def	_get_next_id (self, table)

def	_init_db (self)

def	_load_schema (self)

def	_make_column_value (self, value, typename=None)

def	_make_db_label (self)

Private Attributes
	_id_queue

Detailed Description

Writes messages to a Postgres database.

Output will have:
- table "topics", with topic and message type names
- table "types", with message type definitions

plus:
- table "pkg/MsgType" for each topic message type, with detailed fields,
  BYTEA fields for uint8[], array fields for scalar list attributes,
  and JSONB fields for lists of ROS messages;
  with foreign keys if nesting else subtype values as JSON dictionaries;
  plus underscore-prefixed fields for metadata, like `_topic` as the topic name.
  If not nesting, only topic message type tables are created.
- view "/full/topic/name" for each topic, selecting from the message type table

If a message type table already exists but for a type with a different MD5 hash,
the new table will have its MD5 hash appended to end, as "pkg/MsgType (hash)".

Definition at line 30 of file postgres.py.

Constructor & Destructor Documentation

◆ init()

def grepros.plugins.auto.postgres.PostgresSink.__init__	(		self,
			args = `None`,
		**	kwargs
	)

@param   args                 arguments as namespace or dictionary, case-insensitive;
                      or a single item as the database connection string
@param   args.write           Postgres connection string like "postgresql://user@host/db"
@param   args.write_options   ```
                      {"commit-interval": transaction size (0 is autocommit),
                       "nesting": "array" to recursively insert arrays
                                  of nested types, or "all" for any nesting)}
                      ```
@param   args.meta            whether to emit metainfo
@param   args.verbose         whether to emit debug information
@param   kwargs               any and all arguments as keyword overrides, case-insensitive

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 79 of file postgres.py.

Member Function Documentation

◆ _connect()

def grepros.plugins.auto.postgres.PostgresSink._connect ( self )

private

Returns new database connection.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 137 of file postgres.py.

◆ _execute_insert()

def grepros.plugins.auto.postgres.PostgresSink._execute_insert	(	self,
		sql,
		args
	)

private

Executes INSERT statement, returns inserted ID.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 143 of file postgres.py.

◆ _executemany()

def grepros.plugins.auto.postgres.PostgresSink._executemany	(	self,
		sql,
		argses
	)

private

Executes SQL with all args sequences.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 149 of file postgres.py.

◆ _executescript()

def grepros.plugins.auto.postgres.PostgresSink._executescript	(	self,
		sql
	)

private

Executes SQL with one or more statements.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 154 of file postgres.py.

◆ _get_next_id()

def grepros.plugins.auto.postgres.PostgresSink._get_next_id	(	self,
		table
	)

private

Returns next cached ID value, re-populating empty cache from sequence.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 159 of file postgres.py.

◆ _init_db()

def grepros.plugins.auto.postgres.PostgresSink._init_db ( self )

private

Opens the database file, and populates schema if not already existing.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 116 of file postgres.py.

◆ _load_schema()

def grepros.plugins.auto.postgres.PostgresSink._load_schema ( self )

private

Populates instance attributes with schema metainfo.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 124 of file postgres.py.

◆ _make_column_value()

def grepros.plugins.auto.postgres.PostgresSink._make_column_value	(	self,
		value,
		typename = `None`
	)

private

Returns column value suitable for inserting to database.

Reimplemented from grepros.plugins.auto.sqlbase.SqlMixin.

Definition at line 172 of file postgres.py.

◆ _make_db_label()

def grepros.plugins.auto.postgres.PostgresSink._make_db_label ( self )

private

Returns formatted label for database.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 203 of file postgres.py.

◆ autodetect()

def grepros.plugins.auto.postgres.PostgresSink.autodetect	(	cls,
		target
	)

Returns true if target is recognizable as a Postgres connection string.

Reimplemented from grepros.outputs.Sink.

Definition at line 211 of file postgres.py.

◆ validate()

def grepros.plugins.auto.postgres.PostgresSink.validate ( self )

Returns whether Postgres driver is available,
and "commit-interval" and "nesting" in args.write_options have valid value, if any,
and database is connectable.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 97 of file postgres.py.

Member Data Documentation

◆ _id_queue

grepros.plugins.auto.postgres.PostgresSink._id_queue

private

Definition at line 94 of file postgres.py.

◆ ENGINE

string grepros.plugins.auto.postgres.PostgresSink.ENGINE = "Postgres"

static

Database engine name.

Definition at line 52 of file postgres.py.

◆ ID_SEQUENCE_STEP

int grepros.plugins.auto.postgres.PostgresSink.ID_SEQUENCE_STEP = 100

static

Sequence length per table to reserve for inserted message IDs.

Definition at line 55 of file postgres.py.

◆ MESSAGE_TYPE_BASECOLS

list grepros.plugins.auto.postgres.PostgresSink.MESSAGE_TYPE_BASECOLS

static

Initial value:

=  [("_dt",          "TIMESTAMP"),
                              ("_timestamp",   "NUMERIC"),
                              ("_id",          "BIGSERIAL PRIMARY KEY"), ]

Default columns for pkg/MsgType tables.

Definition at line 71 of file postgres.py.

◆ MESSAGE_TYPE_NESTCOLS

list grepros.plugins.auto.postgres.PostgresSink.MESSAGE_TYPE_NESTCOLS

static

Initial value:

= [("_parent_type", "TEXT"),

("_parent_id", "BIGINT"), ]

Additional default columns for pkg/MsgType tables with nesting output.

Definition at line 75 of file postgres.py.

◆ MESSAGE_TYPE_TOPICCOLS

list grepros.plugins.auto.postgres.PostgresSink.MESSAGE_TYPE_TOPICCOLS

static

Initial value:

= [("_topic", "TEXT"),

("_topic_id", "BIGINT"), ]

Default topic-related columns for pkg/MsgType tables, as [(name, SQL type)].

Definition at line 68 of file postgres.py.

◆ SELECT_TYPE_COLUMNS

string grepros.plugins.auto.postgres.PostgresSink.SELECT_TYPE_COLUMNS

static

Initial value:

=  """
SELECT c.table_name, c.column_name, c.data_type
FROM   information_schema.columns c INNER JOIN information_schema.tables t
ON     t.table_name = c.table_name
WHERE  c.table_schema = current_schema() AND t.table_type = 'BASE TABLE' AND
       c.table_name LIKE '%/%'
ORDER BY c.table_name, CAST(c.dtd_identifier AS INTEGER)
"""

SQL statement for selecting metainfo on pkg/MsgType table columns.

Definition at line 58 of file postgres.py.

◆ valid

grepros.plugins.auto.postgres.PostgresSink.valid

Definition at line 112 of file postgres.py.

The documentation for this class was generated from the following file:

postgres.py

Public Member Functions

Public Attributes

Static Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _connect()

◆ _execute_insert()

◆ _executemany()

◆ _executescript()

◆ _get_next_id()

◆ _init_db()

◆ _load_schema()

◆ _make_column_value()

◆ _make_db_label()

◆ autodetect()

◆ validate()

Member Data Documentation

◆ _id_queue

◆ ENGINE

◆ ID_SEQUENCE_STEP

◆ MESSAGE_TYPE_BASECOLS

◆ MESSAGE_TYPE_NESTCOLS

◆ MESSAGE_TYPE_TOPICCOLS

◆ SELECT_TYPE_COLUMNS

◆ valid

◆ init()