Inheritance diagram for grepros.plugins.auto.sqlite.SqliteSink:

[legend]

Public Member Functions
def	__init__ (self, args=None, **kwargs)

def	emit (self, topic, msg, stamp=None, match=None, index=None)

def	size (self)

def	validate (self)

Public Member Functions inherited from grepros.plugins.auto.dbbase.BaseDataSink
def	close (self)

def	close_output (self)

Public Member Functions inherited from grepros.outputs.Sink
def	__enter__ (self)

def	__exit__ (self, exc_type, exc_value, traceback)

def	autodetect (cls, target)

def	bind (self, source)

def	emit_meta (self)

def	flush (self)

def	is_highlighting (self)

def	thread_excepthook (self, text, exc)

Public Member Functions inherited from grepros.outputs.RolloverSinkMixin
def	close_output (self)

def	ensure_rollover (self, topic, msg, stamp)

def	format_output_meta (self)

def	get_write_options (cls, label)

def	make_filename (self)

Public Attributes
	filename

	valid

Public Attributes inherited from grepros.plugins.auto.dbbase.BaseDataSink
	COMMIT_INTERVAL

	db
	Database connection. More...

Public Attributes inherited from grepros.outputs.Sink
	args

	source
	inputs.Source instance bound to this sink More...

	valid
	Result of validate() More...

Public Attributes inherited from grepros.outputs.RolloverSinkMixin
	filename
	Current output file path. More...

Static Public Attributes
	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, VERBOSE=False)
	Constructor argument defaults. More...

string	ENGINE = "SQLite"
	Database engine name. More...

tuple	FILE_EXTENSIONS = (".sqlite", ".sqlite3")
	Auto-detection file extensions. More...

int	MAX_INT = 2**63 - 1
	Maximum integer size supported in SQLite, higher values inserted as string. More...

Static Public Attributes inherited from grepros.plugins.auto.dbbase.BaseDataSink
int	COMMIT_INTERVAL = 1000
	Number of emits between commits; 0 is autocommit. More...

	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, VERBOSE=False)
	Constructor argument defaults. More...

	ENGINE = None
	Database engine name, overridden in subclasses. More...

list	MESSAGE_TYPE_BASECOLS
	Default columns for pkg/MsgType tables. More...

list	MESSAGE_TYPE_NESTCOLS
	Additional default columns for pkg/MsgType tables with nesting output. More...

list	MESSAGE_TYPE_TOPICCOLS
	Default topic-related columns for pkg/MsgType tables. More...

Static Public Attributes inherited from grepros.outputs.Sink
	DEFAULT_ARGS = dict(META=False)
	Constructor argument defaults. More...

tuple	FILE_EXTENSIONS = ()
	Auto-detection file extensions for subclasses, as (".ext", ) More...

Static Public Attributes inherited from grepros.plugins.auto.sqlbase.SqlMixin
	DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, MATCH_WRAPPER=None, VERBOSE=False)
	Constructor argument defaults. More...

string	DEFAULT_DIALECT = "sqlite"
	Default SQL dialect used if dialect not specified. More...

dictionary	DIALECTS
	Supported SQL dialects and options. More...

list	KEYWORDS
	Words that need quoting if in name context, like table name. More...

Static Public Attributes inherited from grepros.outputs.RolloverSinkMixin
	DEFAULT_ARGS = dict(VERBOSE=False, WRITE=None, WRITE_OPTIONS={})
	Constructor argument defaults. More...

string	FILE_META_TEMPLATE = "{name} ({size})"

string	MULTI_META_TEMPLATE = "\n- {name} ({size}, {mcount}, {tcount})"

list	OPTIONS_TEMPLATES
	Command-line help templates for rollover options, as [(name, text with s label placeholder)]. More...

string	START_META_TEMPLATE = "{mcount} in {tcount} to "

Private Member Functions
def	_connect (self)

def	_execute_insert (self, sql, args)

def	_executemany (self, sql, argses)

def	_executescript (self, sql)

def	_get_next_id (self, table)

def	_init_db (self)

def	_load_schema (self)

def	_make_db_label (self)

def	_process_message (self, topic, msg, stamp)

Private Attributes
	_do_yaml

	_id_counters

	_overwrite

Detailed Description

Writes messages to an SQLite database.

Output will have:
- table "messages", with all messages as serialized binary data
- table "types", with message definitions
- table "topics", with topic information

plus:
- table "pkg/MsgType" for each message type, with detailed fields,
  and JSON fields for arrays of nested subtypes,
  with foreign keys if nesting else subtype values as JSON dictionaries;
  plus underscore-prefixed fields for metadata, like `_topic` as the topic name.

  If launched with nesting-option, tables will also be created for each
  nested message type.

- view "/topic/full/name" for each topic,
  selecting from the message type table

Definition at line 28 of file sqlite.py.

Constructor & Destructor Documentation

◆ init()

def grepros.plugins.auto.sqlite.SqliteSink.__init__	(		self,
			args = `None`,
		**	kwargs
	)

@param   args                 arguments as namespace or dictionary, case-insensitive;
                      or a single path as the name of SQLitefile to write
@param   args.write           name of SQLite file to write, will be appended to if exists
@param   args.write_options   ```
                      {"commit-interval": transaction size (0 is autocommit),
                       "message-yaml": populate messages.yaml (default true),
                       "nesting": "array" to recursively insert arrays
                                  of nested types, or "all" for any nesting),
                       "overwrite": whether to overwrite existing file
                                    (default false),
                       "rollover-size": bytes limit for individual output files,
                       "rollover-count": message limit for individual output files,
                       "rollover-duration": time span limit for individual output files,
                                            as ROS duration or convertible seconds,
                       "rollover-template": output filename template, supporting
                                            strftime format codes like "%H-%M-%S"
                                            and "%(index)s" as output file index}
                      ```
@param   args.meta            whether to emit metainfo
@param   args.verbose         whether to emit debug information
@param   kwargs               any and all arguments as keyword overrides, case-insensitive

Reimplemented from grepros.outputs.RolloverSinkMixin.

Definition at line 64 of file sqlite.py.

Member Function Documentation

◆ _connect()

def grepros.plugins.auto.sqlite.SqliteSink._connect ( self )

private

Returns new database connection.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 176 of file sqlite.py.

◆ _execute_insert()

def grepros.plugins.auto.sqlite.SqliteSink._execute_insert	(	self,
		sql,
		args
	)

private

Executes INSERT statement, returns inserted ID.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 187 of file sqlite.py.

◆ _executemany()

def grepros.plugins.auto.sqlite.SqliteSink._executemany	(	self,
		sql,
		argses
	)

private

Executes SQL with all args sequences.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 192 of file sqlite.py.

◆ _executescript()

def grepros.plugins.auto.sqlite.SqliteSink._executescript	(	self,
		sql
	)

private

Executes SQL with one or more statements.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 197 of file sqlite.py.

◆ _get_next_id()

def grepros.plugins.auto.sqlite.SqliteSink._get_next_id	(	self,
		table
	)

private

Returns next ID value for table, using simple auto-increment.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 202 of file sqlite.py.

◆ _init_db()

def grepros.plugins.auto.sqlite.SqliteSink._init_db ( self )

private

Opens the database file and populates schema if not already existing.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 136 of file sqlite.py.

◆ _load_schema()

def grepros.plugins.auto.sqlite.SqliteSink._load_schema ( self )

private

Populates instance attributes with schema metainfo.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 152 of file sqlite.py.

◆ _make_db_label()

def grepros.plugins.auto.sqlite.SqliteSink._make_db_label ( self )

private

Returns formatted label for database, with file path and size.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 211 of file sqlite.py.

◆ _process_message()

def grepros.plugins.auto.sqlite.SqliteSink._process_message	(	self,
		topic,
		msg,
		stamp
	)

private

Inserts message to messages-table, and to pkg/MsgType tables.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 165 of file sqlite.py.

◆ emit()

def grepros.plugins.auto.sqlite.SqliteSink.emit	(	self,
		topic,
		msg,
		stamp = `None`,
		match = `None`,
		index = `None`
	)

Writes message to database.

Reimplemented from grepros.plugins.auto.dbbase.BaseDataSink.

Definition at line 119 of file sqlite.py.

◆ size()

def grepros.plugins.auto.sqlite.SqliteSink.size ( self )

Returns current file size in bytes, including journals, or None if size lookup failed.

Reimplemented from grepros.outputs.RolloverSinkMixin.

Definition at line 127 of file sqlite.py.

◆ validate()

def grepros.plugins.auto.sqlite.SqliteSink.validate ( self )

Returns whether "commit-interval" and "nesting" in args.write_options have valid value, if any,
and file is writable; parses "message-yaml" and "overwrite" from args.write_options.

Reimplemented from grepros.outputs.RolloverSinkMixin.

Definition at line 96 of file sqlite.py.

Member Data Documentation

◆ _do_yaml

grepros.plugins.auto.sqlite.SqliteSink._do_yaml

private

Definition at line 91 of file sqlite.py.

◆ _id_counters

grepros.plugins.auto.sqlite.SqliteSink._id_counters

private

Definition at line 93 of file sqlite.py.

◆ _overwrite

grepros.plugins.auto.sqlite.SqliteSink._overwrite

private

Definition at line 92 of file sqlite.py.

◆ DEFAULT_ARGS

grepros.plugins.auto.sqlite.SqliteSink.DEFAULT_ARGS = dict(META=False, WRITE_OPTIONS={}, VERBOSE=False)

static

Constructor argument defaults.

Definition at line 61 of file sqlite.py.

◆ ENGINE

string grepros.plugins.auto.sqlite.SqliteSink.ENGINE = "SQLite"

static

Database engine name.

Definition at line 52 of file sqlite.py.

◆ FILE_EXTENSIONS

tuple grepros.plugins.auto.sqlite.SqliteSink.FILE_EXTENSIONS = (".sqlite", ".sqlite3")

static

Auto-detection file extensions.

Definition at line 55 of file sqlite.py.

◆ filename

grepros.plugins.auto.sqlite.SqliteSink.filename

Definition at line 142 of file sqlite.py.

◆ MAX_INT

int grepros.plugins.auto.sqlite.SqliteSink.MAX_INT = 2**63 - 1

static

Maximum integer size supported in SQLite, higher values inserted as string.

Definition at line 58 of file sqlite.py.

◆ valid

grepros.plugins.auto.sqlite.SqliteSink.valid

Definition at line 115 of file sqlite.py.

The documentation for this class was generated from the following file:

sqlite.py

Public Member Functions

Public Attributes

Static Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _connect()

◆ _execute_insert()

◆ _executemany()

◆ _executescript()

◆ _get_next_id()

◆ _init_db()

◆ _load_schema()

◆ _make_db_label()

◆ _process_message()

◆ emit()

◆ size()

◆ validate()

Member Data Documentation

◆ _do_yaml

◆ _id_counters

◆ _overwrite

◆ DEFAULT_ARGS

◆ ENGINE

◆ FILE_EXTENSIONS

◆ filename

◆ MAX_INT

◆ valid

◆ init()