Utility to generate test queries

Generates SQL script that can be used with Test runner: Utility to run generated tests. These scripts are used by The gen_dbt_cwl Module.

Usage

python -m dorieh.platform.dbt.create_test --script SCRIPT [SCRIPT ...]
      [--table TABLE]
      [--db DB]
      [--connection CONNECTION]
      [--autocommit]
      [-h]
      [--verbose]

Options:

Options

Option	Alias	Description
--help	-h	Show this help message and exit
--script SCRIPT [SCRIPT...]	-s SCRIPT [SCRIPT...]	Path to the file to write the test scripts
--autocommit		Use autocommit, default: False
--database DB	--db DB	Path to a database connection parameters file, default: database.ini (in the working directory
--connection CONNECTION	--connection_name CONNECTION	Section in the database connection parameters file
--table TABLE	-t TABLE	Name of the table to being tested, default: None
--verbose		Verbose output, default: False

Details

A utility to generate a test based on a sample table.

The tool introspects a table, given as an input and generates a set of queries each of which tests that the data in a certain column has not changed.

The queries are output into a file that can be executed as a single SQL query producing a table with the following columns:

1. Name of the column being tested

2. What value is being tested not to change: MD5 hash, number of distinct records, mean value or variance

3. Whether the value has changed (indicated by string failed) or remained the same (indicated by string passed)

Individual queries are separated by a comment strings:

• – Test case end

• – Test case start

so a test runner can execute them individually if desired

class CType(value)

An enumeration.

categorical = 'categorical'

text = 'text'

integral = 'integral'

numeric = 'numeric'

date = 'date'

class Column(name: str, ctype: CType, is_indexed: bool)

class TableFingerprint(context: Optional[DBTConfig] = None)

CATEGORICAL_THRESHOLD = 24

get_columns()

get_categories() → List[Column]

generate_tests()

test_column(c: Column)

test_case_sql(name: str, test: str, condition: str) → str

test_exact(sql: str, name: str, test: str) → str

test_approximate(sql: str, name: str, test: str) → str

union()

write_test_script()