The project_loader Module

This module is a command line tool to introspect and ingest into a database a directory, containing CSV (or CSV-like, e.g. FST, JSON, SAS, etc.) files. The directory can be structured, e.g. have nested subdirectories. All files matching a certain name pattern at any nested subdirectory level are included in the data set.

In the database, a schema is crated based on the given project name. For each file in the data set a table is created. The name of the table is constructed from the relative path of the incoming data file with OS path separators (e.g. ‘/’) being replaced with underscores (‘_’).

is_relative_to(p: PurePath, *other)

Return True if the path is relative to another path or False.

remove_ext(p: str) → str

class ProjectLoader(context: Optional[LoaderConfig] = None)

Class, implementing Project Loading functionality.

Parameters:

context – Configuration options, usually provided as command line arguments

classmethod new_domain(name: str, file_path: str)

introspect(entry: str)

save_registry()

run()

Run introspection and ingestion.

Returns:

nothing