Dorieh Data Platform
Contents
Introduction
What is Data Platform
Data Domains
Data Processing Pipelines
Python Packages
Data Modelling for Dorieh Data Platform
Examples
Data Platform Internals
Database Testing Framework
Adding more data
Executing containerized apps
Terms and Acronyms
Indices
Dorieh Data Platform
The introspector Module
View page source
The introspector Module
This module introspects columnar data to infer the types of the columns
class
Introspector
(
data_file
:
str
,
column_name_replacement
:
Optional
[
Dict
]
=
None
)
[source]
static
load_range
(
n
,
f
)
→
int
[source]
static
name
(
path
)
→
str
[source]
static
csv_reader
(
data
,
unquote
=
True
)
[source]
static
unquote
(
s
:
str
)
→
str
[source]
fopen
(
source
)
[source]
handle_csv
(
entry
)
[source]
handle_json
(
entry
)
[source]
classmethod
sas2db_type
(
column
,
rows
)
[source]
handle_sas
(
entry
)
[source]
introspect
(
entry
=
None
)
[source]
append_sql_column
(
name
:
str
)
[source]
load_csv
(
entry
)
[source]
load_fst
(
entry
)
[source]
load_sas
(
reader
:
SAS7BDAT
)
→
List
[
List
]
[source]
load_json
(
entry
)
→
List
[
List
]
[source]
guess_str
(
v
,
v2
:
Optional
[
str
]
,
scale
,
precision
,
max_val
)
[source]
guess_types
(
rows
:
list
,
lines
:
list
)
[source]
static
reconcile
(
cell_type
,
column_type
)
→
str
[source]
static
db_type
(
column_type
,
max_val
,
precision
,
scale
)
→
str
[source]
get_columns
(
)
→
List
[
Dict
]
[source]
append_file_column
(
)
[source]
append_record_column
(
)
[source]
classmethod
classify
(
files
)
[source]
exception
InconsistentTypes
[source]
test
(
)
[source]