Dorieh Data Platform
Contents
Introduction
What is Data Platform
Data Domains
Data Processing Pipelines
Python Packages
Data Modelling for Dorieh Data Platform
Extensions used for creating federated view of different years
The Data Dictionary Generation tool
Medicare: Building a Data Warehouse from ResDac Files
Data dictionary and lineage for Medicare processing
Table Lineage Diagram
Table Lineage SVG Diagram
Table cms.mbsf_ab_2011
Table cms.mbsf_ab_2012
Table cms.mbsf_ab_2013
Table cms.mbsf_ab_2014
Table cms.mbsf_abcd_2015
Table cms.mbsf_abcd_2016
Table cms.mbsf_abcd_2017
Table cms.mbsf_abcd_2018
Table cms.mbsf_d_2011
Table cms.mbsf_d_2012
Table cms.mbsf_d_2013
Table cms.mbsf_d_2014
Table cms.medpar_2011
Table cms.medpar_2012
Table cms.medpar_2013
Table cms.medpar_2014
Table cms.medpar_2015
Table cms.medpar_2016
Table cms.medpar_2017
Table cms.medpar_2018
View medicare._beneficiaries
View medicare._enrollments
Materialized view medicare._ps
Table medicare.admissions
Table medicare.beneficiaries
Table medicare.enrollments
View medicare.ip
Materialized view medicare.mbsf_d
View medicare.ps
View medicare.qc_adm_union
Materialized view medicare.qc_admissions
View medicare.qc_enrl_bene
Materialized view medicare.qc_enrollments
Alphabetic list of all tables
Alphabetic list of all columns in all tables
The Data Dictionary Generation tool
Medicaid: Building a Data Warehouse from ResDac Files
Introduction to data modelling for Dorieh Data Platform
Domain
Table
Column
Multi-column indices
Generation of the database schema (DDL)
Indexing policies
Linking with nomenclature
Ingesting data
Examples
Data Platform Internals
Database Testing Framework
Adding more data
Executing containerized apps
Terms and Acronyms
Indices
Dorieh Data Platform
Data Domains
Dorieh CMS Package (manipulating ResDac Health Data)
Medicare: Building a Data Warehouse from ResDac Files
Data dictionary and lineage for Medicare processing
Table Lineage Diagram
View medicare.ps
Column medicare.ps.hmo_indicators
Column medicare.ps.hmo_indicators Lineage SVG
View page source
Column medicare.ps.hmo_indicators Lineage SVG
medicare.ps.hmo_indicators
medicare.ps.hmo_indicators
Array of indicators if the beneficiary
was enrolled in HMO during a given month
cms.mbsf_ab_2011.HMOIND03
cms.mbsf_ab_2011.HMOIND03
HMO Indicator III
cms.mbsf_ab_2011.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND03
cms.mbsf_ab_2012.HMOIND03
HMO Indicator III
cms.mbsf_ab_2012.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND03
cms.mbsf_ab_2013.HMOIND03
HMO Indicator III
cms.mbsf_ab_2013.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND03
cms.mbsf_ab_2014.HMOIND03
HMO Indicator III
cms.mbsf_ab_2014.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND03
cms.mbsf_abcd_2015.HMOIND03
HMO Indicator: March
cms.mbsf_abcd_2015.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND03
cms.mbsf_abcd_2016.HMOIND03
HMO Indicator: March
cms.mbsf_abcd_2016.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND11
cms.mbsf_ab_2013.HMOIND11
HMO Indicator XI
cms.mbsf_ab_2014.HMOIND11
cms.mbsf_ab_2014.HMOIND11
HMO Indicator XI
cms.mbsf_abcd_2015.HMOIND11
cms.mbsf_abcd_2015.HMOIND11
HMO Indicator: November
cms.mbsf_abcd_2016.HMOIND11
cms.mbsf_abcd_2016.HMOIND11
HMO Indicator: November
cms.mbsf_abcd_2017.HMOIND11
cms.mbsf_abcd_2017.HMOIND11
HMO Indicator: November
cms.mbsf_abcd_2018.HMOIND11
cms.mbsf_abcd_2018.HMOIND11
HMO Indicator: November
cms.mbsf_ab_2011.HMOIND12
cms.mbsf_ab_2011.HMOIND12
HMO Indicator XII
cms.mbsf_ab_2012.HMOIND12
cms.mbsf_ab_2012.HMOIND12
HMO Indicator XII
cms.mbsf_ab_2013.HMOIND12
cms.mbsf_ab_2013.HMOIND12
HMO Indicator XII
cms.mbsf_ab_2014.HMOIND12
cms.mbsf_ab_2014.HMOIND12
HMO Indicator XII
cms.mbsf_abcd_2017.HMOIND03
cms.mbsf_abcd_2017.HMOIND03
HMO Indicator: March
cms.mbsf_abcd_2017.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND03
cms.mbsf_abcd_2018.HMOIND03
HMO Indicator: March
cms.mbsf_abcd_2018.HMOIND03->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND11
cms.mbsf_ab_2011.HMOIND11
HMO Indicator XI
cms.mbsf_ab_2011.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND11
cms.mbsf_ab_2012.HMOIND11
HMO Indicator XI
cms.mbsf_ab_2012.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND11->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND12
cms.mbsf_abcd_2015.HMOIND12
HMO Indicator: December
cms.mbsf_abcd_2016.HMOIND12
cms.mbsf_abcd_2016.HMOIND12
HMO Indicator: December
cms.mbsf_abcd_2017.HMOIND12
cms.mbsf_abcd_2017.HMOIND12
HMO Indicator: December
cms.mbsf_abcd_2018.HMOIND12
cms.mbsf_abcd_2018.HMOIND12
HMO Indicator: December
cms.mbsf_ab_2011.HMOIND05
cms.mbsf_ab_2011.HMOIND05
HMO Indicator V
cms.mbsf_ab_2012.HMOIND05
cms.mbsf_ab_2012.HMOIND05
HMO Indicator V
cms.mbsf_ab_2013.HMOIND05
cms.mbsf_ab_2013.HMOIND05
HMO Indicator V
cms.mbsf_ab_2014.HMOIND05
cms.mbsf_ab_2014.HMOIND05
HMO Indicator V
cms.mbsf_abcd_2015.HMOIND05
cms.mbsf_abcd_2015.HMOIND05
HMO Indicator: May
cms.mbsf_abcd_2016.HMOIND05
cms.mbsf_abcd_2016.HMOIND05
HMO Indicator: May
cms.mbsf_ab_2011.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND12->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND05
cms.mbsf_abcd_2017.HMOIND05
HMO Indicator: May
cms.mbsf_abcd_2018.HMOIND05
cms.mbsf_abcd_2018.HMOIND05
HMO Indicator: May
cms.mbsf_ab_2011.HMOIND09
cms.mbsf_ab_2011.HMOIND09
HMO Indicator IX
cms.mbsf_ab_2012.HMOIND09
cms.mbsf_ab_2012.HMOIND09
HMO Indicator IX
cms.mbsf_ab_2013.HMOIND09
cms.mbsf_ab_2013.HMOIND09
HMO Indicator IX
cms.mbsf_ab_2014.HMOIND09
cms.mbsf_ab_2014.HMOIND09
HMO Indicator IX
cms.mbsf_abcd_2015.HMOIND09
cms.mbsf_abcd_2015.HMOIND09
HMO Indicator: September
cms.mbsf_abcd_2016.HMOIND09
cms.mbsf_abcd_2016.HMOIND09
HMO Indicator: September
cms.mbsf_abcd_2017.HMOIND09
cms.mbsf_abcd_2017.HMOIND09
HMO Indicator: September
cms.mbsf_abcd_2018.HMOIND09
cms.mbsf_abcd_2018.HMOIND09
HMO Indicator: September
cms.mbsf_ab_2013.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND05->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND08
cms.mbsf_ab_2011.HMOIND08
HMO Indicator VIII
cms.mbsf_ab_2012.HMOIND08
cms.mbsf_ab_2012.HMOIND08
HMO Indicator VIII
cms.mbsf_ab_2013.HMOIND08
cms.mbsf_ab_2013.HMOIND08
HMO Indicator VIII
cms.mbsf_ab_2014.HMOIND08
cms.mbsf_ab_2014.HMOIND08
HMO Indicator VIII
cms.mbsf_abcd_2015.HMOIND08
cms.mbsf_abcd_2015.HMOIND08
HMO Indicator: August
cms.mbsf_abcd_2016.HMOIND08
cms.mbsf_abcd_2016.HMOIND08
HMO Indicator: August
cms.mbsf_abcd_2017.HMOIND08
cms.mbsf_abcd_2017.HMOIND08
HMO Indicator: August
cms.mbsf_abcd_2018.HMOIND08
cms.mbsf_abcd_2018.HMOIND08
HMO Indicator: August
cms.mbsf_ab_2011.HMOIND06
cms.mbsf_ab_2011.HMOIND06
HMO Indicator VI
cms.mbsf_ab_2012.HMOIND06
cms.mbsf_ab_2012.HMOIND06
HMO Indicator VI
cms.mbsf_abcd_2015.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND09->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND06
cms.mbsf_ab_2013.HMOIND06
HMO Indicator VI
cms.mbsf_ab_2014.HMOIND06
cms.mbsf_ab_2014.HMOIND06
HMO Indicator VI
cms.mbsf_abcd_2015.HMOIND06
cms.mbsf_abcd_2015.HMOIND06
HMO Indicator: June
cms.mbsf_abcd_2016.HMOIND06
cms.mbsf_abcd_2016.HMOIND06
HMO Indicator: June
cms.mbsf_abcd_2017.HMOIND06
cms.mbsf_abcd_2017.HMOIND06
HMO Indicator: June
cms.mbsf_abcd_2018.HMOIND06
cms.mbsf_abcd_2018.HMOIND06
HMO Indicator: June
cms.mbsf_ab_2011.HMOIND07
cms.mbsf_ab_2011.HMOIND07
HMO Indicator VII
cms.mbsf_ab_2012.HMOIND07
cms.mbsf_ab_2012.HMOIND07
HMO Indicator VII
cms.mbsf_ab_2013.HMOIND07
cms.mbsf_ab_2013.HMOIND07
HMO Indicator VII
cms.mbsf_ab_2014.HMOIND07
cms.mbsf_ab_2014.HMOIND07
HMO Indicator VII
cms.mbsf_abcd_2017.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND08->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND06->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND07
cms.mbsf_abcd_2015.HMOIND07
HMO Indicator: July
cms.mbsf_abcd_2016.HMOIND07
cms.mbsf_abcd_2016.HMOIND07
HMO Indicator: July
cms.mbsf_abcd_2017.HMOIND07
cms.mbsf_abcd_2017.HMOIND07
HMO Indicator: July
cms.mbsf_abcd_2018.HMOIND07
cms.mbsf_abcd_2018.HMOIND07
HMO Indicator: July
cms.mbsf_ab_2011.HMOIND04
cms.mbsf_ab_2011.HMOIND04
HMO Indicator IV
cms.mbsf_ab_2012.HMOIND04
cms.mbsf_ab_2012.HMOIND04
HMO Indicator IV
cms.mbsf_ab_2013.HMOIND04
cms.mbsf_ab_2013.HMOIND04
HMO Indicator IV
cms.mbsf_ab_2014.HMOIND04
cms.mbsf_ab_2014.HMOIND04
HMO Indicator IV
cms.mbsf_abcd_2015.HMOIND04
cms.mbsf_abcd_2015.HMOIND04
HMO Indicator: April
cms.mbsf_abcd_2016.HMOIND04
cms.mbsf_abcd_2016.HMOIND04
HMO Indicator: April
cms.mbsf_ab_2011.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND07->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND04
cms.mbsf_abcd_2017.HMOIND04
HMO Indicator: April
cms.mbsf_abcd_2018.HMOIND04
cms.mbsf_abcd_2018.HMOIND04
HMO Indicator: April
cms.mbsf_ab_2011.HMOIND01
cms.mbsf_ab_2011.HMOIND01
HMO Indicator I
cms.mbsf_ab_2012.HMOIND01
cms.mbsf_ab_2012.HMOIND01
HMO Indicator I
cms.mbsf_ab_2013.HMOIND01
cms.mbsf_ab_2013.HMOIND01
HMO Indicator I
cms.mbsf_ab_2014.HMOIND01
cms.mbsf_ab_2014.HMOIND01
HMO Indicator I
cms.mbsf_abcd_2015.HMOIND01
cms.mbsf_abcd_2015.HMOIND01
HMO Indicator: January
cms.mbsf_abcd_2016.HMOIND01
cms.mbsf_abcd_2016.HMOIND01
HMO Indicator: January
cms.mbsf_abcd_2017.HMOIND01
cms.mbsf_abcd_2017.HMOIND01
HMO Indicator: January
cms.mbsf_abcd_2018.HMOIND01
cms.mbsf_abcd_2018.HMOIND01
HMO Indicator: January
cms.mbsf_ab_2013.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND04->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND02
cms.mbsf_ab_2011.HMOIND02
HMO Indicator II
cms.mbsf_ab_2012.HMOIND02
cms.mbsf_ab_2012.HMOIND02
HMO Indicator II
cms.mbsf_ab_2013.HMOIND02
cms.mbsf_ab_2013.HMOIND02
HMO Indicator II
cms.mbsf_ab_2014.HMOIND02
cms.mbsf_ab_2014.HMOIND02
HMO Indicator II
cms.mbsf_abcd_2015.HMOIND02
cms.mbsf_abcd_2015.HMOIND02
HMO Indicator: February
cms.mbsf_abcd_2016.HMOIND02
cms.mbsf_abcd_2016.HMOIND02
HMO Indicator: February
cms.mbsf_abcd_2017.HMOIND02
cms.mbsf_abcd_2017.HMOIND02
HMO Indicator: February
cms.mbsf_abcd_2018.HMOIND02
cms.mbsf_abcd_2018.HMOIND02
HMO Indicator: February
cms.mbsf_ab_2011.HMOIND10
cms.mbsf_ab_2011.HMOIND10
HMO Indicator X
cms.mbsf_ab_2012.HMOIND10
cms.mbsf_ab_2012.HMOIND10
HMO Indicator X
cms.mbsf_abcd_2015.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND01->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND10
cms.mbsf_ab_2013.HMOIND10
HMO Indicator X
cms.mbsf_ab_2014.HMOIND10
cms.mbsf_ab_2014.HMOIND10
HMO Indicator X
cms.mbsf_abcd_2015.HMOIND10
cms.mbsf_abcd_2015.HMOIND10
HMO Indicator: October
cms.mbsf_abcd_2016.HMOIND10
cms.mbsf_abcd_2016.HMOIND10
HMO Indicator: October
cms.mbsf_abcd_2017.HMOIND10
cms.mbsf_abcd_2017.HMOIND10
HMO Indicator: October
cms.mbsf_abcd_2018.HMOIND10
cms.mbsf_abcd_2018.HMOIND10
HMO Indicator: October
cms.mbsf_abcd_2017.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND02->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2011.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2012.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2013.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_ab_2014.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2015.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2016.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2017.HMOIND10->medicare.ps.hmo_indicators
union
cms.mbsf_abcd_2018.HMOIND10->medicare.ps.hmo_indicators
union