This article details the architecture and process of the Graphite Data Warehouse. The Graphite Data Warehouse allows customers to connect their BI or reporting tools to data stored in graphiteConnect.
Architecture
Graphite - the Graphite cloud application and services running in the cloud
- Supplier Database - graphiteConnect’s supplier data
- Event Bus - graphiteConnect’s event log for real-time data events
- PostgreSQL Database - a dedicated PostgreSQL database housing customer-specific data
Provisioning
The Graphite Data Warehouse is provisioned by Graphite customer support. The initial provisioning time depends on the number of connected suppliers but usually takes about 10 minutes.
When provisioning the Data Warehouse Service will create a new, customer-specific PostgreSQL database and populate from the graphiteConnect database. Only data relevant to one customer is copied into the data warehouse.
The current PostgreSQL version is 14.
After provisioning the customer service representative will give you a PostgreSQL connection string, username, and password you can use to connect to the warehouse.
Update Frequency
Data is synchronized nightly to ensure consistent reporting across all dashboards.
Schema and Organization
The database schema is composed of a few standard tables and several tables that are generated dynamically and are dependent on your graphiteConnect configuration.
Standard Tables
| g_connection | Connections between entities, this table describes the state of the relationship between two entities |
| g_entity | Entities |
| g_event_log | Audit log of all events in graphiteConnect |
| g_task | Tasks |
| g_task_assigned_user | Task assignments |
| g_task_change_log | Task changes |
| g_user | Users |
Dynamic Tables
The bulk of the supplier data is in dynamic tables. These tables have 2 main prefixes, gpt, and gpg.
Gpt tables contain data that is one-to-one with a supplier. Their keys are connection_id and entity_id.
Gpg tables contain data this one-to-many with a supplier. Their keys are connection_id, entity_id, and instance_id. Instance_id is the identifier of a row in the one-to-many relationship between this data and a connection.
A data dictionary containing all columns and tables is available as part of a standard connection export in graphiteConnect (see the glossary tab).
Security and Permissions
No sensitive data is copied to the data warehouse. Bank account information, social security numbers, and other sensitive data is omitted.
The Postgres user account customer use has read-only access to the Graphite managed database tables. This user also has permission to create new tables, wherein customers can co-locate any other data they would like to include in their reports.