Diagrams

The hourglass model is a specific model for the transformation of data sources to a standardized model in a target datastore. It is the simplified implementation of a layered Big Data architecture. The hourglass model can be used to medel specific implementations of transformation of data in a pattern called the datapipe.

In a number of other diagrams a detail view is given of these implementations in projects like Digital Transformation, TDP, MaxLimit and others.

In the context of TDP is the logical application of transformation of data a characteristic pattern.. This pattern includes aspects like extract data from the source, transformation of the model or the protocol and selection of the source and the target model.
Transformation exists at two levels in the hourglass. The first is transformation from the source data to the target data in a standardized model. The second is the transformation of the generic model to a specific model in use by the consumer. The last one is not explicitly modeled in the hourglass model because this is considered as an implementation of the consumer.

Overview of the transformation from a relational database to a data target

Overview of the transformation from file based data to a data target

Description of the generic aspectis of the transformation application function. This has the following elements

• Data sources


• Transformation functions


• Data targets


• Data or object modeling functions

In this diagram a description of the datapipe from an (webservice) or XML file source to the target datamodel is described. This is based on the transformation of a XML model to an intermediate tabular or relational model and this is then processed in an ETL process to transform the source data model in a number of steps to the required target model.