Slowly Changing Dimensions
Provisioning of the Business Intelligence Reporting Tool
Challenges
Slowly Changing Dimensions, or SCD, This is the problem in data warehousing when it comes to tracking changes in the values (facts) of a date. „Slowly“ implies time, but not necessarily „slow“ time; the concepts are the same whether changes occur in seconds or centuries. The interval between changes does not have to be uniform. The key(s) must be unique, and the resulting value is discrete.
Slowly Changing Dimensions are discussed in numerous places with reference to known techniques for storing and accessing such data. Basically, the user can ignore changes, overwrite the existing fact, extend the stored record, or create additional records (tuple versioning) using surrogate keys. This is often a complex process in ETL tools or SQL.
Solutions
IRI took a new approach to reporting on Slowly Changing Dimensions. The SortCL program south CoSort Product uses a powerful, „fuzzy logic search“ for facts in set files.
This core function now also features a new visual SCD job creation assistant for types 1, 2, 3, 4, and 6, which IRI Voracity Users free in the IRI Workbench GUI, that can be used with Eclipse™.
Query for discrete values based on changing information such as date and time. For example, if you specify any search date, find and display the address that was valid before, on, or after that date.
Since you are working with data in fields that change at different times, you can use more than one search argument to determine the returned value.
Fundamentally, IRI's file system approach for slowly changing dimensions offers advantages in simplicity, reduced storage, speed, and increased performance. It enables:
- very fast lookup performance
- SCD Type 0-6 Update Reporting
- searching for strictly increasing values
- complex, multi-stage search criteria
- Easy maintenance and sharing of job scripts
- new values that should be applied and integrated quickly
- Support for built-in comments
- the elimination of DB overheads, reorganizations, etc.
By using SortCL scripts or the Voracity Wizard in Eclipse for SCD reports, you can also integrate sorting, expression evaluation, aggregation, new formatting, encryption, etc. – all within the same job script and I/O pass. Here is the blog series for more details.