Pivot / Unpivot

Denormalize and normalize rows and columns

Challenges

With normalized row and column representations, a consolidation to a de-normalized layout may require rotating rows into columns. This is referred to as a pivot transformation and is a common requirement in statistical cross-tabulation reports. To pivot efficiently and avoid duplicate rows, data should be pre-sorted. In other tools, this step is performed separately and is slow in terms of volume.

Normalizing denormalized content, or unpivoting, introduces further issues. For example, rows might contain unexpected values or be missing altogether if column values were null or zero. Subsequent sorting is, again, typically a separate step.

The commands for pivoting and unpivoting data in SQL, if they exist, are not always straightforward or cross-database. ETL tools may require complex aggregate or union transformations (with „ports“ and „expressions“), or specialized pivot/unpivot transform editors with many properties to configure.

SQL cross-tab scripts are also cumbersome and useless for remapping fields and records in a flat file. XSL „for-each“ transformations for flattening XML files are even more complex. And then there's the question of how all these methods work in volume.

Solutions

The program SortCL in the IRI CoSort Data manipulation utility or the IRI Voracity ETL Platform transposes rows and columns in pivot tables (or flat files) while supporting further transformation, cleansing, masking, and reporting functions. You can perform pivot and unpivot operations via an ergonomic Job Assistant in the IRI Workbench GUI and/or a simple 4GL Job Script define.

The new Pivot Job Assistant in IRI Workbench guides you through the necessary column specifications. for each data source.

The same assistant can also be used for unpivot.