Pump Up Pentaho

Power Transforms, Protect PII, Prototype Operations



While Pentaho Data Integration (PDI) is a powerful tool for preparing and integrating data, it also has some shortcomings:

  • Slow Transforms

    Native sorts, etc. may not run fast enough in high volume.
  • Limited De-ID Features

    Cannot mask or encrypt data flowing through Kettle
  • Limited Test Data

    Cannot prototype ETL jobs without using production data


PDI workflows support system commands, so data can be processed externally without disruption. IRI Voracity or its component software can help Pentaho users in the following ways:

Speed Transforms

  • Use PDI's shell step to call an IRI CoSort job (e.g., SortCL script) to dramatically reduce sort, join, and aggregation times
  • Run multiple jobs in one batch file
  • Get results 14-16 times faster than Pentaho alone

    Using CoSort to Speed up the Sort Process in Pentaho: Read now.

Mask Your Data

  • Run IRI FieldShield jobs from the Shell step in Pentaho to protect data at rest
  • Mask, encrypt, and encode (and others) data in your needed format
  • Secure data at the field-level

    Masking Data in Pentaho: Read now

Test Your Data

  • Run IRI RowGen to populate tables, files and reports with synthetic test data that mimics production data
  • Generate structurally- and referentially-correct DB test data for entire EDW
  • Keep production data safe

    Creating Test Data for Pentaho: Read now.