Pump Up Pentaho
Power Transforms, Protect PII, Prototype Operations
Challenges
While Pentaho Data Integration (PDI) is a powerful tool for preparing and integrating data, it also has some shortcomings:
Slow Transforms
Native sorts, etc. may not run fast enough in high volume.
Limited De-ID Features
Cannot mask or encrypt data flowing through Kettle
Limited Test Data
Cannot prototype ETL jobs without using production data
Solutions
PDI workflows support system commands, so data can be processed externally without disruption. IRI Voracity or its component software can help Pentaho users in the following ways:
Speed Transforms
- Use PDI's shell step to call an IRI CoSort job (e.g., SortCL script) to dramatically reduce sort, join, and aggregation times
- Run multiple jobs in one batch file
- Get results 14-16 times faster than Pentaho alone
Using CoSort to Speed up the Sort Process in Pentaho: Read now.
Mask Your Data
- Run IRI FieldShield jobs from the Shell step in Pentaho to protect data at rest
- Mask, encrypt, and encode (and others) data in your needed format
- Secure data at the field-level
Masking Data in Pentaho: Read now
Test Your Data
- Run IRI RowGen to populate tables, files and reports with synthetic test data that mimics production data
- Generate structurally- and referentially-correct DB test data for entire EDW
- Keep production data safe
Creating Test Data for Pentaho: Read now.