Substring Manipulation
Pattern Matching Expressions and String Functions
String-level expression logic is used to search and manipulate data according to certain patterns. This functionality is often associated with text editors and shell commands, but is rarely integrated into high volume data processing operations.
In other words, string parsing, pattern matching, and other low level manipulations must usually occur in a separate tools and I/O steps, increasing coding and processing overhead.
Solutions
The SortCL program in IRI CoSort and IRI Voracity supports Perl Compatible Regular Expression (PCRE) logic for pattern matching, as well as find and replace, and other string and substring-level manipulations. SortCL also supports field padding and alignment, character validation, and field re-mapping.
These functions are also useful in the context of data discovery, master data management, and data quality improvement.
More importantly, this intricate data transformation can occur in the same job script and I/O pass with all the other simultaneous functions SortCL performs, like:
- Data Transformation (sort, join, aggregate, filter, remap, etc.)
- Data Migration (of data types and file formats)
- Data Protection (field level encryption, de-ID, masking, etc.)
- Reporting (batch, delta, detail and summary BI)
Please let us know below about your specific data manipulation requirements.
