Unix /bin/sort Replacement
How fast would you like to sort today?
Challenges
The native Unix sort command is functionally limited and does not scale well with increasing input volumes. The following cannot be implemented by the operating system's sort:
- sort the largest files efficiently
- process many data types
- filter, reformat, or otherwise transform
- replace old sorting functions or tools
- fulfill DW sorting, aggregation, or join requirements
Nonetheless, you are familiar with Unix sort syntax, and may have an investment in jobs using the /bin/sort commands. You require a more robust sorting engine under Unix under the hood, or the same functionality under Windows.
Solutions
The IRI CoSort packages for Unix, Linux, and Windows include a faster, more robust drop-in replacement for the Unix /bin/sort program.
Use the same Unix sort syntax (but with the CoSort Engine) on the command line or in batch jobs to re-sort large files in parallel. CoSort engine outperforms Unix system sort by multi-orders of magnitude, scales linearly with volume, and does not fail.
Following dramatic improvements in the sorting speed of large files, CoSort users are switching to a more powerful user interface – the Sort Control Language (SortCLSortCL combines sort operations with data:
- Transformation scrubbing, sorting, joining, grouping, etc.
- Conversion (Data types, record layouts, files)
- Data masking (Field-level encryption, de-ID, etc.)
- Reporting (Custom details, delta, and summaries)
In addition to legacy sorting migrations, Oracle unload, ETL acceleration, and some of the related solutions described on this website, such as: data validation and cleansing, pattern matching, complex transformations, etc.