The SortCL Program

Transform, Cleanse, Mask, Map, Prototype, and Report

Overview
Function matrix
Call & Logging
Application examples
Resources

SortCL

SortCL is the powerful data manipulation program and popular front-end that controls almost every IRI software product. Short for „Sort Control Language,“ SortCL is the name of IRI's:

  1. award-winning Fourth-generation language (4GL) data definition and manipulation language
  2. standalone data processing executable that follows these 4GL statements
  3. internal, callable library routine that internally calls the executable file (or your 3GL program) to process data

SortCL scripts and jobs describe and consolidate data mappings and manipulations. They support numerous data source and target formats (including custom reports) in IRI CoSort and its spin-off products:

  • IRI NextForm, for data and DB migration and replication
  • IRI FieldShield and IRI DarkShield for PII classification, masking, and auditing
  • IRI RowGen, for the generation of synthetic test data and population
  • IRI Voracity, which encompasses all of the above points, the support and combination of multiple use cases for data discovery, integration, migration, governance, and analytical applications. The Hadoop-enabled Version of Voracity can also interchangeably execute multiple SortCL programs in MapReduce 2, Spark, Spark Stream, Storm, and Tez.

SortCL therefore makes CoSort and Voracity a one-stop, one-I/O „super tool“ for data transformation, conversion, cleansing, mapping, and reporting. SortCL's extensive data movement and manipulation capabilities—detailed in the tabs above and depicted in this diagram beneath the CoSort logo—can be executed in a single job and process large volumes of data.

SortCL performs the same tasks for Voracity licensees that you use in modern transform mappingDiagrams can be seen represented. ETL and Solution Architects with large amounts of data Assess the ability to perform data integration, remapping, cleansing, masking, and BI-Operations to combine, as well as the dramatic differences in information provision and compliance outcomes:

No experience necessary

SortCL uses open, explicit Metadata, to data in over 150 different File and database sources to define, manipulate, and format. The SortCL syntax is self-documenting; anyone can easily read, recognize, and modify SortCL programs.

SortCL uses simple English instructions that are originally based on VMS sort parameters and familiar SQL, C, and PCRE syntax. Unlike cryptic ETL, 3GL, and MapReduce programs, no training is required to understand what a SortCL script does.

But for those who cannot or do not want to learn another language, there is a powerful GUI, to protect you from it. IRI Workbench is a graphically integrated development environment (IDE) based on Eclipse,™ that is used to create, run, share, and modify SortCL jobs and their „Data Definition File (DDF)“ source and target layouts.

Did you know?

SortCL was introduced in 1992. It remains widely used because it is uniquely explicit, open, and powerful. SortCL has achieved its design goals:

  • modular, so that you can add features iteratively within the same I/O pass
  • expandable enough for complex data transformations and reports
  • clear, so that everyone can create, read, reuse, and extend tasks
  • familiar to JCL users, SQL-savvy DBAs, and DW/BI data architects
  • faster than SQL and ETL tools, without DB or server overhead
  • Real-time, batch, CLI, API, and GUI ops (not compilation!)
  • portable, so that the jobs run on all Unix, Linux, and Windows platforms
  • easy to integrate into new applications, SaaS applications, or ASP models

SortCL job scripts execute, accelerate, and/or combine Big Data transformation, conversion, protection, and BI in the world's largest production environments.

Metadata Interoperability

Quickly find and reuse SortCL metadata across multiple IRI products and projects to meet your data processing, protection, presentation, and prototyping needs. Leverage SortCL job scripts and DDF layouts throughout the IRI ecosystem and manage their usage and lineage directly within Eclipse or available cloud repositories such as Git, CVS and SVN.

SortCL DDF is also compatible with Erwin (formerly AnalytiX DS) Mapping Manager and Meta Integration Model Bridge (MIMB) from Meta Integration Technology, Inc. Convert the metadata already in use by your BI, CRM, database, ETL, and modeling tools into the metadata used by any IRI software. This preserves your existing metadata investments and allows you to more quickly leverage the benefits of IRI software that runs with (or as a replacement for) those applications.