About IRI Metadata

 

Supporting Multiple Objectives

The syntax for business, technical, and master metadata common to all IRI software is the "data definition file" or "DDF" format. It is a primary, but far from the only, form of metadata being handled in IRI software, due to the wide range of jobs and data sources it supports (e.g. SQL DDL, XML ETL .flow.

 

DDF is a subset of the process metadata for the SortCL (.scl) data manipulation and reporting program in IRI CoSort, both of which are also the basis of:

  • IRI Voracity ETL, BI, and other jobs it supports in .flow / .bat / .sh files, including
    • IRI NextForm (.ncl) data migration/replication jobs
    • IRI FieldShield (.fcl) data discovery and masking jobs
    • IRI RowGen (.rcl) test data generation and population jobs

 

These are all simple text files defining the layouts for your data sources and jobs, all of which:

  • Support Data and Application Independence

    Define job commands and data layouts together or separately in explicit, logical text files -- metadata repositories. Centralize the layouts for re-use.

     

    Use the same source layouts in multiple applications. Save and re-use mapped target layouts as metadata sources for new applications.

     

    Specify multiple targets and layouts at once to reflect multiple layouts and views of the same data.

  • Support Master Data Definition and Management

    Specify existing or custom-defined master data formats in files or tables within DDF repositories. Use those layouts in IRI Voracity ETL jobs, as well as any CoSort/SortCL-based transformation, cleansing, federation or reporting application.

     

    Create DDF files manually, or automatically in the IRI Workbench graphical Metadata Discovery wizard.

     

    Manage your DDF files, master data, and other metadata assets in a secure repository for team sharing, version control, and lineage tracking.

     

    Convert to SortCL DDF from 3rd-party file layouts using command line utilities in the CoSort package, the Workbench, or MIMB.

  • Support Compliance and Governance Goals

    IRI metadata supports field-level protection functions like data masking and encryption along with data integration and reporting.

     

    The DDF metadata -- and its inclusion in audit logs -- allow you to map these processes, and help satisfy the requirements of your risk and controls framework.

     

    IRI's metadata and operational infrastructure allow you to replace multiple processes with one product, job script, and I/O pass. Simplicity, transparency, security, compliance, and auditability are included.

For background on enterprise metadata and IRI's points of view around it, see this blog article.