JSON files

Convert, Transform, Mask, Report, Test

Overview
CLF/ELF
CSV
Excel
JSON
LDIF
LS-RS-VS
MF-ISAM
MFVL
Text
Variable blocked
Vision
VSAM
XML
Challenges

JSON is a popular data language and semi-structured file exchange format that is common in big data applications, NoSQL database collections, and IoT data streams. However, conversions between legacy index or flat files and JSON are based on slow parsing technologies that do not allow for simultaneous processing.

Other JSON query or transformation tools on the market cannot or inadequately convert large amounts of JSON data into analytical subsets or gather information quickly or at all. There was no efficient way to quickly convert, process, secure, or create massive JSON files.


For example, it may be necessary:

· Sorting a huge JSON file

Extracting data or reports from a JSON file

· Convert a CSV, LDIF, or other file to JSON

· Convert JSON to Text, CSV, LDIF, ISAM, etc.

Link data in a JSON file with another JSON or another source

Masking, encrypting, or otherwise de-identifying PII in a JSON file

Loading JSON data into a spreadsheet or database

· Creating a JSON file from an Alt or Extract file

Generation of test data in JSON file formats

You may even need to perform more than one of these functions simultaneously, across many massive source and target files.

Solutions

Starting with Ab CoSort v10, IRI offers JSON data conversion and processing capabilities across multiple products. Choose as needed:

Use the IRI NextForm, to move JSON files into DB tables, convert a JSON file to another file format (such as CSV, LDIF, COBOL, XML, etc.), or convert from another file format into JSON.

NextForm includes a JSON file parser to automatically generate field layouts used in file conversion scripts. NextForm also supports field-level data type conversion and record layout remapping. NextForm job definitions also work in SortCL-compatible products like Voracity if you upgrade later.

There are initially known limitations in handling function calls and multiple array elements in unstructured JSON files. Provide the IRI with a sample for analysis if possible.

Use the program SortCL in the IRI Voracity-platform or in IRI CoSort- Package to convert, transform, mask, report, and regenerate JSON files and other targets that represent structured data.

Declare one or more JSON and non-JSON files for input and output as part of any SortCL job that includes data:

Filtering (Selection, Scrubbing, Links to DQ Tools)
Transformation (Sorting, Merging, Aggregation, Calculation, etc.)
Conversion (Data Type and File Format Migrations)
Reporting (CDC, detailed and summary formats)

Protection (Field Encryption, De-ID, Masking)

SortCL makes all these functionalities, one or more at a time, available to data architects working with JSON and other sources must work.

Use IRI FieldShield, to encrypt, mask, or otherwise de-identify values in structured JSON files, or IRI DarkShield, if the data you need to find and mask within them, or NoSQL DBs are less structured. FieldShield and DarkShield are both within the IRI Voracity platform.

Use IRI RowGen, if you need test data in JSON file formats. RowGen is in IRI Voracity contains and uses the same layout metadata as CoSort, NextForm and FieldShield, so that you can easily switch between generating test data and transforming real data.

 
Product links