Test data files and reports

Produce secure, application-specific test data

Overview
DevOps
Benchmarking
Conformity
Database test data
Database Subsetting
Test files/reports
Test Data Virtualization
TDaaS
Challenges

 Database application developers often rely on data in production tables for testing. But there are several drawbacks to this approach, including:

Confidentiality of data in these tables

2. Costs for migration, masking, updating, and/or storage of replicated databases for testing

3. Redundancy of production data, leading to wasted space and insufficient test coverage

4. Only small amounts of data are needed for specific test cases

Sometimes smaller, masked subsets of large files are also needed for quickly testing applications with anonymized data. Most data masking tools cannot support the volume and variety of flat files involved.

Solutions

The IRI RowGen-Software can synthesize and adapt test data directly in many structured file formats as well as in detailed and summary report targets:

Excel (XLS and XLSX, in row or column format) - See examples # 1 and 2 here

RowGen generates field values randomly across more than 100 data types. It can also randomly select (or mask) data from real datasets, so test data can contain both artificial and real (but encrypted or otherwise de-identified) data. RowGen also applies intelligent selection and exclusion criteria, as well as desired data frequency distributions, to enhance data realism.

Assets in these formats can be stored in standalone files in local folders or cloud buckets in Amazon S3, Azure Blob, and Google Cloud Platform (GCP). They can also flow to other test targets through pipes, procedures, or message queues.

Embedded support for standard and complex data transformations, set file (value) lookups, conditional filtering, and custom formatting helps RowGen simulate production files and report formats for a wide range of applications. Once these files or streaming test data sets are created, you have test file management (test data provisioning) capabilities available, which include static, automated, and virtualized test data targets, as well as test data management hubs and CI/CD platforms such as Jenkins, Amazon Code Pipeline, Azure DevOps, and GitLab.