Project Background

In regional deliveries we often face problems with performance of solutions when delivering solutions to production. We push project planning templates to plan for and conduct performance tests on solutions, but often due to project timelines and provision of performance data the level of performance testing is not sufficient and causes issues on production. Also with traditional delivery of solutions not meeting the Experian cloud solutions requirement, performance data is generated from legacy provisioning systems which cannot be brought in-house due to PII sensitivity.

There is a strong need to generate non PII sensitive data in large volumes early in projects to help identify any performance issues, and mixed with automated testing framework can report on solution changes which impact negatively on the overall system performance.

What was my role

My role as a Global Technical Consultant is firstly to investigate the pain points for global delivery teams, and work on innovation to alleviate delivery issues and speed up the implementation time and increase quality of deliveries globally.

For this project, my main role was as the application developer on the solution which handled the identified problems from delivery and to meet all requirements on the specification of the tooling. This solution needed to be flexible to manage all global customer solutions interfacing specifications and platforms.

What I learnt on this project

Revisiting my skills in Java application development

Development with hibernate and java design patterns

Working performance testing into an automated framework to help detect configuration changes impact on performance over time

How the tool works

The tool is quite simple in conception, but through this simplicity it has a lot of uses and applications. A brief summary of how it is developed:

  • Accepts sample account or person records via file interfaces anywhere from 1 record upwards, which is validated to load according to project interfaces. The larger the data sample the more varied the generated dataset.

  • Extracts key/composite key from base records

  • Parameterization of tool allows for sanitization of base record data.

  • User specifies the record volume required. Tool has been tested up to 10 million records, but no technical limit on data generation.

  • Tool then generates the specified volume, and also generates PII compliant contact data interfaces (contact details, phones, email, addresses) from scratch with relationships to account & person records.

Referential integrity

The tool is based on a master entity and generates an unique ID data set for the configurable volume of records specified. As the tool is reading from a small sample data set, it separates the record data from the key or composite key data as a first step and stores that data in memory. Then the computes the looping of data based on the sample set to generate key information for the full requested set.

Along with the sample account data for example, the toll generates contact related interfaces and relationship interfaces to that data, storing the account key array in memory. Likewise storing generated contact key data to memory for its related interfaces. In this way a full dataset is created, with all contact data feed through either sanitized DB tables or as property file array data. The depiction here shows the relationship keys and referential integrity maintained by the tool in the generation of an interface package.

Practical uses

This tool is being used by many regional delivery and global teams:

  • Credit card system in the UK, as credit card information is highly sensitive the tool generates fake IDs and details to test performance for this solution circa 500K accounts.

  • Credit card banking system in India, circa 1million accounts.

  • Major Spanish financial institution circa 1 million accounts.

  • Used in the product benchmarking performance tests by global product development teams circa 10 million records.

  • Used in automated testing suite to monitor and identify performance degradation over time or due to non-performant solution changes.

The tool supports varying date and money formats, adapts to solutions interface structure changes by simple configuration (no coding) and is fully UTF8 compliant thus supporting Arabic, Chinese etc.

It is available as a global building block, thus subject to user guides and release notes.