Q-Bot

Powered By BI3 Technologies



About the Product


Q-Bot(Quality Bot) is a BI/DWH Test automation tool, which is used to validate the quality and accuracy of the data. It loads data from different sources, uses Apache Spark, and some other tools to convert the data as data frames and do different checks, finally produce the result as a report.




Data Validation can be done in different forms


We call the different validation forms as Test Group in the tool, These test groups can be created under different projects.

1. File Validation



  • The file type can be Text or CSV
  • The file can have File Header, Column Header, Detail Record, and Trailer
  • A file without a header also can be validated
  • The source file can be loaded from a remote location or from AWS S3
  • Direct file upload option also available
File Source
File Source


2. File to File Validation



  • The source and target files type can be Text or CSV
  • Both source and target files can have File Header, Column Header, Detail Record, and Trailer
  • File without header also can be validated
  • Both source and target files can be loaded from a remote location or from AWS S3
  • Direct file upload option also available
File To File Source
File to File Source
File To File Target
File to File Target


3. File to DB Validation



  • Validation will be done against a File and Database Table
  • The source file type can be Text or CSV
  • The source file can have File Header, Column Header, Detail Record, and Trailer
  • File without header also can be validated
  • The target can be any database table
  • Option to save the Database Connection String in separate UI, which can be used file execution
  • Option to configure the Connection Details at runtime
  • We currently support the below database servers for the Target Database table

    1. MySQL
    2. SQL Server
    3. Redshift
    4. Snowflake
File To DB Source
File to DB Source
File To DB Target
File to DB Target


4. DB to DB Validation



  • Validation will be done against two database tables
  • Option to configure the filters for source and target tables
  • Option to configure JOIN Conditions for source and target tables
  • Option for source and target tables columns mapping
  • Option to save the Database Connection String in separate UI, which can be used file configuring Test Groups
  • Table column names with space also supported
  • We currently support the below database servers for Source and Target Database tables

    1. MySQL
    2. SQL Server
    3. Redshift
    4. Snowflake
DB to DB Source
DB to DB Source
DB To DB Target
DB to DB Target

5. Custom Query



  • Like DB to DB, validation will be done against two database tables.
  • Here the user can directly pass the SQL Query while configuring the Test Group
  • Can directly write the where conditions, joins, subqueries and etc.
  • We currently support the below database servers for Source and Target Database tables

    1. MySQL
    2. SQL Server
    3. Redshift
    4. Snowflake
Custom Query Source
Custom Query Source
Custom Query Target
Custom Query Target



Different checks handled during Data Validation


Header Check - For file related test groups



Detail Record Check - For file related test groups



Trailer Check and Trailer Aggregation Check - For file related test groups



Primary Key Check



Null Check



Data Type Check



Length Check



Report Statistics



Range Check



Transformation



Row Level Validation Check





Execution Plan





Sample Reports


Report Summary


The Report Summary page will show the below details.


Report Summary
Report Summay

Report Summary with Multiple Test Groups


Report Summary
Report for Multiple Test Groups

Report Details


In the Report Details view, the customer can see the detailed explanation of different Checks and its result for each column.


Report Summary
Report Detail for Source

Report Detail - Failure Scenario


When any checks fail for any columns that check against a column is marked as failed.

When the user clicks on the fail icon, the failed data will be shown and we have given an option to download the failed data as an excel sheet.


Report Summary
Report Detail for Failure Scenario

Report Summary
Report Detail for Failure Scenario - Primary Key Check

Report Summary
Report Detail for Failure Scenario - Detail Record Check

Report Summary
Report Detail for Failure Scenario - Unknown Row Check

Report Details view when we have the concept of Source and Target

Columns from Source and Target with the validation checks results will be shown.


Report Summary
Report Details for Source and Target

Report Statistics


In the Report Statistics view, the Min, Max, Average, Count, Distinct Records count will be shown for each number-type columns.


Report Summary
Report Statistics




Variants of Q-Bot




Q-Bot Standard Edition



High in features, Can be deployed into a Server/Instance.


Q-Bot Standalone Edition(Docker Version)



Can be used as a Docker Image.



Compare editions and top features


Feature

Standard Edition

Description

Standalone Edition

Description

Multiple Projects

Can create multiple projects. Can create multiple projects.

Multiple Test Groups

Can create multiple test groups. Can create multiple test groups.

Test Group Types

Can create all the test group types. Can create all the test group types.

Automated Execution

Execution of test groups are automated Execution should be done by the user manually by downloading and executing the execution package.

Automated Results Upload

Execution Results upload is automated Execution Results should be manually uploaded by the user

Test Groups per execution

Maximum of 5 test groups can be assigned to an Execution Plan Only one test group can be assigned to an Execution Plan

Users

We can add unlimited users to the tool Users are limited to 2



Interested in trying our Product, Reach us at qbot@bi3technologies.com for further assistance.