Test reporting system. Fast and it's not ugly maybe just a bit clumsy.
Why it was created? Because everything that is available usually based on "latest technologies", pages of such reporters usually overloaded with javascript,they loading reports and manipulating with them on the fly in memory, as result such systems usually pretty nifty and pretty slow even when working with small reports and absolutelly unusable with report more than few dozens MB.
This reporter based on php and mysql. The main strategy do not load data when you don't need it.
Using it you can build charts based on stat and view this stat, you can view report logs and it is loaded only when you really want to view it and still there is possibility to load report as file.
There is some integration with testrail you can preview and navigate to suite/testcase.
Start page with list of runs:
Run examples:
Execution statistic (run times and fail success stat):
$ git clone https://github.com/coozoo/FastAutoReporter
$ cd FastAutoReporter
$ git submodule update --init --recursiveCopy it to your webserver directory
Additionally you can update testrail API if required from here
https://github.com/gurock/testrail-api/tree/master/php
And replace stuff/testrail.php with a new one.
Currently there is no one place config.
Open mysqli_connection.php and assign your values instead of this:
$dbhost = "DBHOST";
$dbuser = "DBUSER";
$dbpass = "DBPASSWORD";
$db = "DBNAME";Open initvar.php find and assign your credentials inside:
$testrailhost='https://testrail.com/';
$testrailuser='TESTRAILUSER';
$testrailpass='TESTRAILPASS';For some files it's better to restrict access by password.
For example dbrestricted folder contains killprocess.php that allows to kill DB tasks sure it's better to protect it by password.
To do that you need to create .htaccess file (example):
$ cat .htaccess
Authtype Basic
AuthName "Password Protected"
AuthUserFile /var/www/html/FastAutoReporter/dbrestricted/.htpasswd
Require valid-userYou can add user to .htpasswd by:
$ htpasswd /var/www/html/FastAutoReporter/dbrestricted/.htpasswd newuserCurrent API implemented in a bit strange way for compatibility with old java service.
Each endpoint will return short info how to call it just call this page from browser.
./api/reporter/run/add.php - add new run;
./api/reporter/run/finish.php - mark run as finished
./api/reporter/suite/add.php - add suite;
./api/reporter/test/add.php - add test and logs;
For compatibility with old java service it can be added .htaccess file inside ./api/reporter folder to hide php extensions:
$ cat ./api/reporter/.htaccess
RewriteRule ^([^.?]+)$ %{REQUEST_URI}.php [L]
# Return 404 for php
RewriteCond %{THE_REQUEST} "^[^ ]* .*?\.php[? ].*$"
RewriteRule .* - [L,R=404]
blame.php - view that represents most fails and who is responsible for tests;
dbprocesslist.php - list current DB tasks;
downloadrunlogs.php - download zip file with all logs;
feature.php - simply call suite.php in another view mode;
getblob.php - blob loader from DB;
getrunstatus.php - status of test run;
gettestrailcase.php - preview testrail case;
index.php - main page with filters;
mysqli_connection.php - mysql connector;
runs.php - list of tables;
statuspiechart.php - pie chart with amount of success, fails for suite view;
suite.php - main view to see results of testrun;
systeminfo.php - DB info and task manager;
testdetails.php - info about test;
testhistory.php - show history of some test case;
testresultschart.php - this will build stats for filtered test runs (accept post parameters only);
testvideo.php - video loader.
MariaDB prefarable.
Just keep in mind it's just approximate settings. It's possible that you require less or more memory (really I don't think that you need more. Our reports pretty heavy and it's fast enough) for some them everything depends on amount of your data.
And don't forget to enable events event_scheduler=ON it's important for autofinish runs in case if your system crashed and unable to call run finish endpoint. As well it is used for cleaning purposes to remove old logs and test runs from DB (see Events section below).
Merge this difference to your MariaDB my.ini file
[client]
default-character-set=utf8mb4
[mysqld]
key_buffer=16M
max_allowed_packet=1024M
sort_buffer_size=512K
net_buffer_length=8K
read_buffer_size=256K
read_rnd_buffer_size=512K
event_scheduler=ON
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size=2048M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size=5M
innodb_log_buffer_size=8M
innodb_flush_log_at_trx_commit=1
innodb_lock_wait_timeout=50
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
[mysqldump]
max_allowed_packet=16M
Use phpmyadmin to import tables, procedures and events (all objects important for functionality)
reporterdb.sql - contains everything what you need to import
Those procedures are very important part of frontend reporter.
Here list of procedures used in project, purpose and examples how to call them:
close_running - procedure that estimates common run time for test run and close test run if time is exceeded.
Procedure doesn't require any parameters
count_runs - procedure returns currently visible amount of test runs.
All procedure parameters the same as for get_runs (basically it would be good to merge this precedures and return sets but in such case it is required to change php code a bit).
delete_old_logs - procedure will delete old logs.
Parameters:
IN number_of_days INT - number of days, everything older than this amount of days will be deleted
Examples:
call delete_old_logs(30) - delete logs older than 30 days
delete_old_runs - procedure will delete old test runs.
Parameters:
IN number_of_days INT - number of days, everything older than this amount of days will be deleted
IN is_dev_run BOOL - type of runs to delete;
Examples:
call delete_old_runs(7,true) - delete dev test tuns older than 7 days
get_blamed - procedure returns stats about amount of test cases by developer.
Parameters:
IN days INT - number of days, will gather stat for such amount of days;
IN statuses TEXT - statuses list separated by comma (FAIL,ERROR,PASS,SKIP);
IN who TEXT - developer names separated by comma;
IN teamid INT - team ID, not used in future possible to get stats by teamid;
IN runid INT - run ID, get stats for specified run;
Examples:
call get_blamed(1,'FAIL,ERROR',NULL,NULL,3841) - returns stats of failed test cases by dev for test run 3841
call get_blamed(1,'FAIL,ERROR',NULL,NULL,NULL) - returns stats of failed test cases by dev for one 1 since now
call get_blamed(1,'FAIL,ERROR','Developer Name',NULL,3841) - returns stats of failed test cases just for one dev for test run 3841
call get_blamed(1,'FAIL,ERROR','First DevName,Second DevName',NULL,3841) - returns stats of failed test cases by devs just for two devs in test run 3841
get_feature - procedure returns tests list for run by feature.
Parameters:
IN RUNID INT - get tests by run ID;
IN UUID VARCHAR(255) - get tests by run UUID (for compatibility with Java part);
Examples:
call get_feature(3841,NULL) - returns list of testcases by run ID
call get_feature(NULL,'6a6b304f-6ca7-46c4-a6aa-305258924706') - returns list of testcases by run UUID
get_runs - procedure returns all runs filtered by criterias.
Parameters:
IN userenv VARCHAR(255) - environment name;
IN limitstartrow INT - filter out rows before from (used for pagination);
IN limitnumberofrows INT - filter out rows after this (used for pagination);
IN testtypeid INT - test type ID (Backend, Frontend);
IN teamid INT - filter by team id;
IN featureid INT - not used (and I suppose it is bad idea to use it);
IN startdate DATETIME - filter out everything before this date;
IN enddate DATETIME - filter out everything after this date;
IN isdevrun BOOL - include dev runs tor result;
IN equalrunname VARCHAR(255) - filter by exact run name;
IN likerunname VARCHAR(255) - filter by run names that contains value;
IN equalversion LONGTEXT - filter by exact version;
IN likeversion LONGTEXT - filter by part of version;
Examples:
call get_runs(NULL,0,30,NULL,NULL,NULL,'2021-01-13 08:23',NULL,false,NULL,NULL,'',''); - filter by start date and return only 30 rows
call get_runs('DEV',30,60,NULL,NULL,NULL,'2021-01-13 08:23',2021-02-13 08:23,false,NULL,NULL,'',''); - filter results between date range by environment and return 30 items after first 30
call get_runs(NULL,0,30,1,1,NULL,'2021-01-13 08:23',NULL,true,NULL,NULL,'',''); - return results including dev runs filtered by test type ID and teamid
call get_runs(NULL,0,30,1,1,NULL,'2021-01-13 08:23',NULL,true,NULL,'LBA_DEV_Functional','','') - filter by part of run name
get_run_details - procedure to get brief info about run.
Parameters:
IN RUNID INT - get tests by run ID;
Examples:
call get_run_details(3841); - get run details by run ID
get_suit - procedure to get detailed info by suits.
Parameters:
IN RUNID INT - get tests by run ID;
IN UUID VARCHAR(255) - get tests by run UUID (for compatibility with Java part);
Examples:
call get_suit(3841,NULL) - returns suits by run ID
call get_suit(NULL,'6a6b304f-6ca7-46c4-a6aa-305258924706') - returns suits by run UUID
get_test_details - procedure to return list of logs for test case.
Parameters:
IN TESTID INT - get logs by test ID;
Examples:
call get_test_details(169222) - returns logs by test ID
get_test_history - procedure to return test case execution history.
Parameters:
IN RUNNAME VARCHAR(255) - test run name;
IN TESTNAME VARCHAR(255) - test case name;
IN startdate DATETIME - date from;
IN enddate DATETIME - date till;
Examples:
call get_test_history('DEV_FunctionalTestRun','successfulLogin','2021-01-13 08:23',NULL) - returns execution history by names from date
Events important for DB maintanance and some frontend functionality.
close_runs_in_progress - calls close_running procedure every minute to update test run executiion state.
remove_old_dev_runs - maintanance event to clear dev test runs it calls delete_old_runs procedure.
remove_old_logs - maintanance event to clear old logs it calls delete_old_logs procedure.