logtalk_tester(1)

NAME
logtalk_tester - Logtalk script for automating running unit tests

SYNOPSIS
logtalk_tester -v
logtalk_tester -p prolog [-o output] [-m mode] [-f format] [-d results] [-t timeout] [-n driver] [-s prefix] [-u url] [-c report] [-l level] [-i options] [-g goal] [-r seed] [-w] [-- arguments]
logtalk_tester -h

DESCRIPTION
logtalk_tester is a script for automating running Logtalk unit tests. This script runs unit tests found on the current directory and recursively in its sub-directories by scanning by default for tester.lgt and tester.logtalk source files. The script returns a non-zero exit status in case of failed tests or test set errors. When a tester.sh file exists in the tests directory, the file is sourced before running the tests. The tester.sh file is sourced with all the parameters passed to the script. The script prints by default a full report of test compilation errors and warnings, skipped test sets, broken test sets, crashed test sets, skipped tests, failed tests, and test set and test stats. Note that the existence of broken or crashed test sets may result in some inconsistency on test stats, notably in the number of passed and failed tests.

OPTIONS

-v  Print script version.

-p  Backend Prolog compiler. Required. The following identifiers are used: B-Prolog - b, Ciao Prolog - ciao, CxProlog - cx, ECLiPSe - eclipse, GNU Prolog - gnu, JIProlog - ji, LVM - lvm, Scryer Prolog - scryer, SICStus Prolog - sicstus, SWI-Prolog - swi, SWI-Prolog with Logtalk pack - swipack, Tau Prolog - tau, Trealla Prolog - trealla, XSB - xsb, YAP - yap.

-o  Output verbosity. Default is verbose. Valid values are verbose and minimal.

-m  Compilation mode. Default is normal. Valid values are optimal, normal, debug, and all.

-f  Format for writing the test results. Default is default. Valid values are default, tap, xunit, and xunit_net_v2.

-d  Directory to store the test results. Default is the local ./logtalk_tester_logs directory.

-t  Timeout in seconds for running each test set. Default is 0 (i.e. disabled).

-n  Name of the tests driver and sourced files (minus file name extensions). Default is tester.

-s  Path prefix to suppress when printing test suite and test result paths. Default is the user home directory.

-u  Base URL for generating links to test files (no default).

-c  Code coverage report. Default is none. Valid values are xml and none.

-l  Directory depth level to look for test sets. Level 1 means current directory only. Default is to recurse into all sub-directories.

-i  Integration script command-line options (no default).

-g  Initialization goal. Default is true. Called after loading the Logtalk compiler and runtime.

-r  Random generator starting seed (no default).

-w  Wipe test set default scratch directories (./.lgt_tmp and ./lgt_tmp) before running it. This option should not be used when running parallel processes that use the same test sets.

--  Arguments to be passed to the tests (no default).

-h  Provide help on script usage.

EXIT STATUS

0   All non-skipped tests pass (flaky test failures are ignored) and there are no broken, timedout, or crashed test sets.

1   Some (non-flaky) tests fail but there are no broken, timedout, or crashed test sets.

3   One or more test sets timedout while executing but there are no broken or crashed test sets.

5   One or more test sets are broken and thus could not be executed but there are no crashed test sets.

7   One or more test sets crashed the process running them.

9   The sourced tester.sh file returned a non-zero code.

EXAMPLES

Run all tests on the current directory and sub-directories using the GNU Prolog backend, with results in TAP format, code coverage reports in XML format, and passing arguments foo, bar, and baz to the tests:
    logtalk_tester -p gnu -tap -c xml -- foo bar baz

Run all tests on the current directory only using the ECLiPSe backend, with a timeout of 4 minutes per test set:
    logtalk_tester -p eclipse -l 1 -t 240

Run all tests on the current directory and sub-directories using the SICStus Prolog backend and a random generator starting seed:
    logtalk_tester -p sicstus -r "seed(3172,9814,20125)"

Run all tests on the current directory and sub-directories using the SWI-Prolog backend and looking for tester_debug driver and sourced files:
    logtalk_tester -p swi -n tester_debug

Run all tests on the current directory and sub-directories using the Tau Prolog backend, caching the compilation of the testing tool for better performance, but wiping the test set default scratch directories before running it:
    logtalk_tester -p tau -g "set_logtalk_flag(clean,off)" -w

Run the Logtalk distribution tests using the Trealla Prolog backend, passing a base URL that can be used for creating links to the test files:
    logtalk_tester -p trealla -f xunit_net_v2 -s $HOME/logtalk/ -u https://github.com/LogtalkDotOrg/logtalk3/tree/master/

IMPLEMENTATION NOTES
When the script detects either a timeout or a gtimeout command (provided by the GNU coreutils package), it will use it to run each test set if the timeout option is set to a value greater than zero. This package can be easily installed in POSIX operating-systems (including Linux and macOS) if not provided by default. On Windows operating-systems, the native timeout command is not usable for this purpose. But the Git for Windows installer includes the GNU coreutils package, which can be used from the bundled Bash shell.

Installation of the GNU sed command, when not available by default, is strongly recommended. The Git for Windows installer includes this command, which can be used from the bundled Bash shell.

ENVIRONMENT

LOGTALKHOME
    Specifies the location of the Logtalk installation.

LOGTALKUSER
    Specifies the location of the Logtalk user directory.

FILES

$LOGTALKUSER/library/tester_versions.lgt
    Logtalk source file used for printing the Logtalk and backend Prolog compiler versions.

$LOGTALKUSER/library/parallel_logtalk_processes_setup.pl
    Logtalk source file used for parallel Logtalk processes with selected backend Prolog compilers.

$LOGTALKUSER/tools/lgtunit/automation_report.lgt
    Logtalk source file that intercepts unit test execution messages and generates report files that are parsed by this script.

SEE ALSO
The Logtalk website at https://logtalk.org/

The Logtalk User and Reference Manuals at $LOGTALKHOME/manuals/index.html

AUTHOR
Paulo Moura <pmoura@logtalk.org>

COPYRIGHT
Copyright (c) 1998-2021 Paulo Moura.