test_run

Runs tests

Syntax

• status = test_run()

• status = test_run([])

• status = test_run('minimal_tests')

• status = test_run('-stoponfail')

• status = test_run(modules)

• status = test_run(file_to_test)

• status = test_run(modules, '-stoponfail')

• status = test_run(file_to_test, '-stoponfail')

• status = test_run(modules, option)

• status = test_run(file_to_test, option)

• status = test_run('minimal_tests', '-stoponfail')

• status = test_run('minimal_tests', option)

• status = test_run([], '-stoponfail')

• status = test_run([], option)

• status = test_run(modules, file_output)

• status = test_run(file_to_test, file_output)

• status = test_run([], file_output)

• status = test_run(modules, option, xunitfile)

• status = test_run(modules, '-stoponfail', xunitfile)

• status = test_run(modules, option, xunitfile, '-stoponfail')

Input argument

• module_name - a string or a cell of string: module name or list of modules.

• file_to_test - a string or a cell of string: file to test or list of filenames.

• options - a string or a cell of string: supported options 'all', 'all_tests', 'unitary_tests', 'nonreg_tests' or 'benchs'.

• xunitfile - a string: filename to export results as a .xml or .json file compatible with Xunit format.

• '-stoponfail' - a string: stop tests execution at first 'fails' detected.

Output argument

• status - a logical: true if tests pass.

Description

test_run function searchs 'test_*.m', 'bug_*.m', and 'bench_*.m' files, executes them, and displays a report about success or failures.

Each test or bench is executed in a separated process using the 'unix' command.

That enables the current command to continue, even if the test as created an unstable environment.

It also enables the tests to be independent from one another.

Some special tags can be inserted in the .m files to help the processing of the corresponding test.

These tags are expected to be found in Nelson comments:

<--NOT FIXED--> This test is skipped because it is a reported bug, but it is not yet fixed.

<--INTERACTIVE TEST--> This test is skipped because it is interactive test.

<--CLI MODE--> This test will be executed by nelson-cli executable (default).

<--ADV-CLI MODE--> This test will be executed by nelson-adv-cli executable.

<--GUI MODE--> This test will be executed by nelson-gui executable.

<--CHECK REF--> This test will compare .ref available in same directory with output generated. see test_makeref to generate .ref file.

<--ENGLISH IMPOSED--> This test will be executed with the en_US language.

<--WINDOWS ONLY--> This test will be executed only on Windows.

<--MACOS ONLY--> This test will be executed only on Macos.

<--UNIX ONLY--> This test will be executed only on Unix.

<--WITH DISPLAY--> This test will be executed only if a display output is available.

<--RELEASE ONLY--> This test will be executed only if nelson is an release (not in debug mode).

<--EXCEL REQUIRED--> This test will be executed only if excel is detected (on Windows).

<--MPI MODE--> This test will be executed in MPI mode.

<--AUDIO INPUT REQUIRED--> This test will be executed if an audio input is available.

<--AUDIO OUTPUT REQUIRED--> This test will be executed if an audio output is available.

<--C/C++ COMPILER REQUIRED--> This test will be executed if an C/C++ compiler is available.

<--INDEX 64 BIT REQUIRED--> This test will be executed if 64 bit index is available.

<--NO USER MODULES--> This test will be executed without load user modules.

<--IPC REQUIRED--> This test will be executed if IPC is available.

<--SEQUENTIAL TEST REQUIRED--> This test will be executed sequentialy (1 worker).

<--NATIVE ARCHITECTURE TEST REQUIRED--> This test will be executed if application's build and architecture are same.

<--FILE WATCHER REQUIRED--> This test will be executed if file watcher is available.

<--PYTHON ENVIRONMENT REQUIRED--> This test will be executed if file watcher is available.

Test can also skipped dynamically using skip_testsuite function.

To avoid to block the application, tests have an execution timer of 2 minutes and the benchs have a timer of 6 minutes.

test_run uses n th workers to execute and speed up tests executions.

Tests with <--SEQUENTIAL TEST REQUIRED--> are evaluated last.

Benchs are evaluated sequentialy.

Examples

test_run('string');

test_run({'string', 'time'})

test_run({'string', 'time'}, 'all', [tempdir(), 'tests.xml'])

See also

assert, test_makeref, skip_testsuite.

History

Version	Description
1.0.0	initial version
1.3.0	PYTHON ENVIRONMENT REQUIRED tag added
1.4.0	skip_testsuite function reference

Author

Allan CORNET