README.md 1.25 KB
Newer Older
Paul Cammish's avatar
Paul Cammish committed
1
2
# Sympl Testing Suite

Paul Cammish's avatar
Paul Cammish committed
3
4
5
6
7
8
9
10
11
12
13
14
15
16
A planned suite of tests to confirm various functionality of Sympl is working consistently between revisions.

Tests must:
1. Each individual test must clean up after itself *fully*, even when failing, unless the `--dirty` switch is passed.
2. Be able to be run using `run-parts`, with normal `00-aaaaaaa`
3. Numbering should run tests in order that they are likely to fail. Later tests will be skipped if one fails.
4. Each test must output minimal data (see below) unless the `--debug` switch is passed.
5. Failed tests must report errorcode 1 or higher and output a red (if running in a colour terminal) 'F', followed by the test name, and precicely what failed.
6. Detected errors (things not installed, or other unhandled errors) must report errorcode 255 and output a yellow (if running in a colour terminal) 'E', followed by diagnostic information.
7. Successful tests must exit 0, and echo a green (if running in a colour terminal) '.' with no line break as each part completes okay.
8. Be written in any language which is avilable on all supported Sympl versions, without extra libraries from a minimal install.
9. Fail safe at all times.

Tests will be run by git-pulling this repo, and running a single script within it, and must be self-contained.