3.3 Test Routines

Test routines are required for every VICAR class 2 (R2LIB) application and subroutine. Programs and procedures typically have a procedure PDF as the test routine.

Subroutines must have a test program that calls the subroutine, as well as a procedure PDF that calls the test program. The test program must be portable, and have its own imakefile and its own PDF.

Subroutines often have four test files :the test program, the imakefile, the test program PDF, and the test PDF that calls it. These files are under the TEST file category in vpack. To run the test, first unpack the TEST files, run vimake on the test imakefile, build the program, and execute the test PDF. The test programs and scripts should test all of the major functions of the program, rather than just testing that the program doesn't crash.

The log that's generated when the test is run must in most cases be delivered with the program. This ensures that you run the test before delivery, in order to make sure it still works. Run diff on the new log versus the old log, to make sure nothing changed.

Checking the logs may be difficult because TAE automatically puts usage statistics in the log after each program execution. The date, amount of time, page faults, etc. used will vary every time the test is run, and yet make no difference in the results of the regression test. The usage statistics generate lots of meaningless differences, which can hide any real differences.

For this reason, the TAE global variable $AUTOUSAGE has been added to control the automatic printing of usage statistics. It takes three values: "BATCH", "ALL", or "NONE". Use the TAE command "refgbl $AUTOUSAGE" before the "body" statement, then set it with a standard "let" command. The values are described below:

• BATCH: The default. In this mode, automatic usage statistics are printed after every program execution in batch mode. In interactive mode, usage statistics are not printed unless the usage command is explicitly given.
  
  
• ALL: Automatic usage statistics are printed after every program execution, in both batch and interactive modes.
  
  
• NONE: Suppresses automatic usage statistics in both batch and interactive modes. With "NONE" set, the only way to get the statistics is via the usage command.

We recommend that $AUTOUSAGE be set to "NONE" in every test script, to avoid unneeded usage statistics.

It is possible that the results of a test involving floating-point calculations may be differ on various machines in the least significant few digits. This is due to differences in numerical representation and precision, as well as possible differences in library routines. The acceptable difference between the results is defined in the MSTP Software Requirements Document, by Steve Pohorsky, JPL D-10637.