Ballista® OS Robustness Test Suite:
Expanding to Non-POSIX Functions
The Ballista test suite can be used to test Non-POSIX functions after some modifications. The nature and degree of the modifications vary but are normally relatively manageable. The following steps are guidelines which outline the possible changes you will need to make.
The callTable.all or callTable<system>.all files have a strict format. Each line in the file corresponds with a single function entry. The function entry consists of fields:
|include_file||Exactly one file name of a system header file (.h) to include in the compilation. Do not
include the angle brackets as they are implied. If no system .h file is required,
use "stdlib.h" as a placeholder. User-supplied .h files are specified in the
Occasionally, system files are in sub-directories instead of main directory for system header files. To accomodate these alternately located files prefix the header file name with the sub-directory. (for example sys/stat.h)
|call_type||There are only two options for this field. Either the keyword "function" for calling a c function or the keyword "constructor" for calling a C++ constructor. The only difference in processing the different call_types is that "constructor" inserts the keyword "new" before the function name in the generated code.|
|return_type||C/C++ return type of the function. This field cannot contain any whitespace. For example please use "void*" instead of "void *".|
|function_name||The name of the function to be tested.|
|[+]b_arg||A space-separated list of one or more Ballista datatypes corresponding, in order,
to the arguments of the function.
Any datatype preceded with a "+" is a "phantom" parameter. Phantom parameters are used for variable construction/destruction purposes and do not correspond with a parameter from the function's parameter list. For more on phantom parameters.
|userSetup and userShutdown||is the simplist method. This method creates static scaffolding code that is run before and after each test case. The same state is reached with each test case using this method.|
|phantom parameters||is a bit more complex but is also more versatile. This method treats the setup/tear down code
as variable that can be randomly varied as a parameter to the test case. Phantom parameters are
defined in the same manner as ballista datatypes. The definition of the
Ballista datatypes in this case should also set/tear down the state.
For example, when testing the POSIX function rand() then it might make sense to use a phantom parameter that sets the seed. In order to do this you would need to define a Ballista datatype that varied the seed values, and reference this new datatype in the rand entry in callTable.all. The new Ballista datatype should define different dial settings, and call srand with the setting.
The userIncludes.h file contains a list of header files (“.h” files) and potentially other information needed to compile the function under test. The userIncludes.h file is treated as being entirely C++ code, so comments can be inserted with “/* … */” and “//” as in any C++ program. The userIncludes.h file applies globally to all functions in the callTable.
The Ballista test suite automatically processes anything placed in the file userIncludes.h in the /ballista directory.
The userLibs file contains a listing of object files to be included when compiling, with one filename per line in the the file. The userLibs file can contain comments, indicated by a line starting with ‘#’.
The Ballista test suite automatically processes anything placed in the file userLibs in the /ballista directory.