Game Development Reference
In-Depth Information
Judging Scripts
To achieve a badge beyond Baseline, your application must pass the Baseline tests and the Superior and/or
Exemplary tests. Even though some tests are labeled Superior or Exemplary, they are not the only ones that test
advanced capabilities.
Each test comes with a judging script that looks at the outputs and status returns from the test. The judging
scripts are stored with the test cases in the
StandardDataSets
directory. You have to drill down to the lowest level of
a directory to find them; they're stored at the same level as the .
dae
file input to the test. These files contain useful
information about what the test is looking for in order to satisfy each badge level.
For example, if you look at the multi_contributor judging script (Test ID 0711, #2 on the grid) in the folder
C:\CTS\StandardDataSets\collada\library_geometries\geometry\asset\contributor\
, you will see that this test
involves all three badge levels and deals with them like this:
# JudgeBaseline: just verifies that the standard steps did not crash.
# JudgeSuperior: also verifies that the validation steps are not in error.
# JudgeExemplary: same as intermediate badge.
These comments will quickly tell you whether the test does anything special for the advanced badges. (You may
also want to check the code to make sure they're correct. There are cases in which the comments contradict the code.)
In this case, the Superior level and Exemplary levels require that the target application meet additional criteria.
You see also that at the Baseline level, not only must the test not crash, but import/export/validate must exist and
pass, while render must only exist.
You can verify which operations the test is performing in the code. Only import, export, and validate are used.
Render is absent.
self.__assistant.CheckSteps(context, ["Import", "Export", "Validate"], [])
However, in a script like
C:\CTS\StandardDataSets\collada\library_cameras\camera\_reference\_
reference_optics_orthographic_zfar_z_near
(Test ID 0106, #90 on the grid), all the operations are necessary.
self.__assistant.CheckSteps(context, ["Import", "Export", "Validate"], ["Render"])
The Render operation must exist because at the Superior level, the rendered images are compared, in this line:
self.__assistant.CompareRenderedImages(context)
The multi_contributor script tests all the levels separately, returning unique results for each, but the camera script
tests only the Baseline and Superior levels, returning the same result for both the Superior and Exemplary levels.
The Judging Script Driver
The judging script for each test passes parameters to the judging driver script, FResult, which is stored in the
C:\CTS\Core\Logic
directory. The judging script status codes govern what goes in the badge and results columns in
the GUI; see Listing 6-2.
Listing 6-2.
Judging Script Status Codes and Their Meanings
PASSED_IMAGE = 0
PASSED_ANIMATION = 1
PASSED_VALIDATION = 2
FAILED_IMAGE = 3
FAILED_ANIMATION = 4
