<<TableOfContents(3)>> == Writing Qualification Tests == Developers can use the [[qualification]] system to write a test for almost any robot or robot sub-assembly. Developers can also write configuration scripts, to allow easy reconfiguration of a part (see [[../ComponentConfiguration|Component Configuration]] for details). === Outline Test Structure === To write a qualification test, start by making a folder in the `qualification/tests` directory, with the name of a component. In that folder, make a file called 'test.xml'. In the case of the simple_example test, it looks like this: {{{ <test check-assembly="true" id="my-test" > <instructions>instructions.html</instructions> <pre_startup>pre_startup_script.launch</pre_startup> <startup>startup.launch</startup> <subtest>test1.launch</subtest> <subtest>test2.launch</subtest> <subtest>test3.launch</subtest> <shutdown>shutdown.launch</shutdown> </test> }}} Each launch file is given as the basename of the file in the directory of test.xml. The qualification manager will look for the specified files using the path of your directory. For example, "../caster_test/caster.launch" would load `qualification/tests/caster_test/caster.launch`. == Parts of Qualification Tests == Our test contains five parts: * Instructions, given as an HTML file * Prestartup scripts: anything we need to do before we start our tests * Startup scripts, which generally launch drivers needed such as pr2_etherCAT * Subtests, which run any type of test on the device and send the analysis to the qualification system * Shutdown scripts, which stop any processes we started (mainly the power board) Except for the instructions file, these scripts and subtest are all [[roslaunch]] files. === Instructions === The qualification GUI will show the user the instructions first. All instructions must contain the following items: * Picture of component * Identifying information of the component, to make sure that the user if plugging in the correct one * Connecting instructions (power, etherCAT cables) * Test details, what to look for, etc The instructions should have the test name as an H2 heading, with images roughly 640 by 480. A sample instruction file is below: {{{ <html><body> <H2>Instructions File</H2> <H3>Instructions Subheader</H3> <img src="my_image.png" width="640" height="480"> <H3>Verifying Component Type</H3> <p>Put verification information here...</p> <H3>Running Tests</H3> <p>Put stuff about running tests here.</p> </body></html> }}} Try to follow the above example as a "style guide" for instructions. === Prestartup Scripts === Prestartup scripts generally do things like enable the power supply, and program or configure MCB's. These steps are an important part of checking mechanical components. The qualifcation GUI will launch the prestartup scripts, in order specified in the test.xml file after the instructions. When launching the prestartup scripts, the qualification system will advertise a <<SrvLink(pr2_self_test_msgs/ScriptDone)>> service on the topic `prestartup_done`. All prestartup scripts must call this service when finished. The prestartup script should fill the ScriptDone/result field as follows: * 0/OK: Prestartup completed successfully * 1/FAIL: Prestartup failed (ex: MCB's failed to program) * 2/ERROR: Prestartup recorded an error (ex: Power node wouldn't launch correctly) Failure and error terminations will stop the test (call shutdown script and stop all launched scripts). Prestartup failures will be logged in the inventory system for each part, since they reflect the part's status, but errors are not. Do not let your prestartup script return an error code unless you are sure that it is not due to a bad component. For example, a power node failure is an error, but a bad MCB is a failure. Prestartup scripts do not run tests. Generally, they are done before tests to prepare components for qualification by turning on power and configuring MCB's. === Startup Scripts === Startup launch files are called before subtests and after prestartup scripts. Startup scripts activate drivers such as hokuyo nodes and [[pr2_etherCAT]]. The drivers called in the startup script will remain active during all qualification subtests and are shutdown after a test is finished. {{{ <launch> <!-- robot descriptions param --> <param name="robot_description" command="$(find xacro)/xacro.py '$(find qualifica\ tion)/tests/head_test/head.xml'" /> <!-- Loads realtime machine and PR2_etherCAT --> <include file="$(find qualification)/tests/init.machine" /> <include file="$(find qualification)/tests/pr2_etherCAT.launch" /> <!-- Head Calibration --> <node pkg="mechanism_bringup" type="calibrate.py" args="$(find qualification)/tests/head_test/head_cal.xml" /> </launch> }}} Use the files `qualification/tests/init.machine` and `qualification/tests/pr2_etherCAT.launch` for setting up test machines and using pr2_etherCAT respectively. ==== Qualification Machines ==== The file `init.machine` file loads two machines, `localhost` and a `test_host`. The test host is determined by the environment variable `ROS_TEST_HOST`, which is set by the qualification system. === Subtests === For more details on writing qualification subtests, see the [[../Writing Subtests|Writing Subtests]] page. Subtests form the backbone of the qualification system. Each subtest is started by a launch file, and calls the 'test_result' service of type <<SrvLink(pr2_self_test_msgs/srv/TestResult)>> with its results. Since the qualification GUI does not do any analysis itself, it is up to the subtests to analyze results and output graphs and data. === Shutdown Scripts === Typically shutdown scripts disable the power board after a qualification test. Like prestartup scripts, shutdown scripts call the `shutdown_done` service which takes a <<SrvLink(pr2_self_test_msgs/ScriptDone)>> service. Any ScriptDone.result other than RESULT_OK will send an error message to the user. There can be only one shutdown script. == Adding Test to Qual System == Once a test is written, with a complete test.xml file and associated launch scripts, it must be added to the list of qualification tests. The file `qualification/tests/tests.xml` contains all the tests. Add new tests in the following format: {{{ <test serial="1234567" file="my_test/test.xml" descrip="My Test" /> }}} * "serial" allows the GUI to look up the test by the first seven digits of the serial number. Do not add more digits. * "file" gives the path to the qualification test.xml file, after qualification/tests * "descrip" is a short description, max 80 characters, to allow a user to distinguish two tests for the same serial == Verifying Your Test == After your test file is loaded into the `tests.xml` file, you can call "make test" in the qualification package to verify this file. This will make sure your XML file parses correctly and that your launch files exist. To parse your launch files as part of a build test use the [[rosbuild]] macro `rosbuild_add_roslaunch_check()` to the `CMakeLists.txt` file in qualification. === Debugging Your Test === It can be useful to debug your test by running it without the entire qualification system. You can roslaunch it manually, and use a viewer to view the results. {{{ rosrun qualification display.py }}} This will display the results of the qualification test is the same format as the normal qualification system. === Debug Mode === To run the qualification system in a "debug" mode, use the argument "--debug" to run the qualification GUI. {{{ rosrun qualification qualify --debug }}} Do not use debug mode for production testing. === Stress Testing === You can stress test you qualification test by running it "Continuous" mode. While in debug mode, select "Continuous Testing" in the debug menu. Your test will run continuously, and generate a log of success and fail runs.