Continuous Testing - Continuous Integration - page 158

Summary

How reliable do you want your software to be? Source code is only as
reliable as the test coverage, and tests are only as valuable as their
execution frequency. By segregating tests into four automatable cate-
gories mapping to unit, component, system, and functional, a CI sys-
tem can be configured to execute tests in an efficient manner. Unit
tests can be run during checkins; component, system, and functional
tests on some regular interval—such as with a secondary build.
Table 6-1 summarizes the practices covered in this chapter.

CI Practices Discussed in This Chapter

TABLE 6-1

Practice Description

Automate unit tests Automate your unit tests, preferably with a unit test-

ing framework such as NUnit or JUnit. These unit
tests should have no external dependencies such as
a file system or database.

Automate compo-
nent tests

Automate your component tests with unit testing
frameworks such as JUnit, NUnit, DbUnit, and
NDbUnit if you are using a database. These tests
involve more objects and typically take much longer
to run than unit tests.

Automate system
tests

System tests are longer to run than component tests
and usually involve multiple components.

Automate functional
tests

Functional tests can be automated using tools like
Selenium (for Web applications) and Abbot for GUI
applications. Functional tests operate from a user's
perspective and are typically the longest running
tests in your automated test suite.

Categorize devel-
oper tests

By categorizing your tests into distinct “buckets,” you
can run slower running tests (e.g., component) at dif-
ferent intervals than faster running tests (e.g., unit).
Run faster tests first Run your unit tests prior to running component, sys-
tem, and functional tests. You can achieve this by
categorizing your tests.

Write tests for
defects

Increase your code coverage by writing tests based
on new defects and ensuring that the defect does not
surface again.

Make component
tests repeatable

Use database testing frameworks to make certain
that the data is a “known state,” which helps make
component tests repeatable.