Hardware Reference
In-Depth Information
This experience had also a secondary objective as a check of the acceptance of test
generation based on UML activity diagrams: “correct” solution presented at the end
of the experience was generated using this method. The sample of 71 IT professionals
(discarding unreliable tryouts and incomplete data) who participated in the first wave
is shown in Table 3. Average development experience of respondents is 5.6 years and
average time devoted to the experience was 27 minutes.
Results were presented at [18] and [19] and they can be summarized as follows:
Only 1 participant covered more than 75% of the options of the program. 70.4% of
participants did not reach the 50% of coverage of functional options, 13% did not
detect any of the 4 injected defects and 40.8% detected at least 3. As an average,
50% of defects were detected and participants claim detection of 8 not real defects.
•
•
As an average, around 50% of the cases designed by a tester were oriented to test
program options previously controlled in similar cases executed by him/her. This
was especially intense in test cases oriented to enter data in the program (e.g. insert
new DVD data) rather than when deleting or modifying records.
•
On one hand, among the 10 most executed test cases, there was only one of the ten
most important ones according to participants' own rank of priority. On the other
hand, among the 10 least executed cases, there were 3 of the most important ones.
•
As additional information, it was also shown that practitioners considered a trade-
off to invest in detailed UML models like Activity Diagrams for software specifi-
cations in order to gain productivity and effectiveness in test case generation using
the AQUABUS method and its associated Eclipse plug-in [19].
These results reveal a weak situation and an opportunity for improving both effec-
tiveness and efficiency through a more systematic design of cases. Nor organizational
practices neither individual abilities of developers offer good results for productivity
and quality in software so our next logical step was the investigation of possible
causes of this situation: a detailed study was launched in 2007.
4 Survey on Factors Which Influence Testing Practice
Although some information on which is the state of practice in software testing is
available, it is really difficult to find analysis on which can be the causes of the situa-
tion. In general, it is possible to locate articles (e.g. [20] [21]) based on subjective
personal analysis of experts analyzing or explaining the contributing factors that im-
pede efficient and effective application of software testing best practices. However, it
is difficult to find works based on evidences, quantitative data or, at least, analysis of
experiences (e.g. [22] or [23]) although specific surveys (e.g. [15]) have included
questions on which are some of the barriers for better performance in software testing.
Another interesting approach is the use of ethnographic methods to capture and ana-
lyze the work of software developers in projects [4] as they allow realizing a distance
between theory and practice exists in real testing practices.
In our case, analyzing the results in organizations (Section 2) and the ones of indi-
vidual performance in test case design (Section 3), we decided to investigate the
possible causes of such situation. As part of the research network REPRIS (focused
on software testing in software engineering and funded by the Spanish Ministry of
