Counting Tests of Adobe's Full Screen Navigation Options

 

Summary

In this example we compute the number of possible tests for a small part of Adobe Reader v6.0. We focus on the full screen panel of the preference dialog which provides settings that will affect the behavior of the program in full screen mode.

Application Description

Adobe Reader 6.0 lets you view and print Adobe Portable Document Format (PDF) files on a variety of hardware and operating system platforms. (Source: www.adobe.com).

These are the options that we focus on in this example:

 

The description of these options in the manual is:

Counting the Possible Tests

Let's start by listing the fields that accept input and all possible inputs for them.

Field name
Possible values
Note
Advance every
2
true or false
seconds
65,535^2 *
If Advance every is false, this option is disabled.**
Loop after last page
2
true or false
Escape key exits
2
true or false
Left click to go forward...
2
true or false

* 65,535 is the number of Unicode characters using UTF -16. The field accepts two characters, which can be anything (they should be numbers but the application does not enforce it, so we have to test everything). 65,535 for the first character times 65,535 for the second characters gives us 4,294,836,225 of distinct inputs for this field.

**

 

When "Advance every" is false, "seconds" field is disabled. Looking at the bottom three options, there are 2*2*2= 8 different inputs for this particular options group.

When "Advance every" is true, there are 4,294,836,225*2*2*2= 34,358,689,800 different inputs.

The total number of possible inputs is 8 + 34,358,689,800 = 34,358,689,808.

Results/Relevance

Even though the number of test cases that we have to perform is huge, it is only the number of available inputs. This means that we computed the number of test cases based on what is the input, but we didn't count the any test cases based on:

  1. "how" we enter the input (e.g. repeatedly changing an option to true and false),
  2. "when" we enter the input (e.g. setting an option from true to false and immediately pressing "OK" to see if the application captured the last input).
  3. "what else is running" (e.g. some option might not work if there is program X running at the same time).

and so on...

This means that even if we take the time to perform all 34 billion tests, there is no guarantee that we would catch all bugs related to this area.

Configuration

Testing Adobe Systems' Adobe Reader v6.0:

Created 21 September 2004 for the CSTER. Modified December 2004.

All images and written material ©Copyright Georgi Nikolov 2004

This work is licensed under the Creative Commons Attribution-ShareAlike License.
To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/2.0/
or send a letter to
Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305,
USA.