Specification-Based (Black-Box)

What is Black-Box or Specification-based Testing?

It is also known as behavioral testing techniques!!

  • Specification-based testing technique is also known as ‘black-box’ or input/output driven testing techniques because they view the software as a black-box with inputs and outputs.
  • The testers have no knowledge of how the system or component is structured inside the box. In black-box testing the tester is concentrating on what the software does, not how it does it.
  • The definition mentions both functional and non-functional testing. Functional testing is concerned with what the system does its features or functions. Non-functional testing is concerned with examining how well the system does. Non-functional testing like performance, usability, portability, maintainability, etc.
  • Specification-based techniques are appropriate at all levels of testing (component testing through to acceptance testing) where a specification exists. For example, when performing system or acceptance testing, the requirements specification or functional specification may form the basis of the tests.

There are 4 specification-based or black-box test

1. What is Equivalence Partitioning (EP)?

  • Equivalence partitioning (EP) is a specification-based or black-box technique.
  • It can be applied at any level of testing and is often a good technique to use first.
  • The idea behind this technique is to divide (i.e. to partition) a set of test conditions into groups or sets that can be considered the same (i.e. the system should handle them equivalently), hence ‘equivalence partitioning’. Equivalence partitions are also known as equivalence classes – the two terms mean exactly the same thing.
  • In equivalence-partitioning technique we need to test only one condition from each partition. This is because we are assuming that all the conditions in one partition will be treated in the same way by the software. If one condition in a partition works, we assume all of the conditions in that partition will work, and so there is little point in testing any of these others. Similarly, if one of the conditions in a partition does not work, then we assume that none of the conditions in that partition will work so again there is little point in testing any more in that partition.

Example of Equivalence Partitioning

For example, a savings account in a bank has a different rate of interest depending on the balance amount in the account. For example, 4% rate of interest is for amount balance in range of RM0 to RM150, 6% rate of interest is for RM150 to RM1200, and 8% rate of interest is given if the balance amount is RM1200 and above.

From the above scenario, there are at least three (3) valid equivalence partitions and one (1) invalid partition are required for the test coverage as shown below.

The above equivalence partition is applied on the number (amount). Besides, we also can apply on the interest rate as well, like 4%, 6%, 8%.

2. What is Boundary value analysis (BVA) ?

  • Boundary value analysis (BVA) is based on testing at the boundaries between partitions.
  • Here we have both valid boundaries (in the valid partitions) and invalid boundaries (in the invalid partitions).
  • As an example, consider a printer that has an input option of the number of copies to be made, from 1 to 99. To apply boundary value analysis, we will take the minimum and maximum (boundary) values from the valid partition (1 and 99 in this case) together with the first or last value respectively in each of the invalid partitions adjacent to the valid partition (0 and 100 in this case). In this example we would have three equivalence partitioning tests (one from each of the three partitions) and four boundary value tests.

Consider the bank system described in the previous section in equivalence partitioning.
Because the boundary values are defined as those values on the edge of a partition, we have identified the following boundary values: -RM0.01 (an invalid boundary value because it is at the edge of an invalid partition), RM0.00, RM150.00, RM150.01, RM1199.99 and RM1200.00, all valid boundary values. So by applying boundary value analysis we will have six tests for boundary values.

We can consider another example of Boundary value analysis where we can apply it to the whole of a string of characters (e.g. a name or address). The number of characters in the string is a partition, e.g. between 1 and 30 characters is the valid partition with valid boundaries of 1 and 30. The invalid boundaries would be 0 characters (null, just hit the Return key) and 31 characters. Both of these should produce an error message.

3. What is Decision Table Technique

The techniques of equivalence partitioning and boundary value analysis are often applied to specific situations or inputs. However, if different combinations of inputs result in different actions being taken, this can be more difficult to show using equivalence partitioning and boundary value analysis, which tend to be more focused on the user interface. The other two specification-based software testing techniques, decision tables and state transition testing are more focused on business logic or business rules.

A decision table is a good way to deal with combinations of things (e.g. inputs). This technique is sometimes also referred to as a ’cause-effect’ table. The reason for this is that there is an associated logic diagramming technique called ’cause-effect graphing’ which was sometimes used to help derive the decision table.

  • Decision tables provide a systematic way of stating complex business rules, which is useful for developers as well as for testers.
  • Decision tables can be used in test design whether or not they are used in specifications, as they help testers explore the effects of combinations of different inputs and other software states that must correctly implement business rules.
  • It helps the developers to do a better job can also lead to better relationships with them. Testing combinations can be a challenge, as the number of combinations can often be huge. Testing all combinations may be impractical if not impossible. We have to be satisfied with testing just a small subset of combinations but making the choice of which combinations to test and which to leave out is also important. If you do not have a systematic way of selecting combinations, an arbitrary subset will be used and this may well result in an ineffective test effort.

How to Use decision tables for test designing?

The first task is to identify a suitable function or subsystem which reacts according to a combination of inputs or events. The system should not contain too many inputs otherwise the number of combinations will become unmanageable. It is better to deal with large numbers of conditions by dividing them into subsets and dealing with the subsets one at a time. Once you have identified the aspects that need to be combined, then you put them into a table listing all the combinations of True and False for each of the aspects.

Let us consider an example of a loan application, where you can enter the amount of the monthly repayment or the number of years you want to take to pay it back (the term of the loan). If you enter both, the system will make a compromise between the two if they conflict. The two conditions are the loan amount and the term, so we put them in a table below.

Example of Decision Table

Conditions Rule 1 Rule 2 Rule 3 Rule 4
Text Field A TRUE TRUE FALSE FALSE
Text Field B TRUE FALSE TRUE FALSE
Outcome A TRUE TRUE FALSE FALSE
Outcome B TRUE FALSE TRUE FALSE

4. What is State transition testing?

  • State transition testing is used where some aspect of the system can be described in what is called a ‘finite state machine’. This simply means that the system can be in a (finite) number of different states, and the transitions from one state to another are determined by the rules of the ‘machine’. This is the model on which the system and the tests are based.
  • Any system where you get a different output for the same input, depending on what has happened before, is a finite state system.
  • A finite state system is often shown as a state diagram.
  • One of the advantages of the state transition technique is that the model can be as detailed or as abstract as you need it to be. Where a part of the system is more important (that is, requires more testing) a greater depth of detail can be modeled. Where the system is less important (requires less testing), the model can use a single state to signify what would otherwise be a series of different states.

A state transition model has four basic parts:

  • The states that the software may occupy (open/closed or funded/insufficient funds);
  • The transitions from one state to another (not all transitions are allowed);
  • The events that cause a transition (closing a file or withdrawing money);
  • The actions that result from a transition (an error message or being given your cash).

Test conditions can be derived from the state graph in various ways. Each state can be noted as a test condition, as can each transition. However this state diagram, even though it is incomplete, still gives us information on which to design some useful tests and to explain the state transition technique.

We need to be able to identify the coverage of a set of tests in terms of transitions. We can also consider transition pairs and triples and so on. Coverage of all individual transitions is also known as 0-switch coverage, coverage of transition pairs is l-switch coverage, coverage of transition triples is 2-switch coverage, etc. Deriving test cases from the state transition model is a black-box approach. Measuring how much we have tested (covered) will discuss in a white-box perspective. However, state transition testing is regarded as a black-box technique.