Static Test Technique

What is Static Test Tecnhique?

Static test techniques provide a great way to improve the quality and productivity of software development. It includes the reviews and provides the overview of how they are conducted. The primary objective of static testing is to improve the quality of software products by assisting engineers to recognize and fix their own defects early in the software development process.

What is Static Testing?

  • Static testing is the testing of the software work products manually, or with a set of tools, but they are not executed.
  • It starts early in the Life cycle and so it is done during the verification process.
  • It does not need computer as the testing of program is done without executing the program. For example: reviewing, walk through, inspection, etc.

Example of Static Testing (Type of Static Test):

Why Static Testing is useful?

  • Since static testing can start early in the life cycle so early feedback on quality issues can be established.
  • As the defects are getting detected at an early stage so the rework cost most often relatively low.
  • Development productivity is likely to increase because of the less rework effort.
  • Types of the defects that are easier to find during the static testing are: deviation from standards, missing requirements, design defects, non-maintainable code and inconsistent interface specifications.
  • Static tests contribute to the increased awareness of quality issues.

Two Main Category of Static Testing

1. Informal Reviews

Informal reviews are applied many times during the early stages of the life cycle of the document. A two person team can conduct an informal review. In later stages these reviews often involve more people and a meeting. The goal is to keep the author and to improve the quality of the document. The most important thing to keep in mind about the informal reviews is that they are not documented.

2. Formal Reviews

Formal reviews follow a formal process. It is well structured and regulated.
A formal review process consists of six main steps:

1. Planning
2. Kick-off
3. Preparation
4. Review meeting
5. Rework
6. Follow-up

1. Planning:
The first phase of the formal review is the Planning phase. In this phase thereview process begins with a request for review by the author to the moderator (or inspection leader). A moderator has to take care of the scheduling like date, time, place and invitation of the review. For the formal reviews the moderator performs the entry check and also defines the formal exit criteria. The entry check is done to ensure that the reviewer’s time is not wasted on a document that is not ready for review. After doing the entry check if the document is found to have very little defects then it’s ready to go for the reviews. So, the entry criteria are to check that whether the document is ready to enter the formal review process or not. Hence the entry criteria for any document to go for the reviews are:

- The documents should not reveal a large number of major defects.
- The documents to be reviewed should be with line numbers.
- The documents should be cleaned up by running any automated checks that apply.
- The author should feel confident about the quality of the document so that he can join the review team with that document.

Once, the document clear the entry check the moderator and author decides that which part of the document is to be reviewed. Since the human mind can understand only a limited set of pages at one time so in a review the maximum size is between 10 and 20 pages. Hence checking the documents improves the moderator ability to lead the meeting because it ensures the better understanding.

2. Kick-off:
This kick-off meeting is an optional step in a review procedure. The goal of this step is to give a short introduction on the objectives of the review and the documents to everyone in the meeting. The relationships between the document under review and the other documents are also explained, especially if the numbers of related documents are high. At customer sites, we have measured results up to 70% more major defects found per page as a result of performing a kick-off, [van Veenendaal and van der Zwan, 2000].

3. Preparation: In this step the reviewers review the document individually using the related documents, procedures, rules and checklists provided. Each participant while reviewing individually identifies the defects, questions and comments according to their understanding of the document and role. After that all issues are recorded using a logging form. The success factor for a thorough preparation is the number of pages checked per hour. This is called the checking rate. Usually the checking rate is in the range of 5 to 10 pages per hour.

4. Review meeting:
The review meeting consists of three phases:

- Logging phase: In this phase the issues and the defects that have been identified during the preparation step are logged page by page. The logging is basically done by the author or by a scribe. Scribe is a separate person to do the logging and is especially useful for the formal review types such as an inspection. Every defects and it’s severity should be logged in any of the three severity classes given below:

— Critical:The defects will cause downstream damage.
— Major: The defects could cause a downstream damage.
— Minor: The defects are highly unlikely to cause the downstream damage.

During the logging phase the moderator focuses on logging as many defects as possible within a certain time frame and tries to keep a good logging rate (number of defects logged per minute). In formal review meeting the good logging rate should be between one and two defects logged per minute.

Discussion phase: If any issue needs discussion then the item is logged and then handled in the discussion phase. As chairman of the discussion meeting, the moderator takes care of the people issues and prevents discussion from getting too personal and calls for a break to cool down the heated discussion. The outcome of the discussions is documented for the future reference.

Decision phase: At the end of the meeting a decision on the document under review has to be made by the participants, sometimes based on formal exit criteria. Exit criteria are the average number of critical and/or major defects found per page (for example no more than three critical/major defects per page). If the number of defects found per page is more than a certain level then the document must be reviewed again, after it has been reworked.

5. Rework: In this step if the number of defects found per page exceeds the certain level then the document has to be reworked. Not every defect that is found leads to rework. It is the author’s responsibility to judge whether the defect has to be fixed. If nothing can be done about an issue then at least it should be indicated that the author has considered the issue.

6. Follow-up: In this step the moderator check to make sure that the author has taken action on all known defects. If it is decided that all participants will check the updated documents then the moderator takes care of the distribution and collects the feedback. It is the responsibility of the moderator to ensure that the information is correct and stored for future analysis.

What are the roles and responsibilities involved?

During a review four types of participants take part. They are:

1. The moderator:
- Also known as review leader
- Performs entry check
- Follow-up on the rework
- Schedules the meeting
- Coaches other team
- Leads the possible discussion and stores the data that is collected

2. The author:
- Illuminate the unclear areas and understand the defects found
- Basic goal should be to learn as much as possible with regard to improving the quality of the document.

3. The scribe:
-Scribe is a separate person to do the logging of the defects found during the review.

4. The reviewers:
- Also known as checkers or inspectors
- Check any material for defects, mostly prior to the meeting
- The manager can also be involved in the review depending on his or her background.

5. The managers:
- Manager decides on the execution of reviews
- Allocates time in project schedules and determines whether review process objectives have been met

Real-case scenario.. If Static Test was not done!!

The government was tendering an IT Project to one IT company for developing a complex system which is to transform all their manual daily working activities to system-based. This project should be completed by 2 years. So, all the interaction between public and government are via the online system. The system is quite huge as it involves 12 different sectors or deparments, which means 12 big modules are required and from this 12 modules there are a total of 500 process or features or we can also simply say there are 500 SRS document (System Requirement Specification) are required. The IT contract like this normally worth of RM30Millions.

The IV&V company was not involved in this project because the IT contract didn't mention the need of IV&V company in this project. Guess what happen after the subsequent years?

Year 1: Requirement Gathering Process is very challenging. The development company failed to come up with a good SRS document.
Solution: If IV&V company was involved earlier, all the SRS document standards will have been defined early and will be reviewed and finally signed-off by IV&V company. All the 500 SRS must have reviews and inspection meeting.

Year 1.3: Development company failed to acquire all the requirements because of lack user participation.
Solution: If IV&V company was involved earlier, the IV&V will assist the user and guide them to participate. If this issue has became critical IV&V will raise and highlight this issue to the Software Testing Board and also to higher level government agencies.

Year 1.8: All requirement finally managed to be captured and signed-off. Software Coding has started, but along the way about 50% of the signed-off SRS document need to be changed as User wanted the changes. Developers got tension, a few of them resigned, as such the IT company were hiring new developers which they need to catch up all the domain knowledge and to understand the code written by the resigned developers. User and IT company keep blaming each other for the late change request. Some of the root cause was the disagreement between users from 12 different sectors or departments on some specific solution, and also due to limited skills by developers in User Requirement Gathering phase. Later on, the Project Manager also resigned. But, fortunately, the IT Company managed to find the replacement. The project has slipped-off!
Solution: This are the consequences if the IV&V were not involved in a IT Project. IV&V must get involved since the project kick-off. IV&V will be able to help user to review the SRS document should the planned solution is really workable. IV&V will use their experience in handling all the above mentioned issues.

Year 4: Project has gone LIVE but with a lot major defects (escaped defects). Security and Performance were also the serious issues. This project was slipped away from schedule for an additional 2 years!! It supposed to be delivered or LIVE on last 2 years. Even after 4 years, the quality of the software was at the stake!!
Solution: IV&V involvement is very important in all software development activities. The internal testing team sometimes have less experienced and they were influenced by the developers to follow the developer's decision! There are also some companies out there which still see Testing is just an additional cost but at the end of the day, they had paid double or even more due to serious quality issues. Don't set a high hope on your developers for doing the full testing, why? Because developer and tester (QA) mindset are completely different!!