1. Static Testing Basics and differences with Dynamic Testing
We have mentioned static testing in the first section of this course and said that static techniques test software without executing the software code, while dynamic testing, on the other hand, requires the execution or running of the software. Under test, static testing can be considered the first line of defense in minimizing defects from the final software by removing the defects from the work products that we shall use in building our software or as test basis and hence makes those defects cheaper and easier to fix. Static testing is often a forgotten area of software testing and companies lose a lot by not incorporating static testing techniques in their software development lifecycle. As we have said, static testing relies on examining the software worker products.
The examination is either manual examination of worker products, for example, reviews or using software tools and utilities, for example, static analysis. Whether the examination is manual or tool driven, both types assess the code or other worker product being tested without actually executing the code or work product being tested.
The syllabus concentrates more on the review process that I had to record few values to explain it. Meanwhile, it only mentioned a static analysis by tools. Very briefly, here is all what you need to know about static analysis aesthetic analysis is essential for safety critical computer systems, for example, aviation, medical or nuclear software. But aesthetic analysis has also become essential and common in other settings.
For example, static analysis is an important part of security testing. Static analysis is also often incorporated into automated build and delivery systems, for example in agile development, continuous delivery and continuous deployment. Differences between Static and Dynamic Testing as we are talking about dynamic testing, let’s talk about the differences between static and dynamic testing. As we have mentioned before, static testing dynamic testing can have the same objectives. Objectives such as providing an assessment of the quality of the local products and identifying defects as early as possible. Static and dynamic testing complement each other by defining different types of defects.
Let me give you an example so you can taste the main difference between static and dynamic testing. In regular dynamic testing techniques, the developer will write the code and build an executable software and pass the executable software to the tester to test it. The tester will need to design test cases and test procedures, run the test procedures and compare expected result versus actual result and if they are not the same, the sister will try the scenario a few times to make sure it’s severely a failure. The tester then will create a bug report and send it to the developer to fix it. The developer will try the scenario and might communicate with the tester several times till he can verify the failure. The developer then will try to find the root cause of the failure and believe me, it might take days to find it. And if found, the developer will try to fix it, which again might take a long time as well.
After the bug has been fixed, the developer will send the new executable software with the fix to the tester to confirm the fix in case the bug is not actually fixed. Then the cycle will be repeated several times till the test confirms the fix and so on. So it’s a very long trip. Now imagine a situation where a person or a software tool can examine the source code written by the developer and directly finds the bug. It will be easy to fix the bugs in and we are done. That’s the magic of static testing. Finding defects and not failures is one main distinction between static and dynamic testing. A defect can reside in the work product for a very long time without causing a failure. The best where the defect lies may be rarely exercised or hard to reach, so it will not be easy to construct and execute academic tests that encounters it. Aesthetic testing may be able to find the defect with much less effort. Another distinction is that static testing can be used to improve the consistency and internal quality of worker products. While dynamic testing typically focuses on externally visible behaviors, using static testing techniques to find effects and then fixing those defects.
Bombardier is almost always much cheaper for the organization than using dynamic testing to find defects in the test object and then fixing them, especially when considering the additional costs associated with updating other worker products and performing confirmation and regression testing. Even though static testing sounds like magic, but still we will need to do dynamic testing as running the excitable code can find other types of defects. So static testing is complementary to dynamic testing. Compared to dynamic testing.
Typical defects that are easier and cheaper to find and fix through static testing include requirements defects, for example, inconsistencies, ambiguities contradictions, omissions, inaccuracies and redundancies design defects, for example, inefficient algorithms or database structures. High coupling, low cohesion coding defects. For example, variables with undefined values, variables that are declared but never used, unreachable code and duplicate code deviations from standards, for example, lack of adherence to coding standards.
Incorrect interface specifications, for example, using integers instead of floats when passing parameters between components security vulnerabilities, for example, susceptibility to buffer overflows gaps or inaccuracies in test basis to stability or coverage, for example, missing tests for an acceptance criterion. Moreover, most types of mentality defects can may be found by static testing. For example, the code is too complex to maintain improper modularization poor reusability of components codes that is difficult to analyze and modify without into reducing new defects.
2. More on Static Testing
So the input to static testing is any written document and the output will be defects found in that document. So what type of documents can be examined using static testing? The answer is any written work product can be examined using static testing reviews and or static analysis. For example, verification, including business requirements, function requirements and security requirements. Azurely specific local products like Epics. User stories and acceptance criteria architecture and design specifications source code testware, including test the blends, test cases, test procedures and automated test scripts user guides, web pages, even documents that are not directly software related, like contracts, project plans, schedules and budgets.
Models such as activity diagrams, which may be used for model based testing. Generally, reviews can be applied to any written worker product that a human can understand, whereas static analysis usually requires as an input more formal, more structured, more formatted worker products typically code or models for which an appropriate static analysis tool exists. So a compiler is a type of a static analysis tool where it analyzes the software code written using a specific computer language. A very simple example of a static analysis tools are grammar check tools which work on natural language Voker products such as requirements. Benefits of a static testing There are plenty of benefits to using static testing techniques. Early detection of defects before dynamic test execution the earlier the defect it found, the cheaper and easier it is to fix. This is especially when compared to defects found after the software is deployed and in active use.
Identification of defects not easily found by dynamic testing. This will result in reduced fault levels so the overall severity of the bugs get reduced. Preventing defects in design or coding by uncovering inconsistencies, ambiguities contradictions, omissions in accuracies and redundancies in requirement. By identifying the defect early in the life cycle, it’s a lot easier to identify why it was there in the first place, thus providing information on possible process improvements that could be made to prevent the same defect appearing again. Increasing development productivity, for example, due to improved design, more maintainable code as developers love to work on a stable non buggy software. Reduced development time scales due to the number of bugs get drastically reduced, hence, less time is spent on fixing bugs, hence reducing development cost and time.
Also, the lower the number of bugs will reduce testing and cost as it will result in less time documenting bug reboots, less time retesting fixed bugs and fewer bugs swing back and forth between the developers and pesters, reducing total cost of quality over the software lifetime due to fewer failures later in the life cycle or after delivery into operation. Therefore, ongoing support costs will be lower, which will result in lifetime cost reductions and last improving communication between team members in the course of participating in reviews. Reviews could be the only time in your organization where a senior talks to a junior and points to him what’s wrong with the review document and how to avoid making the same mistake again. Reviews, if done correctly, will improve team communication and knowledge transfer.
3. Review Process
So how can we perform the review? It’s not a random activity. We have a process for that. Reviews can vary widely in the level of formality, where formality relates to the level of structure and documentation associated with the activity. Reviews vary from informal to formal. Informal reviews are characterized by not following a defined process and not having formally documented results. Just like when you ask a colleague passing by to look at one of your documents so there are no written instructions for reviewers. It’s very informal. On the other hand, formal reviews are characterized by team participation, documented results of the review, and documented procedures for conducting the review. There are factors that affect the decision on the appropriate level of formality. Those are organization based factors that affect the level of formality of any review.
It’s usually based on the software development lifecycle model. Waterfall might need a more formal process, while Agile might be okay with informal ones. The maturity of the development Process as the more mature the process is, the more formal reviews tend to be. The complexity of the worker product to be reviewed. The more complex the worker product, the more formal the review process should be. Legal or regularity requirements for example, in safety critical software applications domain, the regularity or legal requirements determine what kinds of review should take place. The need for an audit Trail the level of formality in the different types of review used can help to raise the level of an audit trail to trace backward throughout the software development lifecycle Is standard 20,246 contains more in-depth descriptions of the review bosses for work of products, including roles and review techniques.
Here is another standard number for you to remember 20246 let’s talk now about the work product review process. The different types of reviews vary in its formality, but before discussing the different types of reviews, let’s talk first about the five groups of activities of the review process they are planning initiate review, individual review or individual preparation, issue, communication and analysis and last, fixing and reporting. Again, we need to know which activities happen during which a group and also memorize the sequence of the activities.
As this is one repeated question in the Ice TB exam, let’s talk about each of those activities in detail. Planning reviews are good, but we cannot review every work product we get. So we should be defining the scope, deciding the purpose of the review, what documents or parts of documents to review and the quality characteristics to be evaluated, where to do it, and if there is already any company process, then guidelines or redefine the checklists we could use in the review process.
Also estimating effort and time frame we know when to do it, how long it should take identifying review characteristics such as the review type with roles, activities and checklists to know selecting the people to participate in the review and allocating roles. The reviewers should be skilled to do the job know how to dig for mistakes in the document. They also should be of different background. For example, someone with a design background, someone who is an expert in UI, someone with performance background, another with the standards, knowledge, and so on.
The selected personnel will be assigning roles responsibilities accordingly. Also in planning, we should be defining the entry and exit criteria for more formal review types. For example, inspections entry criteria define what criteria should be fulfilled to start the review, such as making sure the document is still checked before starting the review and the exit criteria. Define what criteria should be fulfilled to stop the review, such as fixing major bugs found in the document. Now we need to sit and wait, checking that the intercriteria are met if we have one, so reviewers won’t waste time on an unready document. The second group activity is initiate review.
Before the actual review, we need to make sure all the reviewers know exactly what’s expected from them to initiate the review. For example, distributing the worker product physically or by electronic means and other material if needed. Hand the reviewers any issue look forms, checklists and related worker products that they might use explaining the scope, objectives, process rules and worker products to the participants and answering any questions that participants might have about their view. After initiate review, we have individual review or individual preparation. Each of the participants alone will review all or part of the local product, noting potential defects, recommendations, questions and comments. This activity could be time boxed, usually two to 4 hours. After individual preparation, it’s time to issue communication and analysis, so it’s time for the participants to communicate the identified potential defects. This could be in a review meeting.
Participants will go through a discussion regarding any defects found. The discussion will usually lead to more defects findings analyzing potential defects assigning ownership and status to them. The viewers may only suggest or recommend fixes, but not an actual discussion on how to fix it. This will be done later by the author. Evaluating and documenting quality characteristics. At the end of the meeting, a decision on the document under review has to be made by the participants evaluating the review findings against the exit criteria. To make a review decision, should we proceed with this document or drop it altogether? Or a simple follow up meeting after fixing the defects found will be enough. Last is fixing and reporting. This is after the meeting.
We could create defect reports for those findings that require changes. The author will have a series of defects to investigate, answering questions and suggestions raised in the review meeting and fixing defects found typically done by the author in the worker product reviewed. We might need to communicate defects to the appropriate person or team when found in a local product related to the worker product reviewed. Recording updated status of defects informal reviews, potentially including the agreement of the command originator gathering metrics is again for more formal reviews. For example, how much time was spent on the review and how many defects were found achieving that exit criteria are met. This is again for more formal reviews and accepting the local product when the exit criteria are reached. The results of the local product review vary depending on the review type and formality.
4. Roles in Formal Review
The participants in any formal review should have adequate knowledge of the review process and have been properly trained as reviewers when necessary. A typical formal review will include the following roles author, management, facilitator or motivator, review leader, reviewers, and last describe or recorders. Let’s talk about each role in in little detail. The author is the person who creates the worker product under review and fixes defects in the worker product under review, if needed.
Management is responsible for review planning, decides on the execution of reviews, assigns staff budget and allocates time in budget schedules monitors ongoing cost effectiveness executes control decisions in the event of inadequate outcomes. Facilitator, which also often called moderator, ensures effective running of review meetings when held, is often the person upon whom the success of the review demands and is responsible for making sure no bug fixing will be discussed in the review meeting, and also responsible for making sure the reviewers will discuss the code objectively, not subjectively.
And last, mediates, if necessary, between the various points of view and the review meetings. Review leader takes over all responsibility for the review, decides who will be involved with and organizes when and where it will take place. Viewers may be subject matter experts, persons working on the project, stakeholders with interest in the worker product, and or individuals with specific technical or business backgrounds who, after the necessary preparation, identify potential defects in the worker product under review and may represent different perspectives. For example, tester, programmer, user, operator, business analyst, usability experts, and so on. And last scribe or recorder collects potential defects found during the individual review activity and records new potential defects, open points and decisions from the review meeting.
When held, some might get confused over the difference between management, review leader and the moderator or facilitator. Well, think of management as management time, cost to resources, very high level decisions, no technicality in the review needed. The review leader is like the team leader in your project he understands technically what’s going on and will make sure everything is executed. The facilitator, on the other hand, usually gets involved in the review meeting only. His job is helping others do their job right and make sure that it runs it smoothly without any interruptions or tension between the participants. Also, the actions associated with each role may vary based on review type.
In addition, with the advent of tools to support the review process, especially the logging of defects, open points and decisions, there is often no need for a scribe. Notice that it’s normal that one person may play more than one role, and one role can be played by more than one person. Again, more detailed roles are possible as described in ISO Standard 20,246. It’s the only standard for everything related to reviews. ISO standard. ISO IEC.
