What is black box testing?

Black Box Testing, as defined by the International Software Testing Qualifications Board (ISTQB), is either functional or non-functional testing, without reference to the internal structure of the component or system. In other words, to expand on this statement, we can define it as testing without looking at the internal code structure, implementation details, and knowledge of internal parts of the software.

This testing type is retrieved solely from technical requirements and product specifications. Its main purpose can be defined as checking whether the software is performing as expected by users and indicated in the specification documents.

What is white box testing?

White Box testing, also known as glass box, clear box testing, code-based testing, or logic-driven testing, is defined by ISTQB as testing based on an analysis of the internal structure of the component or system. In general, White Box testing is considered to be more low-level testing and is derived from the expected internal functioning of the system.

Black box testing categories

Black box testing can be seen as testing from the customer’s point of view, where the tester has an understanding of the expected output without awareness of how that output is achieved by the system. The general categorization of testing into functional and non-functional also applies within Black Box testing. See our article on functional vs. non-functional testing types to read more detail about them, their types and examples, and differences between them.

Functional black box testing

Functional testing in short tests, whether each function of software meets the specified requirements.

Some of the main functional testing types include the following:

• Smoke Testing
• Regression Testing
• User Acceptance Testing

Example:

Testing whether clicking on a button leads you to a page it is supposed to direct to.

Non-functional black box testing

Non-functional testing tests all the non-functional requirements of a system, which comprise of all tests that do not fall into functional ones.

Functional testing types that could be tested via black box techniques are:

• Usability Testing
• Load Testing
• Stress Testing

Example:

Page load and response time could be tested as part of non-functional testing methods.

What are the black box testing types?

Even when a developer has an idea of what needs to be tested, it can be quite tricky to come up with test cases in a systematic and structured way. Furthermore, the process has to ensure that the tests are going to cover the majority of the functionality and expose the system’s potential weaknesses.

Now that we have briefly covered the types of testing that are considered Black Box testing, we are going to explain the techniques used to develop the tests.

1. Equivalence partitioning

Equivalence partitioning, also known as Equivalence Class Partitioning, is a technique that eliminates the number of redundant test cases, where outputs are the same and which would not reveal new or genuine defects of a system.

In other words, its goal is to reduce the test cases, that taking the same input set data makes the system behave the same, i.e., lead to the same outcome when testing the system.

Therefore, when executing a program, inputs that give the same result are identified and classified into groups. This technique is perhaps one of the most commonly used ones within Black Box Testing. Check here for several examples that might give you a better understanding of this technique.

2. Boundary Value Analysis (BVA)

Boundary Value Analysis is a testing process that checks the correct execution of a program given valid and invalid input at extreme ends or boundaries of input domains. So, rather than focusing on the values in the ‘center’ of input, it tests the extreme valid inputs, as well as those falling outside those boundaries.

Equivalence Partitioning and Boundary Value Analysis are linked to each other and are usually tested together. The boundaries are based on the classes defined in the equivalence partitioning and are tested using BVA. Thus, Equivalence Partitioning is applied before Boundary Value Analysis.

For example, extreme ends on input data, such as start and end, lower and upper, as well as so-called ‘just inside’ and ‘just outside’ data sets, are tested in boundary testing.

This technique is particularly useful when there’s a large set of possible inputs for the system, and thus creating individual test cases for all possible data is not feasible.

3. Decision table based testing

Similar to the techniques mentioned above, Decision Table Based testing is categorizing input to the system and expected outputs or system behavior according to these inputs. However, in this technique, different input combinations and their corresponding output are mapped into a table. This table is sometimes called the Cause-Effect table.

A good example would be of a table for the action of uploading an image on a website, where images are the inputs and system behavior is an output. The image file has to be of .jpg format and of size less than 3 MB.

This table easily showcases the system behavior when the input file does not match the specification.

4. Graph-based testing methods

Graph-Based Testing is similar to Decision Table Based Testing, as it maps the input and output of a system. The graph drawing depicts links between the inputs (causes) and output (effects) triggered by the inputs.

Production of a Cause-Effect Graph can be helpful in generating test cases when you think about the software test design and available input and output conditions.

This technique is also useful when trying to understand the performance of a system functionally, as it visualizes actions performed and the flow from inputs into outputs.

5. Error guessing

Error guessing, as the name suggests, is quite a subjective technique. This definition puts it nicely and mentions that it relies on the tester’s intuition and experience to identify defects that might be harder to catch using more formal testing methods.

This method can be more effective when several testers combine their knowledge and experiences of similar systems in coming up with potential issues of the application.

Its drawback is, of course, the subjectivity and total dependency on the knowledge of the tester.

What are the types of white box testing?

White Box testing encompasses testing types which are used to evaluate a specific block of code, software package, and/or to evaluate usability and functionality of an application.

However, in doing so, it takes into consideration how the functionality is written and often tests the particular parts of the implementation. In other words, the implementation of some functionality of a system is known and targeted in these types of testing techniques.

Some of the particularly widely used testing types that fall under the umbrella of White Box Testing include the following.

1. Unit testing

Unit testing is considered to be the lowest level testing method, where individual units or components are tested. Typically automated, unit tests are written and run to verify that small sections of an application meet its requirements and its output matches the expectations.

2. Security testing

Security Testing can sometimes be viewed as a separate branch of testing, as it can be done in many ways and by following a variety of structural approaches.

In general, though, it aims to find potential threats and check the validity of the system in regards to data stored and passed between components. For example, security testing can find how well a system is protected from unauthorized access and code damage.

If done improperly, security testing can undermine a company's reputation and the safety of the stored data. Hence, the methodologies employed within this type need to be quite sophisticated and thorough. While some black-box methods can be used to test parts of a system, knowledge about the internal structure and code is required to perform the tests in-depth.

3. Testing for memory leaks

Another quite sophisticated testing that can be categorized as White Box testing is searching for and detecting memory leaks in an application. Memory leaks are one of the biggest contributors to low application performance. More often than not, specialists in memory management are able to tackle these issues by testing at low to higher levels.

What are the white box testing techniques?

1. Code coverage

One of the main techniques utilized within white box testing is called code coverage. Code coverage aims at finding the areas of a system that has been tested yet and measures the degree to which the program has been tested already. The process also involves creating test cases that increase coverage and continuously measuring the coverage throughout the process. The information gathered includes major characteristics of the running program and its code coverage generated in the form of a report at the end of the process.

There are quite a few levels of code coverage available to developers. However, we are focusing on the three most prevalent ones, namely statement, branch, and path coverage techniques. Other coverage methodologies not included in this article include Toggle Coverage and FSM Coverage.

2. Statement coverage

Statement coverage is a process that involves the execution of all statements of the source code at least once. You can imagine that the total number of total statements to execute can get high very quickly, so the process also tried to cover these statements with a minimal number of tests.

The process is used to calculate the total number of executed statements out of the total present statements in the source code and is based purely on the structure of the code.

Generally, statement coverage tries to uncover the following cases:

• Unused Statements
• Dead Code
• Unused Branches
• Missing Statements

Branch Coverage

Branch Coverage is used to make sure that every possible outcome is tested. The name is derived from the fact that the process aims to execute all “branches” from each decision at least once.

For example, if the outcome of a process is a boolean variable, the functionality has to be tested for both True and False results.

Due to its nature, this method also helps uncover sections of the code that don’t have any branches, thus reducing redundant test cases. Another positive consequence this method brings is that it can help test cases that would not have been tested using otherwise.

Path coverage

To design test cases, Path Coverage uses a program’s control flow graph, designed to identify execution paths within the program. A method called Cyclomatic Complexity is used to determine the number of linearly dependent paths.

Finally, a test case gets generated for each path resulted from the according to control flow graph and calculated cyclomatic complexity.

Differences between white box and black box testing types

The main differences are easy to identify from the clear definition of these testing types. When testing using Black Box methods, the implementation and internal structure of a system are left in a ‘box,’ without looking much into it. Thus, the testers do not care much about the details but rather about the functional outcome of the system. While in White Box testing methods, the internal structure is exactly what is being tested.

Below are given the main differences, as well as what they imply in terms of effort and focused effort within developer teams.

Conclusion

In this article, we have given definitions of two categories of testing - White and Black Box testing, as well as testing types that fall into these categories and techniques of developing test cases for each.

Although each category has its own benefits and drawbacks, it does not mean that one is preferred to another. Both of these types are essential to complete system testing and are applied at different times in a variety of ways.