The next topic that we're going to cover is black box testing. For black box testing, we don't have the source code of the system and we only have the executables of the system. What we're going to do is we're going to feed some inputs into the executable and then generate some output, and then double-check whether the output is going to be the same as the expected output. We call it black box testing because we can now see to source code and the source code is put inside a black box. Black box tests attempt to find incorrect or missing functions on a system or interface incompatibility errors or data structure or externally database access errors, or performance errors, or initialization and termination errors. To achieve reasonable testing, a black box test case should cover a range of input or output values, not just one single value. That is, it should tell us something about the presence or absence of a class of errors instead of just one single error. Here, we focus on equivalence partitioning, which focus on creating subdomains by grouping inputs and outputs by type to create test coverage of a class of errors. We try to divide the input or output data or the component into partitions of equivalent data. Then the equivalent petitions are usually derived from the requirements specification. Then we try to come up with test cases that are designed to cover each equivalent partition at least once. The heuristic that we're going to use here east to try to partition all the inputs into different subdomains based on valid and invalid inputs or outputs. Then when we perform testing, we have to select typical values inside each subdomain. Let's say if the input is a range, for example, the range of student ID that we can use is going to be starting from 0-9,999. Then the valid range is going to be 0-9,999. Then the invalid range is going to be something less than zero my be be negative numbers and something greater than 9,999, maybe 10,000. If the input is a specific value, then you're going to get one valid and two invalid subdomains. For example, the only valid value that we have is going to be 100. Then anything smaller than 100 or anything bigger than 100, they are within the invalid subdomains. If the input is a set of related values, then we have one valid and one invalid subdomain. For example, if we're talking about days of the week, then Monday, Tuesday, Wednesday, Thursday, etc., they are members of the valid subdomain. Then anything outside Monday up to Sunday, they on members of the invalid subdomain. Then if the input is Boolean, then you have one valid and one invalid subdomain. The valid subdomain is going to be true or false. Anything other than true or false, they will be members of the invalid subdomain. Apart from testing the valid and invalid subdomains, we should also test out the boundary situations when we perform black box testing. Why? Because the boundary cases, those are locations that you can easily make mistakes. That's why we focus on those places when we perform testing. That's why here I say there are more errors occur at the boundaries of a subdomain than in the center. That's why we focused on testing the boundaries. These are the reasons for boundary testing, you may get off by-one bugs or forget to handle empty container or empty object or overflow errors in arithmetic or program does not handle aliasing of objects, etc. The small subdomains at the boundaries of the main subdomains have a high probability of finding these errors. That's why instead of simply testing something from the main subdomain, we should also test out the boundary cases. For testing, we have to select values at the boundaries of the main subdomain. For example, this function that we can use for getting absolute value, the boundary situations is going to be zero. Then apart from testing zero, we should also consider the cases next to the boundary. Apart from testing zero, which is the boundary condition, we should also test out those numbers that are next to zero, for example, minus one and one. Those are the boundary cases that we need to consider when we perform boundary testing. If it's a range of value, for example, student ID should be from 0-9,999, then for the lower bound, we should try out, for example, with negative 1, zero, and one. For the upper bound, then we should try it out 9,998, 9,999, and then 10,000. If it's a set of discrete values, then we should test the minimum value and also the maximum value, and also those values that are next to the maximum and minimum. If the input is a set of values, then try to test all the values in the set if possible. For example, if you simply have a small set of inputs, let's say days of the week. It's going to be just Monday, Tuesday, Wednesday, up to Sunday, then you should try out all the possible inputs because you only have a small set of inputs. But if the set of inputs that you get is large, then you may simply try out some of the valid inputs and also one value outside the set, for example, a value which is not day of the week. Then if it's Boolean, then we should try out the Boolean values true or false and also one long Boolean value. You should also apply to guidelines that we have covered before to create output values at the minimum and also maximum expected values. If a data structure have boundaries, for example, if you have an array with bonds from one up to 10, then you should test out the array index at 0, 1, 2 and also 9, 10, 11. We focus on the minimum index and also the maximum index and also those indexes beside a maximum and minimum. Let's say we have this ASU example again. This is the system that students can use to register for courses. One possible input is going to be the student ID. Lets say we know that the valid range of student ID is going to be one million up to 9,999,999. Then how can we come up with the test case is according to this different situations, the boundary situations, typical situation and any special cases. For the boundary situation, we should try the minimum value and also the maximum value and also those values next to the minimum and maximum. For typical values we may test out, for example, 5,000,000, which is a valid student ID, and the student record exists within the database. We may also try it out five million and one which is a valid student ID, but the student record does not exist within the database. Then something below the range for example sample 500,000, and something above the range, for example, 50 million. You should also handle a special situation where students they may enter non numerical values, for example, 5647ABC. You should test out a special value and see what's going to happen on a system. Then for each test case, you have to come up with a name, with a setup, and the test value, and then how to verify whether you have passed a test case on that. You have to do it for all the inputs that you have prepared for black-box testing. We may also perform thread testing in black-box testing. Thread testing is an event-based approach which tests are based on events which trigger system actions, and is particularly important for object oriented system. Because for OO system we may have one object to do something for that object. Is used after classes have been unit tested and integrated into subsystems. But it may not be always possible to complete thread testing because there are usually too many inputs or output combinations. Therefore, we only focus on the most common executed threads. You can extract all those important threads from the basic flow of events within your use case specifications. This is an example of thread testing. Let's say within your system, object number 1 is student and an object number 2 is course, and an object number 5 is a university. Then student, object number 1 may call option number 2. Because object number 1 want to enroll into a particular course within a particular university, then object number 2 may have to call object number 5 to do something. This is one thing that we have to test when we perform thread testing. You may want to test a thread from object number 1 to object number 2 and into object number 5. The types of thread tests may include, for example, a single thread, lets say from object 3 to object 2, to object 4, or multiple thread, lets say object 1 to object 2, and into object 4 and 5. Or multiple input thread, for example, we have multiple things being input to object number 1. We may also focus on state-based testing. We may also focused on state-based testing when we performed black-box testing. Because some programs they can be in many different states simultaneously. For example, when you use to Microsoft Paint, you may put the brush into different colors. Then when we perform testing, we should put the brush into difference states. That means we should put the brush into different colors before we perform testing, before we draw something on the whiteboard. Notice this state-based testing focuses on comparing the resulting state of a class with the expected state. All the states of a particular object that we have within the system can be derived it using state machine diagram. We will talk about how to draw state machine diagram in one of the upcoming lectures. Then we should derive a representative set of stimuli for each transition, and then check the attributes or link of the class after applying a stimuli to determine if the specific state has been reached. We can determine the state of an object based on the attributes or based on to the links to other classes. For example, we can determine whether a class is full or not based on the attribute, the class size. Let's say if the class size is 50, then we know that the class is already full. Then full is a state, is a possible state of the object class. Then we can also use links of the class to determine the state of an object. Let's say we have a set of movies, and if a movie is linked with a particular customer, within the system, then we know that that movie has been, for example, rented by some customer then is no longer available. Then we can double-check whether the movie is available or not, the state of the movie, based on the link from the movie to the customer. We have to put the class into the desired state using attributes or using links before we apply the stimuli.
