We will study the basic concepts and important properties of automatic tests.

First we review the concept of test. According to the dictionary, a test is an “Essay or experiment that is made of something, to know how it will result in its final form.” This means that the purpose of making a test is to know the result of a set of actions.

When we write software, it is for a purpose, typically to solve a problem. Therefore, it is natural that after writing the software we want to check that it solves the problem that we want it to solve. For that, we resort to testing the software. What is testing the software?

Testing the software is using it to see that it works. This test will vary depending on the type of software, but in one way or another, to test a software we must use it. Whether it’s a web page, a mobile application, a console application, or a web service, testing the application involves running it on a device and using it.

The problem is the following, except for trivial applications, real-world applications tend to solve several problems, have many options, and to test these softwares properly, you should try all the functionalities of the softwares. However, this is very tedious and prone to errors. Testing all the functionalities of a software could take long hours or days. And while we do it, we may forget to do certain tests.

These difficulties can be overcome by performing automatic tests.

Automatic Tests

If a test is an experiment that is done of something to know how it will result in its final form, then an automatic test is to automate this experiment. That is to say, an automatic test is a software that tests our software.

One of the main advantages of automatic tests is that they test everything we tell them to test, as many times as we want. This is important because it gives us the confidence to be able to change our software code and know that, by pressing a button, we can automatically test our application to see if everything continues to work correctly. Has it happened to you that you have software that works, then you make a change in the code, and the application stops working? Well with automatic tests you can discover these failures in a very fast way.

There are certain qualities that it is important to ensure that all our tests have. We highlight some of these qualities:

• A good test is reasonably consistent. The idea is that the result of a test will only vary when the software changes. If we run a test, and this is successful, and, without making any changes to the software, we do the same test again, this should be successful again. However, there are occasions where variations may occur in the test result without the software having changed, this can happen when you depend on an external service for the test, such as a web service, for example.
• A good test does not depend on another test. The tests must be independent of each other. It should never be the case that for test B to work it must be executed after the execution of test A.
• A good test does not make external changes. if possible, a test should not make changes to the system, you should try and leave things as they were. In some cases this is not so viable and some flexibility is accepted. For example: When we are going to do tests with a database, it is not viable that each automatic test creates and destroys its own database. In this case we can be flexible, and that a database be constructed during the run of the tests, and that said database be destroyed at the end of the execution of all the tests.

Typically an automatic test is a function which tests some part of our application. If for example you have a sales software, an example of an automatic test could be a function that tests the part of your application where a product is created; another automatic test could be a function that tests the part of the payment of the products that the end user wants to buy. We call a set of automatic tests a test suite.

An automatic test is usually divided into 3 parts or stages: Prepare, test and verify:

• Prepare: The first part of a test tries to prepare the environment to develop the test. This preparation can consist of: Generate the test data, generate test objects, among others.
• Test: In this second stage we execute the functionality of our software which we want to test. This test can consist of: executing a function, triggering a process that interacts with a web page as if it were a user, etc.
• Verify: In this last stage we want to make sure that we obtained the expected result from the test.

Let’s see an example on a conceptual level of automatic tests. Suppose we work for a bank, and we have the task of testing software that makes bank transfers between two accounts. What would be a way to prepare an automatic test to verify a successful transfer? We remember the 3 stages: Prepare, test and verify:

• Prepare: To test a transfer between two accounts, we need two accounts, at least one of them with a balance. Therefore, at this stage we must prepare two fictitious accounts. Maybe we need other things, like a database, whether real or “fictitious” (in a future post we talk about this).
• Test: Since we already have the two accounts, now we need to execute the function which performs the action we want to test. For our case, this function would be a function that takes as parameters the two accounts and the amount to be transferred.
• Verify: We need a way to verify that the money actually passed from one account to another, depending on how our software works, maybe we can see it at the level of the accounts in the database.

In one way or another, the automatic tests you do will have a structure similar to the previous one. However, not all tests are the same, there are tests that test isolated parts of your software, there are other tests that work with different parts of your application simultaneously. That is, there are several types of tests, such as: Unit tests, integration tests, and end-to-end tests. In the next post we will talk about unit tests.

Summary

An automatic test is a function that runs our software to verify that it works correctly. A typical test is divided into 3 stages: Preparation, testing and verification. There are different types of tests, such as: Unit tests, integration tests and end-to-end tests.