In a perfect world I would except to be able to run a load test or a stress test using a cloud provider without being forced to change any line code from my tests.
In this article we will find out how to do this thing using the new cloud service. How will a person without knowledge of cloud be able to run a load test with 50.000 users for 24 hours with minim costs, without being obliged to purchase and configure 10, 20 nodes
What is a load test in fact? An expression that I think fits when we need to answer this question would be: "Are you ready to be so popular?". The main scope of this kind of test is to define and validate the maximum load that a system can have without affecting the performance. Even though the developer says that the system can handle 1000 users in the same time without any problem, the load test can prove the contrary even on simple scenarios.
There is a tiny difference between the load test and stress test. Many times the load test finishes with a stress test that has the purpose to observe the behavior of the system the moment when the load increases above the maximum supported capacity.
At the moment, it is full of different solutions for automated testing and for running load tests. Products like Selenium, LoadStorm or Neoload dominate the market. Besides them, there are many others and I don"t think 10 pages will be enough to enumerate them.
If we use an on-premise solution, then we need to be prepared to resolve things like the acquisition and reservation of the resources for the load-test. In a small company it will be very hard to buy 10 servers just to run the load-test. In a big company, even if it has the financial resources, it will take a while until these resources will be available and sometimes it might be too late. Besides all this factors, the configuration of each machine and deploy can be the labyrinth from which we won"t be able to find a way out.
On this extremely mature market, Microsoft appeared with an ingenious solution. If they already have a powerful and robust infrastructure for cloud why don"t use it to run load tests using Azure. Even though they are not the first ones that are offering this kind of solution, Microsoft has an advantage. They allow you to run load tests using Azure without using a new configuration. All you have to do is to have an account on Visual Studio Team Foundation Service (http://tfs.visualstudio.com/) which will be used for the receipt.
Before, a load test using Microsoft could have been done only through Load Test Ring. This if formed from a controller that controls the tests- Load Test Controller and one or more agents on which our tests run. An architecture that is based on master-slave. The new version of Visual Studio 2013 provides us a new option- instead of running our test in our own Load Test Ring, we can run them on Azure, without a new configuration. We don"t have to deploy any virtual machine or to configure different services.
Load Test Web Service is the new Azure service that helps us. Through this service, Visual Studio 2013 loads our tests on the cloud. Behind this service there is a pool of test agents that is used to run our tests. This thing happens behind the scene and we don"t have to do anything for this to run. All the results from our test, together with other performance counters are available for us.
Before looking over different functionalities that are supported, I propose to first take a look at the most important options that are supported now.
We have the possibility to define UI tests and more. Beside the fact that we can record our UI tests, we can also write our custom tests and hit different endpoints. The tested endpoint doesn"t have to be only HTTP or HTTPS. We can also test a WCF or REST endpoint and also a web service. The only condition is that the endpoint must be accessible via internet.
Visual Studio 2013 brings a project template- Web Performance and Load Test Project. This type of project will allow us to define and run load tests on cloud.
The easiest way to create a load test for a web application is to create one or more Web Performance Tests. This type of test can be UI tests which are very easy to create and to automate. Using the UI recorder that comes with Visual Studio 2013, we can create a Web Performance Test in just a few seconds. This type of test can be modified at any time and, if we want, we can generate code for the test. Through this method, the ones that wish to write code to modify the code can do this very easily. Of course, functionality like automatic detection of the dynamic elements that the page has or the extraction of different constants as parameters is supported out of the box.
For each test of this type we can generate different sources like database, CSV or XML file. You can also use the tests written in Selenium for example. Each test can call other test and, this way, we can reuse the logic that we already have for testing.
Until now we saw how we can create a test that can be used for the load test. It is time for us to see what a load test allows us to do.
The first thing we need to do in order to create a load test is to THINK. We can define different profiles and time intervals through which we should be able to simulate a real user. For example we can simulate a delay of X seconds after each test. There are different profiles available for us to use.
The way we simulate a load test can be made through different methods. We have the possibility to run a test with a specific number of users or we can define the number of users to increase at every step. Of course, most of the times, we will have to run different scenarios at the same time. That why, we can select the tests that we want to run and in what proportion, but also the way that this test has to run. For example we can specify how many times a test should run in a time interval or which is the number of users that have to run a specific test every moment the load test is running.
We can simulate different browsers that the clients might have and different types of connections. The most interesting thing is that we have the possibility to add and access not just performance counters from the clients but also the ones from the servers on which our application is running. This way we can monitor and see the counters from both the client and the servers.
Until now we saw the main options that we have available in order to run a load test, but we didn"t see how to run this type of test. All we have to do to run the load test on cloud is to open the Local.testsettings and on General tab to select "Run tests using Visual Studio Team Foundation Service". This is the only thing we have to do to run the test on cloud. Of course, we have to be logged uni in Visual Studio with an account that is connected to Visual Studio Team Foundation Service.
For now, the service is on preview. Each user has 2000 virtual minutes per month that can be used for the load tests. There aren"t fixed prices for this. If you need more minutes for load tests all you have to do is to enter in the early adoption program without additional costs. You will have 200.000 virtual minutes per month that you can use. Accepting the request takes very little time (in my case, about 6 hours).
The first time I heard about virtual minutes I wondered what these are. A virtual minute is the duration of the load test multiplied with the number of users.
Once we run the tests, we will have access to all the results including performance counters, failed tests, error messages and different diagrams. All the results can be exported into an Excel that will automatically contain the diagrams that you can show to the clients.
I invite you to try the new cloud service for load tests. I was pleasantly surprised by this service and I think it will make our life easier.