Tip:
Highlight text to annotate it
X
SPEAKER 1: Welcome to the how to video on resolving errors
and warnings in LoadComplete tests.
During your execution of load tests, LoadComplete may report
errors and warnings.
Let me explain how you can understand and debug possible
causes of these errors or warnings.
And for this purpose, I'm going to create
a simple load test.
First, I'll start by recording a user scenario that
our test will use.
So I'm going to click on the Record New User Scenario
button right here.
And that brings up this Create New Project Wizard.
This wizard assists you in creating a test project that
will store your load test.
LoadComplete is asking us to create a new project because
there's no currently open project.
So on the first page of the wizard, I'm going to type in a
name for my project, so DiagnoseErrors, and then I'll
click Next.
On this screen, I can select the servers whose performance
LoadComplete would monitor during the test run.
However, I'm not interested in monitoring any server side
metrics in this screen cast, so I'm going to leave these
fields blank and just click Next.
On the final page of the wizard, I'll specify the
settings that help you estimate the tested
application's performance.
So here you can specify a threshold time for request
transferring and page loading processes.
These applications are set based on the application's
specifications.
And if a real time value exceeds the specified value,
LoadComplete will report an error.
And that will mean that the application doesn't meet the
specifications.
So I'm going to enter in the desired
values and click Finish.
So I want 2,500 milliseconds for page load time and 300
milliseconds for a time to first byte.
So now LoadComplete has created a new project.
And you can see in the background on the left hand
side of my screen, the project contents are being displayed
in that Explorer panel.
And it's also invoked the Record User Scenario dialog.
And here, we can specify a name for our scenario and the
name of the test that will be automatically created based on
the recorded scenario that we're generating.
We can also specify the web page that
we're going to be testing.
In our case today, we'll just use Google.
We can also choose what browser will be used
for our load test.
And finally, I'm going to disable this option right here
because I don't want LoadComplete to clear the
browser's cookies and cache before
recording this scenario.
So now I click Record.
You can see here that LoadComplete will
automatically launch our desired web page.
And now I can perform some actions that I expect users to
perform on this site.
So I'm going to type in Smart Bear into the search box here.
All right.
And that's everything that I want to do here.
And so now I'm going to click Stop, and LoadComplete will
save off the recorded traffic in this scenario.
Now, I'm going to verify the recorded scenario.
Verification of the scenario includes a simulation of this
scenario for one virtual user and ensuring that the scenario
is executed successfully.
It's a good practice to verify a recorded scenario for a
single virtual user before you create more robust load tests,
as verification helps you determine a scenario's
bottlenecks and avoid problems which don't relate to the
actual volume of virtual users.
So to verify a scenario, simply right click on it and
select Verify Scenario.
LoadComplete will simulate the scenario
with one virtual user.
OK, I fast forwarded a bit, and now that the verification
has been performed, you can see that LoadComplete is
displaying the scenario results run on screen.
And the summary report here is telling us that there was one
warning that was encountered during the course of the run.
So to get more information, I can click on this View link,
and LoadComplete will bring up the Messages
panel of the test log.
This panel contains various messages that were posted to
the log during the load test run.
Here, we can see messages informing us that various
requests were completed successfully and one message
that tells us that request number one was
completed with warnings.
So let's explore and fix this warning.
LoadComplete reports warnings when it detects that the
server's response code for a given request is different
from the code that was recorded.
So to determine the cause of a warning, we need to compare
the recorded and actual response codes.
And you can see those codes over here in the Recorded
Value and Simulated Value columns of this headers pane.
In these panels, we can see that the recorded response
code differs from the response code that LoadComplete
received during simulation.
A 200 message means that the request has succeeded.
302 means that the content is temporarily
found at another location.
And because the codes differ, LoadComplete has posted a
warning to the log.
Now, a possible cause of this problem is that we did not
clear the browser's cache before recording the scenario.
And when the cache isn't clean, then the browser reuses
cache data to process client requests, instead of sending
that data onto the server and receiving
responses from the server.
However, while running a scenario, LoadComplete doesn't
use the cache.
It sends requests to and receives responses directly
from the server.
And this is what's causing the difference between the
recorded and the actual response codes.
So to avoid such warnings, you need to clear the browser's
cache, cookies, and temporary files, and then
rerecord the scenario.
You can choose to clear the cache manually, or you can use
the Clear Browser Data button in the Record User Options
Scenario right here.
You can just make sure this box is checked.
Now, this approach is meaningful if the log contains
multiple warnings of this type.
However, in our case, we only have one warning in the log.
So we're not going to rerecord this scenario.
We're going to handle this in a different way.
Now, LoadComplete allows you to customize the expected code
for given responses.
So we can change the expected code so that the actual
response will be interpreted as successful in future runs.
And to do that, we just right click on the response header
here and say, Mark Received Response as OK.
So now, LoadComplete will treat any 302 message that
this request receives as successful.
And we can see the expected response codes by coming back
up to the Scenario Editor here, choosing request one,
and then selecting the General tab.
Now, you can see here, we're going to treat
response 302 as OK.
Now, you can also manually add other expected codes to the
expected HTTP responses list by right clicking here and
saying New Item.
And then you can customize whatever response code you're
looking for.
I'm just going to remove that one for now.
Of course, changing the expected codes isn't a
universal remedy.
The very same response can be considered as successful in
one situation, but it may be regarded as a
malfunction in another.
As such, you should treat every warning differently and
resolve them all individually.
To check whether changing the expected response code fixes
the warning, I'm going to verify the scenario once more.
So again, I'm just going to right click
and say Verify Scenario.
OK, I fast forwarded a bit.
And, as you can see, the verification completed
successfully.
There are no errors or warnings listed in the log, as
shown right here.
So now we can know that LoadComplete is reporting that
request number one will complete successfully, even
though the actual response code is different from the one
that we recorded.
So now we can create a load test based on this scenario
that we just verified.
So I've double clicked on my test node right here, and I'm
going to modify this default test to match more what I'm
looking for here.
So we'll do 30 virtual users to start out with, and I want
to emulate a 10 megabit LAN there.
And then I'm also going to add another group of virtual users
called VU2, and I want 20 users of them, and I want them
coming from my workstation at maximum speed.
OK, so now we've got our load test configured,
and I can run it.
So to do that, I just need to click this Run Load Test
button right here.
Now, unlike verification, the ordinary test run is affected
by a number of additional parameters, like workload
policy, think time, and the quality of service criteria
that we specified earlier.
And sometimes, these additional parameters can
cause issues.
So don't be surprised if new errors and warnings are
introduced.
This is normal.
Now, while running the load test, LoadComplete monitors a
number of server parameters and plots them on the runtime
graphs here.
For example, this basic quality graph that you see in
the top left corner of my screen displays the number of
errors and warnings that occurred during simulation.
The errors are plotted in the graph in the red color, and
the warnings are in yellow.
OK, I fast forwarded a bit.
You can see our test run has finished.
And now let's take a look at the messages that were posted
to the log here.
And as you can see, this test log
contains a lot of messages.
And it may be overwhelming to find the ones
you're looking for.
So to check to see it the event log contains informative
error messages, I'll filter the panel's contents simply by
unchecking the OK box right here.
And now, we can see all that's left are these errors.
There are no warnings that were logged, but there were
several errors that occurred during the test run.
Now, all these errors occurred for one reason, the quality of
service criteria were violated.
So these errors mean that the tested web server isn't
operating fast enough.
That is, the appropriate requests were simulated
successfully.
However, the intended response time exceeded the specified
quality of service metrics that we entered at the
beginning of the test.
So, now those things are specified here on the quality
of service metrics on the Load Test Editor page here.
Now, since these are unchecked, then that means
we're using the global settings that are applied at
the project level.
And those are listed back up here, back here on this page.
These settings allow you to enter the times that should be
spent on traffic simulation.
If the actual times exceed the specified threshold time
values, LoadComplete posts errors, as we've seen.
So the quality of service criteria are set from your own
application's specification.
So you come up with what those numbers should be.
And if those numbers are displayed, then we know that
the server isn't performing fast enough, and you need to
find the cause of the server's poor performance.
Once you resolve the error, you can run the test again to
make sure it's fixed.
LoadComplete can post errors to the log for various other
reasons, not just if the quality of service criteria
were violated.
For example, it will post errors to the logs when it
fails to send requests to the server, or if the server
failed to respond.
So let's talk about errors of this type.
I've opened up a new project inside of LoadComplete.
And you can see that I'm testing this V32E server.
I've already run my load test, and I've received this report.
As you can see, this test run has failed because several
errors occurred.
Now, this first error takes place right down here, and you
can see that the message is that the
connection was refused.
And as such, some requests that followed
weren't actually simulated.
You can see here, this particular request has
recorded values, but no simulated values on replay.
That's because the test engine was unable to connect to the
tested host.
Quite often, the connection refused and host not found
errors are occasional errors caused by temporary issues on
the server.
And in subsequent runs, those errors may disappear all by
themselves.
To check this, just run the test again.
But if you find that the error is persisting, you may need to
adjust some server parameters.
For instance, you may need to increase the maximum number of
simultaneous connections that your server allows.
There are situations when LoadComplete does not report
any errors or warnings.
However, you find that the load test
isn't running correctly.
For example, you may notice your database contains the
incorrect data after the test finishes.
This can happen when you load test applications that use
specific data formats and protocols to communicate with
a
web Server.
For example, when you test Flash or Flex applications
that use the action message format, fact is that such
applications return error messages not in the form of
response codes, but as data within the response contents.
So since the response codes are successful, LoadComplete
isn't reporting any errors.
So to illustrate this, I've opened up another LoadComplete
project that contains a load test for a Flex application.
And as you can see here, the LoadComplete results are
saying that everything was successful.
There were no errors and warnings.
However, I know that something didn't work right.
So what I'm going to do is go into the Messages panel here.
And what I can do is click through my messages that are
listed, and then click on the Bodies tab.
And what I want to find is if there are any errors in here.
As you can see, there's a fault string that's been
provided inside the response body.
And so now we know that something went wrong during
the course of our load test.
So to find the cause of problems like this during the
test run, you can also use other sources of information
like web servers logs, your application's logs, and so on.
This concludes our video.
You've seen the basic principles of resolving errors
and warnings in load tests.
Certainly, each real life error and warning would
require its own repair technique.
However, you can find more information in the
LoadComplete documentation that's available on our
website at smartbear.com.
We wish you luck, and hope you enjoy automating your tests
with LoadComplete.