Tip:
Highlight text to annotate it
X
Now I'd like to talk about being great at testing software,
and this involves a number of different elements.
First of all, we need to recognize that testing and development
are fundamentally different roles, even if they're often
played by the same exact people.
A developer's primary attitude is
I want this code to succeed.
A tester's main attitude is I want this code to fail,
and of course, the reason the tester wants the code to fail
is that the tester's end goal is creating stronger code
which later on doesn't fail.
If we look at these requirements a little bit we can see that for the same person
to be a great tester and a great developer
there might be a little bit of doublethink involved,
and so those of you who read Orwell will know that the doublethink is the
ability to hold 2 contradictory beliefs in one's mind simultaneously,
and there is a bit of that required in order to be a great tester and a great developer.
And of course, really the contradictory nature of these 2 beliefs
is only apparent because, of course, the developers and the testers
in the end want the same thing, which is to create a great software.
The second element of being great at testing is to learn to test creatively,
and I showed the example earlier of testing an ARM assembler and disassembler
by exploiting the fact that they were inverses of each other
and also exploiting the fact that the ARM instruction space
could be fully enumerated, and that's something that I consider to be a great example
of creative testing, and if we can think of creative ways to test
we often do a much better job than rote testing,
and furthermore, do a much better job than just
rote testing with the most obvious inputs.
The third thing is that great testers never ignore
weird things that they notice.
It turns out that at least in my experience
it's very often the case that we get little hints of things wrong
with the software that lead to threads that if tugged on
would have led us to discover problems that were really quite serious.
On the other hand, if we see these little things wrong
and we paper over them, we ignore them,
we end up not finding those problems until later.
I'm going to end up today with the claim
that great testing is a lot of fun because
it's fun to break software, and it's very satisfying
to produce software that's really great
because it's been well tested, and great testing can also be profitable.
And what I mean here is that testing is a separate career
at many companies like Microsoft and that companies
like Google and Mozilla offer bug bounties,
and what this means is if you find a security critical bug in,
for example, Google Chrome, they'll pay you up to $20,000,
and this isn't at all theoretical.
I was recently talking to a software testing researcher
who wrote an automatic tester and applied it to Chrome and to Mozilla,
and their research group, over the course of a few months,
made about $50,000.