Tip:
Highlight text to annotate it
X
[Applause] Thank you everybody, I'm Tom.
First of all I'm going to talk a little bit about
Kazoo,
the company we work for. It was
set up almost seven years ago
now and the focus
was on building
personalized public transported
services for
mobile devices
and also for the web. Here's
a picture of
our chaotic offices.
What Shannon was
saying earlier
was very
resonant for
me, because he
was talking
about how
difficult it is to test
mobile services. Over the
years we've
written a few tools
and frameworks to help test mobile devices. We're
going to talk about
today though is
more -- it's
not really about mobile. It's more
general than
that. It's -- what we
call literate
functional tests. And
the
basic idea is to be
able to write tests that are
very understandable by both
developers and by the
customers that we're attempting
to build
for. And this
particularly evidence
because Kazoo is
an XP company?
It's been that way ever since it was set
up. And so we're always
talking about customers. So
what is literate
functional testing? Let's
go through it word by word. Literate comes from Donald
Knuth's idea of
literate programming. Like
many things it can be
a science, it all goes back to
Knuth. His idea there was to have documentation and code embedded in
the same document. We mean it being something slightly different.
We're talking
more about readable code. We're not
talking about
integrated documentation
here. However,
those equate from Knuth, which I'll read out, which
is quite famous. Instead of
imagining that our main
task is to instruct a computer what to
do, let us concentrate rather
on explaining to human beings
what we want the computer
to do. So that's, -- that
shines with this idea of
communicating with
the customer. The
other strand of literate is
to do the linguistics.
Programming languages have always used
ideas from natural language, whether
they
accomplish that explicitly or not. I
think that evolves
quite, --
he talks
about
the fact that the influences from natural
language occur.
So,
in this framework
of
what we're going to talk
about we're going
to acknowledge the different linguistic devices that we're using. So, functional
in
literate functional
testing goes back to XP. So we're talking about the
unit of
delivery of XP, is
a kind of functional unit, it's
a feature.
And the important thing about
that is the
way that
that's in
the customer's
domain of discourse. These things
that customers
talk about are things that
customers talk about. And
so they
understand them and they can -- they
decide what they
are. And so this
is to
distinguish it from
unit testing. And then the testing
is the
ultimate testing bit, which is
where we come in to
this conference. And
we're, we
come from a Java background,
so we're using lots of Java automated testing tools and some
of
which, maybe
all of which,
you know well.
JUnit, for
example, needs no
introduction. JMock is
an objects testing framework and
written in Java
for unit testing and
I think
three of
the authors are in
the audience
today. For
functional testing we've
been focusing on
web applications and certainly
for this
talk we'll mainly focus on
testing
HTML. And so
we're, -- we've used HTTP units and HTML. HTTP units are slightly lower
level
than HTML units, which we
prefer. So, a simple
equation to describe
what we've done
would be like
this, literate style plus HTML unit plus
JMock
gives the literate
functional testing
framework. So that's the
kind of caricature
of the work we've done. However, like
most caricatures, it's suggestive
but wrong. The
-- we started doing this, not because we were
testing a web application. We
were actually testing a proprietary XML
mark up.
So we
weren't using HTML units so --. We didn't
have to
use an HTML unit.
Similarly JMock suggests we're doing mock
objects, and
we're not. Mock objects is really about unit testing, which
we're more functional
testing. The reason I put
JMock on this
slide is because
we used
a flexible assertions
in the constraints of Jmock. So,
those are
two other things
in JMock besides
mock objects that we found
very useful and influential
for
our framework.
Also, we've build on these frameworks.
We haven't just added
them
together and
called it Literal Functional Testing. There are other ideas that have
come in which we'll
go through in
this talk. So,
why are we doing this? Well
our aim
is to be
able to write
tests that developers can read very easily.
That's quite
a common requirement. You want a test that's, when
you come back to it in three
months time, you can read and
get the intent very
quickly without having
to understand the
low level stuff that's going
on. Further more, we
want to be able
to use these
tests as test artifacts
that
we can share with business analysts
and customers.
That's not
something that
happens much these days if
at all. So, these
tests provide a common language for
us to talk
to our
customers and the
idea is that, that
cuts out
us having to
interpret what they've done back to the
customer.
So, instead of saying this area
we've put
your requirements
and we've implemented
it
and this is what it
kind of
does.
We're going to be able to give customers the literate
test and say
this is what we've done;
you can read how
it behaves,
all these
test parts. This is the system.
And then they can
look at those and understand the feedback.
So to do that we've got
five principles that framework
follows. The first
two, very
high level and readable, are important for
the customers. So,
we want them to be
high level so that they're
in
the customers
language, they're
at the user
level as it were. And we
want them
to be readable so
that the customer can
understand them. Those two important for developers
too. Obviously they
should be readable
for developers, for different developers or
for yourself coming back to code. But the
last
three points need to apply to developers
as well.
It should be
easy for a developer
to come along and write
a new
tests. They should
be composed, by which we
mean the constructs of the testing should be applicable in
different contexts.
You shouldn't have to write lots
of stuff
to test your
system. You
should be able
to reuse bits of
space. And it should
be extensible. We
want to be
able
to grow with framework, as
a developer developing
a particular application. I want to be able
to grow
the framework, to
use the vocabulary
of the domain
and
that fits in to the customers
as well.
So
it's talking
in that terms.
So that's
what we're
going to do.
I'm going
to hand
it over now to
Robert who
is going to talk about
how the framework
was born and the --
initial ideas
that got us
started. Okay, thanks Tom.
So, yes, as Tom
said, we didn't
start out doing
web testing. We
actually started out when
we were building, about 9 months
ago now, an application for the transport information over
digital VP,
because obviously you want
to make a transport
information
when you're home, if
front
of the TV.
That's
what we're here do.
And so, this thing was
an excellent mark
up. So we didn't actually
have
an off-the-shelf
tool to
test
this thing. So we needed to build
something and it was sort of took this
as an
opportunity to build something that
was new. So as
Steve mentioned earlier HTTP -- and
a lot of you seem
familiar with
that. And I think, Steve, you
said something like
you can get
it to do --
go to this page and click on the link with
the label
A or something like that.
But if
you use
HTTP unit it can't exactly do that. This
is one of
the tests that we
had from maybe a
year ago.
It models
concepts of
HTML stuff with
web client and
stuff with the actual
requirement that you want
and the thing
that you're testing. And
it's all
a
bit kind of modeled together
and it's
quite a lot
of code.
So you
can make that into a Java program, you
know, it's probably fine. You can work
out what's going on here. But I
can't show this to
my business
analysts,
probably. I
definitely can't show this
to my
customer. They won't
know what's going on
here. It's even
a bit difficult
for me to look
at it now and
see what's going
on.
So we
wanted something
that was a bit
more
expressive, something that was a
bit more in the
domain of the
requirements. Something
that's a bit shorter really. So we started to think
about what
we want and then
build a framework
that will support that.
And so we just
took some paper and we
took some mock up
-- things
like just some
photo shop
mock ups of
what the application
is going
to look like,
the design. As
you can see it's
got sort of
like menus
and stuff. It's got some drop down
boxes, radio buttons and buttons
like that
and it's
got sort of a summary of stuff,
information about it
and things
like this. So looking at these
sort of mock ups, you're like,
okay well how
can we write
a test
just on a
bit of paper that
says how we do know
when we
designed this
feature? And you have to admit
that we can sort
of had a
view that we were going to automate
this using some sort
of program. So
we really don't complete
free form language
but we
tried to write
in English.
It's kind of
stylized but these
are some of
the
things that we wanted to
say. So we said, so
the current page, say, has
three select boxes and then
a region of transport
time and
delays so that the
region sample says
between 9
and 12 lines, it
could be London
or the
Southeast. So hopefully
this will
be on
the same
way and a
non-programmer will be able to check in
to
see the mark
up and they know
that
if we implement
something that passes this test we should have
implemented the
right thing. So, the customers knows that we're writing
the right test
because they can
read it. If we
can make codes
like this, -- which
is tricky. So we
took -- we totaled the words in to
Xpad or something and
said okay, right?
So we've got
in electronic form now. How can we refine
it into something a bit
more code
like? So now trying
not to
think too
much at this point
about how we're going to
implement the framework. But you know,
something that might be implemental, sort of
add brackets
and things
in nice looking places.
You know squash some words together in kind of
Java style -- with some
strings and lots
of the
things that -- literals in
the page. This is
the other example here, the
9 and
12 options one. So
you can kind
of see that this is going towards something that
might pass
as a
bit of code if you
had something to support that.
Okay, so let's
move along then.
So,
actually most
of our
tests that we're
going to show
you look
something like
this. This
is kind
of a common
style. So this is,
yeah, a kind of
form
that
you'll see
commonly repeated. So a little
anatomy of the
different types of
themes that we have;
we have
got a
few different
types of
themes. So all
the things
that you'll
notice is
that before the tests started to
assert that, something, now -- this
is not my invention. I was reading
Joe Waltons --
right. Oh, yeah,
there. We're going
to be
talking about AJAX; sounds very interesting.
So I was reading
one
of the literally 3 or
4 posts on
the blog in
the
last two years
and, about flexible JUnit assertions.
And he says
that you have
all
sorts of different assertions
in Junit. You've
got recalls, you've
got same, you've got true,
and you can
basically replace all
of these
with Assert That
from the JUnit frame
work and a JMock constraint. And thinking
about it, later I
did that
and -- but
basically we
have a lot
of things in
the same
format. It looks a
bit more like
a sentence. And we've also
got,
what I think is the subject
in one session. So normally we're talking about
the page or screen.
So, we're going to talk about the current
page. We've
got things that we want to
be on the
page, or not to
be on the page.
So, kind of like nouns in a sentence.
So you're still on
things, or methods
that describe things.
And
you've
also got, instead of just things, now
you've got
types of things,
so. Rather than just
any select
box, sweep like the select
box, which is laden with regions.
You're about to
be able to pick
out particular qualities
of things. And to
string a whole sentence
together,
we need a verb. So here we've got
'has', so we want to check that the verb 'has' has
a select
box and also
that it
does not have
that select box. Now, in
one of the things I
wrote, we did between 9
and 12 options. And I think
we probably
had something like this between 9, 12. So -- you'll
find -- you're
mostly programmers
here probably,
I think.
It's interesting that
it's between 9 and
12. It doesn't say between 9 and 12. This
says between 9 and
12. So it might not
seem like
much, but actually, we
had thought quite
a lot about,
you know, how
you actually make Java
have the words in
the right
order. That if
you strip out the dots and
brackets that Java forces you
to have, then you can paste
it into a web document and you can
send it to
your customer. And
they can read it and
they think that you're
really well planned, which, you
know this is the
really good
thing about test
reports. If they say what
are your
tests and then print it out, send it
off to them
or just look at the code. We
had BAs and stuff
reading them the code or whatever and
saying -- yeah I can
see what that
does. So just reading
over it, just having
some linguistic
influence, just
working on some
things to try
and make the
Java read properly was quite
hard. But I think quite important in
having the framework
where we really
can write tests
that anyone
hopefully
can
read. So, I'm going to pass
it back
to Tom now.
So, just
to say, so we
started off just as
traditional as can be, as I've said, and
we found out
it worked quite
well and
then, so in our next
project
we're actually doing a
full web application. So
we tried to develop
the framework further so that we
could support more elements of HTML
rather then the
somewhat restrictive subset
you get
with the
TB mark up. So what Tom will demonstrate is
how far
we've got
and what you can
do with
a full
HTML text.
Thank you.
All right, okay so what
we're
going to do
is go to
a famous
website and look at
some images. And I'll do that manually in the browser and
then
I'll switch
to a Java editor and
run the test.
These are the
steps we're going to
follow through. I'll
switch to the
browser straight a way
and come
back to this slide a bit later.
Okay. So,
-- I'm
just going
to say the words
that you might
say when describing
what I'm doing in a fairly
minimal way.
So first
of all go
to the index page,
-- which is Google. Check that
the page has
title the
Google, which is does. Click on
the images
link. Now we'll do some research for
some
images of my company. So
enter 'Kazoom', enter the text
field, then click on the
label Search
Images. Okay, the best
one is
the Kazoom summer party
from a few years ago, which went
pretty well.
So, I'm checking that the page contains
the text 'Kazoom summer party', because it's
an image
so you can also
check that -- there's
an image on
the page. Summer 04,
you probably
can't read
that, summer
04.jpg. So that's a
very simple example of
a
manual
test. And we'll switch to
Eclipse and
write the test
now in code. Okay, we'll call
it GoogleTest.
I am going to
extend the
classical navigating literate
functional
test base. And this is used
whenever you navigate between pages,
which is
most of
the time.
I'm just
going to now use a couple
of these for static mark --
methods. So, you'll see why
we do something like
that. You need to open it first. Okay,
the first thing that
we need
to do is tell
the test
framework what the base
URL is, where do
we start. And
that's the
Google home page.
Now we can
write the test
proper. In case
you haven't noticed, this is
actually based on JUnits
as well. It's somewhere
up the
chain. So,
now I
can start writing the words
I was
reading out when
I was doing
the manual test a few minutes ago. So the
first thing is go to the
index page. That
seems to throw an exception. So,
the next thing I did was
check that
the
page -- has title 'Google',
-- okay. So
I've got an assertion there. I
think I can run the test now and see
if it
works. Yeah, green bar,
that works. Click on Link
Images, enter 'Kazoom' into -- text field,
click on button --
Search Images, assert
at page has text,
'Kazoom summer party',
okay. That should be
a fairly good
test
when
I do
that. Didn't like that, so
let's get that
to work. That's working now.
And finally,
let's just check that
the image name that we
expect is on the
page. So
I'm going to just
slight vary in
here. So that's, it has
image, with
URL that ends with
summer 04.jpg. Okay? Lets program that.
Okay, that passes. So
that's the test,
and that's the same
as the
manual test
I ran
a few minutes ago.
So, finally,
before I go back to
the talk,
I'm going to
turn off punctuation,
so we've
got English that
you
can read now. If
I switch back to the
slide, those words
are the same
as the ones
in the
code. Okay, so what we're
going to
do now is go through
the
features that
the framework has.
We've seen
most of them in that
demo. I'm just
going to go
through
each in
turn and talk
about what it is
really to
get a bit more
detail about
how it works. So those
are the list
of features
that
we think
are
important
in the framework and
the first five
were covered
in that demo; the last
two
weren't.
We'll talk
about
those more in
a minute.
So the first
thing we have are
actions. Actions are
verbs and they do things. So,
for
example, go to
the index page will -- remember we're using
HTML unit and so
it maintains
that the state, the
browser which is connecting
can do
retrieving pages from
the website.
So go to
the index
page
just
retrieves the first
page
URL and
updates an implicit variable called
page. On
the
other side
it was
called the current
page, but
that was the
previous version
of the
framework. So
that's an
implicit variable.
Implicit
variables aren't
a new
idea. For
example, JSPs
have a page variable that
you get when you're launching
a JSP
-- a
Java script.
Don's programming has all of that. It's
the same
concept too. So
page is the
most important
implicit variable in
the framework and
it refers
obviously
to
the current page
that you're on. So actions update
the page
when you
navigate. The second most
important implicit variable
is a variable called,
it. Now linguistically
this
is,
it's
called an anaphora,
which is
a word
that refers to
something
you've already
talked about.
So
I'll
give you
an example in
English. If
I say
the sentence, I've baked
a cake,
would you like
to
taste it? It,
refers to the
cake. Now, first reaction
I'm hearing
with this idea
would be, wait a minute,
isn't that incredibly
ambiguous and the
last thing
you want
to introduce
into a
programming language?
And that's
a good
reaction to
have. And it's
one
that we had. However,
there are -- we've
solved that problem
by making
sure the rules
that govern when, it,
is updated
are very
simple and hopefully
not prone
to ambiguity.
So
the rule,
the
important rule is that
whenever you
do
assert that, the subject of
the assertion is assigned to
the
it implicit variable.
So
if
you look at this example,
the
first line says assert
that box, get fruit,
is not null. When that's
executed the it variable is
assigned to the subject, that is,
box get fruit.
So the
second line can
refer to
the, --
to the fruit in
the box
and say that
it has property
color, which is yellow. So the reason
that's useful is because
it avoids us having to
assign a temporary
variable.
If you wanted to -- so
you've got two
choices actually.
You
could either make
the second
line
say assert that box go
get fruit again,
and just
repeat the
reference, which is okay. But
more forebodes and
often as we
all know these
things get very
long, and
when they do
we typically assign
those to temporary
variables. But
this way avoids
us having to assign
the temporary variable.
And I think
we can
say from our experience of
using it so
far, it
hasn't caused
any
problems. Okay, so the
next, let's
move on to another feature. Matches, we
mentioned those earlier,
those are things on
a page.
So, the
whole point of doing this testing is to assert
things about a
page. We either
want to perform
actions on bits of
the page, for
example, clicking
on a link
labeled --
Images. So link
Images is a match up
for all links on the
page that
are labeled Images.
Or we want to assert
things
about bits of the
page. We want
to say
the
page has a title, which
is 'Google'.
So that's saying that
there's a
match up
for the total
elements on the
page that
have the
line
Google. So it matches
the nouns
in linguistic terms.
Matches can
be
refined and this
is very
useful when, as
we
saw
in the
image
example, there are lots
of
images on the page
and we want to refine the one we are matching. So, if you
look
on the
top line, it says image.
That's the matcher and
we're refining the matcher with a
noun post modifier,
which is the
-- with URL
that, which
takes the constraint
to say how
it finally
works. So
this is
very useful. The other neat thing about
refinements is you
can change them. You can
say
with URL at dot
with name, and you'll
have a
whole list
of refinements allowing
you to
put lots
of conditions on what
you're matching on the
page.
That,
by
the way, is
an example of
what Martin Fowler
would call the affluent interface. And
so, yeah, we -- matches
and refinements
we found. That's
an area
of extensibility that
the team
has exploited. They realize
that
matches they want
don't exist so
they've
written their
own,
which is
quite
easy because
it's
a small issue by which to
build the interface, you have
to implement it not in
a chunk.
So the
next feature
we talk about
is constraints. This is
straight from JMock. So there's
lots of documentation
on how to write constraints in
JMock and you can
do exactly the same
in this framework. The
one that we've found
is useful, I think,
it the
Has constraint.
And what that
does is assert
that the
matcher has at least
one
match. So the
meaning of the
line in the first example
is, assert that the
page has at least
one
title cause it's Google.
Synonyms, so
this is
another linguistic
feature that we've exploited. If you
look at the
code, the
implementation of has
and have, you
would be possibly surprised to see
that they're
the same
one minute,
then it goes
to the
another. So
they have
exactly
the same behavior, which
is slightly unusual in
programming to
deliberately
introduce redundancy
into
your code. And
the reason we've
done it is because
we think
it, as Ron
was saying
earlier, that we've been
trying as much
as we
can to
make these things
as expressive as we
can. So,
if I read
out
the two sentences, you'll see the second one
reads a
bit better. So assert that these
cells has at least one link isn't grammatically correct. Whereas,
assert that
all cells have
at least one link is
correct. It's a small thing
but we've done it in a few
places in the framework. Okay,
-- consequences,
these are
-- this is a
fairly small feature and
this is not to do with functional testing
per say.
It's a full JMock library, when an assertion fails
it's very good
because it can print what
failed, because it uses a
self-describing constraint, so the constraint knows why
it failed. That's normally great
and that's all you
normally need to do. But there are a
few occasions where you
need to
have bit more control over
that to understand exactly
which line has failed.
If you look at
this
example and ignore the
otherwise clauses, then
the
second assertion will fail
but JMock will
just
report
something like, expected true but was false. But that
doesn't give me as a developer even a clue as
to which assertion
it was, the
first or
the second. So we've introduced a
consequence parameter into the
Assert That clause which allows you to give
it an otherwise clause. And so if we
ran it, as is
seen there, the
failure report would say these should
be true, which it has to,
to identity
that line. Okay,
this is the final feature
I'm going to
talk about since
it's probably the most complex.
What
we've been talking about so far is always about the
page really. We've
always focused on
the whole page really, we've always focused on the whole
page or the
whole screen or
the application. We
found
when we were actually
using the framework that we wanted
to
restrict
attention to a
certain part
of the
page and have
all our assertions
just applied to this
small part. An
example that
we use -- sorry
the
place where this
popped up
was when
we had multiple
tables on
a
page and we
wanted to just focus
in on one
table
and say certain tests
and things like that table. If you
look on the screen
you'll see the two, the
two by
two and the
three by three table. Without
the considerate
clause, the assertions would
fail, because firstly
it refers to
the whole page,
in this
case. And the
rows
matcher, this
one here,
would count
the number of rows or match I
should say
the number
of
rows in the
whole page
which,
if you look
in
and imagine those two
were on one page that would
be five rows.
Similarly, cells -- would match
all the
cells on the page rather
than just in one table. So, we
introduce the considerate clause, which takes
matcher to
find the
thing under consideration.
And in this case change it to be a two by
two table.
And then the following
two assertions both pass now
because the rows
-- the
matches use the consideration to match
against not the whole thing.
So it's kind of scope
changing -- clause.
Okay, so, those are
the features.
That's a quick review of
how
the framework
is put
together. Rob
is going
to talk now
about how
we found it
in practice and by
the team at
Kazoom. Thanks
Tom. So,
yes,
so at first we got this framework. We introduced
it
into a team
working on quite
a big project. They were knowing
developers and
a full time
business analyst
and a full
time project manager on a project
and
it was, dare
I
say, a
popular website
in the UK
for making
queries
about trains that traveled nationally.
And so this was
a very restrictive.
It
took about four months to
develop with
all the
people working
full
time.
And it
was written
in Java.
And
we used
-- we didn't
start off using
this framework. We
started
off writing tests
basically in HTML
units. And then
afterwards -- we got
this framework
and we
tried this,
how will this work out?
And so we tried to get
the
team to
adopt --.
We tried
to get the team
to adopt --. And some people really liked it. They
thought, oh this is
good I can write
new text. Some people found it
really
hard. I thought this was
interesting. And it turned out from chatting
with people that it
seemed to be quite, to me, a distinction between the people who found
it easy and the people who
found it hard.
The people who had
always programmed in Java, at least professionally, thought
that it looked
funny. So the people that always
programmed in Java
thought that it looked strange and you would have notice that
none of our examples has
an assignment statement.
People were
looking, where do I assign
this thing so
I can use the data. Where as people who had more
of a
background maybe in
functional programming
where typically expressions
are written without
temporary assignment
found this more natural
and were more able
to compose
things and
sort of call
the three
things. They found that
easier and they
more readily thought, yes
this is good. They
felt happy
at the start.
So for the people who
were more
procedural it
was harder
for them.
So, it
took a while for people
to be
convinced. But what we
found is
that once we had
some examples
and sort
of written some
code fragments, people who were
pushing the frame work
more than others had
written some
examples of
how the test
particular things. You
could pretty much sort of
copy a
test and adapt
it. And you
could see
the style and use
the elements that
were present
in one to write
another. And that seemed to
work pretty well, once we got a fair base of tests that people could
write that's a test that's a bit like this one that
you just need to adapt it this way. And with
those we found that we were able, with a little help, to
start over and
once we got the hang of it to extend
the vocabulary that they
had available to them.
To match the more that
they entered to describe
more
things that they wanted to
talk about
in their text. They
can extend
the match's interface to
create more things that you
might want to identity in the base. I
noticed
particularly when we
handed the framework off to another
team, after a while they said
what are you
doing
over there and
we said oh,
we're using this. And
they wanted to
talk about the styles, the CSS
style that's
being used
in the pages. We thought actually
thought of matching against that sort of
thing. It was
like, oh, you
can't do that what you need
is a match
up with this. And within twenty minutes, they
did one of those and
they were happy
using that.
So they were already able to extend the vocabulary
of the language that they had available
to them in writing the
test to
match things that they
were interested
in
and that a
customer was interested
in. So, one of the other things that we found was principle was good and bad. So you'll notice that when Tom wrote the test in Eclipse he had some completion from place
to show possible things that you could do, plus identified and
matching a link with a URL with ID and it could suggest the possible things that you could do then. So that's quite helpful a reason, but we quite liked having a -- type language so something like Java could read it. Now, we're not doing very well. [Question] [Laughter] So the tool support works well with that. But, the static inputs don't. So what we tried to do was not have everything in a really deep comparisons hierarchy in the framework. So that you might possibly get to use with something else and
possibly it's not going to compose things in, which meant that we decided to
go with static
input from file, which prompted -- which allowed
us to go with finding another
class. But the
source code is a
bit more. You see Tom put in the import style at the top and if you have to
organize your inbox in Eclipse, so you're
not using any
blanks, we'll
just using the one's that you're using, not the star. So therefore I won't suggest the start;
you
have to be a bit perfect with that. I can see there
are some things that might be improved in future
versions. I can
see it's a hard problem to search all possible classes. But -- as it suggests -- not so easy there. So, how could he get on with actually doing the communication pipe? So, I'll just show you.
This is one of the
tests that
Joe came
up with the
code base.
This is about selecting
a particular or going to a particular journey and picking out some
fares and picking the
best ticket. So this is the
sort of thing
where
us inferencing it might have a query, -- like what exactly was the requirement? So we found
a full time business analyst to keep track of
our program.
We said, look,
we're writing this test, you know, all
I want to know,
is it right to that I'm supposed to click on this button should I go to this page or to this button. Am I doing the right thing? And you'd able to use the test as a common discussion after that. So, not being
a full
time programmer, he
had been a programmer in the past, but wasn't
doing it every day. We don't think about that particular advantage. You can redo the test, you can talk about the same test and you can query if test is wrong or right based on just the word style there. So you don't have to say, oh don't worry about that, that's just a programming thing. You can actually talk about this. And also when we had to produce a report for the customer about all of the things you had tested, it's like, take this and do a nice trick. Tom show us how to remove the brackets. You just put in a few spaces between the identifiers. And here you can patch that off and send that to customer saying this is the list of all the things we tested. Here's your requirements against the tests. You can see that we're doing, we're covering those. And if
we have do that again it seems fairly possible that we'd able to write some sort of tool that would just go over it and replace all the punctuation and blanks and do the template case and put some places in there as well. We had to go in and do some dots and spaces and such. And we also do that for manual testing scripts. So we had to get some users off the street to try out some stuff. And we said, okay, you know, you have to come up with a whole load of extra manual testing scripts. Writing them out, they had to be easy to understand. You know we've already got a set of scenarios that came out of the content originally and these steps were what we were going to do. So after doing this in Java -- there was a number of reasons I was doing it in Java, one thing is all the people who work in our company know how to program in Java, which is useful. And not very many know how to program in the same other thing. So it's the common language between them. So the other thing is that we wanted to harness the tools that we already had. So we had JUnit and we had the Eclipse running. So what we wanted to is within there. If we were to go and use another language we'd have to find another tool. We'd have to switch back and forth; it's like switching languages. It's very much a difficult thing to use two different languages, one to write the test and then another to implement it. So we thought by keeping it in the same tool set, in the same -- that would be less of a cognitive difficulty. But it has been suggested that other languages aren't quite difficult as Java to make behave when you put the words in the right order. With out punctuation it's a bit messy. So maybe Ruby might be better? Certainly, you know, if you have fewer punctuation you might have a bit more flexibility and a little bit more syntax, might be other possibility for doing something like that. So we haven't written any of those frameworks. Let's have a go and we have to put this in the past and program these images. So what have we done? So we've shown a Java API for writing tests in what we call a literal style. Hopefully you've seen the code that makes it readable for a program or programamer to read and implement. We've tried to keep the style of the programming that you use as close as possible to natural language and the linguistic process that we used are sometimes better on smaller things. We've gone a long way towards that. Hopefully the tests show clearly the intention of the requirements and are less focused on the actual rate of the pages or the rate of the test or actually mechanics of the test running compared to say the HTTP unit test that I showed you, you know. And we've also shown how the test itself can be a point of discussion between different members of your team or between your team and the customer. Those are kind of, even from one point to another, when you come back to a test that someone else has written -- that it says that click on this, close this page, it's very easy to then replicate that by hand rather than to trace through a low level code to work out what the test is doing. So people like that. And we tried that with our team and initially it was hard work. I think people are more convinced now. It's gotten to the point where we had all of our tests for the same project written in this style and the most people were happy with doing it this way by the end of a couple of months. So, what's next? Hopefully to open source the framework in the next couple of weeks or so. We've got a URL that doesn't have any code on it yet, but we're using it. Time for JUnit, but we weren't actually using any of the mocking features on JUnit, that we don't need. And luckily, or coincidentally some of the other are working on a project called -- where we were working out the constraint part. That way we thought maybe we could just really on that rather than the whole. Other languages as I said may be Ruby. Other domains, we're just testing web applications. Maybe we could adapt this test to all sorts of applications but with slightly different bearings on the framework that matched those sorts of things. Certainly you could tie it to a different set of testing tool so, you know testing tool -- or - documents and that sort of thing. And the other thing is, why are we doing this just for our tests? Why if we're writing bloody clear tests, why aren't we writing bloody clear applications as well? The main reason why I can see for that is that when your, certainly in Java, you're interfacing with a certain number of different APIs you're much more constrained with your application