Tip:
Highlight text to annotate it
X
[MUSIC PLAYING]
PAUL LEWIS: Well, hello everyone and welcome to this GDL
on Hobbit: A Journey Through Middle-earth.
Now I actually have the team with me
from North Kingdom in Sweden who built the Chrome experiment.
So I'll hand you over to those guys,
and they can introduce themselves.
DANIEL ISAKSSON: Hi.
I'm Daniel Isaksson, technical director on the project
And this is my friend, Einar.
EINAR OBERG: Yeah, developer.
PAUL LEWIS: All right, guys.
Thank you for joining us.
So, I guess, the first thing I wanted
to ask is, for anybody who's not actually been to the site
or seen it yet, what kind of things
should they actually expect?
What have you guys put into that experience?
DANIEL ISAKSSON: Well, we wanted to create
a very visual, multimedia exploration of the characters,
locations, and everything from the first two "Hobbit" movies.
And we're doing some small games using WebGL and CSS.
And some dramatizations of different scenes and locations.
PAUL LEWIS: That sounds awesome.
So what kind of technologies or libraries or frameworks
did you have to use to bring this whole thing together?
EINAR OBERG: For the experiences,
we have both the WebGL-based ones using three.js
And we have a couple of locations
as well using CSS only.
So it can be experienced on other browsers
than Chrome and Firefox, on mobile.
So we could reach more people with the actual experiences
as well.
PAUL LEWIS: So is a lot of the code
specific to your own engine?
I mean, you mentioned three.js there.
Did you have to craft a lot of the engine for yourselves?
EINAR OBERG: No, actually we used,
pretty much, three.js as it is, which
is pretty awesome already.
And then we, for example, in Lake-town, we had physics,
and then we used the three.js library and the branch
with the animal.
The animal branch in that one.
PAUL LEWIS: OK.
Awesome.
OK, cool.
So obviously with something like this,
you've got a lot of moving parts.
Like you said, there's CSS 3D transforms.
There's WebGL.
There's audio.
There's imagery.
Just to name a few of these things.
What was the actual workflow for building the whole experience?
Because, as I say, it's a complex beast.
And it has to run well.
So how did you actually kind of build up
to what we actually get to see?
DANIEL ISAKSSON: Well, I think it's
made up of different parts, as you say.
And coming from the really more creative side of it,
we usually do a lot of animatics in After Effects,
pre-rendering 3D work from 3D flows into each section, how
we want it to feel and look.
Then, from that, I think it's very much
going into early prototyping.
Because when you sit there in the room trying
to discuss a game or a project, without having
an actual prototype to play around with,
it's very hard for lots of people
to get a common understanding of what you're talking about.
But that's--
PAUL LEWIS: So did you find that you
were making a lot of prototypes and iterating a lot,
or did you find that you just made one or two prototypes
and that was good enough?
I mean, how difficult was it to arrive at something that you
felt was creatively solid?
DANIEL ISAKSSON: I think for some of the locations,
we've done way too many prototypes.
But it's a very difficult experience
to create stories around and to get something creatively that
works both to make something as a tech demo.
I mean, it's easy to make something that's a good story,
but maybe it doesn't suited for a good tech demo.
And on the other hand, we're making--
so it has been a challenge to create stories
that works in all dimensions.
PAUL LEWIS: Yeah, I mean, for anybody who's not seen it,
I mean it is a phenomenal piece of technology
that you've actually managed to craft.
You know, the WebGL stuff, the CSS 3D stuff, and so on.
But actually, it is, actually, a very interesting
and fun sort of experience from the story side,
to actually go through these parts of the story.
And there are bits where you scrub through the video
back and forth and read about the characters
and the locations, and so on.
I just want to sidestep into a bit of a discussion of WebGL
technology that I have a lot of love for.
But debugging WebGL, particularly,
is notoriously horrible and difficult because sometimes
all you get is this black screen.
And no errors, no warnings.
Just the miserable black screen of nothingness.
So I'm curious-- kind of, what was your, sort of,
your tool chain?
What did debugging look like for you,
in terms of finding, fixing, and squashing bugs?
EINAR OBERG: Yeah.
As opposite to you, that makes your own WebGL engines,
sometimes, since we're using the three.js, much of that
is taken care of.
And with a bit of experience, it's quite straightforward
work, I think, to set up the initial scenes, at least,
and avoiding that black screen that I often
get in the very early stages when
starting to implement the 3D stuff.
But, once you have pointed your camera in the right direction,
set up all the lights and all that basic stuff,
it's pretty straightforward.
But then when moving into debugging,
I most often used the WebGL Inspector,
the Chrome extension.
If I want to see, for example, what's
getting inside the textures and, especially,
if there's dynamic textures and post effects and stuff
like that you need to look at.
But, otherwise, the console log is-- the console output
is pretty straightforward to see if there's any shader issues
and stuff like that that.
PAUL LEWIS: Yeah, you mentioned WebGL Inspector.
If anybody's not actually used that, it's a Chrome extension
by one of the Chrome engineers-- sorry not Chrome engineers,
a Maps engineer called Ben Vanik.
And it basically wraps around all the WebGL context
and gives you a lot of insight into what's
actually going on there.
So definitely worth checking out if you're actually
trying to do any work with WebGL.
So I can open A Journey Through Middle-earth on desktop,
but I can also do it on phones and tablets, which is, frankly,
awesome.
And this is one of the first times
I think I've seen an experience of this quality
available across platforms.
So suppose the obvious question, for me, initially,
is, how did you find the process of building
a genuinely multi-device experience?
DANIEL ISAKSSON: Yeah.
EINAR OBERG: It was a journey.
DANIEL ISAKSSON: It has been a journey.
It has been a learning process for us as well as the company
because we haven't been too much into that before.
But, I mean, from the beginning, it
was a big hurdle to decide what to do with platforms that
didn't support WebGL, for example.
How to handle that.
How to communicate with all different kinds
of mobile devices that couldn't support the experience.
But from that point, I think we found
it pretty similar to normal web development,
in terms of actually building the website.
Of course, there are limitations to it
that you have to abide to.
PAUL LEWIS: Well, that was actually
going to be my next question.
So, kind of, there are obvious constraints.
You know, on the one hand, you've
got these desktops with these meaty GPUs.
On the other hand, you've got little handheld smartphones,
which are going to punish you if you
do the slightest thing wrong.
How did you actually deal with those constraints?
You sort of mentioned that you had
to decide what would go in, what would go out.
Can you just talk a bit more about that process
and how you decided what to include where and so on?
EINAR OBERG: Yeah, I guess the biggest hurdle was the memory
consumption.
because you can pretty much hit the roof
and crash the device if you're not really careful with that.
So loading different assets, for one thing.
And not only for memory, for performance as well.
Because if you're using a smaller asset and scale it up,
it's less video memory, as well, when
you push big [? stuff ?] around.
So that helps the GPU as well.
PAUL LEWIS: You've sort of taken a little bit
of a wander into my next question actually.
Something I wanted to talk to you
about is the fact that this thing's running fantastically
well.
Astonishing, actually, just how well
this thing runs on mobile devices,
in particular, where you'd be a bit concerned--
or I would definitely be a bit concerned when you're
talking WebGL and 3D transforms.
Just generally that you want to keep this stuff running fast
because otherwise there's no point in including it.
It's just going to frustrate the user.
So what were your performance considerations
and how did you actually make sure
that you were going to stay at 60 frames a second
or as close as you possibly could to that?
EINAR OBERG: We thought that-- we didn't really
aim at reaching the 60 frames per second on mobile.
We felt that it's much more important
that we keep a stable frame rate at a lower frame
rate than pushing it up to 60.
So if it could be like around 30,
that's enough if you don't have this jank where
the frame skipping is going on.
So I guess the eye is much more forgiving
if it's not glitches in there.
DANIEL ISAKSSON: I mean, that took a lot of testing.
EINAR OBERG: Yeah, of course.
A lot of iterations to make running smooth.
DANIEL ISAKSSON: Lots of devices,
running from quite low-end devices
to the newest, most performance devices.
EINAR OBERG: But then, of course, we
disabled stuff on low-end devices.
Post effects.
We reduced the size of the canvas that is drawn, too.
And yeah, all the tricks that needed.
PAUL LEWIS: So did you--
DANIEL ISAKSSON: Sorry.
PAUL LEWIS: I was going to ask, did you, then,
have to keep a bunch of these devices
around in your studio to just kind of step through and go,
OK, now how is this running on this device?
And now, how is this running on this device?
I mean that seems like a big process to go through.
Were you able to automate?
Or was this just something that you actually
had to step through and do yourselves
every time you made a release?
DANIEL ISAKSSON: Well, for once, we
had a dedicated tester that actually
was responsible for constantly testing out new versions
that we released.
But in development maybe you can answer that?
EINAR OBERG: Yeah, of course, we used multiple devices
during development.
And there is so big difference between them.
And, for example, the LG Metro with super tiny screen
up to the Nexus 10.
And also with different GPUs and screen sizes and, I mean,
the pixel density and stuff.
So, of course, we had many iterations
of testing on different devices.
DANIEL ISAKSSON: I guess one of the biggest hurdles
is the ratio between amount of pixels,
like density, and screen size and the CPU and GPU.
Because some devices had way too many pixels for their own good.
PAUL LEWIS: They look amazing, but they're not
as forgiving, I guess, when it comes to performance.
Yeah, OK.
So what are you, looking back over this project now,
what would you say you're most proud of from the whole thing?
I guess both from a creative point of view,
but also a technical point of view.
You can, you know, go crazy.
Whatever you like.
DANIEL ISAKSSON: Should I talk?
EINAR OBERG: Yeah.
DANIEL ISAKSSON: I think my thing I'm most proud of
is that we managed to make the experience feel very tight.
Like an app that isn't glitchy.
That really went that extra mile to try to--
even though we have to unload each section between sections
and load the next one, so on, to keep memory consumption low.
I think we managed that by putting hard work with doing
translation, transitions to and from the different sections.
Trying to keep this audio as fluent
throughout the experience as possible.
So I'm very--
EINAR OBERG: Yeah, so you don't feel it's a gap
and then you have to load and load and load.
So we had-- it's, of course, many loaders in this site
as well because it's pretty large.
But since we load smaller chunks and then we
show a splash screen and load the rest,
it feels like you can let the user be immersed instead
of just sit around waiting for a long time.
PAUL LEWIS: It does, to me-- I know
North Kingdom has a very rich history in Flash-- it actually
feels a lot more like the latter stages
that we had with a lot of the Flash work,
where studios were getting very experienced with when to unload
and load to try and keep the user engaged.
And it feels like a lot of that stuff
is still very applicable today.
Even though we're delivering in HTML, JavaScript, CSS,
WebGL, all those technologies, it still
feels like a lot of those techniques
are still valid and still useful and helpful.
What do you guys think?
DANIEL ISAKSSON: Yeah, I mean, often the brief
is still the same.
The brief from the client and the goal
you're trying to achieve.
And so that's not changing.
And that's why we always have thought
that, to get really immersed into an experience,
it has to really have that love in every pixel.
PAUL LEWIS: So before you started on the whole project,
did you have any assumptions about how this was going
to play out or what it was going to be
like that you, then, during the project, you just thought,
wow, we were so wrong with that.
You know.
Did you have any assumptions that were basically
blown to pieces throughout the build
and the creative work that you did?
EINAR OBERG: I guess one thing is we didn't really thought
that the performance would be that good on devices, actually.
So, early in the project, we had a lot
of discussions what we actually could do.
And we need to have super simple version for mobile devices.
But then, gradually, we could add more and more and more.
I'm personally pretty amazed what
you can do devices these days.
And, right now, it takes a really long time
to make a site like this.
But I think that in a very short future,
we will take this for granted, that we use mobile in this way
when making sites like this.
DANIEL ISAKSSON: I think one thing, for example,
is the 3D models we used in WebGL experiments.
From the beginning, we really set the aim
on using much lower poly counts, less complex models.
But that is something that we kept raising for some parts.
It has worked.
PAUL LEWIS: That is awesome.
I asked the question having no idea what you'd actually
answer with and I'm so happy that your answer is, things
perform better than we thought they would.
But, on the other side, if you could
change one thing about the web platform
that would have perhaps made this whole thing
a bunch easier, what would it be?
What would you change about building for the web?
DANIEL ISAKSSON: I think we really discussed hard
when heard you were going to put this question.
And we don't really have a good answer.
One thing, that usually is the big blocker and that always
comes up in discussions is integrated video on mobile.
To be able to create interactive video experiences.
Because that's something we're getting most every brief
from our clients.
But there's a reason why that's the case.
And I'm not sure it would be a better world going that way.
PAUL LEWIS: Ok.
Awesome.
EINAR OBERG: I would say also that to have a better
workflow with animations and vector-based graphics,
there's so many techniques that you can use,
and it's scattered all over the place.
Maybe not changing the platform, but the workflow for us
developers to get all those techniques under the same roof.
Use them easier.
PAUL LEWIS: Awesome.
OK.
So something like this is-- an experience like A Journey
Through Middle-earth is naturally very
cutting edge in what it's trying to do
and the techniques that you used.
Were there any resources that you relied on
or that you looked at to help you build it
as you were going through?
Or was it just something that you just
had to iterate through on your own
and just kind of hope for the best?
EINAR OBERG: I think that how the development community looks
like now is we haven't had it easier
than this to find answers and stuff.
Because you use dependencies and components and GitHub
and Twitter.
It's so easy to get access to not only the ones
that wrote the code from the beginning,
but all the communities behind.
Like IRC channel on three.js.
All these people there are helpful,
can just quickly answer if you have anything.
And so I think it's a great place.
PAUL LEWIS: It sounds like people power is basically
the answer.
You know, IRC, Twitter, GitHub.
Go to the people.
There are loads of really experienced and helpful
developers that will actually help you,
kind of, get to your end goal, yeah?
EINAR OBERG: Yeah.
PAUL LEWIS: Well, there are two articles
by you guys on HTML5 Rocks, which I really recommend,
that steps through, in a bit more detail, some
of the technical challenges that you faced.
And if you've not seen A Journey Through Middle-earth yet,
definitely go and check that out.
It's well worth the time to just explore every nook and cranny.
I had one heck of a time trying to get all the correct answers
to Smaug and not get burned to a crisp.
But that's definitely worth your time, as well.
It's plenty fun listening to Benedict Cumberbatch.
There is his dragon thing.
But all the same, in any case, guys,
thank you so much for taking some time out and catch you
next time.
EINAR OBERG: Thanks for having us.
DANIEL ISAKSSON: Thank you.