Tip:
Highlight text to annotate it
X
ALEX DANILO: Hi, everybody.
My name is Alex Danilo.
I'm in developer relations and on Chrome, and I have today
with me Alexis Deveria from Adobe.
Alexis, of course, is the man who built the site
caniuse.com, which is a fantastic resource for anybody
doing HTML5 development.
We're going to show you a whole lot of stuff today.
We're talking about CSS filters, but most of all,
we're talking about a new next generation reading experience.
So this is built with not just CSS filters, but a whole lot
of web technologies, so we're going to cover stuff that uses
WebGL, uses SVG, uses Canvas, uses a whole lot of stuff, all
working together to bring you a seamless experience.
And of course, this is all about trees, OK?
So when we talk about DOM trees in the
browser, we get excited.
Or at least I do.
I don't know about you guys.
But of course, today, we're talking about the world's
biggest tree in mass, and it's called The President.
And National Geographic has this beautiful, beautiful site
that they've built with Adobe's help, and we're going
to talk all about that today.
So what are CSS filters, I'm sure you're asking?
Well, you build these beautiful DOM tree things
inside your browser, and then the browser comes along and it
goes, well, we need to see that thing.
So the browser lays it out, places all the elements on the
page, and then it gets styled, and CSS comes
along and styles it.
And then we have this post-processing step, which is
a CSS filter.
So if you can imagine baking a cake, you get all your
ingredients, you put them together, you bake it.
That's your web page.
You want to make it look good, so you put a bit of
icing on the top.
And then you sprinkle sugar, and CSS filters are the sugar,
which makes it look even better.
And so once you've done the filter, you then push it to
the screen.
So once you've generated a bitmap, the bitmap can be
filtered inside the browser, and then that eventually ends
up on the page.
And that's what you look at.
So where did this all begin?
This all began with SVG, of course.
Now SVG's a graphics language, which you all know from HTML5.
And within SVG, there are basic filter primitives.
But these filter primitives are like building blocks, so
you kind of combine them all together to end up with the
effect that you want.
So if you have a look at this graphic at the top, we have a
standard graphic on the left.
And then we generate a blur from that.
So you take the blur, and then the second filter offsets that
blur a little bit.
The third filter puts lighting on the blur, which then gives
you this kind of 3D effect.
The next filter along the chain clips the blurred lit
piece to the original graphic so that it only hits the
original graphic.
The next stage of that pipeline combines the original
graphic with the filtered piece we just created, which
gives you this nice little 3D with the second last image.
And then following that, we add the drop shadow from the
second stage again, and we get this nice
drop shadow 3D effect.
And this is a whole lot of individual steps which you
have to write in SVG.
So this bottom right image, of course, is actual text, which
is filtered using this style of technique.
So the beauty of doing something like this is you get
these incredible graphics, but they're accessible, and
they're real text.
So you can select them, you can do all sorts of stuff.
So the thing about trying to do something like this in SVG
is you need quite a lot of markup.
So along came CSS filters, and CSS filters are basically a
beautiful little shorthand that saves you a lot of pain
and a lot of markup.
And this is how easy it is to use.
So here we just have a class selector.
We're saying filter, blur, 4 pixels.
And so this takes the image on the left and
blurs it by 4 pixels.
So that's all the markup you have to do, whereas in SVG,
that would have been a whole ton of markup.
Now, what's nice about these primitives is
you can combine them.
You can actually put multiple filters on your content.
So, for example, you just have to do something
as simple as this.
You just line them up one after the other.
So here we have a hue-rotate followed by sepia.
So we've taken a picture from IO last year, and we've just
applied two filters in order.
Now, the thing to keep in mind is that you can apply as many
filters you like, and they'll be applied from left to right,
one after the other, which is great, because
order actually matters.
So in this case, we've shifted the hue and
we've sepia toned it.
But if we were to flip these around the other way, we'd
have something like that.
So as you can see, the color is actually different.
They're exactly the same filters, just
in a different order.
See?
Sepia and blueish tinge.
So the beauty of these CSS shorthands are that, because
people implementing browsers spend all day hacking code,
because that's what we love to do, because there's a finite
number of these shorthands, we can actually optimize those
inside the browser.
So right now in Chrome, most of these are hardware
accelerated.
So if you do a blur or something, you don't get the
penalty you would've got 12 months ago.
So there's only like 12 shorthands, and here is, of
course, the definitive prefix version with the normal markup
of every single filter being applied to something.
Right.
So, enough of my yakking.
Let's just show you what we've built first.
We're going to start with the desktop experience, because
that, of course, is something a bit special.
And then we're going to show you this all running on
mobile, which hasn't been shown before, by the way.
So anyway, I'll pass it over to Alexis, who will run you
through the desktop experience.
ALEXIS DEVERIA: Thank you, Alex.
So this is the project that Adobe developed using content
from National Geographic.
It's a really beautiful article about the great
sequoia trees.
As I scroll down here, I'm going to go through the
different technologies that we developed to create this next
generation reading experience.
So as I start here, the first thing you see here
is this drop cap.
So what we use here to make this text follow the contour
of the letter O is something called CSS shapes or CSS
exclusions.
And let me show you a little bit about how that works by
inspecting that.
So we set here the webkit-shape-outside property
with an ellipse, and that's what creates the shape.
So this is a floated element, and the inline text just kind
of flows nicely around it, which creates a great
typographical look.
As I scroll down, you see this image comes into view.
And whenever these images come into view, and also this one I
hover over, you can see how we change both the saturation and
also the sepia.
So let me also take a look at that.
So just as Alex mentioned, this is
one of the CSS filters.
So by default, this has a sepia and a grayscale of 0.5.
When we hover over that, we change that to none.
So this is a video that's about the photoshoot that was
done for this.
As you can see, it went to fullscreen HTML5 video.
As I scroll down here, we see this pull quote, and the pull
quote uses something called text balancing.
Now, text balancing is still a proposal, so this was
implemented using a JavaScript polyfill.
And what it does is it ensures that each line is roughly the
same width.
So instead of specifying where you want your breaks, using
text balancing, it will automatically be formatted.
So if I resize the window enough, you'll see that now
the text becomes three lines.
Let me drag that here.
So instead of where the browser would normally move
things over word by word using text balancing, yours get this
great formatted text regardless of the width.
Again, here, this image comes into view.
We change the opacity filter.
I can click on it, put it in full screen.
And I'm going to talk a little bit about CSS regions.
So this is a technology that we use here on the body text.
And what CSS regions do is they let you define your
content in one area in the DOM, but define the elements
that the content flows through elsewhere.
So the author has complete control over where the text
will display without having to modify the markup itself.
So you can see that in action here, where we've got these
two columns.
And if I select this text, you can see it pretty good.
I resize the window.
As you can see, the text flows from one to the next.
So throughout this article, we define all the different
regions and different places to position the text the way
we want it to.
Again, let's scroll this image into view.
As you see, it went from black and white to color.
Here we use another CSS shape.
As you can see, the text here roughly follows
the background image.
That just creates a very nice look in combination with the
photograph.
So I can look at this one real quick.
This was done using
webkit-shape-inside and a polygon.
So that defined the CSS shape that the text flows into.
And now to scroll down here, we get to--
so here you see a thumbnail of the tree that Alex mentioned,
The President.
And what we wanted to do is create a visualization where
you could actually see or kind of get an idea of just how
immense this tree is.
And to talk a bit about the technology behind that, I'm
going to hand things back over to Alex.
ALEX DANILO: Thanks, Alexis.
OK.
Right.
I just want to talk a little bit about triangles now.
Down at the bottom here, you see a Delaunay triangulation.
So normal filters are things that operate on pixels.
So the basic filters we showed at the start-- things like
saturation and the color, the monochrome--
those are pixel operations.
When you get into the world of WebGL and shaders, you have
two types of shaders.
You have per pixel things, which are also called fragment
shaders, but you also have vertex shaders.
And what they do is basically take an object or polygon,
chop it up into triangles, and then you can write the shader
program which manipulates these triangle points.
The beauty of that is it all runs on the GPU, and so in
this case, we're going to introduce WebGL and a proposal
called CSS custom filters.
So a lot of people are already familiar with WebGL, and we're
going to show you WebGL in a moment with one of these
experiences.
But also, CSS custom filters are slightly different.
The difference with CSS custom filters are basically that CSS
custom filters operate on any arbitrary web page content.
So you can take a normal web page and effectively flatten
it to a plane, and then manipulate it
using a shader program.
Now, why would you want to do this rather than use WebGL is
a question people have asked me.
Because it's really easy.
Because WebGL actually takes quite a lot of JavaScript to
set up, and also WebGL has no access to the DOM or any of
the content on the page.
The thing about using something like a CSS custom
filter is that you can take advantage of the really good
typography that's inside the browser.
So most browsers these days rasterize with subpixel
anti-aliasing, and that doesn't happen in WebGL or
Canvas, for that matter.
So by using a custom filter, you can actually take
advantage of that.
So if you can imagine this page here with text and a bit
of SVG graphic, what you would do is you would basically draw
a triangle mesh over the top of it, and then you would
write a write a shader program which would manipulate your
web content.
And if you took this and did that, you'd end up with
something that looks a little bit like this.
So here is a CSS shader.
This is actually rendered by the browser and being
manipulated, and so it's the content that's moving around.
So you can do things like page flips or deformations or
things like that, which is kind of a really nice effect.
So in any case, why don't we actually have a look at this?
I would like you to pay close attention.
So here we have The President, and we wanted to have the
experience where we actually brought forward the sheer size
of this tree.
And when they built this thing, it was like, how do you
actually represent an image?
Like, the thumbnail here, you don't actually get a feel for
how big it is.
So pay close attention.
Look at the start.
I want you to observe something, and watch
the tree fly in.
So what's happening here is there is a gigantic image of
the tree that's been sliced up into tiles, and
they're all flying in.
This is all WebGL.
And as it comes in, you see at the bottom that there's a
person right there.
And there it is.
So it kind of gives you a feel of how big
this actual tree is.
[APPLAUSE]
ALEX DANILO: And as you can see, of course, there are
people up here in this tree somewhere.
And it pans around beautifully.
Look at that.
It's basically nice and smooth, and this is all WebGL
accelerated.
Now, the astute people in the audience may have noticed,
right at the start of that tree flying in, there was a
little graphic that appeared for a fraction of a second.
And what was happening there is that we were trying to get
the texture from CPU memory over into GPU memory.
So the way this has been built is by constructing a giant
texture, slicing it up into rectangles, and then
referencing them from the WebGL context.
But of course, this is one huge texture, and that takes
time to get across into GPU RAM.
So that little spinning thing that you saw just before the
tree appeared was actually referencing the WebGL texture
in a 1 by 1 pixel kind of reference.
So that 1 by 1 pixel reference was enough to pull the whole
texture across so that when the tree flew in, it was
actually nice and fluid.
It's kind of a nice, neat little trick.
Now the other thing is also here that this is very similar
to DOM spriting.
So people who have done DOM spriting, you typically load
one big image, because you don't want to have all these
HTTP fetchers.
So it's a very similar technique, except this is done
with WebGL and textures.
OK, so on that note, I will let Alexis take over and show
you a little bit more about this experience.
Now, this was WebGL.
A little bit later on in the experience, he'll show you CSS
custom filters in action on some of the content.
So over to you.
ALEXIS DEVERIA: OK, thanks, Alex.
So I scroll down through the article, and once again, we
see another image.
We changed the opacity filter once we scrolled into view.
Here, once again, we see another CSS shape.
As you can see, this time the text follows the contour of
the base of these trees here.
Again, looks great as part of the image.
Very hard to do without this technology.
Scrolling down to the end of the article,
I get to this map.
And there's a couple of things I want to show you here.
First of all, we see yet another CSS shape.
This time, we follow the shape of the state of California.
And as you can imagine, if you try to do this today, you
would have to either do this using an image, or by
hand-coding every single line here.
And you can imagine the problems with that.
I mean, anytime you need to make a change, you'd have to
actually go in and change the text and
redo the entire graphic.
You wouldn't get the indexing or searching that you get by
all this just being text, so that just makes it a very
useful technology to have for any kind of typographical
effect like this.
We also have here an infographic.
Click here.
This is a SVG graphic that shows you the elevation range
of the sequoias.
We use more exclusions here for this text.
And now, to show you our CSS filter in action here, once I
click on this button, you can see the page peels away to
reveal this other content.
And let me actually go back and show that again.
So you can see this is all just HTML and CSS content, and
we use a custom shader or a custom filter or a CSS shader
to do that.
So this brings another infographic.
Here we can see the tree.
I can click on these different parts of it to reveal the
different sections.
We did this using 2D Canvas.
And so that about wraps it up for the desktop browser
experience.
And I'd like to mention that actually,
anybody can see this.
We have it on GitHub.
We'll share the link of this with you later.
But when you do--
let's go back to the top here.
At the bottom here, there's a bar that says
show editor's marks.
If you click on that, that shows this annotated view that
actually highlights the different technologies that
I've just been through, so you can see for yourself how we
designed this page.
Great.
Back to you, Alex.
ALEX DANILO: Thanks, Alexis.
All righty.
Now of course, we told you we're going to talk about
mobile, because this stuff was shown last week at [? Max, ?]
which is great.
But today, we're actually going to show you this running
on a mobile device, and it blows your mind to see how far
this has come, and how fast it is, and how beautiful the
experience is.
But of course, when you're building stuff for mobile,
performance really matters, right?
So nothing comes for free.
So the most important thing is to test on many, many devices,
especially your target device.
Really think about how much memory you're using, both for
RAM and texture memory, if you're using
WebGL or custom filters.
There are a couple of sessions coming up.
I highly recommend you see them.
There's a session directly after this one in this room,
which is web page design with the GPU in mind.
I highly recommend you listen to that one and a few others.
So while these guys were building the mobile
experience, it came to how would you actually deploy this
on mobile today?
And of course you can just take Chrome on your mobile
device and point it at that website that we've just shown
you, and it works great.
The thing is that for a next generation reading experience,
people are used to the whole app model.
Maybe we download an app, it gives me my little icon that I
can tap on, and it's a self-contained thing.
I don't have to type a URL or anything like that.
So it gives you a few options.
So there are options, whether you want to go completely
native, or whether you want to have just a website, or
whether you go to some kind of packaged web
container like PhoneGap.
And so this experience we're about to show you was built
using PhoneGap.
It's an experimental build that enables a whole lot of
features that currently aren't in browsers, but it does show
you the capability, and it's pretty cool.
So why don't we take a little bit of a look at the National
Geographic experience on the Nexus 10?
ALEXIS DEVERIA: OK, thanks, Alex.
So, just for those of you unfamiliar with PhoneGap,
PhoneGap is a framework that lets you create applications
using web technologies on different platforms.
It also gives you access to the APIs that you would not
normally have access to in the browser.
So we used that to create this.
And so I'm going to launch this app.
As you can see when it starts, we start with
the same WebGL animation.
As you can see, it's very smooth, very slick, looks
beautiful on the device.
And once that completes, that brings us to this table of
contents view.
So here, I can select our article.
And so for the application experience, we wanted to do
something other than just have some vertically scrolling page
as users expect on the web.
So in this case, we've got a paginated reading experience
where I can scroll
horizontally through the content.
So here we see the same content that we saw in the
browser, so we have the same exclusion here around the
letter O. We also have some interactive elements.
Here, I can use multi-touch to take this
picture and enlarge it.
And as you can see here, it makes a lot of sense to also
use regions to define each container for the
text to flow through.
So that gave us a lot of freedom exactly to define how
we wanted the content to appear without having to
modify the text that was actually in the paragraphs
that make up the content.
So see here, again, we have text balancing regions, and
this brings us to our first use of a CSS shader on mobile.
So what you can see here, once I put two fingers on this
image, we flip to show the other part of the image.
I can move my fingers around, I can open and close, and if I
open them up, that shows this image in full screen.
As you can see, when I scroll, you can also see how the
lighting changes.
So that's all done using the GPU.
It's a beautifully smooth, high frame rate experience.
You move through the content, and we see another beautiful
image-- again, the CSS shapes that we were seeing before on
the web experience.
And once again, here's another image.
If I tap on that, it flips out to show the full screen.
And this one, too.
I can use the multi-touch to open and close.
And so that creates a great experience for the users that
you'd normally only see on native applications, and we're
very happy to be able to bring this to the web.
Here we see another map that we also saw on the desktop.
This one has the same page curl that you saw, except this
one does something a little bit different where if I move
my finger, you can actually see how the page curls
depending on the position of my finger.
As you can see, it runs great.
I let go, and this reveals a Google Map.
Because since we're using web technologies, it means that we
can use APIs like the Google Map API.
And now I can actually zoom in and see the location of the
national park where the sequoias grow.
And after that, I get to our last page.
We see the same infographic before.
I can use multi-touch to make this one bigger.
And we see that in fullscreen.
So that about summarizes the mobile experience.
There's one more thing I'd like to show here, which you
might find interesting.
And just to prove that these are indeed web technologies,
what we can do here is we're going to plug the tablet into
this computer and run an ADB command, which lets us
communicate with the Android device.
So if I switch back to desktop--
ALEX DANILO: OK, so we're just going here to another tab,
which we're going to point at a local host, with--
and hopefully, we'll see the page.
So what we actually see here is a reference to the content
that's actually running on the tablet.
So this lets us pull up all the Chrome Developer Tools and
actually debug the experience.
So this is the huge advantage of taking this approach for
mobile applications.
Because it's built on web technology, you have all the
tools at your fingertips.
You can debug the entire experience, and you've seen
the performance, so you get a really native style
performance built with web technology, and you can debug
it properly.
Like, no more [INAUDIBLE] adb logcat.
You can actually use proper tools.
So why don't I just flip back and forth?
Alexis will highlight a few elements in this, and we'll
just prove to you this is a real thing.
ALEXIS DEVERIA: So this is me turning on and off the
shape-inside property here, for example.
ALEX DANILO: So here you go.
He'll play with the shape-inside property, and
we'll watch it change live on the device.
Go back?
ALEXIS DEVERIA: So that was that right there.
ALEX DANILO: So this is a huge advantage of using web
technologies for this kind of application.
It's a really, really nice way to do it.
Now, let's just--
you have to wait for that one.
OK, so what's next?
The next level of CSS filters?
I've shown you the basic filters that are already
there, the custom filters which are in development at
the moment.
Another thing that is already in the browsers today but is
improving rapidly are SVG filters.
And the reason they're really improving rapidly is because
they've been thrown at the GPU.
So right now, in the channels of Chrome, there's hardware
accelerated SVG filters coming.
They're actually implemented now.
They're just not finished completely yet.
However, what this means is you can take all those
building blocks I showed you at the beginning of the
presentation, and you can chain them together any way
you like, and it all runs on the GPU, which is just
sensational.
So if you can imagine this picture at the bottom-- this
beautiful, clear picture-- this is in actual fact a movie
with a displacement filter on it.
And how do you do that?
You do it with something like this.
So the markup is very simple.
So here's an element selector for video, and it's applying
the filter with a URL reference.
And you can see it references an ID.
So that's basically how you apply it to your content.
And then in the SVG, which I expect people will share
filters between each other, there are two
filters shown here.
One is a convolution filter--
so you can basically use an arbitrary convolution matrix
and apply it to your content.
The second filter applies a blur followed by a
convolution.
And this is all runs in the browser, hardware accelerated,
and I will prove it to you by showing you this.
Here is a nice little movie with no filter on it.
But of course, if we were to put a blur on it, it would
look something like this.
So there you have it.
Full speed video with blur, hardware
accelerated, in the GPU.
But of course, SVG has a whole lot of primitives, so here is
a point light filter.
And what's nice about this is there's a little event
listener on mouse move here, which actually moves the
position of the light.
So this is all happening in your favorite browser, or my
favorite browser, anyway.
So there's also a spotlight.
There are a few different lighting primitives in SVG
which are really quite nice to do all
sorts of amazing effects.
We have a morphology filter, which--
I don't know if it's applicable to video, but
anyway, it works.
You can flip the colors around, you can play with the
color channels, and do all sorts of crazy things.
Now, here's the convolution filter that I showed in the
previous slide.
So this is actually a convolution matrix that is
doing edge detection.
And so it gives you this kind of lineart thing.
And if we actually use the second filter on the previous
slide, there's the blur followed by the convolution.
It gives you this kind of result-- so a
blurry lineart thing.
All right, so we can do offset, so you can basically
offset your graphics.
So you don't even need to use relative positioning.
You could actually do it with a filter, if you
really felt like it.
Then, of course, we can play with a retro feel.
We can go to black and white.
Actually, that's kind of cool, isn't it?
But you can do anything with the color channels.
That's the beauty of it.
You can do all sorts of stuff.
So we could do, like, a noir effect, which is kind of a
funky old style thing, or do a bit of blue.
So you can imagine that you're in the ocean.
And of course, we'll finish up with the displacement filter.
So the displacement filter is a really useful one, and I
can't wait for you guys to get out there and start playing
with this one and build all sorts of crazy sites.
So if you can imagine, when you're on your desktop, and
you've got Windows up, you iconize it, and you see the
rectangular thing go woosh, like a
genie back into a bottle.
Well, the displacement filter lets you do things like that.
So in effect, it's a warp, but what it does is it takes an
input image and uses one of the R or G or B color
channels, and uses the value in the pixel as an offset
that's applied to the content.
And so this is an example.
Actually, we should probably go further
forward in the video.
Why don't I go somewhere-- there.
That's better.
So you can actually see the content is really crazily
warped around with this input image with the
displacement filter.
OK, so what I really want everybody in this room to do
is go to their favorite site and put CSS filters on it,
because you can do some incredible stuff, and we're
hardware accelerating the whole lot of it now.
And so it means that your site just gives you another piece
of creative sugar to put on top of your sites and make
them beautiful.
Now, this National Geographic experience, which we showed
you today-- the whole thing is open source.
So if you go to the URL at the top, you can actually get a
hold of all the content, so you can see how it was built,
see everything, use the content.
The second link there is a link to the actual experience
itself, so you can view it.
I'd encourage everyone to start playing with the stuff,
looking at it, seeing what they can do.
Your browser's there.
You can debug it on mobile if you really want to.
And with that, I think we're pretty much done.
What do you think, Alexis?
ALEXIS DEVERIA: That's right.
[APPLAUSE]
ALEX DANILO: Thank you.
OK, we've got plenty of time for questions.
So, do we have any--
ALEXIS DEVERIA: One other point.
ALEX DANILO: Oh, yes.
One other thing.
We'll be up at the Chrome Sandbox right after this talk,
so if anybody has further questions they don't want to
ask now, come and visit us.
And also, on level three at the Adobe Sandbox, they're
actually showing this on the Nexus 10.
So you can get your own fingers on it, play with the
mobile experience, and see for yourselves.
So anyway, questions.
AUDIENCE: I got here late, so I'm not sure if you already
addressed it, but I was wondering--
I realized it's mostly taking place on the GPU, but how is
battery consumption with all that animation?
ALEX DANILO: Well, in fact, battery consumption is reduced
when you throw it on the GPU.
So typically, the GPU uses a lot less battery power to any
of these kind of effects.
So it's pretty cheap, actually.
AUDIENCE: But do you know what the percent reduction is?
Because I know that playing video on a phone would be a
lot of battery consumption.
ALEX DANILO: Playing it on the phone, I think you'll probably
find a lot of battery consumption is actually the
radio being on and pulling the content across, more than the
GPU itself.
AUDIENCE: OK.
Thank you.
ALEX DANILO: Yes?
AUDIENCE: Is there any way to determine how much texture
memory the device that you're on has for the WebGL stuff?
Because I saw the image for the tree was pretty large.
Do you ever hit a maximum where it just stops working,
or it crashes?
ALEX DANILO: That's why I said test, test, test.
We are actually looking at ways to measure GPU memory.
This has been asked quite a number of times from a lot of
mobile developers, and we don't have a good answer for
that just yet.
But come to the next session, and I'm sure they'll tell you
all sorts of cool tricks.
AUDIENCE: Cool, thanks.
AUDIENCE: I have been playing with CSS filters, and I've
been surprised a few times when some things are
accelerated, and other things aren't.
Is there a rule of thumb that I can follow that will help me
decide whether or not this is going to be GPU accelerated?
ALEX DANILO: I don't know that there's any
clear rule of thumb.
We accelerate all of them in Chrome now, and I know that
Mozilla have just rearchitected the whole back
end of their rendering pipeline, and they'll be
accelerating all those as well.
In general, if you're scared, just don't use blur, because
blur is the only one that really hurts
you in software rendering.
The rest of the filters are pretty good.
Or if you really want to use blur, because it's useful,
make sure you keep the blur small, because it's square.
Like, it's an order n-squared.
So if you do a 2 pixel blur or a 4 pixel blur, the 4 pixel
blur is 4 times the 2 pixel blur in
terms of cost of rendering.
AUDIENCE: Got it, thanks.
ALEX DANILO: John?
AUDIENCE: So what are the requirements on mobile, in
terms of WebKit?
And can you talk a bit about what you've actually
contributed to PhoneGap to make it all work?
ALEX DANILO: In which respect?
AUDIENCE: Actually, I"ll ask a simpler question.
What version of Android is the minimum to run
something like this on?
ALEX DANILO: Well, PhoneGap person--
Alexis, can you tell us, what's the
minimum Android for PhoneGap?
ALEXIS DEVERIA: I think the basic PhoneGap should just run
on, I believe, most Android versions, including 2.x.
But what we did here today that we actually used this
custom version of PhoneGap to make this possible, so we
could have the hardware acceleration working as
swiftly as possible.
AUDIENCE: So does that need a custom version of WebKit?
Or are you using [INAUDIBLE]?
ALEXIS DEVERIA: It's both a custom version of PhoneGap and
a custom version of WebKit, yes.
AUDIENCE: Can we get that version?
ALEXIS DEVERIA: Not yet.
ALEX DANILO: The question was, can you get that version?
ALEXIS DEVERIA: So it's just a very prototyped stage, so we
hope eventually the code that Adobe's been working on-- we
hope that that might get into Blink or WebKit eventually,
but don't quite know for sure, no.
AUDIENCE: Was there any timeline [INAUDIBLE]?
ALEXIS DEVERIA: No.
No time known yet.
ALEX DANILO: Sir?
AUDIENCE: So you mentioned subpixel rendering and then
applying filters, which seems weird to me.
Because once you've subpixeled it, if you filter it, it no
longer hits the subpixels.
ALEX DANILO: Well, what I was saying was that you could do
the subpixel rendering, but then you can do a vertex
shader, which will manipulate the content.
So there's something like the page flip, for example--
the filter is only moving the content around.
AUDIENCE: Oh, I see.
So not when it's static.
ALEX DANILO: Yeah.
So you still get the nice, clean typography, so the
filter's not actually messing with how it looks.
AUDIENCE: But you presumably get sparkle when
you do that, then?
Because the subpixel?
ALEX DANILO: Oh, well, the subpixel rendering is there,
right, so that as the page is flipping, you don't suddenly
see a flip from nice typography to blocky gray
anti-alias as the page flips, right?
So it stays nice.
Do you understand?
AUDIENCE: OK.
Yep.
Makes sense.
Cool.
ALEX DANILO: All right.
AUDIENCE: Hey.
I'm just wondering if you can say anything about the
designer experience of doing a site like the National
Geographic one.
Did you have to design custom tools, or was it done in a
text editor for some of these special effects, or how did
you set that up?
ALEXIS DEVERIA: So are you just talking
about the custom filters?
AUDIENCE: Well, perhaps you guys designed the pages
yourselves, but I'm wondering if maybe a regular graphic
designer came in as well, and [INAUDIBLE]
visual layout tool.
ALEXIS DEVERIA: No, it was all designed by Adobe.
National Geographic gave us content, and gave us a little
bit of idea of what kind of layout they had in mind.
But the whole design process and implementation was done by
us Adobe guys.
And so the CSS filter that we used--
the one on the web is actually--
if you go to CSS Filter Lab, if you just Google CSS Filter
Lab, you can see a whole bunch of filters that we actually
developed at Adobe, and so the page curl one is one of those,
and that's what we included there.
So does that answer your question?
AUDIENCE: I guess so.
I guess I'm just trying to think about it in terms of, to
address somebody who's not a programmer, but who might want
to say, oh, I really would like to have this page pinch
open right here, but I don't know how to do that.
ALEXIS DEVERIA: Well, because this is still in very early
stages, there's not a lot of tooling for it yet.
The spec is still being developed.
But I can promise you that Adobe specifically will make
sure that this will become easier for developers.
ALEX DANILO: I think you need to follow the Adobe Web
Platform blog, because they're developing a ton of tools.
They've got a whole suite of tools, and more and more are
being developed.
So we're showing you leading edge stuff that's still active
in browser development and standards development.
And so once they're locked down, they'll be producing
tools for this stuff for sure.
AUDIENCE: Oh, cool.
Thank you very much.
ALEX DANILO: Yes, sir.
AUDIENCE: So my question pertains to availability and
compatibility.
So I just gave the website that you were demoing a try on
Chrome 28 on my Pixel, and it said I needed to turn on
Experimental WebKit features.
And I did, and it runs great, and that's awesome.
But do you happen to know the timeline on when that will be
in the general Chrome release?
ALEX DANILO: Very good question.
I think that this is one of those open ended questions,
because some of the features, like the simple filters and
the SVG filters, all work currently.
They're all standardized.
The custom filters and some of that stuff, and the regions
and exclusions are currently going through the
standardization process in W3C.
And so that's a kind of open ended thing.
I can't give you an answer of exactly when.
Hopefully soon.
But for now, this is basically an example of where the
technology is being developed in parallel to
standardization, so that when we go to the W3C and other
browser vendors, you can go, look, here's an
implementation.
It works, it's cool.
And then everybody can argue out the fine detail of how
that will finish up.
AUDIENCE: All right.
Thank you very much.
ALEX DANILO: Yep.
Welcome.
John, another question.
AUDIENCE: My second question.
I'm actually excited about this.
One of the issues I've had with mobile web is having to
deal with different versions of WebKit
on different platforms.
So are your contributions to Cordova, does that mean I can
pick a version of WebKit and deploy that?
ALEXIS DEVERIA: No, for Cordova, you're stuck with the
WebKit that is on the device.
I believe there's talk of possibly changing that in the
future, but that's the way it is now.
AUDIENCE: So my first question, then-- doesn't that
mean there'll be minimum requirements for the mobile OS
for me to use these features?
ALEXIS DEVERIA: Yes, that's correct.
AUDIENCE: OK, thanks.
I'm less excited.
ALEX DANILO: Get excited, John.
It's good.
OK, yes sir.
AUDIENCE: On mobile, I understand
that test, test, test.
But I wonder if amongst the different mobile graphics
accelerators, you've noticed that certain of these
operations perhaps tend to work well, and others maybe
not so much?
ALEX DANILO: Yes, very good point.
One of the things about the mobile GPU to keep in mind--
I mean, the next session will dive deep into this stuff.
But the technique of rendering on a mobile GPU is completely
different to a desktop.
So on a desktop GPU, you basically have a ton of
parallel execution units, and so your rendering goes blot in
a huge wide thing, and happens instantaneously.
And it's brute force.
Whereas on a mobile device, because you want to keep the
silicon small, they use this kind of deferred rendering
technique where they buffer up all the things that are going
to be drawn, and kind of build display lists and a stack of
stuff, and try to then eliminate the things they
don't need to draw, and then draw it.
So because of that, you actually end up using a whole
lot of memory in the process.
But one little tip that's really useful when doing this
stuff for mobile is to try and keep the pieces of your
content small.
So, for example, if you're doing a long list, like an
infinite list style of thing, and you have a long, thin
strip like this, if you go put a translate-z hack or
something like that to force it into the GPU layer, the
problem with doing that is that
textures have to be square.
So if you've got a long, thin strip like this, maybe it
consumes a megabyte, but your square texture
could be 100 megabytes.
And so you've blown out all your GPU memory.
And I've seen really janky sites where people have done
this, and then simply by changing the CSS selector to
put that transform on the little pieces of content
within the strip, you would think that would cost more
time, because of all the extra CSS [INAUDIBLE]
recalculation.
But in actual fact, what happens is the tiles end up
these little tiny tiles that then go over to the GPU, and
the thing flies.
So as a general rule of thumb, try and keep content that's
accelerated small.
And especially with the WebGL tree, you saw it was all
chopped into little squares or little rectangles.
That was a similar kind of idea.
AUDIENCE: So following up on that, with the WebGL 3, you
mentioned how difficult it is to get the texture to the GPU,
and that you put a little spinner in there to make it a
better user experience.
Would you have this problem with GPU accelerated CSS
filters, or even custom filters?
And what can you do about it?
Is there a lag due to the texture going through the GPU
when you use those pre-built filters?
ALEX DANILO: You would have that problem with the custom
filters, because it's the same issue.
You have to get the texture of the rendered content.
So effectively, your browser is laying out, rendering, and
then it's creating a bitmap, which it then passes across to
the GPU for it to use a texture.
So you will have that problem for those filters.
For the SVG filters, not so much, because--
well, I suppose you would.
I guess it depends on your content.
But if you're playing video, for example, where I showed
the SVG accelerated filters on the video, the video is
already on the GPU.
So that bitmap of the video is never seen on the CPU side.
It's decoded in hardware.
That's sitting in the GPU memory, and then when the SVG
filter is applied, it's all happening as a
texture in GPU memory.
So there's never a read back to CPU.
So you don't get the same
flip-flop of textures happening.
AUDIENCE: And so if you have the issue with CSS custom
filters, is there anything you can do for displaying a
spinner, for instance?
ALEX DANILO: Probably lots of things.
Well, similar to displaying the spinner.
But I think that you probably find that would just be an
initial load, because if the custom filter is sitting
there, the browser already knows it needs to put it in
the texture.
So the moment you see it, it's ready to go
to do its flip effect.
It's just how quickly that loads.
I think in the actual National Geographic one, because the
custom shader was down the bottom, the custom filter was
right at the bottom of the page, that was just loading in
the background while you're viewing the top.
So your standard techniques--
show something to the user as the fancy content is loading
in the background.
AUDIENCE: OK, thanks.
ALEX DANILO: OK.
Yes?
AUDIENCE: Are you tracking fine grain abilities of what
works on Can I Use for these?
Which filters are on which platforms?
ALEXIS DEVERIA: Hopefully filters is a feature
that's on Can I Use.
I think it's just basic filters right now, but as that
gets more complicated, then I'll definitely add more
subfeatures or whatever.
ALEX DANILO: Yeah?
AUDIENCE: So I don't know whether this is just Moscone
Wi-Fi being Moscone Wi-Fi, but the website that you guys were
using-- the GitHub site to demo the
National Geographic thing--
stopped working.
I refreshed the page, and it says it can't connect to that.
Do you know if that's a connection problem, or is that
being moved somewhere?
ALEX DANILO: I think it's a connection problem.
So this is actually the page.
AUDIENCE: OK.
ALEX DANILO: I'll try and reload it.
Oh, then again, maybe it's Moscone Wi-Fi.
AUDIENCE: OK, thank you.
ALEX DANILO: This is actually a wired connection, so yes.
AUDIENCE: Thank you.
ALEX DANILO: Looks like the network is down, folks.
Sorry about that.
OK, in that case, no more questions?
And anyone quick, one last question?
No one brave?
All right, thanks, everyone, for coming to this session.
[APPLAUSE]