Tip:
Highlight text to annotate it
X
[MUSIC PLAYING]
MALE SPEAKER 1: Shanghai GDG is a very interesting
developer community.
FEMALE SPEAKER 1: I'm glad somebody
has asked this question.
MALE SPEAKER 2: This is where the magic happens.
FEMALE SPEAKER 2: It's primarily a question and
answer show.
So if any of you out there would like to ask questions.
PAUL SAXMAN: Hello, everybody.
Welcome to this week's Google Maps Developers Live, formerly
Google Maps Office Hours.
With me this week, we have a couple of members from the
google.org Crisis Response Team--
Pete Giencke and Ka-Ping Yee.
Do you guys want to introduce yourselves and say what you
work on here at Google, or with google.org?
KA-PING YEE: Sure.
My name is Ping.
I'm a software engineer at Google Crisis Response.
And I'm the tech lead of the Person Finder Project and the
Crisis Map Project, which you're going to see a little
bit about today.
PETE GIENCKE: And I'm Pete Giencke, as Paul mentioned, a
GIS data engineer, and I support the Crisis Map Project
and several other geo efforts to get data
into Map and Earth.
PAUL SAXMAN: Awesome.
So, with that, why don't we just dive into the demo.
So you guys got some pretty interesting stuff to show us
today, things like Crisis Maps, the People Finder,
Person Finder, and let's see it.
PETE GIENCKE: Sure.
We'll see how much time we have, but we'll do as many
demos as we can.
OK, So as I mentioned, I'm Pete and this is Ping.
So we want to spend about 15 minutes today just providing a
high-level overview of Crisis Map and offering some
technical detail along the way.
So before getting started, it's worth mentioning that a
majority of what's been developed and what we'll be
demoing today is based on the same tools on APIs that are
available to external developers.
So a big goal of this project is to open source as much of
the code as we can.
And the number of--
maximizing the amount of external tools we're using and
minimizing the internal tools just means that it's going to
be a lot easier when it's time to push that code out.
So why don't I talk real quickly about the Crisis
Response Team.
So the Crisis Response Team was formed in January of 2011
after the Haiti earthquake, just as a way to better
organize the Google and community resources in
response to an event, really making critical information
more accessible in times of disaster.
Crisis Map, that we're talking about today, is the
application that our team uses to mash up crisis-related
data, such as evacuation routes, flood zones, and
disaster-related imagery.
The goal for us is to take crisis-related geo data that
traditionally has required special software and expertise
to be able to view.
It takes data that also lives scattered across the web in
many different formats and just makes them available to
all users without any kind of special software, so really
just making critical disaster geo data more available.
So when we combine these layers on a map, different
organizations that don't normally communicate with one
another are able to see their map data in context with the
different layers.
So we're going to go over to a quick demo here that Ping is
going to display.
Since we launched for Hurricane Irene in 2011, we've
responded to many different disasters, most recently
Hurricane Isaac.
So what we're seeing here is our Hurricane Isaac map.
Isaac hit New Orleans as a Category 1 storm on the
seventh anniversary, just about the seventh anniversary
of Hurricane Katrina.
And it caused an estimated $2 billion worth of damage.
With hurricanes, we have a little bit of a lead time.
So in this case, we were able to work with the states of
Florida and Mississippi, the Red Cross, NOAA, and other
organizations with really relevant crisis-related data
and prepared this Hurricane Isaac crisis map, preparedness
map, really.
So what Ping is demoing here, we have shelters.
We have many different layers, so he'll
pop a placemark there.
This is coming from the Red Cross.
It's a Fusion Table layer.
It provides nationwide active and open shelters
and number of beds.
We also have things like evacuation routes
that are on the map.
So we have--
from FEMA, it's a Fusion Table.
It's all of the evacuation routes for the coastal states
likely to be impacted by a tropical storm.
We also have, coming from the Google Maps
API, current traffic.
So this is what I mean by providing crisis-related data
in context.
And if you turn on the evacuation routes PDF here,
too, so this is providing more local level data from the
Mississippi DOT.
So really, if you're a user, your city, your town, your
apartment has been evacuated and you're looking to get out
of the city in advance of Isaac, we want to provide that
information in context so that you can provide--
so you have enough information to make an
informed decision here.
Maybe you see that one evacuation route has a lot
more traffic than another.
Maybe you take the less traffic way out of the city.
So this is just really kind of taking information that lives
separately on the web, combining an evaluated way so
that people can really make better decisions.
PAUL SAXMAN: And Pete, is that kind of why you chose Fusion
Tables as a back-end system for--
PETE GIENCKE: So, yeah.
As part of our Crisis Map, we're able to accept many
different formats.
Many of those come directly from the Maps API, so it's
built-in support, so KML.
Fusion Tables is one of those, GEOSS, Google Map Tiles.
And then all of the built-in Maps API layer types that we
find really useful, especially for crisis response.
And as you see Ping turning on and off layers, it's worth
mentioning that much of the application's overall
functionality is built atop Google Maps API here.
So hiding and showing layers is just built into
the Google Maps API.
OK, so one of the questions that we get asked is how do
you find data for the web, or how do you find
data for the map?
What does good data look like?
Really, what is the life of a crisis layer on the map here?
So we thought that we would do a quick demo just to show how
we find-- how the Crisis Response Team finds
information, map data for the map.
So we're going to jump over to the NOAA
National Hurricane Center.
So NHC, they provide--
they're the authoritative source of tropical storm
information for the US.
We're looking at the website now.
You can see that there's one tropical storm, Nadine.
For us, for the map geeks of the world, we're looking for
kind of a data link.
So we're looking where we can actually find data.
And they have a--
I believe it's GIS data sets, so you click on that, and they
provide a slew of different data that you're able to use
and play with.
So in our case, KML is a format that we
really, really like.
It's an open standard.
It's owned by the community.
It's supported by a number of Google products as well as
third-party applications.
It's based on XML.
So you can see here, there's just a version of KML.
It's just XML, like I said.
It includes all of the styling information, and importantly,
it plays really nicely with Google Maps.
So what we do, we see that a provider has a KML link.
It's one of the formats that we support on Crisis Map.
We copy that link.
Crisis Map points at third-party layers.
We do no data hosting.
And what we do, then, is to kind of validate that the data
looks good.
We just paste that URL directly into Google Maps, and
Google Maps will do the rendering for us.
It'll show you what the KML looks like via Maps.
If you pull it in via the Maps API with KML layer, this is
exactly what you'll see.
So in this case, the data looks good.
We see that, thankfully, it's not on
course to make landfall.
And it's a relatively weak storm.
So that's good.
So this is something that we would add to a Crisis Map if
we were doing a Nadine Crisis Map.
So Ping here has added it to the map.
And you can see this is what, indeed, it looks like.
So we have the link there.
The other thing that we do--
for people that--
we provide this for people that want to look at the data
on a Crisis Map.
But for people that want to actually bring it into their
own application or create their own mashup, we provide
links to the source data as well.
So you can see there that we have the download KML link,
and this also works with GEOSS layers as well.
These ones, it's really easy to kind of mash up, so we do
try to provide--
improve the discoverability of the raw data for that.
PAUL SAXMAN: So, quick question about that.
If a developer wanted to do their own mashup with a Crisis
Map and that type of information, would you
recommend they actually embed your map in their website, or
actually just pull the data out and kind of build their
own map around it?
PETE GIENCKE: Yeah, so you have many options.
So one of the things that we provide, we have share
functionality.
So if you want to just take the layers that we have and
provide your own kind of view of that, we do spatial
bookmarking, we affectionately call it.
So your view port and all the layers that you have enabled,
once you click share, those are saved in the URL.
So you can pass that around with Google+, Twitter, or
Facebook, or you could actually--
we provide the iframe code, too.
So if you want to actually just embed that in your
website, you can do that.
If you want to do your own mashup, we definitely
encourage that.
We're developers.
We love to see different people's takes on web mapping.
So we have KML and GEOSS links there, so we encourage people
to do that as well if they want to
customize what we have.
Because we're using open standards, it means that you
can use open layers if you want to do that.
Or we prefer Google Maps, but there's other options as well.
And the one thing that I should re-mention is that we
are looking to open source this, too, so if there's
things that maybe you want to customize this in some way,
just stay tuned and we should be able to give you some new
hotness to play around with.
PAUL SAXMAN: Awesome.
PETE GIENCKE: OK, so I mentioned KML is one of the
layers that we talked about.
Another life of a layer is a Google Map tile.
So Google Map tiles basically--
a Google Map tile is like when you first load Google Maps,
and you see the road data or the satellite,
those are all map tiles.
Google Maps allows you to override the default map tiles
with your custom map tiles.
And that's a format that we support with Crisis Maps.
So what we're looking at now is the--
so NOAA flew a bunch of imagery right after Hurricane
Isaac hit New Orleans.
They're making those data available via their own
website, which is great, to have some cool functionality.
You can download the raw data.
You can toggle the transparency.
With Crisis Map, we think there's a lot of value in kind
of mashing this up with other different data sets.
So one of the things that we did here--
because they're producing map tiles, we're just able to
point at their data using the map tile format.
So I think we have an example of that.
So that's what a map tile looks like.
It's basically the URL, and Google Maps replaces the X, Y,
and Z with the location on the map that those tiles reside.
And so we have an example of that.
This is just the Mercator scheme.
And then you can see that they're organized into 16
tiles, and those increase as you zoom into the math.
So map tiles are-- these work really well.
Map tiles work great for imagery, whereas, let's say
you have maybe a 2-gigabyte image, you can break that into
bite-size pieces, into smaller JPEGs and PNGs.
Also, for vector data, if you have a very large data set and
you don't want to render that as vector, you can turn that
into a raster and then slice and dice that.
And there's many good tools for doing that, many open
source tools.
I believe maptiler.org is one that you can check out if you
want to create your own map tiles.
PAUL SAXMAN: So tiling's kind of an art and a science.
What tools do you guys use specifically?
Is it something that external developers--
PETE GIENCKE: So we love open source tools, so maptiler.org,
I believe, is one of them.
There is an open source tool that lots and lots of people
in the GIS world use.
It's called GDAL, G-D-A-L, and there's something called
GDAL2tiles.
So that's part of the GDAL package.
So people can just download that, and then they can create
their own tiles that work with Google Maps.
So it is kind of an art, but there's high returns in terms
of being able to visualize really complex data sets on a
web map once you're able to do that.
So because the value of a map really has a lot to do with
the underlying data and that knowing that good data comes
in many different formats, we try to support as many of
these kind of geo formats as we can.
Maps API provides a lot of these for free, like I
mentioned, so Fusion Tables, KML, KMZ, GEOSS.
One of the things that we're looking at is supporting
additional formats for the map, especially some of the
more traditional GIS layers and data sets, including PDF.
And we're looking at doing that building on top of Google
App Engine.
So App Engine scales extremely well.
We can serve lots and lots of traffic and sleep well at
night knowing that we don't have to manually tweak the
application just to make sure it stays up.
This is an example here where we're serving--
we have the imagery coming to us.
We also have the PDF, if you wanna maybe zoom into that.
Where is it?
KA-PING YEE: This one?
PETE GIENCKE: Yeah.
So this is the case here.
So we have the Mississippi evacuation route.
We're playing about with converting PDF into map tiles.
[AUDIO LOST]
the ability to kind of toggle the transparency of that.
And this is especially relevant where
you're dealing with--
for map tiles, where you're dealing with
before and after imagery.
So you can turn on the satellite and do the
comparison.
So we mentioned that this application is really about
improving the discoverability and accessibility of
crisis-related geo data on the web.
And a lot of this has to do not just with the data, but
with the presentation and supporting many different
browsers and devices.
We've made a conscious decision to really provide
only one version of Crisis Map.
So there's not a separate desktop and mobile version of
Crisis Map.
And that means that we're really focusing on improving
the scalability of the map.
So what Ping has here, and we'll show you, that the UI
elements on the map dynamically resize themselves
based on the available real estate.
So as we get closer and closer, you'll see that the
search bar there kind of disappeared.
As we get even closer, as the screen real estate becomes
less and less, you'll see here that the Layers panel is
actually like tucked into button form.
So this would be kind of representative of a tablet or
of a phone that has a higher resolution.
And the beauty of this is that we're able to make the same
kind of map data available on a desktop as we are on a
mobile device.
One of the interesting data points that we have, for
Hurricane Isaac, we saw over 20% of access of Crisis Map on
these mobile devices, not desktops.
And really, this reflects the larger kind of macrotrend of
people moving away from the desktop
onto tablets and phones.
But for crisis response especially, where people are
being evacuated, they don't necessarily take their desktop
or laptop with them.
They're looking for information
away from their homes.
So for us, it was a really no-brainer decision to make
sure that mobile support was a core functionality of the
product here.
PAUL SAXMAN: Did you guys see any kind of trade-offs using
the Maps JS API versus, let's say, the native Android
MapView first for building the mobile application?
Or did you guys look at the MapView, Android MapView
itself for--
KA-PING YEE: As Pete was mentioning, we made the
conscious decision to build this one app so that as we
added features to it, as we continue to developing it, all
the features that we developed would be available both on
desktop and mobile at the same time.
So it's actually just the same app.
And when it opens in mobile, it's opening in a browser.
PAUL SAXMAN: Cool.
So performance-wise, though, everything
seems to work fairly--
KA-PING YEE: Yeah.
The V3 of the Maps API did a lot of performance
enhancements specifically to make it work well on mobile
and tablets.
And that's really paid off for us.
PETE GIENCKE: OK, so I mentioned the mobile support.
A big part of that is the functionality, but another
part is the performance of the application itself.
So if you're serving your application out to phones that
don't have the same kind of I7 processor as your desktop
does, it means that you have to pay a lot of attention to
performance and making sure that the application just
performs really well.
So, for us, we used the Closure JavaScript framework
to organize the code, but it also allows us to optimize and
kind of shrink the code so it downloads fast and runs fast.
An interesting kind of step that we have is that our
JavaScript code is about 1.8 megabytes in size so all of
the code that we have uncompressed.
And that's clearly too big to send to a mobile device.
Using the Closure Compiler, we were able to get that down by
over a factor of 10 to 160 kilobytes.
So that's a much smaller.
And then, we're using--
Closure provides the GDAL module, which is basically a
dynamic loading mechanism, which allows us to load only
what we need first.
And that takes the initial JavaScript download size down
to about 45 kilobytes.
So for a mobile device, like just imagine sending that 1.8
chunk of JavaScript versus sending the 45 kilobytes just
to get the application loading and getting the
data into the app.
Another thing that we should mention is that we release
every two weeks.
And we want to make that even--
improve the velocity even more so we
release in shorter intervals.
And for that, we really need automated tasks, and that
really also speaks to the cross browser and mobile
support, like making sure we don't have breakages and that
we're supporting the slew of ways of accessing Crisis Map.
So we run our tests using Google JS Test.
It's an open source project.
It runs in pure V8, which means that the tests run fast.
And that's in italics--
fast.
There's no DOM included with JS Test.
So we have to write fakes for the DOM
documents and elements.
But the trade-off means that we have 330 unit tasks that
run in under 5 seconds.
So we're testing.
And we do this continuously.
We're testing all parts of the application on a unit testing
level, and it's just done really quickly.
So we have tests that run that fast.
We can run them as part of the checking-in code.
So before I am able to check in any code or Ping or anyone
else on the project, we actually have to run
the suite of tests.
So that takes less than 5 seconds, so
it's not too painful.
But it just means that we're not introducing any
regressions under the code base when we're iterating so
quickly on the development side.
PAUL SAXMAN: Wait, what do you guys-- do you know what your
code coverage looks like?
How much of your code is actually being tested?
KA-PING YEE: I don't have a percentage number for that off
the top of my head, but we've covered almost everything.
It's actually pretty good.
PETE GIENCKE: When you have 1.8 megabytes of JavaScript
code, there's a lot that has to get tested.
KA-PING YEE: Can I do the crazy keystroke demo?
PETE GIENCKE: Yes, definitely.
KA-PING YEE: So the really cool thing is that, I mean,
even though all the tests run in 5 seconds, running a test
for just a single file takes like half a second or a
quarter of a second, which is so fast that you can actually
run the tests while you're editing.
So over on the right, I've got a window that's running a test
continuously.
You can see it's actually updating the test several
times per second.
PETE GIENCKE: And green means good.
KA-PING YEE: Yeah, and green's good.
And on the left, I've got the same--
I've got the file that's under test open.
And I can, as I'm typing into to this file, I make a change,
and I can immediately see if the tests are failing or
succeeding.
So I don't even have to go through the process of opening
the file, editing it, saving it, running the tests, and
then opening the file again and changing it.
As I type, every keystroke I type, I can immediately see
the results.
I've just broken that missing character--
PAUL SAXMAN: Going to check that one in?
PETE GIENCKE: Uh-huh.
KA-PING YEE: And it's fixed.
So that makes me feel a lot more confident as I'm working
about making sure that all the tests pass.
PETE GIENCKE: So, JS Test for the web, basically.
KA-PING YEE: Yeah, yeah.
Check it out.
It's really amazingly fast.
PETE GIENCKE: OK.
So we just wanted to provide a few highlights of Crisis Map.
It's good to talk about kind of what's next, what we have
coming next.
So, better format support, I mentioned that, especially for
more traditional GIS formats.
Our ability to be able to handle additional content on
the web means that we're just able to provide the best and
most complete picture of things on the ground and
especially from the preparedness standpoint, just
getting all of the data that people need to make the
decisions on the ground, and just make that available to
them without any kind of special software.
Trust signals and visual cues to improve understanding of
the data so, things like metadata coming
from the GIS world.
Metadata is a four-letter word, but it really is
important, like telling people when the
data were last updated.
So like the shelters for instance, people want to know
that there's a certain number of beds there.
They want to know that the evacuation routes are still
valid, so providing that kind of information to allow people
to really trust the data is something that we want to do.
More support for user-generated content.
On the crisis response space, we're seeing efforts like
Ushahidi really pay off in terms of allowing affected
people-- provide information that's relevant to responders
and the rest of the community.
And we really want to do a better job of surfacing that
data to provide that, like I said, more complete picture of
things on the ground.
And we're doing some of that now.
We have YouTube videos that people are providing.
We have webcams, but there's a lot more that we can do there.
And then open sourcing is another thing that I
mentioned as well.
So I would say check out google.org Crisis Map.
We'd love to hear any feedback that you have.
And before handing it back to Paul, I should also say that
we're hiring.
PAUL SAXMAN: Oh, awesome.
Yeah, it sounds like you guys have the technologies down.
Sounds like a pretty fun job.
So--
PETE GIENCKE: We sleep all the nights.
PAUL SAXMAN: So do you guys get to take vacations around
hurricane seasons or is that--
that's your only--
PETE GIENCKE: The nice thing is that we have a worldwide
team, so we have folks in Sydney.
We have folks in Japan.
We have a New York office.
And we have a Mountain View contingent as well so that we
are able to take vacations.
Knowing that events happen all over the world, there's going
to be someone that can pick up, to help out
when something happens.
PAUL SAXMAN: So where do people go if they want to look
at the job?
Do you guys have it posted on the job site?
PETE GIENCKE: It should be.
Check out google.org, sorry, google.com/jobs and search for
Crisis Response.
Otherwise, just hit me up on Google+ and I'd be happy to
refer you to an open position or just give you the details.
PAUL SAXMAN: Cool.
I'm going to take a quick peek and see what kind of
questions we've got.
Oops, got to reload the page.
PETE GIENCKE: And I'll just watch the YouTube video here
of the caterpillar.
PAUL SAXMAN: I can watch myself right now live.
OK.
Let's see.
Recording.
I don't see anything on here right now.
There's a couple questions that I had just
listening to you guys.
So you did mention open sourcing.
So what are we looking at?
I mean, are you--
sometime this year, maybe early next year?
PETE GIENCKE: So I would say stay tuned.
We're working on it right now.
For a sneak peek, a previous version of Crisis Map, it was
done by the Sydney team.
It's called Map Visage, code.google.com/mapvisage or
mapvisage.appspot.com.
Basically, that's the template for how
we're going to do this.
So to live at code.google.com, it'll be an App Engine
instance, and it's basically everything you need to stand
on your own, stand up your own Crisis Map.
So stay tuned, and we should be able to provide an
announcement of that when it happens.
So hopefully sooner rather than later.
PAUL SAXMAN: Very cool.
I think that's everything.
So, again, this is Pete Giencke and Ka-Ping Yee.
You can find Pete on Google+ if you wanna ask any
questions, and feel free to ask any questions on our
YouTube channel.
And we can try to answer them there.
Coming up next week, I think we're back down in Sydney,
potentially talking with one of the new PMs
on the Places API.
And two weeks out, I think we might have
GEO for Good coming.
So definitely stay tuned.
We'll have everything posted on the Google+ channel.
Thanks for watching, and see you guys in the next week.
KA-PING YEE: Thanks, Paul.
[MUSIC PLAYING]