Tip:
Highlight text to annotate it
X
MANO MARKS: Hi, everyone.
Thanks for coming.
Whoa, that's a little echo-y.
Can we decrease that?
OK.
Hi, I'm Mano Marks.
I'm here with Josh Livni, my colleague on the Google Geo
Developer Relations team, and Brian Flood, who is going to
be talking to you about Arc2Earth.
And we are going to be talking to you about GIS, Geographic
Information Systems, and how you can use Google Earth and
Google Maps working with GIS.
So you all know, I'm guessing, you're all here, you're
probably the hard-core Geo geeks from this conference.
You all know a lot about Google and Google Geo and what
Google Geo is good for.
And a lot of what Google Geo is good for is visualizing
geographic data.
And I'm going to run through-- you've seen all these
statistics before, I'm sure, but we have--
Google Maps is now on over 600,000 websites.
We have over 800 million downloads of Google Earth, and
there are over a billion KML, files, which is a Keyhole
Markup Language that was originally designed for Google
Earth and is now an open standard, over a billion KML
files on the web.
The Google Map's API is the most used mashup
platform in the world.
So that's just a bunch of stuff that you already know.
You see Google Maps.
You see Google Earth everywhere, right?
But most of what people do with Google Maps and KML is
they're showing a point or a set of points.
And there's a whole other world which Google Geo
actually does interact with.
And we're going to talk to you today about some of both the
lesser-known aspects of different parts of our APIs
and also about some exciting new upcoming projects.
So this is the list, and I really dislike agenda slides,
so this is the last time you'll see this, but we will
let you know where you are throughout the talk.
But we're going to be talking a little bit about Google
Earth, Google Maps, As I said, some new upcoming stuff, some
Fusion Tables, and then we're going to talk about some
integration examples using open source libraries really
briefly, and then Brian is going to show you some really
interesting, real-world examples of using Google's
technology to interact with high-end GIS systems.
OK, so let's get started.
And I like to start with this because this
is kind of the basic.
Like if you ask anybody at a party if they know about
Google Earth, and they're going to know
about Google Earth.
Let's try it right now.
How many of you have used Google Earth?
All right.
How many of you knew that Google Earth has a server
component to it?
OK, almost everybody.
Of course, for this class--
for this class!
For this workshop, for this group-- students.
So one of the key components that we have for enterprise
interactions with Google Earth is this Google Earth server
that we've had for a number of years.
And what it does is it fuses together data from a lot of
different geographic data formats and pushes it out into
your own private globe.
And one to the key things that you may not know about, and
this is going to be the extent of my talking about Google
Earth, actually, today, is this product here, or actually
what this represents, which is a portable globe.
So this right here, this is a portable Google Earth globe.
So those people who have the Google Earth server can create
these kind of applications to be used entirely
offline in the field.
So let me describe this for a moment.
This is $150 piece of hardware.
It's a physical--
this is an Ubuntu server.
There's an SD card there, 16 gigabytes, and it's got vector
and imagery data on it.
Now, use that in companion with a Google Earth client--
so this is the Google Earth enterprise client--
and you'll see right over here in the
left panel these layers.
Those are entirely customized, right?
So if you're used to Google Earth, you're used to seeing
our default layers there.
You'll see a lot of these start with JP.
This was data that was actually sent to Japan for use
after the earthquake, where they had portable generators
in local areas, but they didn't have internet access so
that they could access information on servers.
So they shipped servers.
We shipped servers from Mountain View to that area so
that they could use them for disaster response.
And just to give a quick little demo of that, you can
see here that--
where's Japan?
Going into Japan, as we get closer, you'll see there's
actual vector data.
There's roads that are drawn, road and boundaries drawn
within Japan that you then have to.
So let's say major arterial roads.
So this is just one example of using the server.
Oh, and I had to switch because I switched off so I no
longer have this direct connection to the server.
But basically, I can draw roads.
I can use that imagery.
If you go to Japan right now, you'll see that's not the
actual imagery.
And you can connect to it using also the Google Earth
plugin, which gives you another direct
connection to that.
So anybody who's got the plugin installed and the
correct authentication details can connect into
your server as well.
So this is a really interesting--
and I think not widely known outside the enterprise
community--
way to use Google Earth.
So Google Earth server using a portable globe.
As I said, this costs $150.
Google Earth server costs a little bit more than that, but
it allows you to do some really powerful things.
OK, so the next thing I'm going to talk about are some
of the features of the Google Maps API that allow you to do
GIS-like operations.
So how many of you know what a projection is what I'm talking
about maps?
OK, awesome.
So for those of you who are on video, I'll just briefly
describe it.
Imagine you have a globe, say, the Earth.
Imagine that globe is actually like an orange.
You take the orange and you peel the orange and then you
flatten the peel out.
That's a two-dimensional
representation of your 3D globe.
And that's essentially what you have to
do for a map, right?
A two-dimensional map is a two-dimensional representation
of a 3D object, so there's a lot of math that's involved in
that, and there's different choices that you can make.
Now, the default projection is called the Mercator
projection.
And I'm going to give you an example of some of the
differences.
So imagine I'm flying from San Francisco to Paris, and I'm
going to roughly click on where Paris and
San Francisco are.
You'll see I've created two different lines here.
The red line, or this line that goes straight across the
map, is the line that you would naturally draw if you
thought that you could simply draw a line and it would
connect the two closest points together.
The other line is a great circle line.
That great circle line is the actual shortest path between
San Francisco and Paris, or roughly here.
Anybody whose flown internationally knows that
when you want to fly over--
you have to fly over Greenland to get to Europe.
OK, so let's take a look at the code for this.
The code for this is actually fairly simple.
We implemented a new library system in the Maps API, so
that if you don't want to load specific portions of the code
for the Google Maps API, these newer features, you want to
keep your higher performance applications loading as little
code as possible, then you don't have to.
But it would give you the option of loading them in.
So you see this parameter libraries=geometry is in this
geometry library that allows you to do these calculations.
And then you'll see all I did here was in creating this
polyline, I added this option geodesic true, passed in that
option to this polyline that I'm creating and set that
polyline onto the map.
So very simple to create geodesic lines on a map.
So the code for this other kind of a--
for projections is a little bit more complex, but I wanted
to show you an example here.
This is a Gall-Peters projection, and this is using
some tiles that we generated directly for this.
The idea here is you're showing a different kind of
projection using the map's API, and then you're telling
them the map's API how to map points
between the two locations.
So this is what it would look like in standard Mercator.
This is what it looks like Gall-Peters.
And you'll see that Gall-Peters has a fidelity of
area rather than angle here.
That means that you get a much better sense of the actual
size of things.
So you see, for instance, Africa is huge compared to
Greenland, whereas on a Mercator map, they would look
to be roughly the same size.
You all know about projections probably quite a bit so I'm
going to not go too much into detail on that.
But let me just show you a little bit of the code here.
It's actually fairly--
it's a little involved.
We basically provide you with a projections interface, which
you have to implement to tell Google Maps how to map from
what we called world points to latitude and longitude, and
then you have assign tiles to match those coordinates.
So is this is just one of the methods that you have to
implement, which is point to lat long.
You also have to do lat long to point.
So the docs on this are really pretty clear, easy to use.
I'm not going to go too much more into the details, but
just to show you that we do have these features.
So these are the major features of
the geometry library.
We also have some encoding, polyline encoding and decoding
methods that are in there that you can use to make the
definitions of your polylines much shorter.
And then we give you things in the geometry, the spherical
library, to compute area, distance, that sort of thing.
And then as I mentioned before for the projection, we have
these two methods that you have to
implement on top of the--
when you implement that interface.
OK, I'm going to switch over to Josh right now, and Josh is
going to talk to you about Fusion Tables.
JOSH LIVNI: Thanks, Mano.
Hi, everybody.
So you can see we've come quite a long way from putting
points on a map or basic visualization that people
commonly think of Google Maps or Google Earth.
And both of those, Maps and Earth, they've been around a
long time, five years or more.
The stuff I want to talk to you today about are much more
recent developments.
So Fusion Tables, it's the first year at Google IO that
we're talking about it, and I'm going to also talk about
some as yet unlaunched things.
So there's been a couple other talks that have
covered Fusion Tables.
How many of you consider yourselves very familiar with
Fusion Tables at this point?
OK, so that's good.
About half or so.
I'm going to go into not a lot of detail on what season
Fusion Tables is, but I'll give a very brief overview,
and I'm going to focus on some of the kind of spatial queries
that we can do there are more GIS like.
So the first thing that I like to just show off is, hey, we
can have a map here, and this is a cool map showing 2010
census data put together by John Keefe at WNYC, where we
can have a lot of different polygons and some nice
interactive info windows.
And this is the kind of map that a lot of people want to
make, a core plot color map, tons of complex polygons.
Very difficult to do previously without sort of
needing to create your own tiles, have a rendering
library of your own on your server, because the Maps API
can't really handle this number for
vertices all at once.
The browser will keel over.
So Fusion Tables is a great way to put a bunch of data
into our infrastructure and then render it on a map.
That's one way of thinking about it, although it does
quite a bit more.
So let's go on to the API, which is why we're here as
developers.
It has a sequel-like API, which means if you're familiar
with using databases, My Sequel, Postgres and others,
this will be a familiar way of interacting with the data in
Fusion Tables.
So Fusion Tables is a set of tables.
For those of you who aren't familiar, I'll just give a
really quick link here.
What I've done is uploaded a table of restaurant data that
I downloaded from the city of San Francisco.
It kind of looks like a spreadsheet, and it has an
address field here.
And the address field, we're going to automatically geocode
when you upload a table like this so that you can then
visualize it in a variety of different options, including a
map, which I'll get into in a little bit.
So if I was to use the API to do a basic select statement, I
could say select the name, which was one of the columns
in my table from the table ID, which is an ID you can get by
looking at the URL of your Fusion Table or some other
ways, and just give me back five of them, and I'll get
back some information that looks like this.
I'll just give you a really quick example in the browser
so you can see it actually works.
This'll look better if you can see the URL.
This is limit 15.
If I said limit 150, I'd I get back some more, and it's
pretty snappy.
So let's move on to some of the spatial stuff that's a
little bit more GIS like.
A couple of really popular things people like to do
include what's the closest things around me?
And so the way that we've implemented this is using this
order-by-distance idea.
So given a location, so give me everything that you can.
Select some subset of things.
Order it by the distance from the location that I give.
So in this case, I've given the location of where we are,
and say give me back the closest five.
And the best way to show this kind of thing is on a map.
I've got some demo links here so that in the future people
watching a video or yourselves can get the slides and click
on these for the code.
But here's a map of all the restaurants,
quite a lot of points.
I should emphasize that all of these spatial queries also
work with polygons and lines, not just points, although
that's the demo here.
So down at the bottom left, I have a little number you can
barely see.
It says 50.
And if I click on the map, you'll see select statement,
as what happened on my click handler, which is select
address from the table, order by the distance, limit 50.
And you can see if I click over
here, it's pretty clustered.
If I click over here in San Francisco, it tends to be a
little bit less so.
And again, it's pretty quick.
So if I select the nearest 500, we can see we get a
really fast response.
So Fusion Tables lets you do that kind of query.
Oftentimes, you might just select the nearest one if
you're looking for the closest zip code or what have you, but
a great way to be able to do these types of basic
operations.
Another very common operation is your spatial intersection.
And currently, we support circles and bounding boxes.
So here's the syntax for providing a circle and
intersecting it within this case your geometry column,
which is the address, which is then geocoded
automatically for you.
And I'll show you what that looks like on a map.
We'll go ahead and give a radius of 1,000 up here, and
I'll click.
And within this radius, we can see we get a few points back.
We get a few more over here.
And here's the select statement down at the bottom.
So over on the right, I'm not showing all of the restaurants
that have been returned.
I'm actually showing a grouping of these restaurants,
which is pretty neat.
You might imagine you want to do an intersection, and then
get some statistical information or some other type
of query on the result.
And so I'll just show you how I did this.
We can see that if we're downtown here, and say we go
within 2,000 meters, a couple of kilometers, we've got--
I don't know--
1,500 restaurants, 160 bars, et cetera.
That's kind of interesting information.
So I'll show you quickly how I did this, because I like it
when, instead of just using the basic features, folks also
start to interact with the Fusion Tables API to get more
information out.
Some of the other sessions showed how you can have nice
charts in your info windows using the
charts API and so forth.
In this case, the second piece of code is this group-by
bottom line, grouping by description, and you can see
my select statement has changed subtly.
So I'm selecting account and the description and then
grouping by the description, and that gives me back that
kind of aggregate stats on the results that I had.
So I find this kind of stuff a really nice addition to the
side of you map, depending on your application.
So that's a few features of Fusion Tables that weren't
covered in some of the other sessions in too much depth,
and I hope that you guys take advantage of all the other
things that it offers like the styling APIs that were
announced and so on and so forth, but also some of these
more rich spatial features.
So the next item here is Google Earth Builder.
How many do you have heard of Google Earth Builder?
OK, so announced just a few weeks ago at the Where 2.0
Conference, that is not yet launched.
It'll be out in July, and I just want to give a really
quick preview to give some of the vision of where we're
aiming at here.
So a very common problem first in medium and large-sized
organizations is how do you manage all of your spatial
data and give people quick and easy access to it, allow
people to quickly publish and easily share maps either
within the organizations or externally?
So one of the main features is you can upload the data to our
infrastructure.
That's important.
And as you upload it, you can provide various things.
So, for example, if you provide some attribution as
part of the metadata, that attribution will be
automatically baked into the tiles, whether it's in Google
Earth or Google Maps, that we publish out.
You can set up tags for easy searching.
You can easily look over the data that you have using the
kind of search that we try to be good at and easily share it
just like it was a Google doc amongst people within your
organization or externally and set these robust permissions
to make sure that the appropriate people are
authenticated, or if you're sharing publicly, that they
can access as well.
And of course you can do some basic map creation.
You can set nice style templates and apply them to
your data, label your data, and then publish it out in a
variety of formats, whether it's to Maps or Earth, kind of
using the free Earth API plugin, or if you're going a
little more secure route, the Earth Enterprise client that
we saw earlier, and also coming up on Android, Earth
for Android.
So a couple of really, really fast demos in
the interest of time.
I'm not going to go too much into it.
If you're interested, you can search for Google Earth
Builder, and we have a longer video online that shows up
some more features.
But just very, very quickly, first thing you'll see is, as
you log in for your organization, kind of a
dashboard, giving you a real-time view, which is very,
very cool, of who's using your maps, and you'll be able to
dive into this more deeply down the line.
Please keep in mind this is unlaunched, and the UI is
still going to go through some iterations before it comes out
for everyone's use.
But this is just the idea that it is important for you be
able to see who's looking at your data and how.
We have some of the things I talked about, the
attribution, the ACLs.
Let's go ahead and look at some data.
So if we have a bunch of data, in this case, I'm going to
search for my digital ortho quads.
I can see we have a few.
Maybe I'll search for deer because I noticed this was a
deer lodge.
I see we have some topo maps and some digital ortho quads,
some raster data that's been uploaded.
And as you upload it, it's been processed using the same
sort of workflow that we use to ingest the data that we
then publish out to Google Earth.
So you get all of the same nice ingestion workflow, where
we'll take your data, put it into what we call layers if
you're putting a bunch of layers together.
Let's look at our deer lodge layer made up of these topos,
and then take a quick preview.
And if we're lucky, the Earth API should automatically
connect in an authenticated way to Earth Builder in the
background and show you this data, which has been mosaicked
and kind of blended, again using the same infrastructure
that we use to ingest data for our commercial products.
So I'm going to hop out of this demo--
I think I accidentally did not open it in a new window so
you'll pardon me here--
and hop over to the next item, which is Google Earth Engine.
How many of you' have heard of that?
OK.
So one of the things that's very tricky in GIS is remote
sensing, or more generically, image analysis.
It's tricky because it's this interesting overlap between
scientific communities who write really complex
algorithms and then at the same time are trying to manage
sometimes extremely large data sets.
And so we're trying to solve this problem-- this is a
google.org product--
using what we're calling Earth Engine and allowing people to
do near real-time analysis on very large data sets.
So some of the things I'll just point out quickly is the
data that you upload will be kept in this native
projection, which is super important
for doing image analysis.
You're not warping your data and getting different results
because of that.
And that you can define algorithms. We'll launch with
some common algorithms that people might want to run, but
you'll also be able to have an API that allows you to define
an algorithm to run over this data.
And we're also seeding the data repository ourselves to
start with as much as we can, reasonably bringing in of the
Landsat and MODIS imagery.
Right now, we're trying to import the Landsat imagery as
fast as they can get it off tape, and we have about half
of their petabyte or so of data online as well as daily
updates on the MODIS imagery on a whole variety of formats.
And if you check out--
unfortunately, my link's not showing, but search for Google
Earth Engine at code.google.com, and you'll
see that you can interact with all of this data right now, so
it'll give you a hint of some of what we have.
So I'd like to give a little bit of context of why you
might be interested in using Earth Engine.
So that is a satellite view using the historical imagery
that you would see if you loaded up Google Earth right
now in Brazil.
And at the bottom left, you can see the year.
In 1976, it was kind of this nice, big, green patch.
And if you were to progress along, you see some forest
patterns changing a little bit.
And you might ask yourself how much is actually changing?
It looks like a lot, but what if I wanted to run this
analysis over an entire country?
And what if I wanted to know how much was
being cut per year?
And you had access to all of this data, which we're hoping
to allow for.
So if you have access to the Landsat imagery and you have a
lot of it, you can run this type of analysis using Earth
Engine, and here's the basic workflow that we use on the
back end just to give you an idea of how it works.
So we might take this one image, which is a perhaps a
very large image or set of images, a collection of images
that we want to run our analysis over.
And we'll split up into smaller tiles, and we'll send
these tiles off to separate machines, and each machine
will run the algorithm that you've defined over the small
tile in this classic map-produced algorithm, where
we assemble the final result.
Depending on the algorithm you run, you might get back an
image or some statistics.
Perhaps you've clipped to a polygon, and you get the
result back very, very quickly.
If you're zoomed into an area, we run in near real time the
analysis just on the area you're looking at or the area
that you need.
And we're able to do this by sending it in this sort of
distributed fashion to our infrastructure.
So that's Google Earth Engine.
This is a really quick overview of
what we've gone through.
Some of the products have been around a long time,
overlapping in this larger world of GIS and moving out of
just pure visualization and some new things that are
really allowing you to do more robust analysis and map
publishing and all of the other things that you see in
many of our products kind of starting to move more in this
area of GIS.
So to kind of show off what that actually means for you as
a developer, I'm going to invite Mano back up to talk
about some of the integration.
MANO MARKS: Thanks, Josh.
I'm going to talk about a particular set of libraries
that are very commonly used within
the entire GIS community.
And this set of libraries is used for doing things like
interacting with GIS data formats and doing data
conversion.
So this set of libraries is the GDL/OGR libraries.
GDL stands for geographic data abstraction library, and I
haven't actually been able to get anybody to tell me what
OGR stands for yet, so an interesting--
I'm sure somebody will come up to the mike and give me their
opinions on that.
But basically, it's a set of utility libraries for parsing
data, doing data conversion, doing lots of interaction on a
huge range of geographic data formats.
And you can do interesting things like convert data,
reproject it, do data tiling for raster data.
It's a whole wide set of things, and most major
software applications that do these kind of operations rely
on GDL and OGR to do that.
We ourselves use it in some of our back ends for Earth
Builder and also for Google Earth Pro for doing data
conversion.
There's just a number of different ways in which we--
we use these at Google, and almost all the major
applications use these.
These are open source libraries under very liberal
licenses, so I'm really going to encourage you
to check them out.
And I'm going to actually switch over and we had a
little fire drill beforehand, so I didn't get a chance to
open up my terminal.
But I'm going to connect to a server where we set up the
latest version of GDL, that is, GDL 1.9.
This is actually not available in the builds,
in the binary downloads.
It's available only in the nightly build so you have to
actually compile the source for it.
But the feature that I'm going to be showing you--
sorry about that--
is a feature that is available for the first time in GDL.
You can now upload directly to Google Fusion Tables using the
GDL libraries.
So this is the first time that cloud data storage feature has
been incorporated within the GDL libraries.
It's got a very simple mechanism for doing this.
So what I'm going to do right now is I'm going to show you a
command line sample here.
So this is a little hard to read because I have to do
authentication in order to convert a shape file to Google
Fusion tables.
But what you see here is I've got OGR to OGR--
which is your data conversion utility here--
dash f GFT for Google Fusion Tables, and then this start a
very long string.
You've got those four lines on the screen there.
That's my auth token that I got using a
previous command line.
OGR to OGR info, I believe.
And then you'll see the last line right there is
San_Francisco.shp.
And this is a series of bike lanes in San Francisco.
I just downloaded this from the sfdata.gov.
and I run this command line utility, and it's basically
converting from a shape file to a Fusion Tables table, and
you see that's done.
It's actually fairly peppy there, very quick, and I'm
going to refresh my Fusion Tables list, you'll see there
I have this San Francisco table.
And I go there, and it's actually converted the
geometries to KML and then uploaded them.
I clicked on Map, and I have this data
directly in Fusion Tables.
So this is available to you right now in Version 1.9 of
the GDL libraries.
I'm running a little over time, so I'm going to switch
back to Josh and let him finish up.
JOSH LIVNI: Thanks, Mano.
So you'll probably have a good time working with GDL and OGR,
but not all of your end users will for your applications.
And so I just wanted to quickly mention before we
bring Brian on stage the kind of application that we're
hoping you guys might build.
So this is something I put together in a day called
ShpEscape, which lets you take a shape file.
How many of you know what shape files are?
That'd be a good question.
OK, I think we're good then.
Take a shape file and just very, very easily let the end
user upload a shape file, and it does basically the same
thing that Mano just showed off but via WebUI.
And this was built using the Fusion Tables' API and the
same technologies OGR and GDL, built on GeoDjango.
So this is open source code in case you guys want to check it
out, just as kind of an example of how you might do
this type of thing.
You're welcome to grab it and build this type of
application.
But much more interesting in this kind of one-off Stay
application are the really rich and robust applications
that we're hoping you guys will put together using all of
our different services and take
advantage of this overlap.
And a really, really great example of how to do this is
what Brian Flood's put together, I'll switch over to
his slides here.
BRIAN FLOOD: While Josh is getting this ready, my name's
Brian Flood.
I work for a company called Arc2Earth, and we have a piece
of software that is basically a bridge software between the
Esri ArcGIS world and the Google Geo world.
So is everyone in the room familiar with ArcGIS?
Hands up?
So just about everyone.
OK, great.
They clearly are the big name in the GIS space, and what we
try to do is build applications to kind of bridge
the two together.
OK, so basically what Arc2Earth is, it handles
various different aspects of getting your data out of
ArcGIS and into the Google Geo properties.
We do KML exports and imports, map tile cache creation.
We have a new product where we are bringing Google Map data,
the street data and imagery, directly into ArcMap.
We do Fusion Table uploads, and Josh and Mano were talking
about Fusion Table so I'm going to show
you some demos there.
And then we also have some planned support for their
really cool new Google Earth Builder product, which is
coming out.
So basically, I'm just going to run over this
stuff really quickly.
We actually export KML.
We do some very complex type of export.
We take all of your symbology.
If you have very large data sets, we'll break it up into
KML regions for you automatically.
There's a lot of other bells and whistles in there.
We also import KML data, and we're able to get information
out of the pop-up windows and turn it into real
attributes in ArcGIS.
Map tile caches: basically, anything that you can display
inside of ArcMap.
This is raster data or any type of symbology,
TINs or any of those.
You can turn into map tile caches that can then be
displayed directly over the top of Google Maps.
To the right there, you can see another-- this is an open
source application called MapBox.
We will support those types of formats so you can take your
map tiles, put them on an iPad or an Android application and
be disconnected and have them out in the field.
Data services, as I said, is the new product we have, and
it's basically taking Google imagery and bringing
it inside of ArcMap.
Then you can display your data directly over the top.
Worldwide coverage, and it automatically freshes as you
move the map around.
Fusion Tables: obviously, anything you can display and
have inside of ArcMap, you can point our uploader directly at
it, and it'll take care of it and use the Fusion Tables API.
So we're not bound by any of the KML export or import
limits with Fusion Tables right now, so as large as you
need, it'll ask import it up.
So one of the things I want to talk about because I know they
released it at Google IO is the new style API that they're
going to do.
We will handle that in the next rev of Arc2Earth.
Basically, anything you can see.
You have your layers.
The renderers and labeling and symbols will be able to, as
best as possible, get it up into Fusion Tables.
You can define your info windows directly inside of
ArcMap and have them show right up in Fusion Tables.
So basically, the stuff I want to show, the two quick demos,
is how can we make Fusion Tables more GIS like and how
can we make them interoperate with your
standard Enterprise GIS?
With Fusion Tables, you're getting definitely
scaleability, zero configuration, zero
maintenance for it.
So for a lot of our clients, it's just a no-brainer to move
to this type of setup.
So the last point there is probably the most important.
What we want to do is how do we take Fusion Tables and then
make it look like something that a typical ArcGIS server
can interoperate with?
And Esri, they released an open REST specification that
then we took and implemented up on top of AppEngine.
So some of the technology stacks that we're using for
these demos is obviously Google Maps Premier.
We have AppEngine, Fusion Tables,
some ArcGIS and Arc2Cloud.
And Arc2Cloud is--
basically what we have running up on top of AppEngine is a
wrapper around Fusion Tables, and we're adding extra GIS
functionality to it.
Primarily what you want to look at is the data sources up
top, and they are one for one with your Fusion Tables.
We use [? Aloft ?]
to communicate with Fusion Tables and then find out what
your data is and then add this extra functionality.
We have a native REST API and the geospatial REST spec is
how we interoperate with Esri products.
One of the primary things we wanted to be able to do was
every row of data that you have inside of Fusion Tables,
we wanted to make it URL addressable so that you can
get at it and have different formats for it.
So every one of your rows and every one of your searches you
can return as JSON, GeoJSON, Esri's JSON specification.
Some of the lesser-known ones in the GIS world are like GML,
well-known text.
You can get QR codes that are associated with it and that
point back directly at that data.
So basically, we are wrapping Fusion Tables and getting it
more GIS like.
So for the quick demo, I just wanted to be able to show you,
this is Arc2Cloud user interface.
We're able to--
via your settings, you go in and set up your [? Aloft ?]
tokens.
We can do this automatically for you and that's how we tie
into your data.
And then you can start building maps.
And maps are basically collections of layers.
So we'll drill into this.
I've been downloading it all day.
It has a list of layers.
You can see each one of these is pointing directly at a
Fusion Table.
A bunch of polygons.
We've been doing demos in the sandbox all day.
So that's cool.
So we're able to make a collection of layers, and how
can we display this inside some of the ESRI properties?
So let's take that same map and we'll view it in
arcgis.com.
And so right now the arcgis.com JavaScript is
talking to AppEngine and AppEngine is talking to Fusion
Tables to get that data out, and we can go in and use some
of their great editing tools that are inside there.
Let's just draw a polygon out in the middle of the ocean.
We can go in and change attributes.
You can then attach photos and stuff, and those photos will
be stored in AppEngine in the BlobStore.
And the cool part obviously is if we go back and take a look
at that same Fusion Table, we'll see that it
automatically linked up, and it's all ready to go for
everyone else to--
so there's that polygon.
So the idea is we have this thing running on AppEngine
that's a wrapper around F Fusion Tables and we're
synchronizing all the data back and forth.
Any ArcGIS property can then talk to it and then talk
directly to the Fusion Tables.
So another demo I wanted to show real quickly was Task
Queue Geoprocessing.
We wanted to be able to use the Google AppEngine Task
Queue to perform some of the standard geospatial operations
on two or more Fusion Tables at the same time.
This is usually something you want to desktop.
Josh was showing you before how something like the Earth
Engine works on raster data.
This is for vector data in Fusion Tables.
It's kind of experimental now, but we think it has a lot of
legs and something we're going to be working on.
Some of the types of operations you want to do is
full-layer buffering or clipping, merging between two
different layers and coming up with a new Fusion Table.
We're using JTS, Java Topology Suite, and that's running up
on top of AppEngine.
So basically how this would work is--
well, the top request comes in.
It will go to the data store iterator, and basically what
that is is how do you break up processing this in a
map-produced type way?
Depending on the spatial operation you're doing, you
may want to do it in ranges, which is just groups of rows
from Fusion Tables, or if they need to be spatially related,
it needs to be done in grids.
And grids will basically then go out, query both the tables,
bring them back to AppEngine, do the operation, and then
write it back out to Fusion Tables.
So you can see on the right hand of the screen there, you
can see that occurring in the Task Queue, and then when it's
done, it's actually writing back to the new Fusion Table.
So I'll just try and do a quick demo of this to show you
what we're talking about.
So basically you go down to your Analytics section.
We're going to do a spatial join.
And what a spatial join is we have two different tables, a
polygon table in Fusion Tables and a point
layer in Fusion Tables.
We want to be able to add a new field on the polygon
layer, and then sum up the count of one of the properties
on the point layer.
Blockpop is the point layer.
We'll call this zipcode_join_9.
These are the different options for the type of join
we want to do.
Here are the fields.
We want to calculate the total number of households.
The aggregate operation is how you want that to
actually be summed up.
A simple sum is fine.
And then we hit run.
And what this is doing now, well, we saw in the last slide
is it's breaking up into multiple ranges.
It's going off to the Fusion Tables.
It's pulling that data in and actually making
the join for you.
And so we saw a small error there, but for the most part,
we'll be able to drill in and see.
[TYPING]
BRIAN FLOOD: So if we look at this new Fusion Table that was
created, we'll see there's now a households field at the end
that's populated with the sum of all that point
data that came in.
So the idea there is how can we bring the standard
geospatial operations that you do in the Esri and ArcGIS
world and bring it online and do it inside the cloud?
So here are just some of the pros and cons of this.
Distributed dataset analysis is far more attractive.
You can do it on your live data up in Fusion Tables.
The Fusion Table is immediately available for your
online applications to use.
A lot of the stuff in the Task Queue for AppEngine can take
care of small errors and re-queue them.
Obviously, there's a lot of bandwidth consumed.
We think in the future that this is not going to be that
big a deal.
It might be now.
And then there's some other issues with
oversampling of data.
On Arc2Cloud, we do have a feature cache, which helps
with this type of operation.
So that's just my quick demo.
Thanks a lot for listening, and come check us out in the
sandbox if you want to learn a little bit more.
[APPLAUSE]
MANO MARKS: Thanks a lot, Brian.
That's pretty impressive stuff.
BRIAN FLOOD: Cool.
JOSH LIVNI: So just summing up here, you can see a lot of
different ways that we can overlap with the sort of
traditional GIS with a lot of the services that we have and
look forward to some Q&A here.
If you don't mind just coming up to the mikes and lining up
behind them, we'll do our best.
MANO MARKS: Yes.
And also you'll see there's a feedback link on there.
We're on the speaker meter, so please feel free to give us
feedback on the talk.
Please come up to the mikes if you have any questions.
Don't be shy.
AUDIENCE: A dumb question.
What's the price and model for Fusion Tables?
MANO MARKS: So currently, your--
any Google account gets a free quota.
That's 250 megabytes.
If you get Maps API Premier, then you get along with that I
believe it's 10 gigabytes; is that right?
And then you can pay for more by gigabyte.
JOSH LIVNI: We're just starting to add some pricing
to Fusion Tables Premier, and if you're already a Premier
customer, it would probably be best just to
call the sales rep.
I'm not sure the numbers off the top of my head, but we do
have a pricing model that gets you additional QPS and
additional storage.
I'm not sure exactly what it is offhand.
AUDIENCE: So one is a storage and the other one is a hit?
MANO MARKS: So there's both.
So with the free Fusion Tables, with your regular
Google account, you get five queries per second and 250
megabytes of storage.
JOSH LIVNI: And to clarify, that's against the API, not
views of people hitting your stuff.
AUDIENCE: Right.
OK, yeah.
MANO MARKS: And then you get much more with Premier.
And you can pay for additional.
AUDIENCE: When would be the final pricing coming out?
Do you guys--
MANO MARKS: The final pricing is actually out, but you
should talk to a salesperson about it.
AUDIENCE: OK.
All right, thanks.
MANO MARKS: Did you have a question?
AUDIENCE: Well, I just had a comment, that maybe I haven't
found it and you guys are already on it, but you were
talking about getting historical
data into Google Earth.
And one of the sites I've had a lot of fun with is a place
called historicaerials.com, and they have put online a lot
of the USGS aerial mappings from the '40s, '50s
and '60s in the US.
MANO MARKS: Yeah, that's a great site.
We're definitely aware of that site.
So what you're asking about is--
I have to repeat this for the video--
is historical imagery and I guess whether we can integrate
with stuff from Historical Aerials.
Yeah, we are actively pursuing additional historical imagery.
So if you go to Google Earth, you can see there's a historic
imagery button, and there you can view lot of different
imagery for different locations.
Obviously, over the years, we've built up our own
imagery, but we have imagery going back I think into the
'30s, maybe even earlier than that, so aerial imagery
obviously, satellite imagery from the 1930s.
AUDIENCE: All right.
Well, I'll go look some more.
MANO MARKS: And that's also--
in November, I believe, we made that available in the
Google Earth API so you can programmatically interact with
that as well.
AUDIENCE: Nice.
Thanks.
MANO MARKS: Sure.
AUDIENCE: You showed the distance-based queries on the
Fusion Table.
Is it just Euclidean and are there are any plans for
network distance?
JOSH LIVNI: There aren't any plans for network distance as
far as I know.
I should caveat that a little bit with saying that the Maps
API, we just recently launched a distance matrix calculation
that lets you do--
given a set of points, kind of a matrix on them, you can get
the time it would take to travel using various different
bicycling, walking and driving directions.
So that type of network distance we offer in the Maps
API using their sort of directions API.
But Fusion Tables uses--
I don't think it's actually Euclidean distance.
It's--
I'm not actually sure.
I should check up on that before I say something it
might not be.
But yeah, we're probably not going to offer network
distance in that.
AUDIENCE: You mentioned when you upload data to the Fusion
Tables, if you include additional information, that
may get geocoded.
Are there limits in the [UNINTELLIGIBLE].
Not the fact that it's going to get geocoded.
Like if I want to display-- if I want to upload like a real
large data set.
MANO MARKS: Yeah, so the limits are the same as
for the Maps API.
So you actually have to trigger this in Fusion Tables,
when you upload a file and it's got a single column for
your addresses.
And then you go into Maps, it'll start to geocode it.
And the limit is 2,500 a day for the free accounts and then
whatever your quotas are for Premier.
AUDIENCE: So the geocoder is about the same speed as if
you're using Google Maps to geocode?
MANO MARKS: Yes.
AUDIENCE: Good.
OK.
MANO MARKS: You don't need the geocoder if you already have
latitude/longitude information.
Next?
AUDIENCE: This question's for Brian.
You mentioned how your products can take the Google
content and stream it into the ArcGIS.
BRIAN FLOOD: Right, right.
AUDIENCE: Is it the desktop?
BRIAN FLOOD: The desktop.
AUDIENCE: Can you elaborate on that, how it works exactly?
BRIAN FLOOD: Well, basically, we became an OEM partner with
Google so we were allowed to do that.
And what it's doing in the background is interacting with
the Google Maps Premier API.
And as you pan around on the map, it's bringing the data in
and just displaying it as an extra layer.
AUDIENCE: So it's using the actual tiles.
It's not like a WMS or something?
BRIAN FLOOD: It's not a WMS and it's not the actual tiles.
It's kind of some secret sauce that allows us to get the best
of both worlds.
AUDIENCE: OK.
BRIAN FLOOD: Some caching, but doing it the legal, legit way
with Google.
AUDIENCE: OK, thank you.
AUDIENCE: With Fusion Tables, there is a final KML URL that
you can use, right?
MANO MARKS: That's correct.
AUDIENCE: So you can open up KMLs in ArcGIS, can't you?
Or I guess you would just get a snapshot.
If there's editing going on the Fusion Table, you would
just get a snapshot in ArcGIS.
Job
MANO MARKS: So the question is can you open up
KMLs directly in ArcGIS?
And yes, you can.
What you're getting out of Fusion Tables is not--
you're not interacting with a live KML file.
You're downloading.
Yeah, so you're right.
It's either a snapshot or a network link which would
update periodically.
I'm not sure about ArcGIS support for network links.
BRIAN FLOOD: I don't think it has it.
MANO MARKS: I don't think it has it.
Mike?
AUDIENCE: Yeah, this isn't a question.
It's more of a paid advertisement.
I work for the Enterprise team on Geo, so if you want to buy
the Premier API or that portable server or anything, I
can probably hook you up with the right account rep.
So I'll hang out in the back of the room.
If you want to get any more information about pricing or
what we actually sell, I'll sit in the back.
MANO MARKS: Great.
Thanks, Mike.
Any other questions?
All right.
Well, thanks for coming to Google IO, everyone.
[APPLAUSE]
MANO MARKS: I hope you've had a great time.
I certainly have. Josh, Brian and I are going to be hanging
out for a little while afterwards.
So if you have any--
you want to come up and ask us directly.
Thanks a lot, everyone.
JOSH LIVNI: Thank you.