Tip:
Highlight text to annotate it
X
>> NORTH: ...going back to complete his master's degree next week, give or take. And now--now
my stalling time is over, so over the limit. >> Oh, thanks for this wonderful introduction.
I guess I don't have to do any. So--oh, let's dive right in. I'm going to talk about open
sourcing, the Robot API, basically my experiences from the last month. So before I get started,
who of you here ever wrote a Robot for Wave, or used a data API? Okay, good. Not everybody.
So, yeah, I'm going to talk a bit first about what the Robot API actually is. Basically,
there are three components. There's the Passive Robot API, the Active Robot API, and the Data
API. I'm going to talk a bit about what kind of--what code--which code we used from production,
the challenges we had to make it work for the open source server. I'm going to talk
a bit about the implementation, not too much code details. I'll be here all week, so if
you really want to get into the code and you're of the people who signed up for the coding
sessions on Thursday and Friday, feel free to ask me any time. Let me hook up my power
before my screen turns off. There you go. Now, I'm going to give a demo of all the three
different parts of the API. I'm going to talk a bit about the work that's left to be done.
So let's dive right in. Oh, it's the Passive Robot API. On the left side we have--we have
a Wave on which a user does something, like add a robot to it. The server gets this operation
from the user, and then it notes the address [INDISTINCT] to robot so it sends this event,
encode it in a JSON as a POST request to the URL of the robot. The robot then does some
processing. It wants to--it wants to do something to do Wave. The things that it--things that
it wants to do, it will encode it in JSON again. It will send it back to a server. A
server will make operations from these JSONs, and will apply to the Wave, and you too will
see that the Robot has done something. And this is called passive because the robot itself
is not instigating this operation. It's the server who is calling the robot. So if you
take a look at the Active Robot APIs, it's actually the robot--it's just sending us the
JSON to the server, and then the server will apply these operations. But someone who is
quick might have figured out like how do we prevent someone from just cheating and pretending
you're someone else? Well, we use something called OAuth. So during registration, we'll
hand out a security token to the robot, and the robot will use this to authenticate to
our server. And--so we know it is actually the robot that's trying to send this operation
and we can simply apply it as being this robot participant. And then there is a third part
of the API work which is called the Data API. Well, robots are actually considered real
participants. The data API is actually something that allows you to apply operations as someone
else. So, basically, I can log in. I can have any program logged in as me and act as me
on the way. We use three-legged OAuth for this. Now that we--I've given a quick overview
of what the APIs actually are, I'm wondering if there are any questions about it. Okay.
It seems like we're good. So I'm going to talk a bit about--there's one question in
the back. >> Are you planning to add live editing of
the Data API and also playback? >> What--can you repeat that question?
>> Are you planning to add live editing of the Data API or the playback?
>> So the question was, if there is any work going on to add live editing or playback to
the data API? Currently, there are no plans for that. However, we're just trying to port
the existing functionality we have in production now. And then later on, we might be able to
refine and add new operations to it. You're welcome to contribute on how it should actually
work. I believe the playback is actually not working in the client at the moment. But it
is also something we might add later on. Is that correct?
>> Yeah. >> Yeah.
>> INDISTINCT] spoke to it? >> Not yet, but it's--yeah, there's a lot
of work to be done. Okay, let me talk a bit about what kind of code do we use from production?
Well, our production server service scale pretty well, so there is some code we cannot--of
course, cannot use. But important code we reuse is the service add synchronization code
which is pushed out to the open source. It's this thing that takes the JSON that comes
from the robot and transforms it into objects that we can use too, well, create operations
in deltas. Also, there are--in the code itself, I found a lot of--tons of ideas of how it--how
it works. And I tried to improve upon these ideas during my design of the Robot API. Something
that we could not reuse, because we have a slightly different view on Waves, internally,
is the event generations. So we have to write that from scratch. That's why not every event
is yet supported. Also, we're phasing out the blip-submitted event because that's something
that was basically hacked on through our production servers. I had a question about that early
today, and I'm happy to discuss with anyone who wants--really wants something like blips
submitted, but it's--it's kind of tricky. And also, something that I've learned while
making the Robot API work is actually that something that server engineers here might
take home with them is that if you want your code to be usable, you should think about
what types you pass around. I found that the code that we were using was actually passing
around the internal view of our Wave, like the special glass we have and, well, it actually
did not need any of the functionality that it offered. So, do not just use the deepest
type in your hierarchy when you're--when you're writing code. Think about whether you actually
need it or not. So now let's talk a bit about data. I'm going to talk a bit about the challenges
I faced, and I think the most interesting ones lie in the Passive API because if you
have the Passive API covered, you can--you'll also have a large part of the Active API and
any of the API there, except for the OAuth part. So, I'm--I'm going to talk a bit about
how and when to generate events for robots. Should we treat all robots equally? And what
kind of guarantees can we give to robots? Also, some interesting questions came up when
we tried to figure out how a server will know who's your robot, since in production we actually--and
all of the robot addresses where apps product come, so that gave it away to us. But in this
case, we find that robot should be able to be hosted anywhere. So when Wave was first
launched, robots could only run on App Engine, and since we're out in the open source now
and robots have been liberated actually during the last [INDISTINCT], we really want to retain
that capability of just being able to host it in any web server that can speak HTP. Also,
robots shouldn't have to be registered at every public server. We have a federation
for a reason. So, we came up with the following ID that robots--robots will register for an
address at the--on the domain the Wave server is serving on, like the example.com domain.
So if I register a robot at example.com, and I add a robot--and I add this robot to my
Wave on example.com, the server just handles it locally, which makes sense. And also, adding
robot at example.com to acmewave.com, if acmewave is federating with example.com, should basically
be handled by the federation protocol. And so, in allowing this, we actually have enabled
federated robots, and it's actually working. So I'm quite pleased about that. So now that
I've talked a bit about robot registration, I'm going to dive into how [INDISTINCT] Passive
API works. We've seen--we've heard a bit about the Wave bus before. It sends the snapshot
of the latest version and the deltas that had been applied to it recently down the Wave
bus. It makes sense for the robot to--well, anything that works with the Passive API,
to hook up onto this because, well, we need to generate events that come down this Wave
bus. So we call this component the robot's gateway, and its task is basically to generate
events for robots and process their responses. Well, if you would be quick in thinking, well,
we might just hook it up like this, we have the delta coming in from the Wave bus to the
robot gateway, and we just generate events, send out to one robot, get back, apply operations.
It's actually not going to work that well because, first of all, if you do it like this,
well, you're serving HT--when you're calling out to a robot, you're actually installing
a Wave bus in this way. So I--and the other subscribers to the Wave bus have to wait until
the robots are done, which is not a good thing. We want to keep the Wave bus pretty fast,
so we have to do this, well, basically in [INDISTINCT]. Also, multiple robots on the
same Wave have to wait for each other in this way. So if a robot is slow and there's another
robot on the Wave, you have to wait until that slow robot is done if you are next. And
also, because if you do it like this and robot A sends operations back that are applied to
the Wave, robot B will not see them when robot B gets called at this time because that will
be another call to the Wave bus because it's a new set of deltas that have been applied
to it, which leads to stale information going through the robots, which is not really a
good thing. We don't want that. So, when we--we thought a bit about that, and we came up with
somewhat more advance implementation here where we pass along the delta to something
that is like a delayed proxy to a robot. We schedule--we schedule the robot for execution
in another thread. We do this for every robot on the Wave so they run independently. And
then on the execution, we send all the deltas that have been collected for a single robot,
for a single wavelet, to it. So if a robot was doing a lot of work on Wave A and events
came in multiple times to Wave B, when it's time to for Wave B to be updated for that
robot, we will send everything that's happened to Wave B so won't have that--a problem with
stale--really stale Waves anymore. I'm not going to go too deep into code as I said before.
If you really want to know how it works, what the components are--yeah, what the code looks
like, feel free to ask me. So I'm going to go into a little demo now, show you every
aspect of it. Let's hope everything is working. The Wave client is actually up. Let me register
my account quickly. Here we go. Let's see. Hello, people. Well, here we go. So let me
add a robot I've registered before. It should say, hi. Right. So this is basically a passive
API at work. The robot gets told that it has been added to this Wave and it will send a
response back to me. I can also do the active API with this robot. I just have to call a
URL to tell it to actually do something because--but you can also have it do--run in an upfront
con job or whatever you want. It's just a matter of how to tell the robot that it's
time to do something. And we could see here that it's telling me that someone visited
that link. So that's nice. I'm running an old version now so the names are wrong, but
that has been fixed. And now, I will show a bit of the--well, actually, I'm going to
show another robot. I'm going to let--let it call out to App Engine, a registered robot
which is running on App Engine right now. Oh, here we go, robot. It's running on Wave-echo[INDISTINCT].com.
It's actually, it actually echoes whatever you type, which is kind of nice. You can actually--if
you want to attach your survey, you can have--basically add everyone and you can just see how it actually
works. This works, so it's kind of nice. Well, time to dive into the data API using Splash.
Oh, yes. Sure. >> Can you show the--the demo how you register?
>> I will do that in a second after this. Is it good? So, I'm--started up Splash now.
Splash is doing a three-legged OAuth [INDISTINCT], so it sends us back to the Wave in a Box server--servlet
which asks whether or not you want to give that program permission to have access to
your data. So, I will click agree and then it will send me back to Splash. And you will
actually see that our ways will show up. So this is Splash. And it has been mentioned
before, it currently doesn't have the ability to edit Wave, so that is something that might
be added in the future. It's basically because Wave got cancelled right when we wanted to
launch all of this, so. It--the nice thing about this, it runs purely on the data API
and is also pretty fast. Well, I had a request from the public to show registrations. So
I should be able to do that. There we go. It's--page still needs to be beautified, but
it's as easy as just registering any address on your server, typing in the URL of your
robot without the "/_wave". And then your robot has been registered. We hand out a token
and a token secret, which you would need if you want to use the active API. If you download
the latest version of the Client Library, you can actually change the server which it's
calling to now properly, so you can use this for your own servers now. Let me switch back
to my slides. >> A quick question.
>> Yes. >> Is that why the same robot could be registered
under different addresses with different servers? >> Yes. We might need to take a look at what
actually it does to the code, but it should definitely be possible.
>> Do you remember what's--do you remember what that [INDISTINCT] out when, like, your
server is calling them and using them, like, what their address is, [INDISTINCT]?
>> So the question was whether robots find out where--which server is calling them. You
can definitely see the wavelet ID, so you can find out which domain it's on.
>> You can get it [INDISTINCT] >> Wouldn't that--wouldn't that sort of detection
be on the robot code itself rather than being on the server's side? So, you [INDISTINCT]
>> Yeah, the robot can figure out which domain it's being called from. And maybe some extra
code can be added to actually figure out which--well, which URL is calling it, where it's being
called from. Anyway, that code is also--the robot Client Library is also open source and
we actually encourage you to come up with good ideas because we're basically out in
the open now so we can make it better. So, I want to spend another minute on what's the
future work. So there is--normally when you register robots for use in production, we
have an extra step which actually calls out the robot to check whether it's actually you
that's adding it to the server. So that still needs to be added. So we actually know whether
there's a robot running on that address. Support for gadgets inside robots. Currently, robots
can't really act on gadgets. And I've also been asked to point out that we are working
on gadgets to get--to get into the client. We have another patch outstanding which, when
hooked up, makes it show up in a client and then another patch is needed to persist the
data for gadgets. We hope to do that this week during the coding session. But I can't
really guarantee that. You never know what pops up, but there has been a lot of work
going on in gadgets and it's getting there. Yeah, if you check out the code now, you will
see a gadget button actually in the Wave panel if you update the libraries. We want to support
more operations and events. Some of the operations and events we have in production are not supported
yet because they had to be re-written to work with Wave in a Box. Most notably, we need
to make an implementation for the robot notify operation which allows the events to be reset.
And there are some metadata fixes that need to take place because the Wave bus doesn't
currently tell us about the--about the right--the correct time stamp when an event has occurred,
when an event has occurred. So, yeah, I thank you for your attention and if there are any
questions, feel free to ask them now. Yeah. >> Do you have any documentation for setting
up robots on external servers? >> What do you mean by external servers?
>> Well, say, I'm going to host robots on my server and [INDISTINCT] me up to one of
the--how do I go about actually setting up the robot on my server or do I need to do
it? >> Well, the question was whether--is there
any documentation on setting up robots on your own server. I believed that we have an
API page for that on code.google.com. >> Right.
>> So, the original documentation which is used for connection server should also work
for your normal robots. You just need to change the URL that it needs to call to for the active
API. That's basically it. Any other--I saw a question right there.
>> I have a question regarding the [INDISTINCT] in robots.
>> Yes. >> So, if I am using a robot which is on a
different Wave server that means that my Wave server and this different Wave server that
federate, that means they're exchanging all their tasks and to federate [INDISTINCT] to
keep the state of the state, correct? >> What do you mean by--the federated server?
>> Yeah. >> We have an acme com and example com. If
my robot is on example com and I'm using acme [INDISTINCT]...
>> And those servers are federating, okay. >> That means example com must hold the stays
of the entire base. >> Yeah.
>> Now, when I'm using the, let's say, translation robot, it would show my base of [INDISTINCT]
translation robot could be used by many persons around the world, but all of the base, all
of the history has to be stored on my server that runs the translation robot.
>> All right. Yeah. >> So, right. But if you're just using the
robot, why wouldn't you register it on your own server as opposed to using it through
a federated server? >> Well, that was the point, that I registered
from there and instead everybody can use it from there.
>> But the [INDISTINCT] >> If--if the servers are federating, yes.
>> Yeah. >> But you can register robots and not the
servers. >> But nobody will do that. So, under--typically
[INDISTINCT] run--so I have translation servers, it translate an [INDISTINCT]
>> You can program your robot to actually block out Waves from other domains, basically.
If you--you just take the Wave ID you got when the events come in and if it's not under
the domain you want to serve on, you just don't do anything.
>> Okay, you can deny it. >> That's safe sometimes.
>> But [INDISTINCT] to robot API was--is the stateless solution, for you, you get the operation
then you make the decision with that operation [INDISTINCT]
>> That's the robot, yes. And then we--and the federated Waves will actually be stored
on the Wave server not in the robot. >> Yeah. But my [INDISTINCT] full point is,
you cannot really offer the robot service to others because it makes the others frightful.
>> No. But I think you're getting it. >> I think we want to--this--we could take
this discussion off this presentation because it's, for me, dragging along. So we--I'm happy
to talk about that. It sounds like something that might be a problem. Any other questions?
I guess not. That's--I think it's time for a break now, am I right?
>> I guess it's because--so you've been [INDISTINCT] through the content which has been really
good. >> Are there any questions on the Wave, by
the way? >> Not that I saw.
>> Okay. >> I almost put up with, you know, set up
your robot... s discussion off this presentation because
it's, for me, dragging along. So we--I'm happy to talk about that. It sounds like something
that might be a problem. Any other questions? I guess not. That's--I think it's time for
a break now, am I right? >> I guess it's because--so you've been [INDISTINCT]
through the content which has been really good.
>> Are t