Tip:
Highlight text to annotate it
X
>>
BRAKES: Hello, my name is Charles Brakes, and I brought Hanno in here to talk of, little
bit more about some of his latest work with the robots. He's emphasizing robots in kids
this time around, and he currently lives in New Zealand and he's sponsoring a variety
of programs to get kids more interested in electronics and robotics. So without further
ado, here's Hanno. >> SANDER: Thank you, Charles, and thank you,
Google, for having me back here. I was here exactly a year ago, talking about my dance
bots, which is a balancing robot, dances with kids using computer vision. And this year,
I'll talk about 12Blocks which is a simple block language--that's kids program, sophisticated
multi-core robots, very easily. So, first, I'll talk about a little bit, New Zealand.
So, I moved there five years ago, now, and there's a bit more than sheep down there.
I contacted a couple of local companies in Christchurch, New Zealand, which is where
I live. And so, this is what my neighbors and friends are up to. Glenn Martin is building
the first practical jet pack. A hundred thousand dollars buys you a back pack you strap on,
and let's you fly to the air, thousands of feet up for about 30 minutes, and it was shown
at the last Oshkosh very successfully. And Yike Bike could fit in really well here at
Google. With the segways, it's a very light folding bike, fits in the back, strap it on,
and there you go. A friend of mine is doing a GPS Boomerang, so it flies up to edge of
space with a Helium balloon and returns with a GPS sensor right to about three hours later,
hits you on the top of your head, and gets you your sensor readings, and lots of telemetry
information about what's going on in the atmosphere. Another friend of mine is doing the first
color X-rays, so they developed a novel sensor for CERN, and that's currently being used
at the Mayo Clinic to grab color images from X-rays. So, they're still using the same X-ray
emitter but they're using a special detector to count actual photons looking at their energy
level, and with that, getting the composition of what the X-ray went through. Another guy
is doing WhisperGen and that's the four-cylinder Stirling engine which provides residential
heat and power and it's currently in trials in Europe. And, another guy is doing a PivotalEngine,
so, instead of a piston going up and down in an engine, this has a pivot where the explosion
happens in the same chamber but it's stationary on one side, so it's pivoting. And so with
this, you get very high power density, it's quiet, and it can run on hydrogen since you
able to water cool the whole engine and the piston. And, there's another company called
Jade and I searched that it's a very high speed database. So, I have to keep up with
these other guys and so I've been busy too. I'm more busy on the hobby and electronics
and getting kids into robotic side, and sought--take a minute to talk about what I've done there.
And, so I wrote a book with some other co-authors. We're all using the Parallax Propeller, it's
a high-speed, multi-core semi-conductor chip that lets you do a lot of the things that
I'll talk about here today. So, it has memory, it has eight processors running at 80 megahertz
and it has a lot of input and output pins to let you interface with sensors and actuators
and motors. So, this is a five hundred-page book, and I wrote about my balancing robot
that I talked about here last year. I also developed a PropScope. So, this is a $250
little blue box that reads analog signals on one side and connects to your computer
with USB and it also can be used as a function generator, a spectrum analyzer, a logic state
analyzer. So basically, everything that you did in college with a big lab full of instruments
is now in a little box that you can take home and look at synops. Okay? And I've also developed
a tool called ViewPort, that lets you develop very sophisticated projects to build on this
Parallox chip, and I'll show that a little bit on my demos. And, that includes very graphical
debugging tools, so, it includes the standard break point, and call stack, and memory map,
and stepping in and out of syntax, but it also has simulated instruments like in the
celloscope, the logic analyzer that let you look at what's going on in your system. So,
if you've ever wondered why your program isn't running the way it is, but it should, with
this, you can go back in time, so you can see about ten minutes ago, this is what happened.
This is what the sensors were reading, this is what your mathematical model was doing
and that it helps you develop more complex, more sophisticated and edit systems. Integrated
in here is open cd, so a complete computer vision library's in there, as well as physiologic
tools, so, that's all in there. And finally, 12Blocks. So 12Blocks is my answer to my daughter
when she ask me, "What are you doing?", or, "You're always programming, you're always
on the computer. Can I try? Can I do something?" She's five years old and I didn't think it
would be a good idea giving her a C Block and showing her, and teaching in her the complete
language. At the time, she was playing with Legos and putting them together, stacking
them. There are other software out there, MIT Scratch developed the wonderful, graphical
tool. Lego Mindstorm also has a block-based interface, but there's a lot of problems with
those and I tried to come up with the solution to those in 12Blocks. And, 12Blocks is also
a language that is completely focused on embedded systems, so it's a visual language, but it's
developed for robots, for embedded systems that have actuators, that have sensors and
go from there. So, I called it 12Blocks because when everything is contained in a block, so
very object-oriented, then it's very easy to build very sophisticated things in a small
number of blocks. You're no longer looking at pages and pages of code. All the demos
that I'll show you today are in a handful of blocks. And there are many blocks available,
so there is a library; you can create your own blocks and there's about a hundred plus
blocks. I'll show you sampling of those a little bit later. And I like to say that it
has a very gentle learning curve, but it reaches very high. So you can start out using this
when you're still in elementary school and when you're still just dragging a couple of
blocks together. But you can keep using it to University beyond because it's very functional,
it's very sophisticated and there's a lot of power there. It has live edit capabilities
so you can keep editing your program while it's running, I'll show that. Graphical debugging,
which lets you look at graphs instead of looking at variables and looking at lines of code.
And it's integrated with a Wiki, so with an Internet, you can look at videos, you can
look at schematics, share everything with a Wiki; it's all integrated there. And, you
can create stand-alone programs, so, similar to how I built the PropScope by developing
some firm ware and building a PC-user interface that lets you, with 12Blocks, develop both
the firm ware running on an embedded system but also the user interface that other users
that don't have access to 12Blocks can use to run the program. And I mentioned you can
customize your own blocks and it's very easy to transition to other languages. Parallax,
the company behind the propeller, developed an object-oriented language called SPIN and
it's a very nice, very powerful language for writing textural code for the propeller. But
there's also NCEC available. So, if you're used to C, you can program these types of
robots in C as well and you can transition from 12Blocks all the way to writing C code
and going from there. So, I won't read all these off, but there is plenty of capability.
There's a standard computer science where you have arrays, and you have state machines
and events and messages, strings, user interface, graphic capability; so plugging it into a
TV and then building a graphical aim with sprites and vector graphics, sound; so type
in your own score or sound effects, play midi files as well as reading and using actuators
and sensors; so looking at a proximity sensor, looking at a will encoder, all that sort of
stuff that you typically do with robots with embedded systems is always just a block where
you drag the block somewhere connected and away you go. So, the robot that I'll demo
today was built by Chad, and it's a very sophisticated platform that fits in the hand, so it's light,
small, robust, made out of metal. You can screw a mecano and vex components to this
as well as plug in Lego pieces. So, again, going from very simple, basic Lego-type expandability
to really doing a hardware expansion and, this is really designed to get you started
with robots at the simple level so driving it like a remote control car where you're
not programming at all, but also lets you do things like Logo. I started with Logo in
elementary school where you have a turtle and you tell it to move forward, and then
turn right, and you repeat for a while, and then, you move up to using sensors. So, some
of these sensors that I talked about and integrating that to control what the robot's doing. And,
all of these is done with 12Blocks, where you visually program it. And, I'll show how
you can transition to the text-base languages as well. So what I'll do with my demos is,
I'll use 12Blocks, which has its library, which has a worksheet where I assemble programs,
and then has views. And within the view, I can look at what's going on in the robot.
And on the TBot's side, there's shared memory that's shared between the PC and between the
robot, so both have access to it. It's actually in the TBot but I can read and write it from
the PC. And later, I'll show how that same concept of shared memory can be extended when
you have multiple robots. So, again, there is memory on each TBot, but the TBots and
12Blocks can read and write that memory remotely. And on the TBot's side, that memory controls
how the motor moves, how the speaker sounds, what the microphone is doing. So it's all
a very simple architecture that is very functional. I keep saying that--I keep focusing on this
benefit of being able to easily transition from grade school to very advanced things.
But now, there's a lot of robots out there that are targeted at one specific item and
one specific challenge, one specific competition, and I think that this lets people use the
same platform with expanding it either through software, through hardware, through electronically
changing things. And on the software side, starting with 12Blocks but then growing into
Spin and MTC code at the end. Little Ts for the advanced people, so what I'll show at
the end is using wireless to compete, to connect multiple robots, and using sensor information
from one robot to control another robot while monitoring the whole system on the PC and
this can be extended to programming remotely, wirelessly, as well as connecting to Internet,
so causing the robot to move or to play a sound of something on the Internet, a stock
of whether it changes, as well as sending data measurements from the robot if it's video,
if it's sensor reading, sending that back-up to the Internet. And of course, Internet mobile
devices so Bond movies always have a scene where you take a robot and you drive a car
with a iPhone or Android, so that's all possible here to. So, demo-wise, these are some things
that people could do right of the box with us where they start with remote controlling
a car. This is what my daughter would do starting out with or even my son. Then moving on to
drawing logo shapes, moving up to integrating sensors, and using those to control the robot.
Then, line follow is a very common challenge where it is a black line on the white paper
and you try to stay on the line, go as quickly as possible. Communicating with a shared memory,
mail terminal, so a very functional communication networking protocol for sending data back
and forth, and then trying C code and Spin code. So, moving on to demos--I'll clear little
bit of an area here. Here's a robot and I will switch over to 12Blocks. So, 12Blocks
is a Windows application and it has this interface. I can start a new program and as I mentioned,
it has this library section where you have multiple types of blocks to let you do graphic
manipulation, to let you do sensor measurements and even to build user interfaces. And here
in the middle is a work sheet and for this first program, I'll do what the simplest thing
that we can do with the robot just to have it move a little bit. And so, we start with
a start block, so I'm just dragging it from the library onto the worksheet and I take
a block from the TBot section, the move block and it shows me that these are able to connect,
of course, I cannot connect it to the top, and I can't connect it to the library, so
this is the one place where I can connect it. And now, I can hit the run button and
this will load the program into the robot and cause it to move backwards for one second.
So, it's a very subtle program but it's something where I've just taken two blocks, connected
them and right away it did something. So, next demo that I'll show is doing some basic
input, output. So, there's a proximity sensor on here, there is line sensors and there's
will encoders. What we'll do in this demo is look at the proximity sensor to see if
there's something in front of it. And there's a full color LED on there and right now, it's
showing a blue light so, I'll be able to control the color of that light. So, I'll take away
the move block, I'll just throw it back into the library and I have a start and what I'll
do is I'll set the LED to a color and I will get the proximity sensor value. And this is
a specially shaped block which fits into another block. So, here is a variable assignment block.
So, set x to one, that's how it starts out but I can drag this and dock it in there.
So, now I have, three--four blocks. A start block, a block that sets the color of LED
and a block which sets x, the variable x, to the value of a sensor, just to keep this
going, I'm going to use a repeat block. So, this is a loop and I'll dock these inside
of the repeat block. So, this is pretty straight forward and I'll hit the run button and the
LED starts out red. You can see its red there and I mentioned that I can edit things as
its running so I can click here on the red and pull down from these choices and select
a different color. And, so now I have a blue light shining on the robot, and so I can do
that with all of these items that are in yellow. These are parameters and these are changeable
while the program's running. The value of x is supposedly being set to the value of
the proximity sensor. Typically, with embedded systems, we have this problem where it's in
the processor but there's no way of looking into it. So typically, what you do is you
hook up also with some other gadgets to it. The nice thing here is that we're connected
to the PC and on the PC, I have an interface for looking at these values. So, here is x
and here is a value of x. And, it's right now at a certain level and if I--if I put
my hand in front of it I can change that value. And, I can plot this over time so right now,
I'm plotting just the X value and I can plot this over seconds there. And--so, I can look
at the value as it changes overtime. What I'll do in this case is move on to—-showing
how to build a user interface based on this. So, instead of having to go in here, and clicking
these values and changing from blue to purple or some other color, I'm going to build a
simple user interface for that. So, I'll go into the user interface section, and there's
a background, and I can use a textbox, and just drag that on top of there, and this text
box will set the variable end to whatever is type into it. So, instead of setting the
LAD to the color purple, I'll set the LAD to the number end. And now when I run the
program again, I have—-I can type into this number, into this textbox, a different number
so zero is red and now when I type a 120 into here, it changes the LAD to a different color.
I can save these 12Block files, so I'll just give a file name, and I can also save it as
a stand-alone executable. So, I can save as and in here I can call it Google, and in this
case, a Google EXE file. And, I can hit save and that file is now saved to this directory,
so I will go there--12Blocks tutorials. And here is a new file, just created right now,
254. I will close 12Blocks so I've now given this to a colleague of mine and they want
to change the LAD on my robot, and so, they could just run this megabyte file, and what
this first does is it loads the firmware to the robot, and then it gives me this little
user interface that I've designed in 12Blocks. So, it just shows me N equals this, I could
type a value in here, so 120 is what I have last time, and then it changes the color of
that LAD. So, it's building the firmware inside of 12Blocks as well as a user interface to
work with the sensors, work with actuators, and bundling all of that into an executable
that you can then run somewhere else. So, that's with 12Blocks, and with 12Blocks we
can do more. I'll now move into transitioning from the 12Block visual language to Spin.
So, when you're more advanced or when you want to go into building your own blocks,
or when you want to use a more powerful debugger, then you can do that with the text language.
So, I'll take the same code that I had before. So opening up 12Blocks again. Here we go.
Okay. And, I'll take our Google One that we just had. And, what I'll do now is just like
you do on the Internet, you have an Internet browser, you have view code button, and so
you have right here. And what view code does is bring up a tool, it's called ViewPort,
that has a syntax highlighting editor in there as well as lots of graphical tools to let
you debug that program. And inside of here, I have the Spin equivalent code of my program
so you can see down here that it's doing a repeat loop and it setting the HSL to end.
And in here, I can make changes, I can do that. But first thing—-what I'll do is,
I'll just load this program, so from a different environment, I'm now loading this to the robot.
And, I have a tool like the DSO viewer, so in here I can look into the values or I can
change a value, so I get a textbox again for setting the color of my LAD, so it started
with zero and 120 is a nice color blue. So, I can do something like that as well as graphing
that over time and so, is that and an overtime, it goes down. And, what's also in here is
I can set break points, so here is 989, and I can now compile this with debugging information,
and load this to the propeller to the robot, and this will now stop on that break point
and give me all sorts of debugging tools for this. So, I can look at the files, I can look
at the code but I also have a memory map of all the information that's in the propeller
as well as a watch list of variables, a call stack, a profiler, seeing word that time is
being spent, a view of the pins the sort of actual hardware interface of what's going
on with the propeller. And I have a command interpreter so I can for example take 10 steps
and it'll take 10 steps in there. I can take one step at a time, so sort of going between
those two. So, the traditional visual studio-type of debugging is integrated in here. Okay.
And the last thing that I'll show here is that I can make changes in here. So, I can
change the intensity of the LAD so I'll make it very dim, I'll set it to five. And I can
save this file and now when I go back to 12Blocks, it says, "Do you want to reload it?" So, it
notice that the file that I was viewing had change and now I can say yes to this, and
then the values are changed, and I can go from there. Next demo. So I can transition
to C. So, in the same ViewPort tool that I showed earlier, I can look at a C project
and this is the standard code blocks project file. So, I can change the "Compile to" options,
the "Link for" options, Environment options. And, I can look at the standard NCC that I
can compile for the same robot. And this–this is very simple, it blinks the back LAD on
here, so, there's a little LAD. Right now, it's blinking very, very fast but also for
the NAC part, there is a compiler–a debugger. And so, I can compile up with debugging information.
And now, in the C compiler--C debugger, I can send a break point and run up to that
point and keep stepping around here. And now, you can see the LAD is blinking every time
I go pass that instruction. So far, we've seen at the very high level using a visual
programming language and using--and interfacing with the sensors, a higher level, which is
the Spin language, and then I'll going all the way down to C code, and stepping line
at a time to blink a LAD. Okay. Okay. Now, shared memory requires two robots, so I'll
have two robots running here. And, what I can do here is I can open a single program.
So, this is a slightly longer program but it's still usually fits on the page. And what
this program does is it runs on two robots and uses a shared memory structure that I
can monitor with 12Blocks to set the LAD color on one robot to the proximity sensor reading
of the other robot. So, I'm going to change the proximity value on one to change the color
on the other. And I need to load this into both robots, so I'll start by loading into
one of them. Let's see which one. First, I have to...okay. Both robots are on.
So, this robot on the right now, I can look at what the sensor reading is doing here.
And what's--what this one is doing is in I3, it's putting the value of the proximity sensor.
So, you can see that it's going from 30ish, when I'm close to it to a 180 or so when I'm
further away, you out-graph it so we can look at it. So, you can see on the graph that as
I'm moving my hand back and forth, that's the sensor value, is changing. And this is--the
sensor reading is put into the shared memory of I3. So, I3 is where this one is putting
its reading. I can now connect to the other robot. That was–and load this robot with
the same program, but that robot is looking at I3 to set its LAD and storing its sensor
reading into I0. Okay. So, look at both I0 and I3. And now, if you look at the other
robots as I'm moving the--I saw them, you know, disconnect the--so, I make this one
truly wireless, and so, what's going on is it's taking readings with the proximity sensor,
and I can look at those readings on my graph over here. So, you'll see the red line goes
up and down as I'm moving my hand back and forth over here, and the other robot is receiving
that information, and using that to change its state. So, we're taking readings from
one and affecting the behavior of the other, and it goes both ways so I can change the
reading over here to change the LAD over here. Okay. So, that's what I was here to show,
12Blocks going all the way from starting a program, adding a move block, moving the robot
all the way to using shared memory to control a robot remotely with wireless and looking
at the result on the PC. Any questions? >> So, I noticed your blocks was--looked a lot
like a flow chart rather than a dataflow program. Did you think about a data flow model at all?
>> So, there are many different visual ways of creating a program. This one is still somewhat
linear, so it starts at the top and goes down. It is in two-dimensions so if--so some of
these programs that I have or I can sort of show you so, of course, you have two dimensions
and so you can put it together. This is what I thought would be the easiest for a beginner
to get started with reading from top to bottom and naturally reading what's going on. So...
>> So, I have a couple of questions. One is--I know this is very different from the Arduino
microcontroller but could you say any comments on how this compares? And with your programming
language, could that be ported to the Arduino? >> SANDER: Yeah, okay. So, yes, at the moment,
everything I showed here used the Parallax Propeller multi-core processor. The nice thing
with that architecture is that it has eight processors in that chip and so I'm using one
of those processors to establish the shared memory connection. The connection is just
a serial interface and there is a documented protocol of how that information is shared
between the two. I currently--so, on this device list over here, I currently support
several robots as well as a simulator, as well as Java for the Lego Mindstorm NXT. So,
already I'm taking this visual programming language and outputting different codes depending
on what device you want to interface with. So, if you want to interface with one of these
robots, then it would output the Spin code that runs natively on the device. If you want
to run in a Python simulator then the same program that you have here on screen, can
be saved and run inside of a Python environment, or it can be saved as Java code. So, you're
just mapping blocks to the native language that you want to support. So, in terms of
Arduino support, I don't have it right now, but the interface--that communication link
is serial so that's straight forward. And, instead of outputting whatever commands I
use here to move the robot back and forth, I would have to or someone else would have
to write the library to do that and map that back to these blocks.
>> Okay. And I notice that you have primitives like setLED, which is different from your
set, you know, variable. Is that setLED--how is that tied to the hardware and...?
>> SANDER: So, there is a block manager under Tools and what this lets you do is map the
blocks and what they look like and how many children they have. So if you're looking at
nested items like an F or repeat and it lets you map those to your language that you want
to support, so if you want to go to spin code what you have is you have object files that
you include and you need to set up some global variables or variables to pass information
and you need to set something up at the beginning of a program and do something in between.
So, when I'm blinking an LED this is driven by i2C. So there's an i2C chip in there and
so when you set the LED to a certain value and the driver that's actually doing that
at the end of the day is written in assembly and doing some bit banging to set the i2C
device to a certain state. Whereas the variable assignment in this particular language is
straightforward like it is MO2setX(21) does not require any other variables, it does not
require any other objects, it's just sitting a variable to one. So, some of these blocks
are very complex and some of them are very simple.
>> And how was the robots where they communicating over Bluetooth or...?
>> SANDER: So, we have different ways that they communicate. Today they're communicating
over XP. It's a $19 solution--different ways and you can have multiple robots up to thousands
and this particular one has a range of about a kilometer and works very nicely 100 to 110
kilobyte per second. And what we did is we developed a very flexible communication tool
that lets you share memory but also send messages back and forth, like, male or your force to
respond and to where you're able to command other robot to do something as well as terminal
emulation so all robots running on the same hardware.
>> Very nice. >> So, I might have missed this but is any
of these for sale or is this just a project right now?
>> SANDER: Getting very close. So, about a month or so away from selling Tbots and they
were initially designed for summer camps for students go to summer camp [INDISTINCT] and
at the end of a camp there's a competition. So, that's what these are built for. 12Blocks
has been for sale I've been selling out for six months now.
>> Okay. >> SANDER: And the props called parallax [INDISTINCT]
and view port I sell that as well. >> Uh-hmm.
>> SANDER: If you go to handover.com then you'll get a link to all of those.
>> Okay, thanks. >> SANDER: Okay, thank you very much.