Tip:
Highlight text to annotate it
X
MANDY WAITE: Last week I had to rename an entity kind name
for Google App Engine Java.
I had to write one time code in my application to duplicate
each entity to the new kind name, delete old instances,
then remove the one time code and re-deploy the application.
Is there an easier way of doing this?
FRED SAUER: I think this is a little
bit of a longer question.
But I think the person who asked it is actually on the
right track in how to do this.
So first let me contrast this with how you would do things
in a relational database world, where you have a fixed
schema for your database.
So if you have, for example, a table in a relational database
that was called Customer and you decide for whatever reason
you want to restructure that and you want to call it
Customer 2, what you would actually have to do is you
would have to declare a maintenance window.
You'd have to take the website down.
And you would then perform the database maintenance.
You would rename that table, which locks the entire table
and all data in it.
You would modify the code in the application server.
You'd bring everything back up.
And maybe half an hour later, maybe a few hours later, you'd
be up and running.
And that was maybe acceptable a number of years ago when
websites regularly went under maintenance.
And a lot of systems were just 8:00 to 5:00.
But now, we live in a world where everyone is connected to
the internet.
It's always 5:00 somewhere.
Everyone needs to get to your site all the time.
And so what you really want is a system where you can make
changes while the system's up and running.
And part of the question actually has the answer here
in how you do this.
So the specific question was, how do I rename an entity?
But you can generalize the question.
What if you wanted to add new required properties to a
particular entity in the data store?
Or you wanted to restructure?
You're going from three separate entities and you're
kind of combining them into this new data structure where
you put everything in one entity.
And these are common refactorings that will happen
with your application.
And so the way to do this is to think about this live
system that's always handling transactions.
And it's actually sort of a three step process.
In the first step, what you want to do is change your
application so that any time that you write out an entity,
you're writing it out in the new format or with the new
entity name, with the new properties, the new
constraints.
But at the same time, that every time you do a read, you
first look for the new kind of entity.
And if you don't find it, you look for the old kind.
And you deploy that.
And as soon as you deploy that, your application will
slowly begin migrating data.
Any user that logs in and touches their data, those
entities, those rows in the data store, will
begin to get migrated.
And now actually, you're in a perfectly happy state.
Your application will continue to function.
There's a little bit of extra read overhead for people as
they're migrating.
But you now have all the time in the world to make the full
migration happen.
You're still serving live traffic.
And now, depending on what you want to do, you can either
say, we'll just leave it like that.
But oftentimes, that extra code there is complexity you
don't want.
And you might want to get rid of that.
And so what you do is then you run what we call a MapReduce.
But you essentially iterate over all the entities of the
old kind in the data store.
And you just touch each one.
You can do that in application logic.
And as soon as you touch each one, it's
migrated to the new structure.
And you can take an hour to do that, a day.
You can take weeks or months, really, however long it takes.
And once everything's been migrated, then you go back
into your application and you just remove that extra code in
the read handler.
And your app's migrated.
And many developers have many of these migrations kind of in
place at once.
They might introduce a few migrations incrementally.
And then maybe once a quarter, do a clean up and get rid of
some of the code.
It's not the case that you have to migrate everything in
a short window.
Think about doing things live and in real time.
MANDY WAITE: Wow, OK.
That's cool.