Tip:
Highlight text to annotate it
X
Please stand by for realtime transcript. >> Good morning. I am Karen Trebon from the
challenge .gov team from the General service administration office of Citizen services
and innovative technology. We are happy to have with us today, Clay Johnson. There are
two famous Clay Johnson's and were happy to have one of the them with us today. Our webinar
today is going to cover the screen door -- ScreenDoor. Clay will tell us about his time at the presidential
innovation fellow and how this project came out of (indiscernible). ScreenDoor allows
you to accept and review challenge submissions, participants can ask questions and submit
responses and I think you'll be impressed with a customized submission form. [Introductions]
>> IM Clay Johnson of sunlight labs, not the Clay Johnson that his former deputy director
of OMB which means you guys can just call me Clay. Not Mr. Johnson as some people are
prone to do when they don't know the difference between the two of us. He is also a lot taller
than I am. Although I am very tall. I am Clay and with the Department of better technology,
a company dedicated to give the government great software they love using and we understand
the government. To six months ago, we were federal employees working hard from within
to drive innovation from inside our agencies. We are a unique company and that we have a
strong understanding of the struggles employees have so our innovation is mainly around creating
software that solves those pain points in that people love using. During our fellowship,
I led team RFP easy. This is a system that helps make acquisitions similar -- simpler
--
we had inside of the White House to make this technology widely available to states and
local governments as well as local agencies that want to use it for things other than
acquisitions. If you want to use this kind of technology for things like acquisitions,
right now would be a good time to talk to the current RFP-EZ that is working ***
their fellowship. My personal background that is relevant to this webinar, I also happen
to have a lot of experience managing and running contests both with and for government. Before,
I was at the like -- at the White House, I led (indiscernible) in partnership with (indiscernible)
and before that I helped create apps for America and at for America to which were contests
around open data and dated .gov respectively. I also managed app's for communities. We are
very familiar with the contests base. So familiar that last year I wrote a mini book on how
to run apps contest and even though it is a year old it is still relevant today for
a lot of what we are doing if you are doing something more developer centric. Before we
get completely started with the ScreenDoor demo, I wanted to add that ScreenDoor is 508
compliant. In fact, not only are we 508 compliant but we took the framework called bootstrap
that we used to make ScreenDoor and we made it compliant and released it back to the public.
It was just accepted into the bootstrap library earlier this week and that makes it so that
anybody using bootstrap which represents 1% of the (indiscernible) now can get 2508 compatibility
and compliance very quickly. Let's get started on taking a look at ScreenDoor. This is what
you will see when you click on the new Project button. I want to show you how easy it is
to create a new contest inside of ScreenDoor. Let's dream big. Let's create our own Apollo
project and put a human on the moon and can. We will create that project. Every page has
a two or on a soap in case you get lost. We will take it to work up every element on the
page and what we are expected to do. We see now that we are on the listing and description
page where we want to add a description for the project. I won't take you through the
whole tour because that's my job to two or you through it manually. Now we need to create
an abstract for the contests, this should just be about a paragraph long that's a general
description of the contests. I won't bore you guys with making it up so I will take
text from an existing contests running now. This is the hypertension control challenge.
We will copy and paste that into here and for the project description, we are looking
for a dummy text and I want to show you the compatibility of this so let's copy and paste
this into our system and note that it maintained our headers and bullets and list items and
things like that. We are not actually writing a description of the contest to put a man
on the moon. The next thing we do is note that you can add amendments later to this
if you want to add more attachments. You can add specific file attachments as well if you
need to upload a PDF or a movie file. We have two fields called enable Q&A and question
period, enable Q&A means people will be able to ask questions in public and you can answer
them in public. You can, each contest comes with its own help form. Let's make it not
listed because we don't want this, because this is a sample project, we don't want it
to be in the actual library of available projects right now. Let's click on, update project.
And we now have that information successfully saved. In terms of response preferences, we
want to receive responses. If we turn this off, we need to provide instructions that
say e-mail your responses or meal be your responses. Let's allow for anonymous responses.
Now I will show you the real interesting thing. This is called a form builder. This is how
we build the response form for the contest. All you do to create forms inside a ScreenDoor
to select an entry on the left and it appears on the right. It's a very visual and quick
way to create response forms for challenges. Let me show you how easy that is. First we
create a text area. Click on text. Now it says untitled appear and up here and over
here we can change it and notice if I change it on the left, the field name changes on
the right in real time. Let's ask people for their name. We can add a coaching text if
we want at the bottom. . Let's add another field that asks for people's e-mail address.
Let's create one more field that asks people for their organization. But let's make that
not required. So we know if people are working for a company. Let's go ahead, that is our personal
details part, let's create a section break and call it planned plan to proposal. How people
will actually yet to the moon and back. Let's ask why they would be good at this. We will
create a new. Graph. We will make it large and say, why would you be good at this? Instead
of going back to adding a new field, we will just duplicate this and change the name as
a shortcut. We will say, how are you going to get to the moon? and we will require that
answer. Let's add duplicate that again and we will ask, how are you going to get back?
Since this is about putting a person on the moon, this will not be required. They may
want to stay on the moon and colonize. Let's ask for a date. A date for the launch. Let's
add duplicate that again for a date for the landing. So we know how much time it will
take them. Let's add one other field that asks for a web link to their semantics. That
will validate for a URL. Let's make that large. All of this is saved in real time. That means
there is no clicking on save buttons between the forms. We have created a nice form for
people to view. Let's see whether or not our HTML formatting and form has managed to make
it through. Here is our contest. We created a contest, we are not live so it's not open
for bids yet. We can make it live if we want that this is just a preview. Let's make it
live. Now we are accepting responses. and here is our lives page. and here is where
people can submit responses. This is the form we designed. That's how to create a form.
You can also preview the form live as well. You can do all kinds of things with this form
page so if you want to change form fields you can do that and drop them where we want
them to be. It has all kinds of different things, you can have people upload files if
you like and people can upload files onto the server or videos or an audio clip or blueprints
or something like that. They can add prices and multiple-choice checkbox items and all
of that stuff validates. The one question is, once we get proposals, how do we rate
them? There are a couple of ways you can do that inside of ScreenDoor. The first thing
is each field that is inside has a, ask reviewers to rate this field, checkbox. If you do that,
you can ask your reviewers to review it on any scoring mechanism you could possibly want.
You can do it by stars, you can do it through a number range if you want to assign points
to each individual field, you can get a free response and ask for a qualified response
but let's go ahead and make our required fields here stars based. We will meet the stars based.
Actually the let's make this one red, yellow, green. And let's make this one the same and
this one stars. I'm making this up as I go along. The other way you can do ratings, if
you want people to rate outside of the fill -- outside of the form itself and maybe more
qualitatively is by creating custom ratings fields. I will show you how that works in
the second. Let's go back to our main project dashboard and pretend that we've got some
responses. These are some fake responses. I'm afraid there are people with NASA on the
line so I did not try to create real actual responses because I did not want to get called
out for how to land on the moon. Let's pretend these are our responses for our contest. You
can take a look at all these responses. If you click on one of them, you can see that
we've got our ratings fields down. People can rate like we've asked them to. I created
a one through 100 scoring thing and notes for each of these. These are red yellow and
green checkmarks and stars. That's all it takes in order to rate things and if we wanted
to add a custom ratings field, we can just date overall quality and we will make that
a free response and add that field. Let's get our judges and. In order to get judges
then, we have to create a team of judges and we do that by clicking on, teams. We create
a new team. They will be called judges. They will be reviewers and we have different levels
of permission. You want administrators to have different levels of access than reviewers.
You don't want reviewers to delete or change the contest criteria you just want them to
judge the submissions. We will click on, create team. and then we add people's e-mails addresses
in here by copying and pasting. I will add myself. to show you again the responses, we
created a customer response field and now the overall quality thing is down there. As
another field that people can review outside of the basic response. the other thing you
can do is, it you can dismiss bids for being crazy or quickly, accept button and if you
accept, the person who is bidding is notified that they win so be careful of the except
button. You can also label things. You can also label things. (indiscernible) thinks
she can launch in 2013 by October, that does not seem realistic so I will checkbox that
and put it under wacky or I will go over and take a look at the good ones. Now (indiscernible)
is labeled as wacky and let's show you again, how we label Jim as wacky as well. All kinds
of other things and you can create you're own labels. So we can go, totally pragmatic
and create a new label with the color purple and now we have a totally pragmatic label.
You can also add whatever one of the columns you want into your reviewing list and is reviewing
with can handle hundreds of reviews. You can pull out and do searches in real time. You
can type in a name and get only those names out and you can add an overall rating quality
average rating since nobody has entered that. You don't see that since nobody has entered
it but if we go down here and enter, this sounds great, then you can find in the scroll
down less. If these were stars, they would show as average. You can then sort by those
ratings so you can see that Jane Smith is the one rated highest for launch date. It
allows you to sort through, filter and get to the right responses at the right time.
One other thing is we are constantly experimenting with this product. So we have this lab functionality.
You can turn on a reviewer leaderboard. This, makes a little leaderboard tab pop-up and
that show is -- shows us a leaderboard form. the one thing this system does not have right
now although we are working on it should be around in the October, November timeframe,
as we take ScreenDoor and specialize it into, we are taking screen do -- ScreenDoor and
splitting it into two different products as we exit our data period in late October early
November there will be public voting but for now there is not public voting on contest.
Only suitable for certain closed judge panel type voting. The whole system has an API built
in so you can extract data out. The one thing about us, it's important to us from my background,
it's important to us that we are fireable. There are no long-term contracts with us.
There are no data lock-in. You can export all of your data at any time and walk away
with it. And go and imported into somebody else's system. You just get a plain CSB not
a proprietary format somebody else has to beef up. All of your data belongs to you and
it is cost free to get your data out of your system. We will not charge you a dime for
that. In fact, right now were not charging you a dime for anything. We have a free beta
period open between now and October when the real contest period ends. If you run a contest,
start a contest between now and October, during that beta the beta period then it will be
free until your contest is over. I'd love to hear more about how long your contest ends.
I don't want to maintain a server for the next decade or something like that but for
the most part, we are looking to collect feedback and get ideas for how we can make a platform
work better for you and how we can ease more plain -- pain points for you. One thing that
has been brought up a lot, is does (indiscernible) and the paperwork reduction act, it does that
give there blessing first off like this? My answer to that is, and agencies relationship
with (indiscernible) is between the agency and (indiscernible). But we are working on
making it so that this form builder also generates real-time burden estimations that are accurate
and fills out paperwork for you so that at the very least you can generate, should you're
agency and council decide you need clearance from (indiscernible) to create forms like
this, at least that paperwork is done for you taste on real data coming from the form.
It should be stuff that (indiscernible) really liked and they are working or talking with
(indiscernible) about how to make it easier but we're probably not going to have the software
level create restrictions on what it is that agencies can and cannot ask because we think
that's between the agency and (indiscernible) on a case-by-case basis, not for us to decide.
That's it. That is the demo of ScreenDoor. Now I think I would like to open up for questions
and answers if that's okay for you guys. I hope you enjoyed the demo. We are free and
willing and eager to help you out and learn what we can. We are really experienced are
running contests and we think we've developed a tool that solves many problems inside of
government for creating contest. Thanks a lot for your time and attention.
>> Thank you. That was a great demo. We do have some questions. We might've missed it
but what are the photo and video hosting options for contest we maybe if you could address
how somebody would submit a photo or video and a challenger contest?
>> Sure. It depends on how you want to collect that information. Often times for instance
when people are submitting videos, they are not really submitting a video file they are
sometimes submitting a link to YouTube or two (indiscernible) are one of the different
online video providers. In that case, you just use the website button and ask for it
YouTube URL and they can copy and paste that in there. As well, if somebody wants to upload
a photograph you could ask for a Flickr URL or if you want people to upload videos it
you can ask for people to use this file thing and they can type and enter or upload your
photo. Either of those things would work. We don't have the capacity right now to create
in-line viewing and the reviewing interface for the videos although it is something we
will take note of and I think that's a neat feature we could add is recognizing a YouTube
URL and playing the interface. Now that you've given us that idea, I'm excited about it.
>> On (indiscernible) you will see that some of the challenges the agency recorded a kickoff
video with her secretary or something like that. Or they have a photo or an image that
is representative of what the contest is all about. Does ScreenDoor allow for photos and
videos? >> a couple of things. You are able to embed,
copy and paste the YouTube embed inside of this and added to the editor and the project
description which would show that. The other thing that I want to add to that is we don't
have it now, this is rolling out I think on Friday so that's tomorrow. One thing that
has been very important is for people to be able to collect analytics on visitors and
people viewing their contest description pages. So we will add the ability for you to add
a Google analytics account to this and then you can embed code into the contest pages
and you can track entrants into the contest. We want you to have as much data about what's
going on and access to as much data about what's going on as you would have if you are
hosting this internal eight with you're own IT team.
>> You mention anonymous submissions, it doesn't seem like that would be something you would
use in a case where there was a prize. >> Anonymous is not the right word to use.
We are probably using the wrong language there. By anonymous responses, we don't mean that
people don't have identity attached to their submission, we mean they don't have accounts
inside of ScreenDoor. If anonymous responses were off and a person had registered with
ScreenDoor, then we would send you all of the information that they have the inside
the ScreenDoor accounts. But we don't want to put agencies in the position of driving
up our user base for the sake of their accounts. So instead, a way to do that is to allow for
anonymous responses and ask people for their information inside of the form builder. So
if that was turned off, really what we should do is change this to say account list responses
rather than anonymous. There can be anonymous responses but it depends on how you build
the form. If you don't ask this information then you would have anonymous responses or
at these fields were not required, you would have anonymous responses. But I can't imagine
anybody would ever run a contest or enter a contest without the ability to win the contest.
>> a two-part question, one of his questions you addressed -- public voting not available
quite yet but coming in the future right? >> Yes, it is coming late September early
October timeframe. We are taking ScreenDoor as we know it. We found that we can't create
a platform that speaks the language of contest and speaks the language of RFPs at the same
time and they are not the same customers. You never want to have public voting on an
RFP. You never want to have (indiscernible) Incorporated by reference inside a contest
description. So we need to split off these two projects. Or you knowing a specific ScreenDoor
RFP product and a specific ScreenDoor contest product. Those will be launched in October
and when we launch them, that's when our -- when public voting will see the light of day.
>> He is also asking about what do you think about using ScreenDoor for a open-ended idea
gathering exercises as opposed to a challenge that has a start and end date?
>> ScreenDoor is great at all kinds of sourcing. ScreenDoor has been used presently for lots
of things that don't have to do with anything to do with either contest or per Chairman.
There is one federal agency out there that is using it to have people, have hosed supply
to host a fellowship program at another that is having fellows apply to those hosts to
be excepted into the fellowship programs. They are using it for any kind of sourcing.
There's all kinds of different ways you can use ScreenDoor. I think it is an excellent
tool for lots of us. As we move forward, I think you are going to see ScreenDoor split
into different specific kinds of technologies. Because it is very hard to develop a technology
that pleases everyone and all instances. We are probably going to have three products
by the end of this year. ScreenDoor for RFP, ScreenDoor for contest, and ScreenDoor for
a generic sourcing platform that's mainly used for things like ideation. If you can
get past the language of amendments and RFPs and stuff like that, ScreenDoor right now
is a great idea for sourcing stuff. It's great looking and easy to use software for getting
that stuff. Think about it this way, anytime any time you're asking people something and
the management of it is through Microsoft outlook or through your Gmail account or something
like that, this is a different kind of box for you to process this information that is
more collaborative, that you can have multiple people collaborating with you on say you can
make comments on the inside and outside of every kind of project. Who you can speak to
one another and get step out of your thoughts and into a place where people can make selections
quickly
and easily. >> Can you come to federal agencies when you're
in town for demos or do them online? >> I am more than happy to. I am in DC about
once a month or once every two months. I have a one -year-old and during the fellowship,
my wife was at home alone with a one week old so I'm trying to be a good spouse in sort
of be in town so that I don't end up getting divorced. Or murdered. I'm up there about
once a week, I'm sorry once a month and sometimes twice a month. I will be up there on Monday
and have a little bit of free time if somebody would like to schedule something with me.
You can e-mail me. If this Monday does not work out, we can come up with another time
to chat. I can also do screen cast like this. >> Can you give us a story about a successful
challenging prize competition run on this platform?
>> Not yet. I can do that yet. We are about two months old so we don't have, what we are
looking for first customers now. We are looking for beta testers who want to try something
and see how it works. So if you are looking for something that has a lot of agency experience,
they were probably not the right product that I can give you a story of the software which
is based on our XP easy and what we did for RFP-EZ is we saw our software resulted in
a 30% reduction in average bid coming in through Turkey are met. Through -- procurement. This
was a big success and something worth exploring whether it has applications in other ends.
While I can't give you, the software right now can't give you a good experience or a
good track record, you will not find many individuals with more of a track record than
I have been running and delivering successful apps contest. Hopefully I am able to personally
make up for what our software history is unable presently to provide because we want for our
initial customers to succeed. As a customer of ScreenDoor, you get me as a resource as
well. >> Can you give us a short story from apps
from America or one of the challenges you ran previously?
>> Sure. Design for America has created all kinds of amazing things, in particular the
visualizing community health data from the Department of Health and Human Services. This
was the seed that created the (indiscernible). If you ask (indiscernible) where the idea
came from it came from visualizing community health data of design from America. We partnered
on this and we ran it and now as a result of running this, that little nexus of community
which a lot of people may have gone to that first date at (indiscernible) the top part
announced the results of health data challenges. That came from this and those results were
done there. Many of these links are sadly now dead but the amazing results came out
of design for America. Apps for America to which was focused around data .gov, created
a network of engineers interested in hacking. That's where the code for America got conceived
of. That was inside the apps for America to challenge for the community. The apps They
apps for community challenge, I wonder if they have some of their entries up. This was
exciting to me, the text to work application which allowed people to submit applications
and resumes directly through text messages so they could store the message and store
the resume online and text the number to send the resume to a job they were interested in.
While I have not seen an SMS short code outside of the waffle House down the street, I think
it has a term in this amount of business potential for some of the lower-level jobs. I think
there are lot of great stories around contest and the best way to investigate it is talk
to the FCC and asked them if they felt like apps for communities was successful. There
are a lot of great stories out there. The stories I liked are the ones I can point to
that are five years old like the ones from design for America that we are still seeing
ripple effects now and those are the kinds of contests I like to see, the ones that don't
just award something but the ones that create communities that are empowered and in the
long-term create the facts you could not imagine seeing. It's very rewarding work.
>> Those are good things for all of us to keep in mind. We have another question, relating
to regulations. What questions do you get about personally identifiable information
and are restoring that and are we in violation of anything?
>> You are leveraging a service which is ScreenDoor so the federal agency is not storing anything.
You have to look at engaging with ScreenDoor more like engaging with social media like
Facebook or toward her or ideas scale than you would look at it like a traditional contractor.
Speaking of that, we do have terms of service, our terms of service are straight nearly copy
and pasted from GSA so we have the amendment that GSA wants and that has been hammered
out about a million times from various other vendors. We hope to be as compliant with what
the federal government needs in order to do business with us as humanly possible and we
will work with you and your agency to figure out what the right way of dealing with us
is. But other agencies have found success and work with us under the auspices but engaging
as a social media platform not through a traditional high-end vendor relationship. Part of that
is because our pricing is quite low and it's going to be quite low in the future.
>> That's a good segue into a question from Josh about approximate costs.
>> Right now, our cost for the beta period which ends in October, we don't have anything
in writing that we can put something in writing for you, the beta period will end on October
31. If you get a contest up and running by October 31 it will be free for the life of
the contest for that contest. the general pricing that we are using is well under purchase
card limits. Were not even aiming for simplified acquisition -- we charge on a per active project
basis. So we are trying to figure out a lot of what our pricing is it we need your feedback
for contest platform. We are interested in exploring because contest last and indeterminate
amount often we sometimes get extended, there's not a lot of con current contest or agencies
running 50 different contests at a time. The question really is is it better for us to
just charge a one-time fee to run a contest to ScreenDoor, let's say the one-time fee
is $500 to run a contest or is it better to charge $50 per month or $100 per month to
run a contest. Is it easier for you guys the program managers who want access to this,
how do we make it easy for you to subscribe to the service. Is it easier to pay annually
or pay with a purchase card but what we don't know is whether or not it is easier for you
to pay annually or pay month to month. We'd love your feedback on that.
>> If you have feedback on the pricing, please e-mail Clay. We also have other questions,
(indiscernible) is asking about your opinion on some of the other tools on the market and
what makes this one better? >> I'm not familiar with a lot of other tools
on the market other than (indiscernible). And the challenge posed is a great piece of
software and if your used to using it and continue using it, then do that by all means.
It's my understanding they are headed straight for a developer only market. Which I think
is a smart move for them but it's also an opportunity for us. The other thing that makes
us better other than being broad-based and making it so it's easy for government to run
any kind of contest at once is that it's being run by people who have experience inside of
government so I don't think you'll find many tools or many software developers out there
that are like, let's make it easy for Crete people took (indiscernible) were let's make
sure terms of service are completely compliant with federal guidelines people or let's make
our product by the way compatible. You won't find people as dedicated to making great technology
that is accessible to government than the department of better technology.
>> Also, we are very good-looking and charming. I think the attractiveness of the office is
pretty good. >> Thank you for showing us a picture.
>> We all like your sense of humor and this has been a more fund them of the most. I notice
you are in the judging. You mentioned the judging is hidden from the public is that
correct? >> Right now we don't have public judging.
It is non- transparent judging and you can see who's in the leader that kind of thing.
Or very good for expert panels of judges, not so great for public panels of judges.
>> I was wondering about when you have judges rating and you can do stars or numbers, can
you do two things? Can you have stars and an open box for people to the comments?
>> You sure can. the other thing I should mention is in terms of flexibility, that's
the other thing I've got that nobody else has. If your agency has the money and the
kind of long-term dedication to this kind of thing, then one thing that I should point
out is that the software we are using that this is based on, is open source. That means
we can take that software, we can create you're own ScreenDoor instance under your control
and deploy it on your IT systems for you to have. That's not going to be something that
will fall under the purchase card payment but it's not something that will be to expensive,
you are talking about a 50 to 100,000-dollar expense depending on the customizations you
need and because it's open-source, you can have us take it, install it, put it behind
the IT systems and fire us and have you're own people maintain it however you would like
an add or subtract to it. You get the ultimate level of flexibility. If ScreenDoor does not
work the way you wanted to in particular, you can always invested in your own challenge
platform and have that inside you're agency that works the way you wanted to that works
and is compliant with your agency's regulations. Any kind of customization, if you've got the
budget for it, is available to you. >> I think that is all the questions. All the ones
that I had to. We can probably go ahead and wrap up. I want to say before we sign up -- before
we sign off, the people listening today will get a follow-up e-mail and it will have some
resources, a lake to an evaluation survey, he will have Clay's e-mail address in case
you did not get it earlier and the recording will be up shortly. Please send Clay feedback
on pricing and any other feedback you have an follow-up with him if you would like to
use the tool. If you have follow-up questions for the challenge that -- with the challenge
team in general e-mail them. With that, we will wrap up today's webinar. Thank you all
for listening. >> [Event Concluded]