Tip:
Highlight text to annotate it
X
>> Good morning everyone, thank you for joining us today for the NCI CBIIT Speaker Series,
My name is Tony Kerlavage. I'm the head of the cancer informatics program here at CBIIT.
I'll remind everyone that today's presentation is being recorded and will be available on the wiki for the Speaker Series as a screencast
with voiceover and also posted on the Speaker Series YouTube playlist. You can find that by Googling NCI Speaker Series.
Information about future speakers is also available via Twitter and our blog so you can check those sites for the latest information,
you can Google NCI blog and our Twitter handle is NCI_NCIP.
Today we're quite happy to welcome Bill Elgie, who's the director of IT at the NRG-Buffalo and the Gynecologic Oncology Group
at the Roswell Park Cancer Institute.
Mr. Elgie's 30 year career in the IT field includes over 20 years of directorship in management of computer professionals
in the areas involving cancer and AIDS clinical trials research.
He received a B.A. in computer science in 1984 and an M.B.A. in 1994, both from Canisius College.
The title of today's presentation is "Evolution of Gynecologic Oncology Group Use of caDSR from eCRFs to Rave."
And with that I'll turn the floor over to Bill.
>> Thank you very much, thank you for the honor. I'm going to go through the agenda today, I'll go through a quick background on myself
and the groups that I'm working for, describe the Legacy data entry system that we developed, discuss a little bit...[audio edited]... I'll go through
my background, the data system that we created, discuss a little bit about standardization and RAVE, using the CDE browser,
quick introduction to RAVE so that we can understand where that is coming into play, the solution that we've put in place and then I will
wrap it up with a CDE compliant PDF that we will populate with RAVE data and web services.
My background is 30 years...I can't believe I'm doing this for 30 years,
I came to Roswell Park in 2000 to work with the group formally known as the Gynecologic Oncology Group
and now we are now the NRG Oncology, Buffalo office.
Before coming to Roswell Park, I worked at Frontier Science from '84 to 2000, these and other cooperative group functions.
GOG was organized in 1970, way, way, way back when, they had 11 original institutions and it grew over the years.
We ended up with about 45 GYN clinical trials active at any one time and most recently we
formed with two other cooperative groups to form the NRG Oncology LPO. Brought in NSABP, RTOG and GOG
together and the amount of years that everyone has worked on clinical trials is stunning at times.
The Legacy system that we built at the GOG was based upon Cardiff Software TELEforms package,
now this is a PDF designer where you can put on forms and basic controls and build a very nice user friendly interface, PDF and present it to the user in
a web browser where they can key in data. There are some data checks, range checks that are built within the software and it also came with
an internet server that once once you presented the PDF and saved it would come back and the internet server would process it and pass
that to the database. What we also found was that we needed some more heavy duty checking,
so we employed the active PDFToolkit, that would open up a TELEform designed PDF and allowed us to add even more Javascript to it
to create even more range checking and logic checking within our PDF. A little bit of the background of the things that are behind
the scenes, under the hood, is we have two database systems that we use, the SQL server and Ingres from Computer Associates.
All the permissions were based upon a username/password challenge with CTEP-IAM challenges.
A patient or a user could only be mapped to specific patients based upon roles
that were assigned to that user and the institution that user was coming from. An electronic audit trail was created where we captured every image,
we had the staging table that would capture every transaction and that was the basics of our system.
As always, we start out with registration, we originally had what we would refer to as Web Reg, which was our basic registration system but we
moved all of our protocols to Open and once you register a patient there would be obviously assigned a patient identifier
and that patient identifier then would trigger what would be a patient calendar or a series of events that we would expect.
The system would display this calender for the user, there was color coding so that you knew when you had overdue forms
or forms that were received or forms that were pending. The due dates and then a place of hyperlinks to those PDFs that were created
through TELEforms and obviously you know some printed images of prior submissions and forms instructions were available.
That was our basic set up and what we would end up with would be a eCDE compliant eCRF.
The eCDE compliant I'll get into in a second in a couple sections but those were
where we go in, grab all the eCDE compliant questions and build this particular PDF.
We had the ability to preload data of the user, preload drugs for the specific protocol and do some validation checking.
So that was all the system that we had created and then we started to move into the realms of forms and ECRF standardizations that would come
across all the groups and moving towards a unified data entry system called RAVE.
This process has been in the works for many, many years and this one I grabbed just as a flow of how things have been flowing.
We're into Round 5 now, the standardization of all of the forms, while these are the standards we've found that the LPOs aren't really using
each one of these and some of these are mandatory and there's many, many working groups trying to figure out how we're going to
employ and take all of these CRFs and put those into RAVE and coming up with the mandatory fields that have to be completed and the optionals, the
valid values so we're working towards implementing this across all of the groups. We have found that as we're beginning to come
together through our working groups in Rave that there was little standardization
and use of caDSRs across all of the cooperative groups or the LPOs. LPO obviously is the lead protocol organization;
I'm so used to calling us cooperative groups that I may flip back and forth between those two.
The implementation of Medidata Rave across all the LPOs is moving us towards more compliance with eCRFs
and more use and a greater use of the CDE database.
Once the Rave was obtained and purchased we created with the help of CTSU
and various other people a number of working groups to come up with standards
of not only using eCRFs but how to use Rave to the best of our ability.
To take this data entry system and apply it uniformly across all of the LPOs.
Three working groups in particular were really pivotal in coming up with use of caDSR and CDE compliance.
The data element working group, we haven't met in a while but this was one that really spearheaded the idea
that I'm going to present. The study build working group that's where we're going to come
up with how we're going to use RAVE and how we're going to use RAVE in a uniform method.
Now the options that we were presented because we were trying
to implement these new eCRFs and the caDSR how we were going to do it
so a couple of the options of the LPOs being able to retain their way
of collecting specific information did not really fall into the caDSR and we were trying to figure out how we were going to continue
to collect information that was not within the caDSR or the CDE but our cooperative groups, our institutions had become so familiar
with how we designed forms and how we added questions we had to come up with,
how we were going to deal with using eCRFs and still collecting information on our protocols.
Came up with three options to modify the caDSR data base structure to accommodate the LPOs needs, modify the RAVE software
or come up with a solution based by the...from the LPOs.
Obviously the first one, the amount of effort to modify the caDSR database, there was just no way that was going
to happen. Plus there are other entities outside the LPOs that use the caDSR and there's no way that we would be able to change that.
The idea of moving and changing the Medidata Rave software, that was just not
within the contract and here again there's other people besides the LPOs that were using Rave and to change that is just not going to happen.
So the LPOs, through those working groups, we identified two options.
One is strictly use only the NCI eCFR standard modules. Now, there's a lot of debate about that and that was something that we just felt
that through the working groups and through all of the discussion we needed to come up with a different solution.
So the idea was to be able to capture and collect the information that we would normally create on our own data entry systems but then be able
to take that data and put it into a standardized format where NCI and CTEP can know where to go grab the data.
Once when you have the data within Rave there are standard naming conventions
that you use and you can...once you pull the data back to a source you can pull out those specific items.
So the idea was to come up with a method of how the LPOs can take the data and put it over to where it is properly formatted for governmental
reporting. So the idea is take the current structure, take what you can from Rave, use the best functions from Rave that you can,
take the LPO data that we already have and that we are very familiar with, merge it with the NCI data and create custom functions
to build the new forms and the new folders. Custom functions is a Rave module or terminology, that is programs
that are written in C Sharp, in our world, to run within real time of Rave.
Now we're going to just quickly jump into the CDE browser to see
where the standard questions and answers are coming from.
Within our organization what we've always done is we've had a protocol team
neighbors they were assigned to deal with the protocol,
it consisted of various members of the statistical staff,
clinical data coordinators, IT, translational research scientists,
[inaudible] we would all get together and discuss the protocol.
Once the requirements of the protocol are determined the forms
and schedules decided upon by the form builder and the curator,
the clinical data coordinator would drive the preparation of forms
and our curator would use the CDE browser to find out and search through forms
that were already existing or for new questions and then for the requirements
that were brand new, would bring those forward for addition to the caDSR.
Very quickly, CDE is basically you have a question that is associated
with answers of a very specific type and they are only permitted values
so you have questions, answers and those answers are predefined and they're in the database in the catalog.
Now to drill into the CDE browser what we're going to be looking
for is what question are we going to try to match to a specific requirement
on protocol and those values have to be of a certain set.
All of the data within the caDSR and CDE browser you can download one
of the ways of also gathering out this information is through the form builder.
Now to drill into the CDE browser you would go into and find the group
that you are looking for, for us it is the energy oncology group,
we're going to look for a specific protocol, we've gone to 240 and we're going to look at the performance data's field.
By drilling down into the CDE browser you can get right to a specific field
and find out the information about that specific data element.
I picked this one and I'm going to try to use this throughout the presentation as my one of my example fields.
This specific data element has specific values with specific meanings; this is what we would present to the user in order to collect that data on our forms.
That's just bringing up the permissible values. That performance field is actually on this one study...or
on our CDE compliant form it is in the middle of this form and in RAVE it also appears like this.
So while we're trying to build forms I wanted to show you exactly what type
of forms we were looking at, you have on the left you have the PDF
and on the right you have what a form looks like in RAVE. Now I want to go through RAVE really quickly,
RAVE has a number of modules that drive the system and one of them, one of the most important ones is the architect.
That is where you build your matrices, your folders that will contain all the forms that you are going to present
to the user and all the forms have all of the fields that the user will actually key on.
The fields can be dates, integer values and a number of them are the data dictionaries
and this is where it ties back into the CDE browser.
The custom functions within are the driving force, they are the ones that do all the very serious heavy lifting,
being able to take full advantage of medidata RAVE.
When you're looking at RAVE you have to know how to navigate RAVE to get down into the actual form.
There's always the home button, the protocol, the actual institution,
now you're looking at the patient, the folder, the form and the fields.
This is the presentation that would happen to the institution user
where they would be able to jump back to the home button at any point in time
and see all the protocols that they have access to.
Within the protocol some times we have parent institutions
which will have affiliates and so that's why there's a way of drilling
into an institution tab and then the folders apply to the patient
and so the actual CDE data element is the field that is presented to the user for keying.
Then the architect screen you drill down into the folders and then the forms,
this is the view of the data management staff, the IT staff while we're trying
to work with RAVE and create CDE compliant forms.
Here I'm going to concentrate on this cycle patient information,
if you see that there are two forms here the cycle patient information
and the cycle drug information and there's also one on here called the labs
and chemistries, those three fields...three forms were created form the PDF
that I showed you earlier with the performance status on it due to certain ways
of dealing with medidata RAVE we had to break that one form up into three specific forms.
So the performance status is now exists on the cycle patient information field.
Each form within a medidata RAVE will have the name, the label and the format.
The label is the actual presentation of the question to the user and the form OID...the form OID that's actually where the data is actually stored.
Within the cycle patient information form you'll see
that there is a data dictionary associated with the performance status,
this data dictionary is what comes from the CDE browser so we are able
to take this data and put it in here,
additionally the LPO's can have their own data dictionary
for their specific questions and that's where we're going to try
to determine how we're going to take the data dictionaries that the LPO's want
to use and convert those into the expected NCI caDSR browser expected data dictionaries.
Okay our solution we had to come up with a multiple approach here
because it is never just a one to one, can be many to one or across folders.
Now when I say one to one I'm talking about you have one question that's going
to map to another question on the same form
or we can have multiple RAVE forms populate a single NCI form
and we can also create folders to populate across the folders.
The driving of the data is within RAVE you have data dictionaries that's behind
the field that dictates the valid values to capture.
With our approach we've developed a way to map data onto another field with a different dictionary.
We accomplished this through using custom functions written in C sharp
and they're stored in RAVE and they're executed at run time.
The data can be on a completely different form and a completely different dictionary.
The field that you see here is the cycle patient information,
the field that we want to map it to is the NCI registration form at the bottom
and you'll see that on this NCI registration form there is also a field called performance status.
The dictionary names are different.
What we are trying to take from the cycle patient information that we presented to the user, have them key the data and then transfer
that to an NCI registration form.
The way that we've done this is we have an underlying mapping dictionary.
This mapping dictionary is a RAVE data dictionary that exists within the protocol instance.
The LPO dictionary is at the top, the NCI dictionary is at the bottom
and the mapping dictionary is how we transpose the data and so we will derive
from the LPO using this data and place it into the NCI dictionary.
Here we have multiple fields that are on multiple forms that want
to go...that we're going to direct into a single NCI form.
We're able to do this by creating a data dictionary layer
that not only defines what field goes to what field but by tagging
and coding the actual form we're able to take from one field and map it to the form.
Here you'll see that we're going to take the data that's we've coded as zero,
this reg NCI 1 we're going to take these two fields and map it to there and then we're going to be able to map to 1.
By creating a data dictionary where you not only take and map the values
but you also create other data dictionaries where you have the fields defined
to go to a specific form, so you have your source and you have your destination folders.
The foldering option is one where it's not really something
that we were...well hang on a second I'll show you that in second, what we've done in this example is that we have
at the patient root level there is a baseline folder but there is no NCI folder,
what we're going to do that we have a custom function that will fire
when the performance status is loaded by the LPO,
the custom function understands that this data will then trigger the need
for an NCI folder to be created for the patient and what happens
when we create the NCI folder is that we will take and copy the baseline folders
that are applicable to the NCI folder and place those into the baseline folder.
So we've come up with a way to actually go and create a folder within a folder.
Now this is not a recommended method for medidata but we're working with them
on this and it's definitely worth exploring because this way we can create an NCI folder
and populate that with all the necessary standardized eCRF's that are required
for a given protocol then have them located in one,
one region and that way any reporting that needs to be done can be isolated
into a single folder with a copy of all the information that is necessary.
The real, the real basics of this is that you have a data dictionary
that has the LPO values and the LPO questions that we are used to using
and you also have the data dictionary of the NCI, the eCRF standards, caDSR,
you link those two data dictionaries, fire that off with an edit check
on the LPO form and that LPO form calls custom function...that custom function
then populates the necessary fields for the NCI CTEP reporting.
This is a kind of a benign slide but I just wanted to show you that it is not a, once the custom function is placed within the build,
there is not a lot of IT work that has to go through that the...real quickly just to summarize the approaches is
that if you have the LPO and the NCI question and answer on the same form,
you can hide the NCI question and populate it from the LPO form, that way you have on one form,
you have the questions that the LPO has always presented to the user
and then that populates the NCI the corresponding answer to the question and it remains on the exact same form.
You can have a eCRF that is a standardized NCI form where the LPO may have
that form spread across multiple forms just from historical reasons
or the way that you are trying to work with institution used to key data so that you have multiple LPO questions
and you need to populate a single NCI form or vice-versa.
Multiple forms go into a single form and then the folder structure is something
that we're going to continue to explore to be able to populate a NCI eCRF
with all the standardized eCRF's for this patient in an NCI folder
and then copy all the LPO information and place it into a single location.
Couple of the caveats for this is that while the custom function is easily copied across all the LPOs because the code is very easy to share.
Any of the data dictionaries, the folder names that we would use to create a NCI specific folder has to be followed to a T so the conventions
of naming these must remain the same across all the LPO's
and how to use the data dictionary, the source and the destination in the user
and in the code of those has to remain the same in order for the custom function to run.
But once those are standardized that custom function can be moved...can be used across anybody.
This will require the study builders and not IT staff to maintain,
so you can have your study builders without having to have to deal with any of the IT staff.
This also has a great benefit is that once when you create these data dictionaries you can copy them from trial to trial.
Once a LPO sets it up for one trial they can copy it into what's called a global library of RAVE and then transport
that from one trial to the other without having to have to re-code anything.
And finally there's one last item that I would like to show you
and that is the ability to take a standardized eCRF of PDF and then populate
that with data collected in RAVE and then make that available to the RAVE user for printing.
The institution staff, the flow of this is the institution staff log in to RAVE,
they'll key their data, upon submission a web service pushes that back
to the data center, that data is then populated onto to a PDF
and that PDF is then pushed back onto that same form that has a hyperlink for the user to click on it and be able to print it.
Now we've used this on our translational research form, this is our manifest,
our way of shipping specimens for every specimen that is collected
and shipped to a tissue bank it correspond with a trasmittal form has to go
with that specimen and collecting that data in RAVE is fairly easy
but trying to populate a single form so the institution user can ship it is next
to impossible with all the information that we're collecting,
it would take two to three pages of printed form and the printout is not user friendly.
So what we've decided upon doing is we've created this process
where a specimen consent form exists within our base line,
there's three standard questions to confirm that the patient has acknowledged
and is willing to participate with [inaudible] specimens.
Once that form is submitted that creates a translational research folder
and within that translational folder are our transmittal forms.
These are the mandatory transmittal forms that we're requiring for this particular protocol.
We allow some optional once, the user will then tell us which forms they have collected
and if they don't collect a mandatory one why not then every...upon the save every...we populate the transmittal forms and present this to the user.
The user will then key the RAVE form, all of these are CDE complaint questions,
once they complete the form that is then pushed back to the data center
where we take this data that they've keyed and push it back onto a PDF
and we push that PDF back into RAVE for that patient,
this way the user can then click on the link for the PDF and be presented
with a transmittal form that was keyed in RAVE to CDE compliant questions pushed
to us presented as a PDF and then the shipping goes and life is good.
So that's all that I have for you today, thank you very much for your time.
>> Great, thanks, so let's thank Mr. Elgie for a very informative presentation.
[ Applause ]
At this time, we'll open the floor to questions so if you are in the room please use the microphone that's on your desk,
there's a push button there to turn that on and off and if you're one the WebEx please indicate by raising your hand on the dashboard
and we will unmute your line
so [inaudible] are there any questions here in the room?
>> I've got one, this is Christo Andonyadis with CBIIT and I think we've been on some of those working group calls together.
>> Yeah. >> I was wondering early on in your slides you had some options that talked
about modifying caDSR structure and then modifying you know RAVE software
and I guess my question was about that modifying the structure
and my understanding or at least my take on those discussions was more
about the content, not the structure so was there some structural items that need...?
>> No, no I may have just, I think I may have just mistyped but no all of those were...you're correct.
>> It's about the content not the...
>> More of the content, right.
Though there is as you mentioned there was that one form that you had
to break apart into three pieces and that is actually a structural issue
with Medidata RAVE that it does not support something that we have done
in the caDSR quite often which is a form that has multiple modules in it
and medidata RAVE doesn't support that notion so in order
to make it make sense you have to break it into three smaller forms in RAVE
and just wanted to clarify or get confirmation on that.
>> Yeah, I didn't know how much to go into you know standard fields
versus log fields and only being able to have one log field on a form,
that was the limitation of RAVE is that you can only have one log field
with multiple values within it but only one log field per form and that's where we had to split up our forms.
We've actually had to split up one form into five because of that limitation.
>> And just for those who may be familiar with the caDSR
and how it's forms are built we have a notion of repeating question groups
or repeating modules and again in our caDSR we don't have a limitation so you could build a form that had 5, 6,
10 probably infinite number of repeating groups of questions which again
as Bill was saying, in RAVE you can only have one of those on each form so that's led to this sort of splitting apart.
>> Yeah, that web service that I referred to at the end with the TR from we're looking into the idea of being able to bring back
and stitch together multiple forms onto a single form so that we can have back the single page from 5 RAVE forms can build back on it
and stitch into one single PDF, those are the types of things that we're trying to research and figure out how to do.
>> Hi Bill, this is Jose over here at CBIIT,
for of all thanks for the presentation I think it's a great presentation, I just wanted to clarify a couple points.
One...if I heard you correctly once the custom functions are created
and assuming people adhere to the standards that you've laid
out in those custom functions they can actually be shared with other members
of the LPO so they don't have to recreate what you've already done.
>> Right.
>> And then the other item which I don't know if other people got was that the data then actually gets stored twice, one in an LPO friendly manner
where statisticians are seeing the data represented the way they're used to seeing it.
But also in an NCI specific manner so again it's stored as a standard
so it can be shared across and compared across studies in which ever way ultimately needs to be done.
>> Right.
[ Background noise ]
>> Bill if I could ask how many trials are actually now using RAVE, using this process that you're engaged with in the GOG
or I should say in the NRG?
>> Right now I'm going to just refer to the GYN portion of it,
I know that we have 5 trials that are in RAVE right now,
the actual use of the first process that I was talking about being able to transfer from one folder to another for NCI,
we haven't implemented that on any of the trials we're still...this is a we're in like a proposal to the [inaudible] committee
to adopt this as a standard process.
We've come up with the ability to do this but we still are a little ways away
from figuring out how we're exactly going to report it.
From our standpoint we pretty much use you know CDE compliant forms, the...for us it was not a very hard transition
since we were always using the CDE browser, this was more something that we came up with where we were trying to help the other LPO's with their
problems. So we haven't put it into place yet but we're hopeful that we will move this along.
>> What's been the early feedback from your community of users, have you gathered input on this process and how it impacts them,
and have they seen this as an improvement, as a step forward? Or they see particular hurdles in using this process?
>> Right now, like I said, we're still waiting to get back into the committee meetings, we've had some delays in getting back together
but as soon as we can get this out to people we're willing to open it up, discuss it.
We've presented this to the LPOs on some conference calls before, feedback was very positive but we still have,
like I said, we still have to get things rolling. We also have...what we've put forward is being able to determine query free,
but that digresses into a whole different set of issues.
What we really need to do is to get back in and get the meetings started again and create these standards.
>> Great, thanks, terrific.
>> Okay I'm not taking any other questions here in the room, anything on line?
>> We have...
>> Tony this is Diane.
>> Okay, go ahead Diane.
>> So Bill, that was wonderful, thank you. I'm one of the co-leads of the DWG, we have postponed our meetings for a number
of months while NCI looks at the various options but I just want to point
out that Bill and his colleagues have been extremely collaborative in showing this kind of approach.
I can't speak for the other LPOs about their uptake of this approach,
it certainly has been greeted positively in our DWG meetings, he's also shown it in a number of other settings,
you know they have shared their time very unselfishly with a number
of other groups looking at this solution so we are anxious to move forward with this in hopefully a very short period of time,
thank you Bill that was very nice.
>> And I didn't even have to pay you to say that.
>> No you didn't at all.
>> Okay so we have one more question, Andrea.
>> Yeah, hi this is Andrea [inaudible] from CTEP,
I'm sorry I couldn't get on to this WebEx right when it started
because I had another conflict so I don't know if this was already covered
but I was interested in the mapping solution with the data dictionary.
In CTEP even during our protocol reviews and we were reviewing more requests for certain types of data and will continue to do so in the future
with our new NCTN network and people wanting to analyze samples
and have clinical data and so with this potential mapping and data dictionary
and you talked about working with other groups.
If there was an investigator in one of the groups that wanted to look across disease for certain clinical features
with some genomic information is this type of how you have the mapping from your LPO, with your data dictionary to NCI something that you're talking
with the other groups so they could all do this, so that would enhance that ability?
Or maybe I'm not asking the question correctly.
>> If I can follow you correctly it would be as long
as all the LPOs populated the OIDS that you are trying
to find you would be able to collect from every RAVE instance a single OID and then collect that in a larger repository
so that everybody's one OID would be in one spot where you could take a look at it, now how you would populate
that if the LPOs were collecting it all a little bit differently as long as they transfer it into a single common standard location,
I would think that would be possible.
>> Andrea, this is Jose, I mean basically what Bill is suggesting if it's done correctly, actually disambiguates the data
so we can do exactly what you articulated. I mean without doing this there's always the question
of when you collect those data having to transform them or what does that transformation look like so you know what you are comparing.
By allowing everything to be transformed at the very beginning into a common standard then the data is completely disambiguated
and eventually we would be able to do a cross study analysis, something that I don't think we are able to do right now very easily.
>> So what we've done with our five studies is that we pull all the data back
to the data center and we store it in a single data base so that we can go across our studies and find a specific data item
that may exist across all five studies.
That's fairly easy to do once we pull everything out because you can't go
across when you're in RAVE but RAVE has this web service where it's very easy to pull the data down and use it across studies.
>> And I'm sorry this is Andrea again so you've presented this to all the other LPO's.
>> The idea of pulling data down or the process that I've displayed them
or showed them today because the one that I've shown today I know
that we've done this on a number of calls, we also did this in Boston at the FCT but as far as the ability of how to pull data back
to the data center I think all the LPOs have discussed it in various forms on different conference calls.
>> And how about the mapping with the...your data dictionary mapping to [inaudible].
>> Yeah that I've shown on different webinars and at the FCT.
>> And how has that been received by the other NCTN groups?
>> Quite well I think Diane spoke to that earlier.
>> Thanks.
>> You're welcome.
>> I think we have one more question from Ravi
but he doesn't seem to be connected via phone.
>> Ravi are you there?
>> Can you type us your question please [inaudible] nothing.
>> Okay we're unable to connect...
>> Well, Ravi knows how to get a hold of me.
>> Yeah, there you go so great, thanks, it looks like there are no further questions so I'm going to remind everyone
that the Speaker Series is going to be taking a hiatus for the month of August, but we hope you can join us for our next presentation which will be
on September 3, when Jim Cimino will be talking
about BTRIS, the NIH Biomedical Translational Research Information System, so we hope to see you all back then.
Once again thanks everybody who joined today here in the room or on the phone
and once more thanks to Bill Elgie for a very interesting and informative presentation, thanks very much Bill.
>> Thanks for the opportunity.
[ Applause ]
Take care everyone.
>> So long.
>> Bye-bye.