Tip:
Highlight text to annotate it
X
>> MS. SARAH AXELSON: Welcome to the Pre-Application webinar for the Tribal Personal Responsibility
Education Program or Tribal PREP program. This webinar has been pre-recorded for your access.
My name is Sarah Axelson. I’m a Management Analyst in the Adolescent Pregnancy Prevention
Program within the Family and Youth Services Bureau. I'm also joined today by Kathleen
McCoy, Social Science Research Analyst with the Office of Planning, Research and Evaluation
within the Administration for Children and Families. We're pleased to welcome you to
today’s webinar. The purpose of the webinar is to review the tribal PREP’s Funding Opportunity
Announcement, known as the FOA, including the program design, eligibility requirements
and application instructions. As you can see in the agenda, we’ll also provide a brief
overview of the Administration for Children Youth and Families as well as FYSB before
diving into the specifics of the FOA. The webinar should answer most of the questions
and concerns that you may have as applicants. Because the webinar is pre-recorded, there
will not be an opportunity to ask questions. Additionally, ACF does not provide direct
guidance or instruction in the development of an applicant's project design or in writing
their application. Applicants should use their best judgment in determining whether they're
able to meet the requirements contained in the FOA, whether they're able to develop an
application they believe to be responsive to the FOA and in designing and writing their
application. Applications will be reviewed and evaluated by objective review panels using
the criteria described in Section 5.1 of the FOA. The review panels will use the FOA as
their principal guidance available to them in the same way that it is the principal for
all other applicants. We would like to begin by providing you with a brief overview of
ACYF and FYSB, the administration and bureau that published the tribal PREP FOA. The Administration
on Children Youth and Families, ACYF, is within the Administration for Children and Families.
It administers the major federal programs that support social services that promote
the positive growth and development of children and youth and their families, protective services
and shelter for children and youth in at-risk situations adoption for children with special
needs. These programs provide financial assistance, community-based organizations and academic
institutions to provide services, carry out research and demonstration activities and
undertake training, technical assistance and information dissemination. ACYF houses the
Family and Youth Services Bureau. The mission of FYSB is to support the organizations and
communities that work every day to put an end to youth homelessness, adolescent pregnancy
and domestic violence. FYSB’s vision is a future in which all our nation's youth,
individuals and families, no matter what challenges they may face can live healthy, productive,
violence-free lives. FYSB hopes to achieve this vision by promoting positive outcomes
for children, youth and families by supporting a wide range of comprehensive services and
collaborations at the local, tribal, state and national level. The Adolescent Pregnancy
Prevention program is one of the programs within FYSB. The APP program began in 2010
and currently manages the personal responsibility education program and abstinence education
program, which jointly run the six different funding streams for pregnancy prevention education
that you see listed on the screen. More information on the APP program is available on the APP
website. Next we'll move into a discussion of the tribal PREP’s Funding Opportunity
Announcement. FYSB anticipates awarding ten cooperative agreements through the tribal
PREP FOA for a total amount of $3.436 million. Awards will be a minimum of $300,000 per budget
period and a maximum of $700,000 per budget period, with an average of award of $400,000
per budget period. There is no cost sharing or matching requirement for the Tribal PREP
Program. The project period will be five years with five twelve-months budget periods. Awards
will be made no later than September 30th 2016. It's very important that applicants
meet the eligibility requirements outlined in the FOA. Applicants must fall into one
of two categories of eligibility to be able to receive Tribal PREP Funding. Eligible applicants
are Indian tribes or tribal organizations as defined in Section 4 of the Indian Health
Care Improvement Act. Applicants that are applying as Indian tribes must be acknowledged
by the Bureau of Indian Affairs as federally recognized. For a list of federally recognized
tribes, you can access the Bureau of Indian Affairs web page listed on the slide. For
those applicants that are applying as tribal organizations rather than as federally recognized
tribes, tribal organizations must submit documentation that supports their classification as a recognized
governing body of any federally recognized Indian tribe, any legally established organization
of Indians that is controlled, sanctioned or chartered by such governing bodies or that
is democratically elected by the adult members of the Indian community to be served by such
organizations and that includes the maximum participation of Indians in all phases of
its activities. A tribe that is federally recognized but is not on the Bureau of Indian
Affairs web page or a tribal organization that does not meet the requirements that I
just described above must submit documentation that is signed and dated by an official of
the governing body. Written board resolutions, meeting minutes from the governing body and/or
letters from the authorizing official requesting approval are acceptable documentation, depending
on what is appropriate for the applicant’s governance structure. This documentation must
be included with the applicant’s submission to FYSB. Without FYSB’s receipt, a signed
and dated documentation, the application cannot be reviewed. PREP is authorized and funded
by Section 513 of the Social Security Act. The program was originally funded from FY2010
through FY2014. It was then extended through FY2015 by Section 206 of the protecting access
to Medicare Access of 2014. Section 215 of the Medicare Access and the Children's Health
Insurance Program Reauthorization Act of 2015 extended that funding further through FY2017.
Section 513 CQA of the legislation specifically addresses the tribal program, particularly
the requirement for HHS to consult with tribes and tribal organization to review requirements
for awarding the five percent set aside. Tribal consultation regarding the travel PREP program
was conducted in June through September, 2010 and again in September, 2015. As outlined
in the FOA, the goal of the tribal PREP program is to reduce pregnancy and birth rates and
the spread of STIs for Alaska Natives, American Indian youth populations. Tribal PREP programs
are to accomplish this goal by replicating evidence based programs using culturally and
linguistically appropriate adaptations as necessary, substantially incorporating elements
of effective programs that have been proven on the basis of scientific research to change
behavior or substantially incorporating elements of practices from programs that have demonstrated
effectiveness within AIA and tribal communities. Tribal PREP programs must promote changes
in behaviors associated with pregnancy or STIs through methods such as delaying ***
activity, increasing *** or contraceptive use and reducing the number of partners. The
Tribal PREP program also has several objectives which you see listed here. These include designing
programs that educate adolescents on abstinence and contraception, providing medically accurate,
age appropriate programming that is inclusive and culturally appropriate, using best practices
to replicate or substantially incorporate elements of evidence-based or effective programs
within tribal communities, promoting successful healthy transitions to adulthood through the
implementation of adulthood preparation subjects or APS and targeting youth between the ages
of ten to nineteen who are at high-risk for becoming pregnant or who have special circumstances.
Tribal PREP efforts are dedicated to developing comprehensive, culturally appropriate interventions
and services that rely on best available research to inform and guide practice. Tribal PREP
programs must serve AIA and youth populations. Youth are defined as those who are between
ten to nineteen years old, except in the case of pregnant and parenting youth, which may
include youth under twenty-one. Tribal PREP programs may serve AIA and youth who have
additional risk factors for teen pregnancy or otherwise have special circumstances, including
youth in or aging out of foster care, homeless youth, youth with *** or AIDS, systems involved
youth, rural youth, runaway youth, out of school youth or youth who are at risk of or
have experienced sex trafficking. Finally, Tribal PREP programs must consider the needs
of lesbian, gay, bisexual, transgender and *** or questioning youth, sometimes referred
to as Q spirit youth in tribal context. There are several programmatic components that must
be addressed in tribal PREP programs. These are listed on the slide you’ll see here
and we're going to go through each item individually as well. First, Tribal PREP programs must
include activities to educate AIA and youth on both abstinence and contraception and must
place substantial emphasis on both abstinence and contraception for preventing pregnancy
and STIs among AIA and Indian youth. Second, Tribal PREP programs must address at least
three of six adulthood preparation subjects, or APS, as outlined in the PREP legislation.
The APS subjects include healthy relationships such as positive self-esteem and relationship
dynamics, friendship, dating, romantic involvement, marriage and family interaction, adolescent
development, such as the development of healthy attitudes and values about adolescent growth
and development, body image, racial and ethnic diversity and other related subjects, financial
literacy, such as budgeting, income and financial planning, parent child communications such
as strategies to increase parents’ capacity to talk with their children about sexuality,
education and career success, such as developing skills or employment preparation, job seeking,
independent living, financial self-sufficiency and workplace productivity, healthy life skills
such as goal setting, decision-making, negotiation, communication and interpersonal skills and
stress management. Topics must be tailored and developed in appropriate
ways to reflect the cultural linguistic and regional practices of the tribal communities
that are being served. Third, allow states that PREP programs are legislatively required
to replicate evidence-based effective programs or substantially incorporate elements of effective
programs that have been proven on the basis of rigorous scientific research to change
behavior, which means delaying *** activity, increasing *** or contraceptive use for
sexually active youth or reducing pregnancy among youth. HHS recognizes that to date there
is little empirical research in AIA and communities. To the extent possible, Tribal PREP programs
should replicate evidence-based programs or substantially incorporate elements of effective
programs that have been proven on the basis of rigorous scientific research to change
behavior. Tribal PREP programs may include practices that AIA and communities know to
be effective for delaying *** activity, increasing *** or contraceptive use for
sexually active youth or reducing pregnancy among youth. HHS established through a systematic
review a list of 44 program models that are considered evidence-based and have been found
to be effective in preventing teen pregnancies or births, reducing rates of STIs or reducing
associated *** risk behaviors defined by *** activity, contraceptive use or number
of partners. This list is available at the website that you see listed on the slide.
Tribes may use this list as a guide in the development of their proposed project. However,
they should note that the programs in the systematic review vary in their structure,
lessons and content. They may not adequately meet the PREP requirements to place equal
emphasis on abstinence and contraceptive education. Such programs would require adaptation in
order to meet this PREP requirement. Also, some evidence based programs may need additional
supplementary lessons in order to address at least three adulthood preparation subjects.
FYSB is also aware that of the 44 programs identified as evidence-based, only five were
validated with AIA and youth. Therefore, evidence-based programs may also require adaptations to address
the unique cultural and linguistic needs of AIA and tribal communities. Fourth, Tribal
PREP programs must address fidelity and adaptations. Fidelity refers to how well the program implemented
without compromising core content that is essential for program effectiveness. In general,
adaptations to evidence-based or effective programs should be minimal. However, as mentioned
previously, more significant adaptations may be needed to address several mandatory components
of the Tribal PREP program, including equal emphasis on abstinence and contraception,
the inclusion of at least three adulthood preparation subjects and/or meeting the unique
cultural and linguistic needs of AIA and tribal communities. Even when such adaptations are
made, they should not alter the core of program model. Fifth, Tribal PREP programs must be
medically accurate and age appropriate. Medically accurate and complete programs are verified
or supported by the weight of research conducted in compliance with accepted scientific methods
and published in peer reviewed journals where applicable or comprised of information that
professional organizations and agencies with relevant expertise in the field recognize
as accurate, objective and complete. Tribal PREP programs must provide medically accurate
information, including up-to-date contraceptive information and age appropriate information
and activities. Topics, messages and teaching methods must be suitable to specific age groups
of youth based upon varying capacities of cognitive, emotional and behavioral development.
Sixth, Tribal PREP programs must consider trauma informed approaches to programming.
FYSB recognizes that AIA and youth may have unique service needs related to the history
of trauma in Indian country. The Indian Country Childhood Trauma Center defines trauma with
the AIA and context as a unique individual experience associated with a traumatic event
or enduring condition, which can involve an actual death or other loss, serious injury
or threat to a child's well-being, often related to the cultural trauma, historical trauma
and intergenerational trauma that has accumulated in AIA and communities through centuries of
exposure to racism, warfare, violence and catastrophic incidents. Several distinct forms
of trauma have been identified in Indian country, including cultural trauma, historical trauma,
intergenerational trauma and current trauma. Trauma experiences are significant and prevalent
in the lives of AIA and youth. For example, compared to their non-Indian peers, AIA and
children are 2.5 times more likely to experience trauma. AIA and children experience a rate
of child abuse and neglect of 11.4 per 1,000 children compared to the rate for all children
of 9.1 per 1,000. Alcohol abuse related to child abuse and neglect is more likely to
be reported for AIA and families. Violence is more likely to be reported among AIA and
families, both as an element of abuse and/or neglect and in general. Repeated exposure
to trauma results in the impaired ability to have cognitive relationships, communicate
well and demonstrate resiliency, skills that have a direct connection to adolescent health
behaviors and that are emphasized within PREP programs. Broadly speaking, Tribal PREP programs
should consider trauma informed approaches that may recognize the high prevalence of
trauma in the target populations that they're serving, assess for traumatic histories and
symptoms, recognize and reject practice that are re-traumatizing, place increased emphasis
on the importance of culture, focus on collaboration with caregivers and support systems, address
training needs of staff to improve knowledge and sensitivity, ensure that program staff
understand the function of coping behaviors and use objective neutral language. Lastly,
Tribal PREP programs must provide teenage pregnancy prevention related health care service
referrals to other providers of healthcare services. Examples of such services include
substance abuse, including alcohol, tobacco cessation, family-planning, mental health
issues and intimate partner violence. Programs should also encourage enrolling eligible youth
in health assistance programs such as the Affordable Care Act, funded Medicaid and CHIP
or any other federal or state assistance program for which they may be eligible. Tribal PREP
programs must assist program participants in locating services and making referrals,
but such health services cannot be purchased with Tribal PREP funds. FYSB recognizes the
importance of engaging tribal communities in health planning and programming that affects
AIA and youth. Therefore, Tribal PREP programs will have a project period of five years divided
into two phases. Phase one of the project will begin upon award of the grant and is
expected to be completed no later than June 30, 2017.
Phase one activities include assessing the needs of the identified target community or
communities in the areas of teen pregnancy prevention and youth preparation for adulthood,
developing a program implementation plan that includes a selection of a culturally appropriate
teen pregnancy prevention program and beginning to build capacity to respond to the needs
that are identified in the needs assessment. The needs assessment is to be conducted during
phase one activities during phase one must assess the quality and capacity of existing
teen pregnancy prevention programs, numbers and characteristics of youth and families
who receive services or potentially may receive services, the extent to which these program
models are evidence-based, high quality and meeting the needs of eligible families and
the factors which may limit capacity in providing teen pregnancy prevention and adulthood preparation
subject material. Planning and capacity-building activities during phase one may include identifying
populations to be served on the basis of the needs assessment, building administrative
and management capacity for the program, such as hiring key staff and locating space, strengthening
relationships and developing formal agreements with partners and stakeholders, selecting
evidencebased or effective teen pregnancy prevention programs for implementation, collaborating
with the developers of the selected teen pregnancy prevention program or programs and the community
to make culturally and linguistically appropriate adaptations to the program if needed, establishing
formal agreements with developers and locating and securing partners such as local evaluators
to plan for and participate in ongoing research and program evaluation activities. Programs
will transition from phase one to phase two upon submission and approval of their implementation
plan. In Phase 2, Tribal PREP programs will implement the various components of their
approved implementation plan and work closely with FYSB to ensure that high quality services
are implemented in their selected tribal community. Phase two activities include building infrastructure
to implement effective teen pregnancy prevention programs in the community, providing high
quality Tribal PREP programs to AIA and youth in the selected communities, measuring and
reporting on progress in meeting benchmarks and participating in ongoing performance measurement,
data collection and program evaluation activities that will result in building the knowledge
base around evidence-based teen pregnancy prevention programs for AIA and youth populations.
I’m now going to hand it over to Kathleen McCoy from 0PRE who will discuss performance
measures and evaluation. Kathleen.
>> DR. KATHLEEN McCOY: Thanks, Sarah. So as she said, I’m going to be giving an overview
of performance measures and evaluation. Starting with performance measures, FYSB uses one standard
set of performance measures to collect data from all PREP programs. The purpose of these
performance measures is to support program monitoring, improvement and reporting. It
does not evaluate program effectiveness or the impact of the program. In terms of post-grant
requirements related to performance measures, you need to report performance measures on
a semi-annual basis. You also need to ensure that all staff involved in data collection,
management and reporting activities are adequately trained. There are several categories and
types of performance measures. First, performance measures assess structure, cost and support.
An example is a number of facilitators who receive training, a number of intended program
delivery hours and the target populations. Performance measures also assess participants’
characteristics, perceptions of program experiences and perceptions of program effects, such as
participants’ characteristics in measures of *** risk behavior. They also measure
attendance region dosage. Examples of this are the number of youth served, number of
program hours delivered and the number of youth completing at least 75 percent of the
program. We also are requiring local evaluations to be conducted. In terms of the general requirements
for local evaluations, during phase one, grantees must design and further develop their capacity
to implement a plan for local evaluation to answer one or more grantee-specific evaluation
questions. These local evaluations must be conducted by a local evaluator. It has to
be someone outside of the organization. It needs to be a minimum of ten percent and a
maximum of 25 percent of annual budget for the local evaluation. In terms of more general
information, local evaluations they can either be descriptive or impact evaluations. Descriptive
evaluations, they do not involve a control/comparison group. Whereas, impact evaluations do have
a control/comparison group. At a minimum, grantees must conduct a descriptive evaluation.
Awards that are less than $400,000 can only conduct a descriptive evaluation. Awards that
are $400,000 or more, they can either be descriptive or impact evaluations. All impact local evaluations
must plan to document program implementation as well. Local evaluations may only conduct
one local evaluation. Evaluation questions must drive the local evaluation in its design,
methods, data and analysis. It can focus on the entire Tribal PREP program, the programming
of one or more implementation sites or simply a component of the program. It must be designed
to help inform current and future programming and expand the evidence base. Examples of
evaluation topics are for the following. For example, one may want to look at populations.
An example of this is to describe the populations reached and whether programming addresses
the perceived needs of the target population. Another example of evaluation topics are adaptations.
For example, determining the differential outcomes or impacts of programming that adhere
to standard curricula versus programming with significant adaptations. Another example is
looking at adulthood preparation subjects, APS. An example of this is the impact of adding
specific programming related to APS’s. And lastly, another example is looking at the
overall program effectiveness of the program. Post-reward requirements for local evaluation
are the following. During the planning period, work with local evaluations to develop evaluation
questions and identify evaluation designs. This will require several things, such as
involving collaborating with ACF to refine, improve, design or select assessments and
to potentially pilot and make necessary changes as needed. ACF approval is required prior
to implementation of the evaluation plan. And ACF may work with programs to adjust funding
for evaluation activities based on the final approved plan. If the local evaluation is
not approved by ACF, budget and staff that would have supported the local evaluation
will be directed toward other approved Tribal PREP program activity. After evaluation plans
have been approved, it’s expected that it will be implemented robust evaluations throughout
the grant period, including instruments relevant, data collection, analysis, drafting of the
reports and dissemination. The only undertake evaluation activities with PREP funds that
are described in this FOA. ACF will provide technical assistance throughout the grant
period to help support the local evaluations. Application requirements related to the program
performance evaluation plan are as follows. You must describe the plan for the program
performance evaluation that will contribute to continuous quality improvement. The program
performance evaluation should monitor ongoing processes and the progress towards the goals
and objectives of the projects. And the application should include descriptions of the inputs,
processes and expected outcomes of the funded activities. It’s also required that the
application must be supported by a logic model and must explain how the inputs, processes
and outcomes will be measured and how the resulting information will be used to inform
improvement of funded activities. You must also describe systems and processes that will
support the organization's performance and management's requirement through effective
tracking of performance outcomes. And you must describe potential obstacles for implementing
the program performance evaluation and how those obstacles popsicles will be addressed.
And application requirements related to the funded activities evaluation plan related
to the local evaluations are the following. Such as you must describe the plan for rigorous
evaluation of funded activities. You must describe how the evaluation will assess processes
and progress towards goals and objectives of the project. You must specify expected
outcomes in any research questions. You must discuss how the results of the evaluation
will provide greater understanding and improvement of funded activities. It must include a valid
and reliable measurement plan and sound methodological design. It also must contain details regarding
proposed data collection activities, participants, data management and analyses plans. It must
describe any potential obstacles foreseen in implementation and evaluation and how those
obstacles will be addressed. It must also describe the approach in the planning period
to plan for evaluation. It must describe the approach to building capacity to engage in
the local evaluation activities. And it must clearly describe how the local evaluation
will inform and improve current programming at the application organization and improve
future programming and expand the evidence base. Application requirements related to
funded activities. There are some additional ones such as they must describe two to five
key evaluation questions which are feasible and acceptable in tribal context. It must
include plans for incorporating input from tribal organizations and other partner organizations
and community partners in the development of these local evaluations. It must provide
letters of support from all partner organizations and tribal leadership or community advisory
boards. It must identify local evaluators and include signed letters of agreement with
the local evaluators. Additionally, it must include the CV of the leader of the research
project and you must identify the IRB that will be involved in your local evaluation.
I will turn this back over to Sarah for to discuss the FOA evaluation criteria. Sarah.
>> MS. SARAH AXELSON: Thank you, Kathleen. As Kathleen said, we will now move into a
discussion of the evaluation criteria for applications for the Tribal PREP program.
As a reminder, the evaluation criteria in the FOA outline how applications for the Tribal
PREP program will be scored. This FOA will award points, as you see on the slide, for
objectives, outcomes and needs for assistance, approach, performance measurements, evaluation,
organizational profile and budget and budget justification. The detailed items under each
category are listed in the evaluation criteria section of the FOA. As a reminder, tribal
PREP grantees once awarded will have a planning period to fully develop their program plan.
Therefore, it is not expected that their application will contain a fully developed plan. Applications
for the Tribal PREP program must focus on providing a clear description of the applicant's
plan and capacity to conduct a comprehensive needs assessment during phase one and demonstrating
the applicant’s capacity to implement programming that needs to identify components for the
tribal PREP program during phase two. Lastly, we’ll discuss application submission
instructions. The deadline for electronic application submission is 11:59 p.m. Eastern
Time on the due date listed in the FOA which is July 8, 2016. Electronic applications submitted
to grants.gov after 11:59 p.m. on the due date, as indicated by a dated and time-stamped
email from grants.gov, will be disqualified from competitive review and from funding under
this announcement. It is very important that all applicants submit by the identified deadline
or their applications will not be reviewed. Remember that the FOA is the primary guide
to programmatic requirements. Applicants are strongly encouraged to read the entire FOA
and application tip before writing their applications and to submit their applications by the identified
due date listed in the FOA. For further assistance, the FOA lists several reference websites that
you can see on this slide, as well as here. It also lists several FYSB specific resources
that may be helpful in writing applications. These include the PREP evaluation page with
resources on how to design evaluation plans, the HHS evidence interview with a list of
44 programs, the Tribal PREP fact sheet with an overview of the Tribal PREP program, a
TIP sheet on adulthood preparation, such as a link to other APP-specific TIP sheets and
the FYSB grantee directory of currently funded projects. For questions about the grants.gov
website, please use the email address or phone number listed on the slide to contact the
grants.gov team directly. This concludes the Tribal PREP pre-application webinar. If you
have any additional questions, please contact myself, Sarah Axelson, using the email address
listed in the slide. Thank you for your time.