Ask NCFY Live Support
Monday-Friday 9-5 pm Eastern

Toolkits

4. Monitoring and Evaluation

Who should use this section:

  • programs that have never monitored or evaluated their ARA prevention efforts; and
  • programs that are planning to implement ARA prevention and are considering ways to monitor their approaches.

As part of their overall planning process, APP programs that want to address ARA should consider plans to monitor program implementation and evaluate participant outcomes. Programs could also make decisions to initiate or improve monitoring and evaluation efforts after program implementation has begun. Regardless of the timing, the tools in this section will help with the process of planning for monitoring and evaluation of ARA prevention activities. The sample evaluation plan is the most general of the tools, presenting the various types of decisions that programs should make prior to beginning their monitoring and evaluation efforts. The remaining tools can help programs make decisions about specific measurement tools to use.

  • The Adaptation Guide (PDF, 1.8MB)
    Centers for Disease Control and Prevention. (2010). Atlanta, GA: Centers for Disease Control and Prevention.
    Pages 107-110 of this tool provide guidelines for developing a monitoring and evaluation plan. 

    Pages 111-112 of the tool show a planning table that APP programs could complete as they finalize specific elements of their plan. Although the examples provided were developed for projects adapting HIV interventions for another population, they can be used as a general guide to how APP programs could complete this table.
     
  • Implementation Plan for ABCD PREP for Teen Pregnancy Prevention Project 
    The example APP evaluation plan on pages 8-10 shows how a program might document the proposed plan for evaluating ARA prevention activities. Programs may adapt this example plan for their own use. The evaluation plan may include information about involving a local evaluator with expertise to help with further planning. A written evaluation plan is critical for communicating with staff, partner organizations, stakeholders, and the FYSB project officer about the proposed approach; the plan should be updated as changes or improvements are made.
     
  • Adolescent Health Programs ARA and Sexual Assault Quality Assessment/Quality Improvement Tool (PDF, 8.9MB)
    Miller, E., & Levenson, R. (2013). San Francisco: Futures Without Violence.
    This tool, found in Appendix B (pages 49-53), is designed to provide project managers and trainers with a guide to assess ARA prevention efforts. The tool uses a checklist format and addresses topics such as assessment method, intervention strategies, project networking and training, self-care and support, data and evaluation, education and prevention, and environmental resources. This tool can be used as an initial assessment for project services and to measure ongoing project progress. It can be administered to program staff or administrators as part of monitoring or process evaluation efforts.           

APP programs can use the tools below to identify survey items to administer to youth as part of outcome evaluation surveys. Here are some tips for how to decide between the many potential youth survey items:

  • Choose measures that directly correspond to the project's ARA prevention goals, objectives, and logic model.
     
  • Choose measures that have been tested with respondents who are similar to the project's target population; correspondence on age group and literacy level are particularly important.
     
  • Keep validated scales intact if possible; minimize deletions and wording changes. Choose shorter scales if time is limited.
     
  • Pilot or pre-test tools to ensure comprehension and honest reporting.
     
  • Use data collection best practices to maintain respondent confidentiality during survey administration.
     
  • Measuring Intimate Partner Violence Victimization and Perpetration: A Compendium of Assessment Tools (PDF, 499KB)
    Thompson, M. P., Basile, K. C., Hertz, M. F., & Sitterle, D. (2006). Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.
    This compendium includes more than 20 measurement tools that are designed to measure intimate partner violence perpetration and victimization. The measures assess various types of intimate partner violence, including physical, psychological/emotional, and sexual abuse, as well as stalking. The compendium provides target group, reliability/validity, and source information for each measure, so that users can assess their fit and quality and know where to go for additional information. Note that it is particularly important for APP projects to consider the target population for measures of interest, given that some measures are designed for individuals not in the general adolescent population, but rather for specific groups such as college students, adults, married individuals, and partners of inmates. 

    If an APP project decides to include outcomes of ARA prevention efforts in their local evaluation, this compendium can be used to identify established ways to measure violent behavior. These measures can also be used for needs assessment or surveillance purposes. One ARA prevention goal might be to advocate for widespread ongoing assessment of ARA prevalence in their community; certain measures from this compendium might be appropriate for inclusion in a school district’s recurring youth risk behavior assessment survey.
     
  • Measuring Violence-Related Attitudes, Behaviors, and Influences Among Youths: A Compendium of Assessment Tools, 2nd ed. (PDF, 6MB)
    Dahlberg, L. L., Toal, S. B., Swahn, M., & Behrens, C. B. (2005). Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.
    This compendium includes more than 170 measurement tools that can be used to assess attitudes, beliefs, behaviors, and contextual factors related to various forms of youth violence, including ARA. The compendium provides target group, reliability/validity, and source information for each measure so that users can assess their fit and quality and know where to go for additional information. 

    If an APP project decides to include outcomes of ARA prevention efforts in their local evaluation, this compendium can be used to identify established ways to measure short-term outcomes (also known as mediators or risk/protective factors) and long-term outcomes. These measures can also be used for project needs assessment or surveillance purposes. For example, during the planning stage of their ARA prevention efforts, an APP project might choose to measure several possible risk and protective factors among a sample of youth in their community to determine which have the most room for improvement through intervention. 
     
  • Research Findings: Tech Abuse in Teen Relationships Study (PDF, 351KB)
    Picard, P. (2007).
    The above compendia are comprehensive but do not include measures of electronic aggression (i.e., the use of technology or electronic media—such as cell phones, texting, instant messaging, social network sites, e-mail, Web chat, or blogs—to perpetrate psychological abuse), which has increased in visibility in recent years. Research has shown that electronic aggression is associated with other types of ARA, such as psychological, physical, and sexual abuse, as well as stalking.(16

    APP programs that wish to assess electronic aggression among their project participants (e.g., to determine the prevalence in their communities or to assess the impacts of ARA prevention activities targeting electronic aggression) can use some of the same items that were used in this national study on tech abuse in teen relationships. The questions are shown at the bottom of each slide. In particular, slides 11 through 14 show questions that assess electronic aggression perpetration and victimization. Programs interested in collecting data from parents about adolescent electronic aggression can also find useful items in this document.

 

 

The Adolescent Pregnancy Prevention Program’s Training and Technical Assistance and Meeting Logistical Support project led by RTI International for the U.S. Department of Health and Human Services, Administration on Children, Youth and Families, Family and Youth Services Bureau, under Contract No. HHSP23320035651WC developed this toolkit.

Monday-Friday
9-5 pm Eastern