Strategies for Building the Evidence About Social Service Programs That Work

Ever thought about evaluating an intervention you designed? To do that, you'll need funding. So how can you make the case to funders that your program shows sufficient promise and merits evaluation? Meant for funders, the Coalition for Evidence-Based Policy's new "Practical Evaluation Strategies for Building a Body of Proven-Effective Social Programs Guide" (PDF, 114KB), released in October, provides five recommendations that you can use to bolster your case in a grant proposal. They are:

  1. Don't evaluate an intervention too soon.
  2. Choose your evaluator wisely. Choose evalutors with a demonstrated success record conducting studies.
  3. Cut costs by using data that has already been collected and consider using credible evaluations that are already being implemented.
  4. Be selective when funding expensive, comprehensive evaluations of an intervention.
  5. Measure the impact of the intervention's main goal first.

Read the guide (PDF, 114KB) for more about each recommendation.

Right on the Money: Show Donors Your Impact

Charity Navigator, the nation’s largest rater of nonprofit organizations, has long rated charities based on their  financial health, accountability and transparency. Soon, an organization’s score will also include the impact its work has had on its overall mission.

The change is meant to benefit people who make donations to charities, says Sandra Miniutti, vice president of marketing and chief financial officer of Charity Navigator.

“All donors want to know the impact of their contribution,” Miniutti says. “This will help them understand what the charity actually spends its money on.”

Transparency for All

While Charity Navigator only rates charities with budgets of $1 million or more, all groups seeking donations from the public should consider making this level of transparency a key part of operations.

Charity Navigator’s results reporting score will be based on a number of factors, including:

  • Alignment of mission, solicitation and resources: Whether the charity’s funding solicitation materials are in line with how it allocates its resources.
  • Published evaluation reports: Whether the charity is publishing regular evaluation reports that explain the results of a majority of their programs.
  • Use of feedback from the people it serves: How well a charity collects and publishes feedback from its clients and others who benefit from its services.

The results won’t be included in scores until 2016, but there’s nothing to stop groups from making sure this information is visible on their websites sooner.

Sara Brown, CEO of The National Campaign to End Teen Pregnancy, recognizes that the idea of evaluating impact isn’t a new one. 

“It’s good to challenge people to think about these questions,” Brown said. “It’s not demanding a clean number, but I think everyone’s saying, you have to think about this. You have to come up with something. It doesn’t have to be perfect.”

Miniutti agrees.

“Whether it’s for your board or your donors, whatever the case, you should collect and share information on your impact,” she said. “There’s going to be a growing demand for this type of data.”

Start Sharing

Most charities aren’t measuring and reporting their impact publicly, but by instituting this new measure, Miniutti says the hope is that they’ll start to. Over the next few years, Charity Navigator staff will contact charities and allow them to send in information for consideration.

“But in the future, we want to see it on their website,” Miniutti said.

Specifics about how this measurement will impact a group’s overall score are still being decided. There may be different benchmarks for charities of different sizes or different types, Miniutti says. For example, a shelter directly providing a service may be scored differently than an arts organization.

Do you share information about your impact with your donors and the general public? Tell us how on Facebook and Twitter.

Primary Sources: Surveying the Latest Research on Intimate Partner Violence

"Systemic Perspectives on Intimate Partner Violence Treatment" (abstract). Journal of Marital and Family Therapy, Vol. 38, No. 1 (2012).

What it’s about: Research on intimate partner violence has evolved significantly over the last decade. After surveying the literature in 2003, Sandra M. Stith, Eric E. McCollum, Yvonne Amanor-Boadu, and Douglas Smith decided to take another look 10 years later to see what is now known about intimate partner violence and developing treatment approaches.   

Why read it: According to the authors, early research on intimate partner violence was based on data from the criminal justice and domestic violence shelter systems, where perpetrators of domestic violence tended to be largely male and victims largely female. However, a number of recent studies, which have taken a broader look inside communities as a whole, have found that both men and women are abusers. In some violent relationships, both partners are violent. To better understand the dynamics of domestic violence, researchers are exploring the different kinds of violent offenders –- both men and women -- and the number of different ways relationships can be abusive. The goal, the authors say, is to develop screening tools that could give therapists a better understanding of which couples may be able to safely undergo marital and family therapies designed to address interpersonal violence.

Biggest takeaways for family and youth workers: According to the authors, there is growing consensus in the research community on two major types of perpetrators: those who are “characterological,” also referred to as "intimate terrorists" by some researchers, and those who are “situational.” Characterological perpetrators use violence to dominate and control a partner, and they may also be violent outside the family. Situational perpetrators are more likely to act violently in specific situations because they lack communication and conflict resolution skills. Many of those violent incidents are also fueled by drug and alcohol use.

The authors suggest that situational perpetrators may be good candidates for marital and family therapies that address domestic violence. In their review, they found a handful of therapies that have shown some promise, despite very limited research in this area: Behavioral Couples Treatment, Domestic Violence Focused Couples Treatment, Couples Abuse Prevention Program, Circles of Peace and Motivational Interviewing. Each therapy addresses conflict resolution and communication skills, and some have a substance abuse treatment component. For those who are "characterological" perpetrators, individual and gender-specific group treatments may be safer and more appropriate than the more systemic treatments described above, the authors say.

Additional reference: The Domestic Violence Evidence Project has comprehensive information about what works in domestic violence screening and intervention.

Use Video Scenarios to Open the Lines of Communication Between Girls and Their Families

Research has shown that when young women have open lines of communication with parents and caregivers, they fare better in school and are less likely to abuse substances or get pregnant. But sometimes parents have a tough time talking to daughters as they approach puberty.

Family and youth workers can help by offering "Communication Skills Building for Parents of Preteen Girls," a program developed by the Office on Women's Health at the Department of Health and Human Services. The program is meant for small groups of parents and caregivers led by a facilitator. Through videos tailored toward African-American and Hispanic families, you can show parents how to discuss topics such as

  • dating
  • appropriate dress
  • Internet dangers
  • peer pressure
  • time management

The facilitators' guide will guide you through the process of developing constructive conversations around each topic. The guide also includes discussion questions, information about how to use the videos, and a planning checklist.

 

Work to Be Done: As Teen Pregnancy Rates Decline, Georgia Youth Workers Broaden Focus

The past two decades have been busy for Georgia's sexual health advocates. The state that had the highest teen-pregnancy rate in the country in 1995 (96.5 pregnancies per 1,000 young women) has cut its rate by more than half. Today, 37.9 Georgia teens out of every thousand get pregnant, only the 13th teen-pregnancy rate in the nation.

Of course, Georgia’s success reflects a national trend. The Centers for Disease Control recently reported that American teen pregnancy rates have declined steadily for decades now. In fact, the 2012 rate was less than half of 1991’s. The U.S teen pregnancy rate is still higher than those in Canada and Western Europe, and disparities remain: the pregnancy rate for white teens is less than half that for Hispanic and black youth, for example, and a similar imbalance exists between New England and the Deep South.

But generally the news is good, particularly in Georgia, where advocates say they continue to focus on preventing pregnancies while also evolving and meeting new challenges. That means framing teen’s sexual health and safety in a greater context of personal well-being.

“We’re trying to move away from pregnancy prevention as one set-apart thing, a taboo subject, and get to where it’s intertwined with the rest of young people’s education and experience,” says Daniela Whitaker, executive director of Kids Restart, a teen pregnancy prevention program in Augusta, GA.

Nurturing Adolescent Health

To reflect the wider focus on adolescent health, the Georgia Campaign for Adolescent Pregnancy Prevention, known as GCAPP, earlier this year changed its name to the Georgia Campaign for Adolescent Power & Potential.

“We started looking at the other issues facing young people,” says Kim Nolte, vice president of programs and training. “Obesity is a big one. So we’ve started expanding our work into adolescent health and well-being.”

Along those lines, new GCAPP initiatives include a curriculum for parents to improve their children’s eating habits. The organization is also collaborating with seven Atlanta churches to develop a physical activity plan for young people living in “food deserts,” where there are few grocery stores and other sources of healthy food.

Nolte says the organization’s original mission and its new nutritional focus are complementary ways of ensuring healthy futures for young people. “Whether it’s preventing a pregnancy, developing healthy relationships, or eating better, it’s about helping young people grow into happy, healthy adults.”

Embracing ‘Evidence-Based’

In the last five years Georgia organizations have also embraced the federal government’s focus on evidence-based practices in adolescent pregnancy prevention, which Nolte says has sent their already successful campaign against teen pregnancy into overdrive. She notes that five years ago there were no agencies using such practices in Georgia. Now, she claims over 140 agencies do, reaching nearly 30,000 young people.

For example, thanks to a Personal Responsibility and Education Program sub-grant from GCAPP, Kids Restart has begun using the evidence-based Making Proud Choices curriculum, which teaches youth about avoiding sexually transmitted infections, resisting peer pressure and generally living healthily. And GCAPP is developing a community of practice—an information-sharing network of service providers—to standardize the work being done in community organizations like Kids Restart.

Before the evidence-based culture took hold, Nolte says, many Georgia programs worked in isolation. “Now, through PREP, we have an opportunity to align our funding and measure our success together.”

 

Q&A: Pamela Wiseman of the New Mexico Coalition Against Domestic Violence

This is the third in a series of articles highlighting statewide efforts to combat family and sexual violence. The first two installments looked at Alaska's Choose Respect campaign and the Pennsylvania Coalition Against Domestic Violence.

Pamela Wiseman rejects the idea that domestic violence is a single, discrete issue. As executive director of the New Mexico Coalition Against Domestic Violence, she knows that young people who witness violence at home are less likely to succeed in school, and more likely to develop health problems and commit violence themselves as adults.

Like Alaska (whose anti-domestic violence campaign we recently profiled), New Mexico is largely rural and ethnically diverse. So Wiseman says her private nonprofit that trains and supports a network of 31 direct care providers focuses on empowering them to help their neighbors. Wiseman spoke with NCFY about her efforts to get New Mexicans to tackle this issue from the ground up, from holding online trainings to hiring children’s advocates for family shelters.

NCFY: What is your most urgent work in New Mexico right now?

Wiseman: The Annie E. Casey foundation ranks states according to overall welfare of children, and we’d been 49th for a number of years. But this year we became 50th. Domestic violence is at the root of all the things that plague this state. Trauma leads to drug abuse, low educational attainment, a sense of hopelessness. And the experience of Native Americans here obviously includes hundreds of years of trauma and abuse as well.

There’s also a lot of serious isolation. You can drive for hours and not see anyone. So we work with 31 partner agencies around the state: we train, we do consultations, and we also advocate policy issues and legislation to the state government. Through our conferences and webinars, we provide a way for all these agencies to collaborate with each other and with state-level organizations. But mainly we’re focused on children right now, because if you can help them early on you might be able to prevent some of these negative consequences 40, 50 years down the road.

NCFY: How have you gone about focusing on children?

Wiseman: We received funding to put a children’s counselor in many of our programs. Even though these places see quite a few children, most couldn’t afford to have someone in there who deals solely with kids. But children need that kind of intervention [when they witness domestic violence]. We’ve also worked with Head Start and different in-home caretakers and visitors, training them in how to identify and deal with domestic violence.

We’ve also had several meetings with animal welfare agencies, because animals are part of the family and so important to children. In such a rural state, people have everything from horses and goats to smaller pets. If your child has a puppy and the abuser threatens to kill it, it’s traumatizing and it’s a barrier to leaving for help if you don’t have a place to put your animal. So we’re fighting for regional animal shelters so that victims can put them there for a short time if they relocate.

NCFY: How have family and youth workers been involved in this fight?

Wiseman: A lot of the work today has to do with improving communication—messaging. We know that 70 percent of victims never go anywhere for help, and that’s sometimes because the way we talk about it doesn’t seem relevant to them. Many of our community partners are attending our upcoming summit on how to talk about domestic violence effectively.

Rural communities are close-knit, so if someone’s being abused, they may find the community turns against them if they tell. Families may turn against them. People don’t identify as victims. If we show pictures of people who have been abused it might alienate people who haven’t reached that level. So our family and youth workers are trying to be more open and flexible in the way we define domestic violence.

Enter the 2014 Mural Contest for Runaway and Homeless Youth Programs – Inspire Creativity and Beautify Your Program

The Family and Youth Services Bureau is pleased to announce its 2014 Runaway and Homeless Youth Mural Contest. If your agency is a grantee of FYSB's Runaway and Homeless Youth programs, this is your chance to engage youth in a creative activity and win the materials to paint a mural in your program.

As in years past, entries will be judged by a panel of FYSB staff and youth based on the meaningfulness and creativity of the design and the extent to which Positive Youth Development was used in the process. 

We look forward to seeing what ideas youth come up with to showcase their talents and experiences! If you’re looking for additional inspiration, visit our slideshow of last year’s entries, including the winning submission from Bridge Over Troubled Waters.

Interested in applying?

Please contact Joelle Ruben, community manager at the National Clearinghouse on Families & Youth, to receive an application. Materials are due by 5 p.m. Eastern on Friday, January 31, 2014.

Good luck!

Q&A: Peg Dierkers of the Pennsylvania Coalition Against Domestic Violence

This is the second in a series of articles highlighting statewide efforts to combat family and sexual violence. The first installment looked at Alaska's Choose Respect campaign.

The nonprofit Pennsylvania Coalition Against Domestic Violence was the first agency of its kind when it opened in 1976. A liaison between the Pennsylvania government and a network of local services providers, the coalition disburses state funding for domestic violence services and ensures that such services are available near every resident. It also collaborates with the Departments of Public Welfare, Crime and Delinquency, and Health to provide training and technical assistance to public and private organizations. Above all, the coalition is a central resource for Pennsylvania’s family and youth workers, and a way for local advocates to have their concerns represented at the state level.

Executive Director Peg Dierkers spoke with NCFY about the progress her organization has made over the decades, and the new challenges facing family and youth workers in Pennsylvania, from the recession to gas drilling.

NCFY: How have your priorities and mission changed since your organization was founded in 1976?

Dierkers: We collaborate with 60 community organizations with a goal of providing services within 50 miles of anyone experiencing domestic violence in Pennsylvania. That used to mean just a shelter, but now we know the housing needs of violence victims go beyond short-term housing.

So we work with state housing agencies and supply money for relocation and security deposits and rent for victims who need to relocate. We also educate our network and community partners about federal housing laws, because these victims are often discriminated against by landlords. Landlords don’t want the noise from a fight or regular visits from the police, but you should be able to call for help whenever you need it.

We also work with the child welfare system to help case workers understand domestic violence and promote the protective, non-offending parent’s relationship with the children. The long-term resiliency of the children really depends on maintaining that familial bond with the protective parents. Often the child welfare system and the courts don’t have the training around the dynamics of abuse, like the barriers to a woman leaving—from economics to the threat of death. It results in dual custody after an incident. So we help train all case managers in county children and youth agencies.

NCFY: What new challenges or opportunities exist today that weren’t around decades ago?

Dierkers: Natural gas drilling is one. The drilling companies come in and rent or buy up all the available housing stock for their workers, often at an inflated price because they need a lot of it quickly. So then there’s either not housing available for victims, or it’s so expensive that the agencies can only help a few people where they used to help many. At least one recent study documented this.

And of course in Pennsylvania, the brightest spotlight possible is currently on child sexual abuse in light of the Sandusky case. There’s been a huge expansion of who counts as a mandated reporter and proposals to increase the liability for those who see something. We are right in there, explaining how dynamics of abuse work and how the protective parent is often the sole mitigating factor. There is a tendency to blame the mom instead of really focusing on holding the abuser accountable for the behavior. Our concern is that if we don’t advocate really hard for those dynamics to be considered, many children will be removed from their caregivers, endangering their long-term health and success.

Bright Idea: A Free Teen Clinic Reduces Barriers to Health Care

For unaccompanied youth, pregnant teens and other young people independently seeking free health care, going to an adult clinic can be daunting. Those wary of authority may be nervous about sharing personal information. Others don’t know their health history, or fear they’ll be talked down to by a doctor or nurse.

“This age group is really stuck,” says Eddie Bonin, instructor of clinical pediatrics at Tulane University’s Section of Adolescent Medicine. “They’re not quite adults, but they’re also not children.”

Bonin oversees Tulane’s Drop-in Clinic, which provides free medical care to New Orleans youth age 23 and under. Giving teens their own space, Bonin says, has helped the clinic earn young people’s trust and increased word of mouth. Yet even in communities without a teen clinic, family and youth workers can benefit from learning what to look for when connecting youth to health services.

Breaking Barriers

One of the most important factors in helping disconnected youth, Bonin says, is reducing the barriers to trust. At the Drop-in Clinic, staff members limit the personal details they ask patients to share before an appointment. Young people may be asked to give their name and birthday, for example, but they don’t have to show identification and won’t be turned away for providing false information.

Staff at the Drop-in Clinic, Bonin says, have also learned to ask youth the right questions to encourage them to open up about their lives. Asking where they sleep at night, for example, is generally more effective than asking if they’re homeless.

Youth are also asked if they’d like information about birth control and safe sex, topics that they may be too embarrassed to bring up on their own. Patients 13 and older are typically screened for HIV and sexually transmitted infections, and young parents are encouraged to bring their children to see the onsite pediatrician.

Even the clinic’s physical setup, Bonin says, can make a difference. When Hurricane Katrina forced the Drop-in Clinic to relocate to Covenant House New Orleans in 2005, some patients were put off by tightened security at the new building. Staff at both organizations worked together to make the facility more welcoming without compromising safety, even removing a gate around the building’s perimeter.

“Every time you throw up a door or gate, that’s a barrier,” Bonin says. “It’s a physical barrier and it’s intimidating to people.”

Asking the Right Questions

Here are some things you might want to think about when connecting youth in your program to health centers and clinics:

  • Are they flexible? At the Tulane clinic, most appointments last 45 minutes or longer. Doctors and nurses may shorten the appointment, however, if they see a young person is too nervous or agitated to stay that long comfortably.
  • Do their staff members like working with teens? Seems like a given, but clinic staffs’ attitude toward young people could make or break a youth’s decision to get medical attention.
  • Can youth give feedback? At the Drop-in Clinic, patients are invited to share their opinions at quarterly feedback meetings. The sessions enable clinic staff to tailor services to their youth clientele.
  • Do staff empower youth to take care of their health? In New Orleans, clinic staff try to give youth enough information to manage their health on their own while encouraging them to come back as often as they’d like for follow-ups.

Pages

Subscribe to National Clearinghouse on Families & Youth RSS
Monday-Friday
9-5 pm Eastern