A New Journal Article Tackles Myths and Misconceptions About Evidence-based Programs

Photograph of a notepad and a pen.

“Evidence-based Programs in Children's Services: A Critical Appraisal.” Children and Youth Services Review Vol. 35, No. 2. (February 2013).

What it’s about: Proponents of “evidence-based programs” in social services for children and youth and their families take on the debate about the use of such programs.

Why read it: Momentum is growing behind efforts to promote more widespread use of programs that have been studied by researchers and found to result in better outcomes for families and youth. Many private and public funders, including the Family and Youth Services Bureau, are encouraging or requiring their grantees to take evidence into account when planning and implementing programs for youth and families. While many practitioners have embraced evidence-based programs, others are skeptical. By looking at both sides and responding to arguments that have been made against emphasizing evidence-based programs, this article makes a compelling case for their use.

​Biggest takeaways for youth workers: The authors look at five different types of critiques of evidence-based programs: scientific, ideological, cultural, organizational and professional.

Among the scientific critiques are those that question the validity of and methods behind the research that has been done to determine whether programs are effective or not, and the idea that programs can be declared to work or not. The authors argue that though some studies may be flawed and the answer may not be cut and dry, scientific study does enable us to have a better understanding of what works. They write that “the most promising innovations should be strengthened and tested with a level of rigor appropriate to their stage of gestation as part of a logical process of program development.”

Among the many arguments the authors take on are contentions that evidence-based programs:

  • Only treat symptoms of larger problems and ignore bigger-picture issues like poverty and racism. To the contrary, the authors say, evidence-based programs can contribute to greater social justice.
  • Do not translate across cultures. If designed with enough flexibility, the authors write, evidence-based programs can be used with families and youth of different cultures.
  • Are too expensive and time-consuming to set up. Not all are expensive, but even so, their ultimate cost savings may justify their upfront cost.  
  • Hamper innovation and creativity on the part of practitioners. Evidence-based practices are the product of innovation, often by practitioners.

The article ends with concrete suggestions that could bridge the gap between proponents of evidence-based programs and those with reservations about their use. The authors write:

Researchers need to learn more about what makes programs transportable and understand cultural variation in impact. Methods for adapting [evidence-based programs] need to be developed and tested. When introducing programs to systems it is imperative to stress the importance of the practitioner, demonstrating how professional knowledge and experience contribute to outcomes. The place for innovation should be highlighted, as should the radical nature of programs that reach the most needy and reduce inequality, and the significant amount of ‘face time’ with users. Program training should be seen as an opportunity to master new skills and be certified as such. These are all things that practitioners are reported to find motivating.

Additional references: A new NCFY report looks at FYSB’s position on the importance of evidence-based practice, one grantee’s experience implementing an evidence-based program, and how some programs are partnering with researchers to show the effectiveness of their work.

(Publications discussed here do not necessarily reflect the views of NCFY, FYSB, or the Administration for Children & Families. Go to the NCFY literature database for abstracts of this and other publications.)

 

Monday-Friday
9-5 pm Eastern