2019 State Standard of Excellence
10. Evidence Definition and Program Inventory
Did the state or any of its agencies release a common evidence framework, guidelines, or standards to inform its research and funding decisions and make publicly available an inventory of state-funded programs categorized based on at least two tiers of evidence?
Why is this important?
Defining evidence and conducting program inventories can help state governments ensure that their programs are using proven practices and that their interventions are effective in meeting the state’s desired goals as well as improving outcomes for residents.
Under a 2015 Minnesota law (section 13), Minnesota Management and Budget has developed numerous inventories of evidence-based programs, including in the areas of criminal justice, mental health, child welfare, and higher education. Minnesota Management and Budget also maintains the Minnesota Inventory, a searchable clearinghouse of more than 400 programs operating in the state. As part of the inventory, the state developed a guide for using evidence in policymaking and evidence definitions to categorize interventions as proven effective, promising, theory based, or no effect. These resources helped inform funding decisions in the state, including $87 million in new or expanded evidence-based programming in the FY 2020-2021 budget.
The California Department of Social Services created an Evidence-Based Clearinghouse for Child Welfare, which allows child welfare providers and professionals to “identify, select, and implement evidence-based child welfare practices that will improve child safety, increase permanency, increase family and community stability, and promote child and family well-being.” This tool helps identify best practices and provides guidance and support for program implementation. The Clearinghouse’s numerical rating scale categorizes programs into six tiers of evidence and uses a relevance scale as a complement to the scientific rating scale and to demonstrate applicability for client populations.
The Colorado Governor’s Office of State Planning and Budgeting publishes periodic Results First reports that categorize all state-funded prevention programs in the areas of child welfare, criminal justice, and juvenile justice according to three tiers of evidence: evidence-based, promising, and needs additional research. In 2018, a findings report applied the cost-benefit analysis framework to health programs.
A 2015 Connecticut law defines the following three tiers of evidence for programs operated by the Connecticut Departments of Correction, Children and Families, and Mental Health and Addiction Services, and the Court Support Services Division of the Judicial Branch: evidence-based; research-based; and promising. The law requires these same agencies to categorize their programs by these evidence tiers in even-numbered fiscal years. Additionally, the law charges the Institute for Municipal and Regional Policy at Central Connecticut State University with submitting a report containing a cost-benefit analysis of the programs, which in 2018 was entitled Benefit-Cost Analyses of Evidence-Based Programs.
A 2014 Mississippi state law required the Mississippi Departments of Corrections, Health, Education, and Transportation to (1) develop an inventory of their programs based on four levels of evidence (evidence-based program, research-based program, promising practice, or other programs and activities) and (2) report during the budget process about their programs’ cost-benefit ratios and effectiveness. The Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review publishes cost-benefit analyses and reports on program effectiveness.
New Mexico has published a series of inventory and cost-benefit reports in the areas of children’s behavioral health, adult behavioral health, early education, child welfare, criminal justice, healthcare, infant and maternal health, and education. The Legislative Finance Committee’s Evaluation Unit publishes Results First reports, program evaluations, information technology reviews, and health policy reviews publicly. The state has also conducted cost-benefit analyses of its programs and published guidance on Legislating for Results.
In 2019, the North Carolina Office of State Budget and Management published a child and family health program inventory and cost-benefit analysis. The state also publishes periodic Results First progress reports and has released a tiered evidence definition.
Under a 2003 Oregon law the Oregon Department of Corrections, the Oregon Youth Authority, the Oregon Youth Development Division, and “the part of the Oregon Health Authority that deals with mental health and addiction issues” are required to compile a biennial program inventory with results from funded programs and to perform cost-benefit analyses. The law also defines an evidence-based program as being cost effective and incorporating “significant and relevant practices based on scientifically based research,” including “[u]tilizing randomized controlled trials when possible and appropriate.” In 2018, the Oregon Youth Authority and the Oregon Department of Corrections published reports on the costs and benefits of their programs.
In 2015, in accordance with the state’s Alcoholic Beverage Control Act (section f), the Utah Department of Human Services issued a rule to create a statewide registry of approved evidence-based substance abuse prevention interventions. The Department’s Division of Substance Abuse and Mental Health Evidence-based Workgroup defined the following four levels of effectiveness: well supported, supported, promising, and emerging. In addition to creating these definitions, the Working Group provides resources to support program design and implementation including a process for approving a program as evidence-based and a logic model guide and template.
A 2012 Washington law: (1) stated that “prevention and intervention services delivered to children and juveniles in the areas of mental health, child welfare, and juvenile justice [must] be primarily evidence-based and research-based” (p. 2); (2) directed the Washington State Institute for Public Policy (WSIPP) to develop definitions for three levels of evidence: evidence-based, research-based, and promising practices (p. 4); and (3) tasked WSIPP with creating an inventory of evidence-based programs, which was released in 2012 with subsequent periodic updates, including in 2018. Other legislatively directed inventories include: Corrections; Behavioral Health (originally published in 2014 and updated in 2016); and K-12 education programs, including the Learning Assistance Program (most recently published in 2018 and featured on the state superintendent’s website as Menus of Best Practices and Strategies).