Grading the States 2006:  Introduction - Methodology

Three landmark documents guide the report's overall vision:

• The U.S. Surgeon General's Report on Mental Health (1999)
• The report of the President's New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Healthcare in America (2003)
• Improving the Quality of Health Care for Mental and Substance Abuse Conditions, by the Institute of Medicine of the National Academy of Sciences (2005)

"Evidence-based practices" (EBPs), as based on those promoted by the U.S. Department of Health & Human Services (HHS) Substance Abuse & Mental Health Services Administration (SAMHSA), are those in which treatment methods have been measured against outcomes. These EBPs also provide a foundation for the report. They include Assertive Community Treatment (ACT), integrated dual diagnosis treatment (IDDT) for co-occurring disorders, illness management and recovery, supported employment, and family psycho-education.

Each state's progress toward a proven, cost-effective system of mental healthcare is indicated by a letter grade of A, B, C, D, or F. These are discussed in individual narratives in the state by state section of this site. Each grade is calculated in part from 39 specific criteria organized in four categories:

• Infrastructure
• Information Access
• Services
• Recovery Supports

Specific scores for each of the 39 criteria for each state appear in tables accompanying each state narrative.

The national grade was calculated as an average of the state grades, both overall and by category. In addition, the reader can quickly compare state systems by referring to tables in the report that list states grouped by grade and that list all states' scores for each of the 39 criteria.

The evaluation of existing state systems relied on four principal sources:

• Written responses from state mental health agencies to detailed questionnaires submitted in October - December 2005. To some degree, the questionnaires resembled "take-home tests" in contributing to each state's final grade. Colorado and New York were the only states to decline to participate in the survey, for which they have been graded "U" for "Unresponsive."
• Public information, such as state community mental health services block grant applications, agency reports, Web site content, and newspaper articles.
• A "Consumer and Family Test Drive" of every state agency's Web site and telephone routing system for information accessibility - which was then incorporated into the scoring system with a weight of 10 percent. To some degree, the test represented a "pop quiz."
• Interviews conducted with consumer and family advocates, which provided additional information for the state narratives.

A more detailed explanation of the scoring methodology can be found in the Methodology Section. A discussion of the Consumer and Family Test Drive can also be found there.

Information constantly changes. We have worked to make the report as accurate and up-to-date as possible. We also worked with states to address concerns or clarifications to improve the report during the research process. We look forward to receiving additional written comments and refining the methodology over time. It is important to see the grades as a baseline or starting point for the future.

Keep in mind also that NAMI members represent people with serious mental illnesses and their families. We are the "customers" whom state agencies serve. The perspective of this report is ultimately that of the people for whom the criteria, overall, may make the difference between recovery or premature death.

continue to Grading the States Introduction: Common Trends

Back