1. Overview
2. Background
3. Development
1. Goals
2. Development Team
3. Domains
4. Components
   1. Target Population
   2. Initial Patient Interview
   3. Initial Informant Interview
   1. Outcomes
   2. General Functioning
   3. Prognostic Variables
   4. Follow-up Patient Interview
   5. Follow-up Informant Interview
   6. Medical Record Review
   1. Treatment Components
5. Reliability & Validity
   1. Reliability
   2. Validity
   3. Sensitivity to Change
   4. Subject-Informant Agreement
6. Potential Uses
   1. Strengths
   2. Limitations
7. Data Analysis and Presentation
   1. Individual Patient Reports
   2. Aggregate Patient Reports
   3. Scoring
   4. Usage
8. Summary
9. Bibliography

The introduction of the enhanced Schizophrenia Outcomes Module has prompted minor text revisions to the schizophrenia section of the NetOutcomes^TM website. While certain variations now exist between the updated web text and the hard copy version of the NetOutcomes^TM Primer for Behavioral Health Outcomes Assessment, they are not considered fundamental in nature.

Overview  Top

Schizophrenia affects more than 2 million people in the United States and accounts for approximately 40% of all expenditures for mental illness. The Schizophrenia Outcomes Module (SCHIZOM) measures the types of care received by patients with schizophrenia, the outcomes of that care, and the patient characteristics that influence either the types or the outcomes of care. Data generated by the SCHIZOM will enable users to examine the relationships between patient characteristics, treatment, and outcomes. The module may be used to determine the most effective and highest quality services by comparing outcomes of different treatments in similar settings or of similar treatments in different settings.

Background  Top

The National Institutes of Mental Health states that schizophrenia is the most chronic and disabling of the severe mental disorders, with devastating effects on individuals, their families, and also the communities in which they live.

More than 2 million Americans are affected by schizophrenia in any given year, and only one in five recovers completely. Even with available treatment, most continue to suffer chronically or episodically from their illness throughout a large part of their lives. One measure of the anguish of schizophrenia may be inferred from its lethality; an estimated 1 of every 10 people with the illness dies by suicide.(1)

One study estimated the total aggregate economic cost of schizophrenia, including use of medical resources, lost productivity, and incidence of death, to be $33 billion in 1990.(2) Although persons with schizophrenia comprise only 1% of the adult population, they consume about 2.5% of the total annual health care expenditures. They also constitute 10% of the permanently disabled and as much as 14% of the homeless population in some urban areas. (3)

Approximately 50% of those afflicted with schizophrenia experience lifelong, chronic, and disabling symptoms.(3) Patients with the disorder may experience time distortion, suspiciousness, difficulty concentrating, and difficulty with abstract ideas. A wide variety of factors is known to influence the treatment outcomes of persons with schizophrenia, including a patient's social environment, adherence to treatment regimens, comorbid conditions such as substance use,(4) awareness of illness, and ability to recognize and articulate symptoms,(5) as well as the clinical therapies received.

Development  Top

Goal

The SCHIZOM is designed for use in routine clinical care settings as part of an outcomes management system for continuous quality improvement. The module provides information on a comprehensive set of variables that are relevant to understanding how treatment affects patients' symptom severity and functioning over time. The module is used to measure the process and type of care patients with schizophrenia receive, the outcomes of that care, and patient characteristics that influence the type or the outcome of care.

Development Team

A multi-institutional, multidisciplinary panel advised UAMS on the development and methodological assessment of relevant measures regarding the treatment and outcomes of schizophrenia. The module was developed by Ellen P. Fischer, Ph.D.; Brian J. Cuffel, Ph.D.; Richard R. Owen, MD; Barbara J. Burns, Ph.D.; William Hargreaves, MD; Craig Karson, MD; Anthony Lehman, MD; David Shern, Ph.D.; G. Richard Smith, Jr., MD; and Greer Sullivan.

Domains

Because the diagnostic process in schizophrenia is so complex, the module currently does not include a diagnostic measure or screen. However, the module does include symptoms of schizophrenia to assist providers in monitoring clinical change. The patient's general functioning is also assessed using the SF-36. Change in patient symptoms and general functioning from baseline to follow-up comprise the outcome domains. Prognostic variables allow meaningful comparison of outcomes across patient groups. Treatment components are also included in the module.

Components  Top

The SCHIZOM consists of five components: an Initial Patient Interview, which includes a baseline assessment; Initial Informant Interview; Follow-up Patient Interview; Follow-up Informant Interview; and an optional Medical Record Review. Most questions in the patient and information assessments require either a yes/no response or response on a four- to seven-point Likert scale. While the patient and informant components are designed to be administered by lay interviewers, some patients and informants may be able to complete them as self-administered questionnaires.

Target Population

Adult patients are entered into the SCHIZOM protocol if they have a clinical diagnosis of schizophrenia, are at least 18 years of age, are able to cognitively understand the assessment process, and consent to participate.

Administration & Sources of Information

The SCHIZOM uses interviews, rather than relying on patients' self-reports, because individuals with schizophrenia may not be able to focus on the task of completing a self-administered questionnaire. Because the credibility and reliability of self-report by patients with schizophrenia may be questionable at times, the module includes forms for data collection from "informants" or friends/family members who know the patient well. The usefulness of this complementary data outweighs the associated costs of obtaining it.(6) Current work is ongoing concerning the feasibility of allowing the interview to be self-administered.

Initial Patient Interview

The Initial Patient Interview is analogous to the Patient Baseline Assessment in other modules. Administered as an interview with the patient, it takes approximately 30 to 35 minutes to complete. The 127-items provide information on general health, disorder-specific symptoms, functioning, sociodemographics, prognostic factors, and sources of care. Work is ongoing concerning the feasibility of this interview to be self-administered.

Initial Informant Interview

At the time of the Initial Patient Interview, one or more family members, friends, or mental health providers associated with the patient complete either a telephone or in-person interview. The interview parallels the interview with the patient, providing complementary information about the patient's symptoms, functioning, history, and co-existing conditions. Information about the effect of the patient's illness on family members is also gathered. The 85-item interview requires approximately 25 minutes to administer.

Outcomes

The SCHIZOM covers four categories of outcomes. (See Table 9.1 for a full listing of measures and sources.)

Clinical Status. Clinical status measures include symptom severity, hospitalizations, and death.

Rehabilitative or Functional Status. Indicators assess the ability to function in various vocational and social settings by measuring instrumental activities of daily living, living arrangements, social relationships and activities, and occupational and educational activities.

Humanitarian Status. This category, which covers life satisfaction and fulfillment, includes a subjective appraisal of well-being and personal fulfillment. Patient indicators include self-reports of general life satisfaction, satisfaction with friendships and social relations, and satisfaction with living arrangements. Family indicators include measures of the extent of instrumental support provided and feelings about the patient.

Public Welfare Status. Indicators include illegal activities and other problems with the legal system and suicidal or violent behaviors.

Table 9.1 Sources of Outcome and Prognostic Items and Scales for the SCHIZOM

CIDI-SAM=Substance Abuse Module of the Composite International Diagnostic Interview

General Functioning

The patient's general health is measured during the Patient Interview using the SF-36.(21)

Prognostic Variables

The SCHIZOM includes measures of those variables, other than services provided, that have been shown in one or more studies to influence treatment effectiveness and can be easily obtained under routine clinical condition. These measures include the following.

Follow-up Patient Interview

Due to the chronic nature of this disorder, follow-up is conducted every six months post-baseline as long as the patient remains in the treatment system or catchment area. The 127-item follow-up interview is completed in person and takes approximately 30 to 35 minutes to finish. Current work is ongoing concerning the feasibility that this interview may be competed as a self-report questionnaire.

Follow-up Informant Interview

The Follow-up Informant Interview is completed at the same time as the Follow-up Patient Interview to provide complementary information on the patient's current symptoms, functioning, and services received. About 25 minutes are needed to complete the interview, which consists of 83 items.

Medical Record Review

Characterization of the frequency and types of treatment provided and the medications used for schizophrenia is completed through the optional Medical Record Review. If used, the Medical Record Review is completed every six months during follow-up. The Medical Record Review is adapted for compliance with the user's medical record system

Treatment Components

Treatment information includes data on medication management and the patient's use of outpatient, emergency room, and hospital facilities. Medication management includes information on the type, amount, and mode of administration of psychotropic medications. Utilization data include the number and length of hospitalizations, frequency of use of other mental health services, including residential program days, partial hospital days, outpatient hours, case management hours, and emergency services contacts.

Reliability & Validity  Top

 The reliability and validity of the SCHIZOM were tested using longitudinal data from 100 individuals with schizophrenia age 18 to 55 years. Patients were recruited from the Central Arkansas Veterans Healthcare System (North Little Rock, AR) or the Arkansas State Hospital (Little Rock, AR).(22)

Reliability

Test-retest correlations for six of the eight outcomes variables were excellent (r�=0.70) and were moderate for suicidal and violent behaviors (r=0.42). Correlations were also excellent (r�=0.70) for all prognostic variables. The internal consistency of multi-item scales was assessed using Cronbach's alpha. Symptom severity and the Instrumental Activities of Daily Living (IADL) both had high internal consistency (a�=0.70). The social relations scale had an alpha consistency of 0.59.(6)

Validity

Instruments used in assessing criterion and concurrent validity were:

Table 9.2 Validity of Outcomes Measures

(a) Spearman correlation unless otherwise specified
(b) Pearson correlation.
(c) Reported for previous week.
(d) Reported for previous month.
(e) Reported for previous six months.

Reproduced with permission of Sage Publications Inc. (Cuffel B.J., Fischer E.P., Owen R.R., Jr., Smith G.R., Jr. An Instrument for Measurement of Outcomes of Care for Schizophrenia Issues in Development and Implementation. Evaluation & the Health Professions, Vol 20 No. 1, March 1997 96-108.)

Sensitivity to Change

The SCHIZOM detected statistically significant change(p<0.05) between baseline and follow-up for outcomes relating to symptom severity and violent behavior and prognostic measures of   alcohol and drug use/abuse. Overall, the SCHIZOM detected change in the same areas and same directions as the validation instruments.

Subject-Informant Agreement

Intraclass correlation coefficients and kappas were calculated to assess agreement between patient and informant data at baseline and follow-up. Agreement was fair (ICCs �=0.30) for legal problems, housing, occupational and educational activity, age of symptom onset, frequency of drug and alcohol use, and medication adherence. Agreement was poor for all other measures, and there was no statistically significant correlation between patient and informant reports on suicidal behavior or violent behavior at follow-up.(6) Agreement was higher for objective variables then for subjective variables and those unlikely to be directly observed.

Potential Uses  Top

Strengths

The SCHIZOM is necessarily broad in scope to address all aspects of a person's life that are affected by schizophrenia. It has been tested in a patient population undergoing routine treatment for schizophrenia in large public mental heath care settings. The SCHIZOM was shown to be useful for measuring the full range of outcome domains that are relevant to schizophrenia treatment. Field-testing has shown the module to be relatively brief to administer and acceptable to respondents. It also has good reliability and concurrent validity and is able to detect change and the absence of change in a patient's functioning and health status.

Limitations

The SCHIZOM module is not a static product. Additional work is currently underway to enhance, expand, and further validate it and assess its performance in clinical and demographic subgroups of the population with schizophrenia. Although the SCHIZOM has been translated into Spanish, there have been no validation studies conducted of this version. 

Data Analysis and Presentation  Top

Feedback of patient data can be critical for provider buy-in to quality improvement activities. The SCHIZOM can provide important information about individual patients as well as targeted groups of patients.

Individual Patient Reports

Individual reports can be generated from the SCHIZOM to provide relevant information about the patient's symptoms, general and mental health functioning, critical items such as suicidality, and prognostic indicators. Individual patient reports may be obtained from www.NetOutcomes.net.

Aggregate Patient Reports

The NetOutcomes^TM outcomes management systems have three types of aggregate reports-provider, program, and facility reports. Provider reports present individual providers with quarterly summaries of baseline and follow-up information as well as derived data such as the percentage of patients who receive guideline concordant care. It also compares the provider's performance to the overall performance of his/her program and to national benchmarks.

Program reports present data on each provider within a program as well as the aggregate data for the entire programmed national benchmark. Each provider is distinguished by a code known only to the provider and the clinical director.

Facility reports are used in situations where similar programs are to be compared within a facility or system of care. National benchmarks are also presented. Aggregate reports from the SCHIZOM provide baseline data on patient characteristics and initial symptoms. This information, when combined with follow-up assessments, can inform providers about the effectiveness of care for their schizophrenic patients.

Scoring

Scoring is available via the NetOutcomes^TM website at www.NetOutcomes.net

Usage

The SCHIZOM is protected under copyright and is available for unlimited free use, provided that there is no charge associated with its administration. To that end, UAMS permits unlimited reproduction and distribution of the module by the public for nonprofit, educational, or research purposes. However, any commercial use of our work, including the creation of electronic versions or other derivative works of the module for sale, constitutes a violation of the copyright of the university unless prior authorization has been granted in writing

Summary  Top

Background.

The Schizophrenia Outcomes Module (SCHIZOM) seeks to improve care for individuals suffering from schizophrenia, which affects more than 2 million people in the US and accounts for approximately 40% of all mental illness expenditures. Although persons with schizophrenia comprise only 1% of the adult population, they consume about 2.5% of the total annual health care expenditures, constitute 10% of the permanently disabled, and make up as much as 14% of the homeless population in some urban areas. The total aggregate economic cost of schizophrenia was estimated at $33 billion in 1990. That figure includes use of medical resources, lost productivity, and incidence of death (an estimated 1 out of 10 people with schizophrenia die by suicide). Most individuals suffer chronically or episodically from their illness throughout a large part of their lives, even with available treatment. Treatment outcomes are influenced by a wide variety of factors, such as social environment, adherence to treatment regimens, comorbid conditions, and clinical therapies.

Development.

The SCHIZOM is specifically designed for use in routine clinical care settings as part of an outcomes management system for continuous quality improvement. A multi-institutional, multidisciplinary panel served as advisors to the University of Arkansas for Medical Sciences professionals who developed the SCHIZOM's clinical and methodological assessments for issues that surround the treatment and outcomes of schizophrenia. The assessments cover symptoms, general functioning, outcomes, prognostic variables, and treatment components. Because of the complexity of the diagnostic process in schizophrenia, the module currently does not include a diagnostic measure.

Components.

To be eligible for the SCHIZOM protocol, patients must be at least 18 years of age, able to cognitively understand the assessment process, and consent to participate. The module consists of an Initial Patient Interview, an Initial Informant Interview, Follow-up Patient and Informant Interviews, a Medical Record Review, and a User's Manual. Individual patient and aggregate provider reports are also available. Interviewers administer the patient and informant components, with informant interviews paralleling patient interviews. Follow-up interviews and the optional medical record review are conducted every six months after baseline for as long as the patient remains in treatment. In keeping with the SCHIZOM's emphasis on use in routine clinical settings, its basic components are relatively quick to complete. The initial interviews take 30 to 35 minutes to administer, and follow-up interviews take approximately 25 to 35 minutes.

Work is ongoing concerning the feasibility of self-administering the initial interviews and completing the Follow-up Patient Interview as a self-report questionnaire.

SCHIZOM individual reports can provide information about a patient's symptoms, general and mental health functioning, critical items such as suicidality, and prognostic indicators. Aggregate patient reports can provide both descriptive and inferential statistics, although the more complex data analyses for inferential statistics require data from large groups of patients. Aggregate reports provide baseline data on patient characteristics and initial symptoms. This information, when combined with follow-up assessments, can inform providers about the effectiveness of care for their schizophrenic patients. In the outcomes management systems, which are available via the NetOutcomes^TM website at www.NetOutcomes.net, aggregate reports are available for providers, programs, and facilities.

Reliability and Validity.

SCHIZOM's reliability and validity were tested using longitudinal data from 100 patients with schizophrenia who were recruited from a VA medical center and a state hospital. Ages ranged from 18 to 55 years. Test-retest correlations were excellent for six of the eight outcomes variables and moderate for suicidal and violent behaviors. Correlations were also excellent for all prognostic variables. Internal consistency was high for both symptom severity and the instrumental activities of daily living (IADL). The social relations variables had an alpha consistency of 0.59.

When the SCHIZOM's responses were compared with six different instruments, almost all measures were significantly correlated. Subject-informant agreement was fair between patient and informant data at baseline and follow-up for several items (legal problems, housing, occupational and educational activity, age of symptom onset, frequency of drug and alcohol use, and medication adherence) and poor for all other measures. There was no statistically significant correlation between patient and informant reports on suicidal behavior or violent behavior at follow-up. Agreement was higher for objective variables than for subjective variables and those unlikely to be directly observed. The SCHIZOM measured statistically significant change between baseline and follow-up for outcomes relating to symptom severity, violent behavior, and alcohol and drug use/abuse. Overall, the SCHIZOM detected change in the same areas and same directions as the validation instruments.

Potential Uses.

The SCHIZOM is broad enough in scope to address all aspects of a person's life that are affected by schizophrenia. It was shown to be useful for measuring the full range of outcome domains that are relevant to schizophrenia treatment, relatively brief to administer, and acceptable to respondents. The SCHIZOM has good reliability and concurrent validity and is able to detect change and absence of change in a patient's functioning and health status. Additional work is underway to enhance, expand and further validate the SCHIZOM, assessing its performance in clinical and demographic subgroups of the population with schizophrenia. Although the module has been translated into Spanish, there have been no validation studies conducted of this version.

Bibliography  Top

1. National Institute of Mental Health. Mental illness in America: The National Institute of Mental Health Agenda. 1997.

2. Rice DP, Miller LS. The economic burden of schizophrenia. Research Conference on the Economics of Mental Health. 1992;1-17.

3. Rupp A, Keith SJ. The costs of schizophrenia: Assessing the burden. Psychiatr Clin North Am. 1993;16(2):413-423.

4. Owen RR, Fischer EP, Booth BM, Cuffel BJ. Medication noncompliance and substance abuse among patients with schizophrenia. Psychiatr Serv. 1996;47(8):853-858.

5. Cuffel BJ, Alford J, Fischer EP, Owen RR. Awareness of Illness in Schizophrenia and Outpatient Treatment Adherence. J Nerv Ment Dis. 1996;184(11):653-659.

6. Fischer EP, Cuffel BJ, Owen RR, Jr., Burns BJ, Hargreaves W, Karson C, Lehman A, Shern D, Smith GR, Sullivan G. Schizophrenia outcomes module user's manual. 1996.

7. Cottler LB, Robins LN, Helzer JE. The reliability of the CIDI-SAM: A comprehensive substance abuse interview. Br J Addict. 1989;84:801-814.

8. Mayfield D, McLeod G, Hall P. The CAGE Questionnaire: Validation of a new alcoholism screening instrument. Am J Psychiatry. 1974;131(10):1121-1123.

9. Ewing JA. Detecting alcoholism: The CAGE questionnaire. J Am Med Assoc. 1984;252(14):1905-1907.

10. Hargreaves W, Cuffel BJ. Personal Profile. San Francisco, CA: University of California, Department of Psychiatry; 1992.

11. Gordon E, Kraiuhin C, Kelly P, Meares R, Howson A. A neurophysiological study of somatization disorder. Compr Psychiatry. 1986;27(4):295-301.

12. Tessler, R. and Gamache, G. Toolkit for Evaluating Family Experiences with Severe Mental Illness: To be used in conjunction with the family experiences interview schedule (FEIS). 1-135. 1994. Amherst,MA. The Evaluation Center@HSRI Toolkits.

13. Salkever D, Domino ME, Burns BJ, Santos AB, Deci PA, Dias J, Wagner HR, Faldowski RA, Paolone J. Assertive community treatment for people with severe mental illness: The effect on hospital use and costs. Health Serv Res. 2000;34(2):577-601.

14. Lehman AF. A quality of life interview for the chronically mentally ill. Eval Prog Plann. 1988;11:51-62.

15. McLellan AT, Kushner H, Metzger D, et al. The fifth edition of the addiction severity index. J Subst Abuse Treat 1992; 9: 199-213.

16. Miklowitz DJ, Goldstein MJ, Nuechterlein KH, Snyder KS, Doane JA. Expressed emotion, affective style, lithium compliance, and relapse in recent onset mania. Psychopharmacol Bull. 1986;22(3):628-632.

17. Miklowitz DJ, Goldstein MJ, Nuechterlein KH, Snyder KS, Mintz J. Family factors and the course of bipolar affective disorder. Arch Gen Psychiatry. 1988;45:225-231.

18. Derogatis LR, Lipman RS, Covi L. SCL-90: An outpatient psychiatric rating scale--Preliminary report. Psychopharmacol Bull. 1973;9(1):13-17.

19. Derogatis LR, Cleary PA. Confirmation of the dimensional structure of the SCL-90: A study in construct validation. J Clin Psychol. 1977;33(4):981-989.

20. Katon W. Somatization in primary care. J Fam Pract. 1985;21(4):257-258.

21. Ware JE, Jr., Snow KK, Kosinski M, Gandek B. SF-36 health survey manual and interpretation guide. Boston: Health Institute, New England Medical Center; 1993.

22. Cuffel BJ, Fischer EP, Owen RR, Jr., Smith GR, Jr. An instrument for measurement of outcomes of care for schizophrenia: Issues in development and implementation. Eval Health Professions. 1997;20(1):96-108.

Top