Challenges in Assessing Substance Use Patterns in Persons With Comorbid Mental and Addictive Disorders

Kate B. Carey

Assessment of substance use patterns can be distinguished from two related assessment goals. These consist of screening (i.e., identifying persons with addiction problems) and diagnosis (i.e., determining whether abuse or dependence criteria are met).

All three goals are important and relevant to persons with comorbid disorders. Nonetheless, this chapter focuses on substance use assessment for three reasons.

One, it is the least studied assessment goal in the comorbidity literature; very few published studies address this topic.

Two, substance use assessment is applicable to all persons in treatment for mental disorders. Use of illicit drugs or alcohol is more common than abuse, and information about use patterns may be desired to determine risk for medication-drug interactions and other health concerns.

Finally, substance use assessment should play a central role in the treatment of comorbid disorders. It serves as the basis for treatment planning and as a point of departure for outcome assessment.

It also constitutes the first step in conducting a functional analysis of drinking and/or drug use for an individual (Sobell et al. 1988).

Quantifying patterns of substance use allows for determination of increased versus decreased use, an outcome measure consistent with harm reduction approaches to treatment of substance misuse in the context of major mental disorders (Carey, in press).

In the addictions literature, a rich tradition of research exists on topics related to assessing alcohol and drug use patterns. Ample sets of instruments and guidelines for their use have been developed and standardized in substance abuse treatment settings.

However, importing such tools for use with persons with major mental disorders raises questions about their psychometric properties and other potential limitations.

This chapter briefly summarizes current approaches to, and problems with, substance use assessment. Because self-report measures continue to be widely used, emphasis is placed on factors generally considered to affect the accuracy of self-reported substance use. Next, concerns about the reliability and validity of self-reported substance use in persons with major mental disorders are discussed.

Finally, recommendations for enhancing the reliability and validity of assessment instruments are presented, highlighting areas in need of empirical research.

CURRENT APPROACHES TO SUBSTANCE USE ASSESSMENT

To borrow a scheme used by Skinner (1984), options available for sub- stance use assessment include (a) prospective methods, (b) retrospective methods, and (c) objective indicators.

Prospective methods consist of variants on self-monitoring. Self-monitoring reduces reliance on memory, and is generally regarded as the most accurate alternative to direct observation. Successful self-monitoring does, however, require a subject with the skills and motivation to complete the task.

Prospective information gathering also requires time. Retrospective methods involve asking the subject to report on past substance use over a designated time interval. Examples include the Addiction Severity Index (ASI) (McLellan et al. 1980), the Time Line Follow-back interview (TLFB) (Sobell and Sobell 1992), and various quantity-frequency methods (e.g., Cahalan and Room 1974; Polich et al. 1981).

Retrospective self-report is practical for most settings and is the most frequently used. However, its drawbacks include the potential for memory failure or other sources of distortion. Objective indicators include blood- or urine-based drug screens, breathalyzer tests, laboratory tests (e.g., gamma-glutamyl transpeptidadase, high density lipoproteins, mean corpuscular volume), collateral reports, and official records.

Each of these information sources has limitations. Breathalyzer tests and blood/urine screens yield information about recent use only (Schwartz 1988). Other laboratory tests identify medical consequences of substance use, but are generally sensitive only to prolonged high levels of use; furthermore, elevations are nonspecific to substance use.

None of these indices yields data on the pattern of substance use.

Collateral reports or other official records tend to be limited due to incomplete knowledge or representation of actual use history, and collaterals may be unavailable for some socially isolated subjects (Drake et al. 1993; O’Farrell et al. 1984). In the absence of a gold standard, confidence in the accuracy of assessment can be enhanced by adopting a convergent validity approach (Sobell and Sobell 1980).

This involves using multiple indicators that will tend to converge on a consistent picture of actuality. Significant discrepancies must be evaluated from a methodological perspective as well as allowing for subject-specific factors. In any given population, consideration must be given to 18 appropriate selection of measures as well as to ways in which their accuracy can be enhanced. Since retrospective self-reports continue to serve as the cornerstone of assessment, factors affecting the validity of self-reported substance use will be considered next.

Factors Affecting Validity of Self-Reported Substance Use

For substance use assessment to be useful in a treatment context, measures must be both accurate and sensitive to change. The literature on accuracy indicates that acceptable levels of reliability and validity are found for alcoholics' self-reports when recommended procedures are followed (e.g., O'Farrell and Maisto 1987; Sobell and Sobell 1980).

However, some samples and procedures have yielded less than impressive findings. Furthermore, the reliability and validity of self-reported drug use varies across both studies and types of drugs (Maisto et al. 1990). Test-retest reliability is infrequently reported and, when it is, shows only modest reliability coefficients.

Validity coefficients tend to be similarly moderate. A reasonable conclusion is that self-report data are "inherently neither valid nor invalid, but vary with the methodological sophistication of the data gatherer and the personal characteristics of the respondent" (Babor et al. 1990, p. 8). In the substance abuse field, questions have thus moved beyond "Are self-reports valid?" and are framed more as "When, and under what conditions, are self-reports valid?" (Brown et al. 1992).

It is this approach to evaluating the accuracy of self-reports that provides a framework for understanding the process of substance use assessment among persons with major mental disorders. In a discussion of the factors affecting the accuracy of self-reported substance use, Babor and colleagues (1990) highlighted four classes of variables. The first class yields characteristics of the respondent.

These include the respondent's state of sobriety at the time of assessment and the possible influence of a social desirability response set. The second class of variables includes aspects of the task that might enhance or detract from accurate responding. These include the degree of rapport between assessor and respondent, whether assurances of confidentiality can be made, the likelihood that selfreports will be verified, the criterion interval reported on, and the clarity of the questions.

Motivational factors constitute the third class of variables affecting accuracy of self-report. Obvious short-term goals (e.g., to obtain treatment or to avoid arrest) must be considered, as well as the fear of potentially judgmental attitudes or other threats to the respondent's self-esteem. Cognitive processes constitute the 19 fourth type of variables influencing self-report accuracy.

Impairment of attentional processes, verbal comprehension, or retrieval will interfere with the accuracy of an assessment. Cognitive processes may be impaired due to recent alcohol or drug use, to situational stress or anxiety, or to associated psychiatric syndromes such as depression.

In addition, recall of past behavior is subject to information-processing biases, so that recall is disproportionately influenced by salient and/or very recent events.

Application to Persons With Severe Mental Disorders

There are reasons for concern about the reliability and validity of substance use assessment in persons with severe mental disorders (e.g., schizophrenia, bipolar disorder, major depressive disorder).

Several studies suggest significant underreporting of substance abuse in this population (e.g., Safer 1987; Shaner et al. 1993; Test et al. 1989). No formal reliability or validity studies have been published on substance use assessment, but the accuracy of screening measures has been empirically addressed. For example, the criterion validity of screening measures (e.g., the Michigan Alcoholism Screening Test) has been evaluated in psychiatric settings.

Results indicate adequate sensitivity but low specificity (Drake et al. 1990; Teitelbaum and Carey, in press), suggesting that population differences may affect the psychometric properties of many of the standard industry tools.

A recently completed literature review (Teitelbaum and Carey, in press) found surprisingly few studies addressing the reliability of alcohol/drug screening or diagnostic measures used with psychiatric patients.

In this population, which is characterized by fluctuating mental status, the issue of test-retest stability is a fundamental psychometric concern. 

~~~
Continued on source document:
NIDA Research Monograph, Number 172