Stereotype Threat and College Academic Performance: A Latent Variables Approach *

Stereotype threat theory has gained experimental and survey-based support in helping explain the academic underperformance of minority students at selective colleges and universities. Stereotype threat theory states that minority students underperform because of pressures created by negative stereotypes about their racial group. Past survey-based studies, however, are characterized by methodological inefficiencies and potential biases: key theoretical constructs have only been measured using summed indicators and predicted relationships modeled using ordinary least squares. Using the National Longitudinal Survey of Freshman, this study overcomes previous methodological shortcomings by developing a latent construct model of stereotype threat. Theoretical constructs and equations are estimated simultaneously from multiple indicators, yielding a more reliable, valid, and parsimonious test of key propositions. Findings additionally support the view that social stigma can indeed have strong negative effects on the academic performance of pejoratively stereotyped racial-minority group members, not only in laboratory settings, but also in the real world.

The academic underperformance of racial and ethnic minorities at selective colleges and universities is well-documented (Charles et al. 2009; Bowen and Bok 1998; Massey et al. 2003). Minority students, particularly black and Hispanic students, tend to perform less well on exams, graduate at lower rates, and earn lower grades than one would predict on the basis of objective characteristics such as family income, parental education, and SAT scores (Espenshade and Walton-Radford 2009). In particular, black students perform less well than equally-qualified whites, even after introducing exhaustive controls for background differences (Bowen and Bok 1998). A variety of explanations have been advanced to account for racial gaps in academic performance, including intergroup differences in resources (Massey et al. 2003), oppositional identity (Fordham and Ogbu 1986), and stereotype threat (Steele 1988a).

Evidence suggests that differences in access to financial, social, human, and cultural capital play an important role in explaining academic performance differentials; but they by no means eliminate intergroup gaps (Massey et al. 2003). Quantitative evidence in support of oppositional identity—the idea that minorities resist educational achievement as a racial betrayal and “acting white”—is remarkably weak (Downey 2008). More support has been found for stereotype threat—the view that minority students underperform because of pressures created by negative stereotypes about their group. Steele’s original studies examine how black undergraduates at Stanford responded to race primes in carrying out everyday academic tasks (Steele 1988a; Steele 1988b). Since then, the stereotype threat model has been validated in numerous laboratory experiments (Major and O’Brien 2005; Spencer, Logel, and Davies forthcoming), intervention field experiments (Aronson, Fried, and Good 2002; Cohen et al. 2006, 2009; Steele 1997; Walton and Cohen 2007), and meta-analytic studies of laboratory and field experiments (Walton and Spencer 2009).

At the same time, other studies have called into question the validity of stereotype threat. For example, a recent field experiment primed high school students in a treatment group to think about their race and/or gender before taking higher-stakes Advanced Placement exams or Computerized Placement Tests and found no effects of race/gender priming on performance (Stricker and Ward 2004). Re-analysis of the Stricker and Ward (2004) study by Danaher and Crandall (2008), however, questioned Stricker and Ward’s conclusions, instead arguing that their findings indeed support stereotype threat if a different interpretation of the results is applied. In general, then, experimental studies tend to support the theory of stereotype threat, and laboratory studies strongly support it.

In terms of non-experimental research, some observational studies point to a rather tenuous association between stereotypes and academic or occupational performance. For example, one recent study exploits patterns in military performance data and college grade performance—adjusting for scores on the Armed Services Vocational Aptitude Battery (ASVAB) and the Scholastic Aptitude Test (SAT), respectively—to see if they predict performance in military jobs and college GPA differentially for blacks and whites. It concluded that the relationships posited by stereotype threat do not hold in observational settings as well as they do in the laboratory (Cullen, Hardison, and Sackett 2004). Other studies use different domains, such as women in mathematics and geographic differences in the extent of stereotype threat, but come to similar conclusions, finding limited support for stereotype threat theory (Cullen, Waters, and Sackett 2006; Pope and Sydnor 2010). Although these studies underscore the fact that eliminating stereotypes cannot be expected to erase the black-white achievement gap in real-world settings entirely, they acknowledge that further testing of stereotype threat in non-experimental settings is needed (Cullen, Hardison, and Sackett 2004).

Other survey-based studies on stereotype threat have come to different conclusions, detecting relatively strong support for stereotype threat in real world settings in selective colleges and universities (Charles et al. 2009; Fischer and Massey 2007; Massey and Fischer 2005; Massey and Mooney 2007). Although these studies appear to confirm hypotheses derived from the stereotype threat model, they are nonetheless characterized by methodological shortcomings. For example, they measure key theoretical constructs using only summed indicators and predict relationships using ordinary least squares. These techniques lead not only to inefficiencies but also potential biases. Here we overcome these shortcomings by drawing on data from the National Longitudinal Survey of Freshmen (NLSF) to develop a latent construct model of stereotype threat in which theoretical constructs and equations are estimated simultaneously from multiple indicators, yielding a more reliable, valid, and parsimonious test of key propositions. By elaborating a model of the psychological processes by which stereotype threat undermines performance, our analysis lends additional support to the view that social stigma can indeed have strong negative effects on the academic performance of pejoratively stereotyped minority group members, not only in laboratory settings, but also in the real world.

STEROTYPE THREAT AND ACADEMIC PERFORMANCE

In their original analysis of NLSF data, Massey and Fischer (2005) synthesized findings from prior laboratory experiments to derive an Ordinary Least Squares model of stereotype threat that could be estimated using survey data. In their model specification, the existence of negative societal beliefs about a minority’s intellectual ability was hypothesized to affect members’ academic performance through two psychological pathways. Figure 1 shows the two pathways along with the direction of the effects hypothesized under the theory of stereotype threat. These pathways occur to the extent that individual minority group members internalize or externalize negative stereotypes about their group’s intelligence. Externalization (EXT) occurs when minority students believe that members of the majority perceive them stereotypically as less intelligent and thus judge them invidiously in the performance of academic tasks. Internalization (INT) occurs when minority students themselves buy into the stereotype of intellectual inferiority at some level and come to fear that it may pertain to their own academic abilities.

NOTE: The boxes overlaying arrows contain ‘+’ or’−’ signs that indicate the expected direction of the effect based on the theory of stereotype threat.’+’ represents an expected positive relationship between the two concepts, whereas’−’ represents an expected negative relationship.

The internalization and externalization of negative stereotypes yield two separate pathways to underachievement. Minority students follow the internalization pathway when they “disidentify” with academic performance as a determinant of self-esteem to relieve the psychological distress of potentially confirming the negative stereotype (Steele and Aronson 1995; Crocker, Major, and Steele 1998), typically achieved through a reduction of academic effort (EFF). The reduction of effort relieves the psychological pressure, because when minority students earn low grades they can attribute their failure to measure up to their lack of effort rather than any ostensible limitations of their intellectual abilities. They follow the externalization pathway when they become preoccupied with possibly confirming the negative stereotype rather than concentrating on the task at hand, yielding an academic performance burden (APB) that majority students do not face. We refer to the effects of internalization and externalization on academic performance as they operate through academic effort and performance burden as the “internalization mechanism” and “externalization mechanism,” respectively. Although Massey and Fischer (2005) thought that externalization could affect disinvestment and internalization might affect the performance burden, this proved not to be the case empirically.

As noted above, most prior research on stereotype threat consists of laboratory experiments in which minority students are randomly assigned to a treatment group in which race is primed before administering an intellectual task, and a control group where race is not primed. These studies demonstrate the ease with which stereotype threat can be created and manipulated experimentally to push academic performance up or down (see Spencer et al. forthcoming for a review). Although laboratory experiments confirm the hypothesis of stereotype threat with a high degree of internal validity, such studies lack external validity in the sense that results cannot readily be generalized to real world settings such as a college campus or classroom (Campbell and Stanley 1963).

It is not much of a conceptual leap to speculate that minority students attending a range of selective, four-year, residential colleges might underperform academically owing to stereotype threat. Minority college students can expect to experience racial priming from a variety of sources in the course of daily life—from teachers, students, administrators, the media, and society in general. Moreover, because selective colleges and universities exhibit many characteristics of what Goffman (1962) calls total institutions, stereotypes are likely to carry particular weight on campus. With young students segregated in dorms, subject to a strong collective labeling, exposed to social sanctions from each other, and vulnerable to academic sanctions from faculty, even subtle evocations of minority status may trigger heightened self-awareness to undermine academic performance (McGuire, McGuire, and Winton 1979).

In order to detect the potential influence of stereotype threat on campus, researchers began to add items to student surveys to measure constructs such as internalization, externalization, academic performance burden, and academic effort. Studies based on survey data generally provide empirical support for the view that the internalization and externalization of stereotypes independently influence academic underperformance (Charles et al. 2009; Fischer and Massey 2007; Massey and Fischer 2005; Massey and Mooney 2007). At the same time, unlike laboratory experiments, it is difficult to explicitly measure the extent of racial priming that may influence college performance.

As already noted, previous social survey-based studies of stereotype threat suffer from serious methodological and theoretical problems. First, they employ simple summed indices to measure internalization, externalization, and the academic performance burden, implicitly assuming that each item in the summed index carries equal weight in the underlying construct when, in fact, different items are likely have different measurement properties with different errors, especially across groups. A latent variables approach to model each stereotype threat concept, on the other hand, allows us to estimate factor loadings for the indicators of each latent variable—maximizing efficiency and accuracy when fitting measurement models for each latent construct and empirically examining each indicator’s statistical significance and magnitude (i.e. the weights of each indicator) varies.

Second, previous studies using ordinary least square regressions rely on the theoretical assumption that the mechanisms of internalization and externalization are empirically distinct paths. Using multiple, distinct regressions to test paths between each construct not only prevents a formal, empirically-based test of whether the mechanisms of internalization and externalization are theoretically and statistically distinct, modeling constructs and equations separately also reduces statistical efficiency and introduces potential bias if in fact the internalization and externalization mechanisms are not distinct in practice. A method like structural equation modeling that relies on latent variables fit using measurement models, on the other hand, allows for the simultaneous modeling of relationships between all stereotype threat constructs, estimating error correlations between internalization and externalization and academic effort and performance burden to empirically test whether the mechanisms are distinct.

Third, previous studies rely on just a single variable—number of hours studied—to measure academic effort, assuming without justification that this single indicator fully captures the complex psychological reality of disidentification. The use of a single measure to capture academic effort is problematic for both substantive and theoretical reasons. Substantively, it is well-known that multiple measures yield more reliable scales than single indicators. Theoretically, we argue that disidentitifcation--the removal of academic achievement from the domain of self-evaluation and self-esteem—is likely to be expressed in both subjective and objective ways. Hence, in addition to hours studied, we add two subjective indicators of academic effort: self-assessed academic effort and the self-rated importance of learning course material.

Although we believe using multiple indicators to measure academic effort offers a more accurate strategy, we acknowledge the relative subjectivity of all indicators—not only of academic effort, but of each stereotype threat construct. We address this limitation by leveraging one of the advantages of latent variables analysis: the ability to test whether each indicator improves the fit of its respective latent construct. Throughout this study we make the fundamental assumption that students with higher levels of stereotype internalization and/or externalization have been more “primed” to think about race in their daily lives than those who report lower levels. As a broader check to our model, we also fit our model for white students by way of comparison to show that stereotype threat from negative racial stereotypes is indeed unique to minority students.

DATA AND METHODS

Sample

The present study seeks to remedy the foregoing limitations to obtain more accurate estimates of effects associated with two the pathways specified in the Massey-Fischer (2005) model. As before, our data come from the National Longitudinal Survey of Freshmen (NLSF), a stratified random sample of 3,924 college students who entered 28 selective four-year colleges and universities throughout the United States in the fall of 1999 (see Massey et al. 2003; Charles et al. 2009). Students were interviewed in-person during the fall of their freshman year to collect retrospective information about social and educational experiences from childhood through high school. They were subsequently re-interviewed by phone every spring from 2000 through 2003 to learn about their social and academic experiences in college. Here we draw upon data gathered in the baseline and first two follow-up surveys, which had respective response rates of 86%, 96%, and 90%. We selected Hispanic and black students for study because, as stigmatized minorities, they are most susceptible to stereotype threat. For comparison purposes, however, we run the full structural model described below for white students, confirming that the stereotype threat model does not hold for these groups not historically stigmatized with negative stereotypes about their academic or intellectual abilities. Multiple imputation of five datasets was used to deal with item non-response (which ranged between .5% and 5%, as shown in Table 2 ), yielding a sample of 1,710 students, 918 black and 794 Hispanic. Comparisons of the stereotype threat model estimated for white students rely on a sample of 853 whites after multiple imputation. To make sure that results were not driven by imputation procedures, analyses were replicated on a sample of 1,579 students based on list-wise deletion of cases with missing data, for a final count of 839 black students and 740 Hispanic students. Results from analyses carried out using the imputed data or the list-wise deleted data yield the same substantive findings (results available upon request).

Table 2

Means and Standard Deviations of Stereotype Threat Constructs and Control Variables, by Race, with Stereotype Threat Constructs Represented as Summed Indexes of Their Respective Indicators and Control Variables 1

NOTE: Standard deviations in parentheses. Descriptive statistics based on sample resulting after list-wise deletion.

1 Means of each construct of stereotype threat and the index of racial ingroup exposure (IEX) are shown as indexes based on sums of their respective indicators.

2 This item was reverse-coded to accurately reflect its negativelyl valenced properties in comparison to the other items in the externalization construct.

Variables

The dependent variable in our analyses is academic performance, which we measure as average GPA earned by students during the spring and fall of 2000-or the spring of their freshman year and the fall of their sophomore year. We derive estimates of the theoretical constructs shown in Figure 1 —INT, EXT, EFF, and APB—using multiple indicators drawn from the survey, which are listed in Table 1 . The items we drew upon were explicitly developed by the survey’s authors in a conscious effort to capture different facets of stereotype threat. They are not a convenience sample of items pressed into the service of measurement ex post facto, but were intended from the start to get at the issue of stereotype threat.

Table 1

Indicators and Dimensions of Stereotype Threat Used in the Analyses

We operationalize the internalization construct (INT) using three items that assess the degree to which respondents believe that members of their own group are lazy (v1 in Table 1 ), unintelligent (v2), and give up easily (v3). Externalization (EXT) is indexed by four items on whether respondents think whites (v4) and Asians (v5) discriminate and the degree to which they think other instructors (v6) and students (v7) base academic evaluations on group stereotypes.

Unlike Massey and Fischer, here we measure academic effort (EFF) using three items: the average number of hours studied per seven-day week during the academic year (v8), how important the respondent believes it is to try hard to learn the course material in college classes (v9), and how much self-reported effort they estimate they have put into their studies over the past year (v10). Finally, academic performance burden (APB) is assessed by five items that ask respondents to report on the degree that instructors will think less of them if they have difficulty in class (v11), their individual performance reflects positively (v12) or negatively (v13) on their group, their own apprehensions about appearing foolish or stupid before others (v14), and the extent to which they believe not doing well academically will cause people to look down on others like them (v15).

In addition to measuring these theoretical constructs, all models include institution-level fixed effects. Our models also control for a variety of background variables that prior work using the NLSF has shown to be important in determining academic performance (see Charles et al. 2009; Massey et al. 2003), including gender, having a foreign-born parent, number of siblings in the household of origin, whether the student was raised in an intact, two-adult household, parental educational attainment, percent of college paid by family members, and the average number of hours worked for pay during the typical academic week in the first two years of college. We also control for differences in racial exposure and identity—the percentage of high school friends who were black or Hispanic, perceived social distance from whites, the strength of in-group identity, and skin tone. Differences in the degree of academic preparation are held constant by measuring the number of advanced placement courses taken in high school, cumulative high school GPA, and self-rated academic preparation.

We deploy these controls to help eliminate equally plausible alternative interpretations of the study’s results. For example, it is possible that students clutch to negative stereotypes as a way to rationalize their own poor performance, but we try to eliminate this alternative by controlling for students’ prior academic performance. Similarly, we try to control for the possibility that students attended poorer-performing schools and therefore arrive less-well prepared even if they received high grades by controlling for measures of social class—like neighborhood and school racial composition and percent of college paid for by family. Of course these are by no means perfect or exhaustive measures, but they allow us to help isolate the effects of internalization and externalization, via academic effort and academic performance burden, respectively, on college academic performance.

Methods

Figure 2 shows the path diagram used to model stereotype threat. Each circle represents a latent construct from the theoretical model shown in Figure 1 , and arrows between these latent constructs represent regression paths to be estimated. Squares represent observed variables that are used to measure each construct and are identified by their corresponding variable number and explained above, as well as in Table 1 .

Path Diagram for Stereotype Threat 1

1 See Table 1 for specification of measures v1–v16. Arrows signify regression paths. Lines without arrows signify estimated error correlations.

We use multiple-group structural equation modeling (SEM) to investigate our research questions (see Bollen 1989). Broadly, SEM analysis allows us to take into account the measurement properties of the indicators underlying each of our key theoretical constructs—internalization, externalization, academic effort, and academic performance burden—and to assess whether they function similarly for black and Hispanic students. More specifically, multiple-group SEM simultaneously fits the model shown in Figure 2 (consisting of measurement models of internalization, externalization, etc., regression paths between each of these constructs, and error correlations between the internalization and externalization mechanisms) for each race group, leading to two different sets of estimates for the factor loadings on the indicators of each construct and for the regression paths between constructs. If the measurement properties of the indicators for each theoretical construct do not function similarly for blacks and Hispanics, and measurement differences are ignored, then apparent differences between groups may not, in fact, reflect real differences. Instead, they may result from measurement differences between groups.

Prior work investigating stereotype threat using OLS regression modeling with summed multi-item indices assumes that the unique variance of each indicator is itself a robust measure of the construct for which it is a proxy, essentially meaning that OLS takes all measures of latent constructs to be equally valid. SEM allows us to relax that key assumption by measuring the latent constructs based on only the overlapping variation between indicators of each latent construct, as shown in Figure 3 . Measuring each latent variable based only on the joint variation between all its indicators, each indicator’s factor loading serves as a weight for the latent variable. As the unique variance of an indicator approaches zero, its standardized factor loading approaches one, which equals a perfect correlation between that indicator and the latent variable it measures. The result is that SEM controls for measurement error, leading to more accurate measures of the latent constructs if in fact factor loadings are all significant and vary in magnitude, indicating uneven contributions to their respective latent construct.

Measurement of a Latent Variable as the Joint Variance of Three Indicators (u1, u2, and u3)

In addition, by allowing the factor loadings to vary between blacks and Hispanics, a latent variables approach provides additional flexibility by allowing the “contributions” of each indicator to the measurement of each latent construct to vary by race. For example, in the measurement of internalization, perceptions of one’s own group’s intelligence may receive a larger weight for blacks than for Hispanics, for whom perceptions of one’s own group’s hard work may contribute more heavily to the measurement of internalization.

Finally, fitting the full SEM shown in Figure 2 simultaneously for blacks and Hispanics (each with their own sets of measurement models, regression paths between constructs, and error correlations) and SEM’s use of multiple dependent variables (i.e. internalization, externalization, etc.) allow us to examine error correlations between relevant latent variables (internalization with externalization and effort with academic performance burden) and to test whether the internalization and externalization mechanisms are truly distinct, which has been implicitly assumed in prior work. If the two mechanisms are not truly distinct, this paper and future work will need to investigate additional paths through which stereotype threat might operate.

Because structural relationships are conventional regression paths estimated using maximum likelihood methods, SEM makes many of the same assumptions as ordinary least squares regression. For example, a key assumption of SEM is that, taken together, indicators of a given latent variable and the latent variable itself have a joint multivariate normal distribution. However, violating this assumption does not seem to be consequential for the integrity of the structural model (see Bollen 1989). In addition, whereas OLS regression loses efficiency by making the assumption that error terms between variables are uncorrelated, SEM does not. Indeed, because SEM simultaneously models multiple structural relationships, it estimates correlations between the error terms of endogenous variables and thereby increases efficiency in the estimates themselves (see Bollen 1989).

To test whether a latent variables approach offers a more accurate and efficient modeling strategy, we carried out two preliminary steps. First, we tested whether each indicator improved the fit of each latent stereotype threat construct, which we accomplished by adding each indicator to the measurement model one at a time. We found that the fit improved with each additional indicator (results available upon request) and that, together, all indicators of a given construct more accurately capture the concept being modeled. Second, we also tested whether it is sufficient to assign equal weight to each indicator in a given latent construct by examining the magnitude and significance of individual factor loadings. Here we found that magnitudes vary across indicators, highlighting the need for latent variables as opposed to summed indices (factor loadings shown in Appendix B).

We begin by using separate measurement models to estimate internalization, externalization, academic effort, and academic performance burden for each group. After validating our measurement models—i.e. establishing their reliability and validity for the aggregate group of blacks and Hispanics—we compare our results to measurement models corresponding to summed indices that have been used in the past. Once the measurement components of the analyses are complete, we estimate the full multiple-group SEM model shown in Figure 2 to investigate directly whether the model holds for both blacks and Hispanics, and identify whether there are intergroup differences in the structural effects of stereotype threat. The SEM is based on the following equations, modeled simultaneously:

PERF ∗ i = β 0 i + β 1 i ( INT ∗ ) + β 2 i ( EXT ∗ ) + β 3 i ( EFF ∗ ) + β 4 i ( APB ∗ ) + β 5 i ( male ) + … + β 17 i ( self - rated prep ) + e 1 EFF ∗ i = γ 0 i + γ 1 i ( INT ∗ ) + γ 2 i ( male ) + … + γ 14 i ( self - rated prep ) + e 2 APB ∗ i = α 0 i + α 1 i ( EXT ∗ ) + α 2 i ( male ) + … + α 14 i ( self - rated prep ) + e 3 INT ∗ i = δ 0 i + δ 1 i ( male ) + … + δ 13 i ( self - rated prep ) + e 4 EXT ∗ i = ξ 0 i + ξ 1 i ( male ) + … + ξ 13 i ( self - rated prep ) + e 5

where i represents a separate estimation equation for blacks and Hispanics, respectively, and * represents latent variables/constructs.

After determining that intergroup differences do exist, we estimate final models for both groups. In the full SEM models, we control a number of background characteristics, which are shown in Table 2 . Finally, we carry out a number of sensitivity analyses to examine the robustness of the stereotype threat model given the subjective measures we use to operationalize each latent construct. Although the variables we used were designed specifically to measure each stereotype threat concept, we first verify the reliability of each latent construct even if different indicators are used to measure each construct. Our goal is to ensure that each construct is not heavily reliant on a single indicator to capture the complex underlying concept. Fitting the full SEM fifteen times, each time excluding one of the fifteen indicators of stereotype threat described in the variables section, an examination of the magnitudes and significances of the paths indicate robustness against slight changes in latent variable specification (shown in Appendix C).

These results indicate that no single indicator is driving the relationship between each latent measure of stereotype threat (results available upon request). Second, we verify the value-added of each indicator by examining the magnitudes and significances of the factor loadings of each indicator on its respective latent construct, as shown in Appendix B. The significance of each indicator suggests that each additional item used adds important variation to the measurement of each latent variable, thus more fully capturing each underlying latent concept. The differential magnitude of each factor loading confirms that differential weights on indicators are indeed necessary in order to most accurately model each latent construct of stereotype threat.

MEASURING THEORETICAL CONTRUCTS

Table 2 shows means of indicators for each theoretical construct and the corresponding summed indices (assuming no measurement differences between groups) for blacks and Hispanics with whites shown for comparative purposes. Although there are some differences in the means of summed indices and their constituent items between blacks and Hispanics (most noticeably with Hispanics having lower mean levels of externalization and performance burden), the largest overall differences are between whites on the one hand and blacks and Hispanics on the other. Whites have significantly lower levels of externalization and academic performance burden than either blacks or Hispanics. In terms of the summed indicator for internalization and academic effort, however, the mean values for Hispanics, blacks, and whites are nearly identical.

Even though the means of the summed scales for internalization and academic effort are similar for blacks, Hispanics, and whites, it is nonetheless possible that the constituent items have different weights in the latent constructs. Table 3 shows results comparing each of the four theoretical constructs estimated by group using two measurement models: a full model that estimates errors and factor loadings empirically (a confirmatory factor analysis) and a restricted model that constrains factor loadings to be 1 and assumes measurement error to be 0 (the constrained factor analysis, which is comparable to a summed indices approach). Table 3 reports three common goodness-of-fit measures for each of the models: the model chi-square, the Root Mean Square Error of Approximation (RMSEA), and the Comparative Fit Index, or CFI (Bollen 1989). Appendix B further demonstrates the improved accuracy of using latent variables versus summed indices by documenting the significance and varied magnitudes of each of the indicators of the given stereotype threat constructs.

Table 3

Results of Multiple Group Analysis Comparing Full Measurement Models to Summed Indexes for Black and Hispanic Students 1

1 Blacks and Hispanics are estimated separately for purposes of estimating the fit of both the confirmatory factor analyses and the summed indexes. The Difference in chi-square statistic displays a goodness-of-fit test between the confirmatory and constrained factor analyses for each latent construct using the Satorra-Bentler Scaled Chi-squared Difference Test.

2 Factor loadings for all indicator variables are set to 1 (within and across race and immigrant generation groups). Residual variances are equal across groups for each indicator variable.

3 Academic performance is not shown here because it has a single indicator and therefore a CFA-summed index comparison does not apply.

All in all, these results suggest that using a measurement model that allows for differences in measurement properties across groups is warranted when assessing the relationship between theoretical constructs pertaining to the stereotype threat model. Our findings imply that if measurement differences are ignored, estimates of the structural relations between constructs may be biased because of measurement differences. Thus we employ unconstrained latent variables in our SEM analysis of stereotype threat, yielding estimated effects that should be free of this bias.

PATHWAYS TO UNDERPERFORMANCE

Table 4

Changes in Structural Equation Model Fits by Estimation Strategies for Black and Hispanic Students (N=1380) 1

1 List-wise deletion leads to a sample of 735 black students and 645 Hispanic students. Models include institution-level fixed effects.

2 Difference in Chi-square statistic indicates difference from the model where blacks and Hispanics are esimated separately.

Table 5 presents estimates the three different models estimating the effects of internalization and externalization on academic performance through academic effort and the academic performance burden, thus fully operationalizing the diagram shown in Figure 1 . Given the nature of social survey studies, these results do not establish causal mechanisms, of course, but causality has already gained strong support in Steele’s original experimental work (Steele 1988a, 1988b). Here we seek only to detect evidence of stereotype threat in the real world. Nonetheless, by deploying a variety of methods that come to similar conclusions our findings withstand various tests of robustness.

Table 5

Comparison of Structural Equation Model (SEM) and Path Analysis Results Showing Relationships Between Internalization, Externalization, Disidentification, Academic Performance Burden and Academic Performance with and without Racial Exposure Control Measures, by Race

NOTE: Reporting Standardized coefficients from sample resulting from multiple imputation of five datasets. Standard errors in parentheses. Models include institution-level fixed effects.

1 Structural residual error correlations are estimated between internalization and externalization and between academic performance burden and disidentification. Measurement error correlations are estimated between indicators v6 and v7 and v10 and v11 (See Table 1 for list of variables by number).

2 Factor loadings for the indicators of each construct of steretoype threat in this full model are shown in Appendix B.

3 Measures of Ingroup Exposure (IEX) include: Strength of racial ingroup identity (0–10), social distance from whites growing up (0–10), percent of friends of same race as respondent while growing up (scaled 0–10), and percent black or Hispanic neighborhood composition growing up (scaled 0–10). IEX measures reported in Appendix A.

In order to examine whether the stereotype threat model holds for non-minority students, model 1 fits latent variables for stereotype threat for whites, blacks and Hispanics, but excludes the racial in-group exposure variables. Racial in-group exposure variables are only measured for blacks and Hispanics and are therefore excluded so that whites can be included in the model. Model 2 of Table 5 , together with Figure 4 , show the results of a final latent variables model for only blacks and Hispanics, including the racial in-group exposure variables, in which parameters were allowed to vary by racial group. Regression coefficients capturing the effects of each control variable on latent theoretical constructs of model 2 are shown in Appendix A. No systematic differences in the influence of controls were found between Hispanics and blacks. Factor loadings for each of the indicators of the latent variables are shown in Appendix B. The significance and varied magnitudes of the indicators confirm the value-added of each indicator in measuring the given latent variable. Finally, model 3 of Table 5 presents results from a path analysis model of stereotype threat, in which summed indices of respective indicators are used in place of latent variables when estimating relationships between the constructs postulated by stereotype threat. Though magnitudes and significances between paths are generally attenuated and weaker, results from the path analysis indicate similar patterns of relationships between constructs.

Direction of Significant Paths in Full Structural Equation Model, by Race, Compared to Expected Direction of Effects Based on Stereotype Threat Theory

Note: E=expected direction of effect based on stereotype threat theory, B=black students, H=Hispanic students. Only significant paths are shown with a + or − sign; all 0’s indicate statistically zero coefficients.

As noted above, the full structural model shown in model 2 of Table 5 and in Figure 4 fits the data quite well. The externalization of negative stereotypes is associated with a significant increase in academic performance burden for both black and Hispanic students. A black or Hispanic student whose level of externalization is one standard deviation above that of another student’s of his/her same racial group is, on average, expected to experience a 0.08 and 0.03 standard deviation higher level of performance burden, respectively, net of controls—consistent with expectations derived from stereotype threat theory. Moreover, as hypothesized by the model, a black or Hispanic student whose level of performance burden is one standard deviation above that of another student of his/her same race is expected to perform 0.06 and 0.03 standard deviations lower, respectively, net of controls. The direct effect of externalization on academic performance is statistically zero for both groups, meaning that externalization affects academic performance through the posited mechanism of academic performance burden.

As expected, the effect of internalization on academic effort is negative for both groups. Among black students, one standard deviation more internalization on average yields a 0.18 standard deviation decline in academic effort. Among Hispanic students, the association is slightly larger—a Hispanic student who is one standard deviation higher in his/her level of internalization is expected to put forth 0.21 standard deviations less academic effort on average, net of controls, compared to a student whose level of internalization is one standard deviation lower. Consistent with theoretical predictions, the effect of academic effort on performance is positive and significant for both groups, so that the indirect effect of internalization through academic effort is negative. Internalizing negative stereotypes brings about disidentification and a reduction of academic effort which, in turn, yields lower grades. The effect, however, is greater for Hispanics than blacks. Whereas a black student who puts forth one standard deviation more effort receives a grade-point that is on average 0.17 standard deviations higher than another student with the same background characteristics, the same relationship is associated with a grade-point that is 0.36 standard deviations higher for a Hispanic student with the same background characteristics as another Hispanic student but for putting forth one standard deviation worth more academic effort.

In addition, the direct effect of internalization on academic performance is positive and significant for both black and Hispanic students, implying that the internalization mechanism accounts for a significant but nonetheless incomplete portion of the total effect of internalization on academic performance. A black student who experiences one standard deviation more internalization than another black student with the same background characteristics on average earns a grade-point that is 0.11 standard deviations higher. On the other hand, a Hispanic student who experiences one standard deviation more internalization than another Hispanic student with the same background characteristics earns a grade-point that is an even larger and more highly significant 0.17 standard deviations higher on average. The fact that internalization has a positive direct effect on performance runs counter to the theory of stereotype threat and will be explored more in the discussion below. Here we simply note that the magnitudes of the direct paths between internalization and performance are of notably smaller magnitude for both blacks and Hispanics than are the paths posited through the internalization mechanism via academic effort. Finally, error correlations between internalization and externalization on the one hand and academic effort and performance burden on the other are non-significant and of small magnitude, confirming that the dual-pronged model of stereotype threat via the internalization and externalization mechanisms operate as two distinct pathways to underperformance among minorities.

Unlike for blacks and Hispanics, when the stereotype threat model is fit for whites, results from model 1 of Table 5 show significant structural paths only for the internalization mechanism. Among whites, a student who perceives one standard deviation more negative stereotypes about his/her group’s intelligence, hard work, and persistence puts forth, on average and net of controls, 0.14 standard deviations less academic effort than an otherwise comparable white student who does not perceive these stereotypes. The magnitude of the effect, however, is about 75% as large as that for blacks and Hispanics, on average. As logically expected, however, a one standard deviation increase in academic effort is associated with an increase in performance of roughly similar magnitude and significance as for Hispanics, on average, net of controls. The lower magnitude of the effect of effort on performance for blacks, however, is salient—a one standard deviation increase in academic effort is associated with an increase in performance that is only about half as large as the average of that for whites and Hispanics (0.15 for blacks compared to roughly 0.30 for whites and Hispanics, on average).

As with blacks and Hispanics, the magnitude and significance of the direct association between internalization and performance is positive, counter to the theory of stereotype threat, again suggesting that the internalization of negative stereotypes influences grade-performance through channels other than a reduction in academic effort. Some of this positive, direct effect may be due to minorities wanting to show that they are “not like others” of the same, negatively stereotyped racial group. The externalization mechanism, however, is entirely non-significant for whites. Furthermore, the internalization and externalization mechanisms do not appear to be theoretically distinct concepts. As discussed in more detail in the discussion, model 1 results for whites indicate whites’ general lack of fit within the stereotype threat framework, particularly with regard to the externalization of negative stereotypes.

CONCLUSION: STEREOTYPE THREAT IN REAL WORLD SETTINGS

Stereotype threat theory has historically been tested in laboratory experiments using domestic college students engaging mundane academic tasks within a short period of time. More recently studies have examined the degree to which stereotype threat functions in natural social settings using survey data spanning multiple years of college. Although some of these studies have generally confirmed the operation of stereotype threat on campuses of selective colleges and universities (Charles et al. 2009; Massey and Fischer 2005; Massey and Mooney 2007), they have relied on relatively weak modeling strategies that do not exploit the full variation in the measures of stereotype threat and thus limit the accuracy of the model and more complete testing of its validity. This study was the first to improve measurement by deploying a latent variable structural equation model with more robust and realistic measurement assumptions.

Because the causal effects of negative stereotypes have been generally supported through randomized experiments, this study aims to accurately model and test whether the causal mechanisms demonstrated in laboratory experiments can be detected in the real world. To this end, this analysis lends further credibility to survey-based research on stereotype threat and lends further credibility to the notion that the internalization and externalization of negative stereotypes undermines the academic performance of minority students in theoretically expected ways, even after measurement error is effectively modeled and controlled.

This growing body of survey-based research on stereotype threat should be viewed as a complement to existing experimental research. Although experimental research is of utmost importance in establishing the causal relationships between race primes, negative racial stereotypes, and academic performance, its reliance on relatively small numbers of students brings with it a loss of generalizability outside labs and across universities. Survey-based studies offer a valuable complement in their ability to gain generalizability across institutions once the validity of experimental results is highly supported. Instead of using direct race primes, this study uses as its starting point estimates of how much negative stereotypes have been internalized and externalized, and then assesses the influence of these psychological states on academic outcomes after controlling for relevant background factors possibly associated both with grades and internalization and externalization (for example, factors like growing up in neighborhoods with high proportions of black residents or having the same race friends as oneself).

Specifically, we show that the externalization of negative stereotypes—expecting to be judged invidiously by majority group members on the basis of a stereotypical belief in minority intellectual inferiority—increases the performance burden experienced by individual minority group members and that this extra psychological burn, in turn, lowers grade performance. Moreover, the lack of a direct effect between externalization and academic performance suggests that the entire effect operates through the hypothesized indirect pathway. Although statistically significant using a one-tailed test, however, the externalization pathway is not particularly strong.

For blacks and Hispanics, the hypothesized pathway through internalization appears to be much stronger in certain respects. To the extent that minority group members internalize negative stereotypes—believing at some level that the canard of intellectual inferiority might actually apply to them—they reduce their academic efforts in keeping with a psychological process of disidentification, which involves disengaging from grade achievement as a domain of self-evaluation. However, the strong, direct relationship between internalization and academic performance suggests other mechanisms are at work beyond those specified by stereotype threat theory.

That internalization is directly associated with higher grades calls into question the validity of oppositional culture theory (Fordham and Ogbu 1986). In other words, academic disengagement does not seem to be the only response to the internalization of negative stereotypes among minority students. In fact, a recent study suggests that if the reframing of threats as opportunities to overcome challenges is taking place, internalization may actually not effect, or even improve, performance for threatened groups (Alter et al. 2010). What this strong, direct relationship between internalization and academic performance really means should be the focus of additional work.

In addition, our findings with regard to blacks and Hispanics suggest that aggregation across minority groups misses significant variation along the lines of ethnicity and national origin (Portes and Rumbaut 2001), in this case between African Americans and Hispanics. Unfortunately, owing to sample size limitations, we were unable to disaggregate the black and Hispanic students by ethnic origin. There is reason to believe, however, that ethnic origin may emerge as a primary line along which students experience differences in the effects of stereotype threat. African Americans of immigrant origin students may experience stereotype threat differently than those of native origin, for example, and Mexicans may be more or less susceptible to stereotype threat than Cubans. Future research should endeavor to consider differences by national origin as well as broader racial-ethnic categories.

For whites, the coherence of the internalization mechanism is perhaps best understood in the context of research related to the effects of stereotype threat on athletes and women in mathematics and science—the majority of whom are white (Harrison et al. 2009; Huguet and Regner 2009; Yopyk and Prentice 2005). To the extent whites who identify as athletes or “women in the sciences and mathematics” fields (a la former Harvard President Larry Summers’ comments in 2005), the broad items used to measure internalization in this study would allow white respondents to identify “his/her group” as that of athletes or women in particular male-dominated fields. If these sub-groups of white students experience depressed performance on the basis of negative stereotypes about athletes, women in mathematics, sciences, engineering or other male-dominated fields, or otherwise, negative stereotypes about these groups’ academic intelligence may account in large part for the apparent relationships between internalization, academic effort, and performance we find.

The lack of the influence of the externalization mechanism, on the other hand, may be explained if whites in the particular sub-groups susceptible to threat do not perceive negative stereotypes to be widely accepted by members outside the university community in general, or that they are not readily identifiable as members of the subgroups in question. In either case, it would follow logically that whites in these sub-groups would not experience a strong enough sense of externalization to drive an overall effect of stereotype threat.

Overall, the primary findings of the current research pertain to minority students attending selective colleges and universities. Future survey-based work might seek to expand the analysis to minority students at non-selective schools, including both two-year and four-year institutions, who comprise the bulk of minority students in higher education. Doing so will reveal whether stereotype threat is exacerbated or diminished by institutional selectivity and enable generalization to a population of students that more closely represent minority students attending college and university in the United States today. It may also help shed further light on the potentially contradictory findings between this study and Stricker and Ward’s (2004) field experiment. Even though both measure the effects of stereotype threat—such as through grades or AP scores—in ways that reflect students’ sustained academic achievement over time, they come to conclusions that are not immediately reconcilable.

Acknowledgments

The authors would like to thank Scott M. Lynch and three anonymous reviewers for their invaluable feedback in the preparation of this manuscript.

Appendix A. Regression Coefficients for Effects of Controls on Structural Equation Model of Relationships Between Internalization, Externalization, Disidentification, Academic Performance Burden and Academic Performance

NOTE: Reporting Standardized coefficients. Model includes university-level fixed effects. Effects of regression paths between constructs in full SEM shown in Table 5 , model 2.

Appendix B. Factor Loadings for Each Indicator of the Constructs of Stereotype Threat, by Race

NOTE: Results from the full structural equation model are shown in Table 5 , Model 3 and are based on multiple imputation of 5 datasets.

1 Each indicator is statistically significant at .001, indicating a significant contribution to the variance of its respective latent construct.

2 This item was reverse-coded to accurately reflect its negativelyl valenced properties in comparison to the other items in the externalization construct.

Appendix C. Fifteen Structural Equation Model (SEM) Results for Black and Hispanic Students, Each Model Omitting One Indicator of Stereotype Threat to Test the Robustness of Structural Paths to Changes in Model Specification (N=1,710)

NOTE: Reporting Standardized coefficients from sample resulting from multiple imputation of five datasets. Standard errors in parentheses. Models include institution-level fixed effects.

1 Structural residual error correlations are estimated between internalization and externalization and between academic performance burden and disidentification. Measurement error correlations are estimated between indicators v6 and v7 and v10 and v11 (See Table 1 for list of variables by number).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

• Alter Adam L, Aronson Joshua, Darley John M, Rodriguez Cordardo, Ruble Diane N. Rising to the threat: Reducing stereotype threat by reframing the threat as a challenge. Journal of Experimental Social Psychology. 2010; 46 (1):166–171. [Google Scholar]
• Aronson Joshua, Fried Carrie B, Good Catherine. Reducing the Effects of Stereotype Threat on African American College Students by Shaping Theories of Intelligence. Journal of Experimental Social Psychology. 2002; 38 :113–125. [Google Scholar]
• Bollen Kenneth A. Applied probability and statistics section, wiley series in probability and mathematical statistics. 1. New York: Wiley; 1989. Structural equations with latent variables. [Google Scholar]
• Bowen William G, Bok Derek C. The shape of the river: Long-term consequences of considering race in college and university admissions. Princeton, N.J: Princeton University Press; 1998. [Google Scholar]
• Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally; 1963. [Google Scholar]
• Charles Camille Z, Fischer Mary J, Mooney Margarita, Massey Douglas S. Taming the river: Negotiating the academic, financial, and social currents in selective colleges and universities. 1. Princeton, NJ: Princeton University Press; 2009. [Google Scholar]
• Cohen Geoffrey, Garcia Julio, Apfel Nancy, Master Allison. Reducing the Racial Achievement Gap: A Social-Psychological Intervention. Science. 2006; 313 (5791):1307–1310. [PubMed] [Google Scholar]
• Cohen Geoffrey, Garcia Julio, Purdie-Vaughns Valerie, Apfel Nancy, Brzustoski Patricia. Recursive Processes in Self-Affirmation: Intervening to Close the Minority Achievement Gap. Science. 2009; 324 :400–403. [PubMed] [Google Scholar]
• Crocker Jennifer, Major Brenda, Steele Claude M. Social stigma. In: Gilbert Daniel T, Fiske Susan T, Gardner Lindzey., editors. Handbook of social psychology. 4. Vol. 2. Boston, MA: McGraw-Hill; 1998. pp. 504–553. [Google Scholar]
• Cullen Michael J, Hardison Chaitra M, Sackett Paul R. Using SAT-Grade and Ability-Job Performance Relationships to Test Predictions Derived from Stereotype Threat Theory. Journal of Applied Psychology. 2004; 89 (2):220–30. [PubMed] [Google Scholar]
• Cullen Michael J, Waters Shonna D, Sackett Paul R. Testing Stereotype Threat Theory Predictions for Math-Identified and Non-Math-Identified Students by Gender. Human Performance. 2006; 19 (4):421–40. [Google Scholar]
• Danaher Kelly, Crandall Christian S. Stereotype Threat in Applied Settings Re-Examined. Journal of Applied Social Psychology. 2008; 38 (6):1639–1655. [Google Scholar]
• Downey Douglas B. Black/White differences in school performance: The oppositional culture explanation. Annual Review of Sociology. 2008; 34 (1):107–26. [Google Scholar]
• Espenshade Thomas J, Walton-Radford Alexandria. No longer separate, not yet equal: Race and class in elite college admission and campus life. Princeton, NJ: Princeton University Press; 2009. [Google Scholar]
• Fischer Mary J, Massey Douglas S. The effects of affirmative action in higher education. Social Science Research. 2007; 36 (2):531–49. [Google Scholar]
• Fordham Signithia, Ogbu John U. Black students’ school success: Coping with the burden of ‘acting white’ The Urban Review. 1986; 18 :176–195. [Google Scholar]
• Goffman Erving. Asylums: Essays on the social situation of mental patients and other inmates. Chicago, IL: Aldine; 1962. [Google Scholar]
• Harrison C Keith, Stone Jeff, Shapiro Jenessa, Yee Sharon, Boyd Jean A, Rullan Vashti. The Role of Gender Identities and Stereotype Salience With the Academic Performance of Male and Female College Athletes. Journal of Sport and Social Issue. 2009; 33 (1):78–96. [Google Scholar]
• Huguet Pascal, Regner Isabelle. Counter-Stereotypic Beliefs in Math do Not Protect School Girls from Stereotype Threat. Journal of experimental social psychology. 2009; 45 (4):1024–1027. [Google Scholar]
• Major Brenda, O’Brien Laure T. Social psychology of stigma. Annual Review of Psychology. 2005; 56 (1):393–421. [PubMed] [Google Scholar]
• Massey Douglas S, Charles Camille Z, Lundy Garvey F, Fischer Mary J. The source of the river: The social origins of freshmen at America’s selective colleges and universities. Princeton, NJ: Princeton University Press; 2003. [Google Scholar]
• Massey Douglas S, Fischer Mary J. Stereotype threat and academic performance: New findings from a racially diverse sample of college freshmen. Du Bois Review. 2005; 2 (1):45–68. [Google Scholar]
• Massey Douglas S, Mooney Margarita. The effects of America’s three affirmative action programs on academic performance. Social Problems. 2007; 54 (1):99–117. [Google Scholar]
• McGuire WJ, McGuire CV, Winton W. Effects of household sex composition on the salience of one’s own gender in the spontaneous self-concept. Journal of Experimental Psychology. 1979; 15 :77–90. [Google Scholar]
• Pope Devin G, Sydnor Justin R. Geographic Variation in the Gender Differences in Test Scores. Journal of Economic Perspectives. 2010; 24 (2):95–108. [Google Scholar]
• Portes Alejandro, Rumbaut Rubén G. Legacies: The story of the immigrant second generation. Berkeley, CA: University of California Press; 2001. [Google Scholar]
• Spencer Steven, Logel Christine, Davies Paul G. Stereotype threat. Annual Review of Psychology. 61 (1) Forthcoming. [PubMed] [Google Scholar]
• Steele Claude M. The psychology of self-affirmation: Sustaining the integrity of the self. Advances in Experimental Social Psychology. 1988a; 21 :261–302. [Google Scholar]
• Steele Claude M. A threat in the air: How stereotypes shape intellectual identity and performance. In: Eberhardt Jennifer L, Fiske Susan T., editors. Confronting racism: The problem and the response. Thousand Oaks, CA: Sage Publications; 1988b. p. 234. [Google Scholar]
• Steele Claude M. A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance. American Psychologist. 1997; 52 :613–629. [PubMed] [Google Scholar]
• Steele Claude M, Aronson Joshua. Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology. 1995; 69 (5):797–811. [PubMed] [Google Scholar]
• Stricker Lawrence J, Ward William C. Stereotype Threat, Inquiring about Test Takers’ Ethnicity and Gender, and Standardized Test Performance. Journal of Applied Social Psychology. 2004; 34 (4):665–93. [Google Scholar]
• Walton Gregory M, Cohen Geoffrey L. A Question of Belonging: Race, Social Fit, and Achievement. Journal of Personality and Social Psychology. 2007; 92 :82–96. [PubMed] [Google Scholar]
• Walton Gregory M, Spencer Steven J. Latent Ability: Grades and Test Scores Systematically Underestimate the Intellectual Ability of Negatively Stereotyped Students. Psychological Science. 2009; 20 (9):1132–1139. [PubMed] [Google Scholar]
• Yopyk Darren JA, Prentice Deborah A. Am I an Athlete or a Student? Identity Salience and Stereotype Threat in Student-Athletes. Basic and Applied Social Psychology. 2005; 27 (4):329–336. [Google Scholar]