NYC's Entry-Level Firefighter Exams Unlawfully Discriminated Against Hispanics and African-Americans
In USA v. City of New York, 07-cv-2067 (NGG)(RLM) (E.D.N.Y. July 22, 2009) (download), Judge Nicholas Garaufis of the Eastern District of New York issued a 93-page decision endorsing in wholesale fashion the statistical analysis of two experts on behalf of Plaintiffs to hold that entry-level firefighter exams used by the City of New York from 1999 to 2007 had an unlawful discriminatory impact due to the disproportionate failure rates of African-American and Hispanic test takers in violation of Title VII. The court held that the statistical conclusions of experts Bernard R. Siskin, Ph.D. and Joel P. Wiesen, Ph.D, relied upon by the Plaintiffs, were sufficient to set forth a prima facie case of disparate impact.
The hiring of entry-level firefighters was based on Exams 7029 and 2043 each of which were 85-question multiple choice tests. (Exam 7029 was administered from 1999 through 2002, and Exam 2043 was administered from 2002 through 2007). Candidates who passed the written examination were allowed to take the physical performance test (“PPT”). The PPT consisted of eight physical tasks, and a candidate had to pass a minimum of six tasks to achieve a passing score overall. Candidates who passed both the written examination and the PPT were placed on a “rank-order” eligibility list. The ordering of the eligibility list was based upon an elaborate process of “standardizing,” “combining,” and “transforming” the raw scores.
Exam 7029
The cutoff passing score for Written Examination 7029 was 84.705%. Based on that cutoff score, the pass rate of white candidates for Exam 7029 was 89.9%, while the pass rate of black candidates was 60.3%. In other words, out of 12,915 white test takers, 11,613 received a passing score of at least 84.705, whereas out of 1,749 black test takers, only 1,054 received a passing score. The pass rate of black candidates was, therefore, 67% of the pass rate of white candidates.
Both Dr. Siskin’s and Dr. Wiesen’s standard deviation analysis found that this disparity is equivalent to 33.9 units of standard deviation, meaning that the likelihood it occurred by chance is less than 1 in 4.5 million-billion. The practical effect of this disparity, according to Dr. Siskin, is that 519 black candidates who failed the examination—74.7% of the black applicants who failed—were eliminated from consideration. Dr. Wiesen estimated that 457 black candidates would have passed the examination but for the effect of this disparity. Based on Dr. Siskin’s calculation, 114 additional black firefighters would have been appointed absent the disparity.
The pass rate for Hispanic candidates taking Exam 7029 was 76.7%, compared with a pass rate of 89.9% for white candidates. Accordingly, the pass rate of Hispanic candidates was 85.3% of the pass rate of white candidates. Dr. Siskin’s standard deviation analysis found that this disparity is equivalent to 17.4 units of standard deviation, meaning that the likelihood it occurred by chance is less than 1 in 4.5 million-billion. The practical effect of this deviation, according to Dr. Siskin, is that 282 Hispanic candidates who failed the examination—56.9% of the Hispanic applicants who failed—were eliminated from consideration. Based on Dr. Siskin’s calculation, 62 additional Hispanic firefighters would have been appointed absent the disparity.
Exam 2043
The cutoff passing score for Written Examination 2043 was 70%. Based on this cutoff score, the pass rate of white candidates taking Exam 2043 was 97.2%, while the pass rate of black candidates was 85.4%. In other words, out of 13,877 white test takers, 13,495 received a passing score of at least 70, whereas, out of 1,393 black test takers, 1,190 received a passing score. The pass rate of black candidates was, therefore, 87.8% of the pass rate of white candidates. Both Dr. Siskin’s and Dr. Wiesen’s standard deviation analysis found that this disparity is equivalent to 21.8 units of standard deviation, meaning that the likelihood that it occurred by chance is less than 1 in 4.5 million-billion. The practical effect of this deviation, according to Dr. Siskin, is that 165 black candidates who failed the examination—81.3% of the black applicants who failed—were eliminated from consideration.
Dr. Wiesen estimated that 150 black candidates would have passed the examination absent the disparity. Based on Dr. Siskin’s calculation, 30 additional black firefighters would have been appointed absent the disparity. This last calculation was based on the assumption that the black applicants who failed Exam 2043 would have passed the PPT at the same rate as other similarly situated passers, and would have met the other qualifications and been appointed at the same rate as other passers.
The pass rate for Hispanic candidates taking Exam 2043 was 92.8%, compared with a pass rate of 97.2% for white candidates. The pass rate of Hispanic candidates was, therefore, 95.5% of the pass rate of white candidates. Dr. Siskin’s standard deviation analysis found that this disparity is equivalent to 10.5 units of standard deviation, meaning that the likelihood it occurred by chance is less than 1 in 4.5 million-billion.
The practical effect of this deviation, according to Dr. Siskin, is that 94 Hispanic candidates who failed the examination—61.8% of the Hispanic applicants who failed—were eliminated from consideration. Based on Dr. Siskin’s calculation, 17 additional Hispanic firefighters would have been appointed absent the disparity.
Prima Facie Disparate Impact Based on Statistical Significance Testing
Noting that "standard deviations of more than 2 or 3 units can give rise to a prima facie case of disparate impact because of the low likelihood that such disparities have resulted from chance," the court held:
"Plaintiffs have presented analyses from two experts that thoroughly demonstrate the statistical significance of the disparities between groups of candidates. For each of the pass/fail uses of the examinations, these analyses demonstrate that the disparities between the pass rates of whites and minority candidates were between 10.5 and 33.9 units of standard deviation. For each of the rank-ordering uses of the examinations, the analyses demonstrate that the disparities between the rankings of whites and minority candidates were between 4.6 and 9.7 units of standard deviation. These statistical disparities show that black and Hispanic candidates disproportionately failed Written Exams 7029 and 2043, and were placed disproportionately lower on the eligibility lists created from those examinations."
Notably, the court declined to apply the "80% Rule". (The 'four-fifths rule' provides that a selection tool that yields a selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or 80%) of the rate for the group with the highest rate will generally be regarded by the EEOC as evidence of adverse impact, while a greater than four-fifths rate will generally not be regarded as evidence of adverse impact). The 80% Rule would have resulted in a finding of no disparate impact on some of the Plaintiffs' challenges, specifically: the pass/fail uses of Exam 2043 with respect to black candidates, and the pass/fail uses of Exam 7029 and 2043 with respect to Hispanic candidates. The court was unpersuaded that it must rely on the 80% Rule to the exclusion of statistical significance testing. Citing case law, the court held that "[c]ontrolling precedent holds that the 80% Rule is not an exclusive means of proof, and that alternative statistical tests should be considered."
City Failed to Show Business Necessity
Assessing the validity of the City's tests under Guardians Association of the New York City Police Department, Inc. v. Civil Service Commission, 630 F.2d 79, 82 (2d Cir. 1980), the court applied the Guardians five-part test:
(1) the test-makers must have conducted a suitable job analysis;
(2) they must have used reasonable competence in constructing the test itself;
(3) the content of the test must be related to the content of the job;
(4) the content of the test must be representative of the content of the job; and
(5) there must be a scoring system that usefully selects from among the applicants those who can better perform the job.
The court considered each seriatim and concluded that the City "has made an inadequate showing that the tests contained appropriate content and were properly constructed pursuant to the EEOC Guidelines and Guardians. . . . [and] has not justified its ranking of thousands of candidates for the job of entry-level firefighter."
In the court's opinion, the City failed to demonstrate a sufficient relationship between the tasks of a firefighter and the abilities it intended to test on Exams 7029 and 2043, and the City also failed to:
- take measures to ensure the reliability of those examinations;
- take steps to ensure that that the reading level of the examinations was appropriate;
- test for various recognized important abilities of a firefighter;
- test for abilities needed upon entry into the Fire Academy, rather than abilities to be learned on the job;
- retain testing professionals to devise the examination questions;
- demonstrate that the examinations it administered actually tested the abilities it intended to test.
Relying on Guardians, the court held that the City "improperly relied upon these poorly constructed examinations in the face of a disparate impact upon minority candidates."
Ricci v. DeStefano - Irrelevant
Judge Garaufis, cognizant of the media hype surrounding Ricci v. DeStefano, 129 S. Ct. 2658 (June 29, 2009), properly observed that Ricci has no application in this case:
"In Ricci, the City of New Haven had set aside the results of a promotional examination, and the Supreme Court confronted the narrow issue of whether New Haven could defend a violation of Title VII’s disparate treatment provision by asserting that its challenged employment action was an attempt to comply with Title VII’s disparate impact provision. The Court held that such a defense is only available when 'the employer can demonstrate a strong basis in evidence that, had it not taken the action, it would have been liable under the disparate-impact statute.' Id. at 2664. In contrast, this case presents the entirely separate question of whether Plaintiffs have shown that the City’s use of Exams 7029 and 2043 has actually had a disparate impact upon black and Hispanic applicants for positions as entry-level firefighters. Ricci did not confront that issue."
Aftermath
The media, always sensitive to legal news touching on matters of race especially on the heels of the Hon. Sonia Sotomayor confirmation hearings which provided wide exposure of Ricci, is rapidly reporting on this case. It will take patience and perhaps a primer in statistical analysis to really get your hands around this decision; however, what stands out as most significant is the court's adoption, essentially wholesale, of the findings and conclusions of two experts on behalf of the plaintiffs in this case to solidify their prima facie case. The court had broad discretion to rule on the admissibility of the experts' evidence in the summary judgment context, and its ruling will be sustained unless manifestly erroneous and/or whether the court abused its discretion.
