A Way to Detect Bias

Paul Graham’s essay A Way to Detect Bias proposes a method to identify bias in selection processes.

The Core Logic

Bias means “it’s harder for them to make it through” for certain applicant types. This implies that successful applicants from disadvantaged groups must be inherently better—so “they outperform other successful applicants.”

Three Required Conditions

A random sample of selected applicants exists
Their subsequent performance is measurable
Comparison groups have roughly equal ability distributions

Real-World Example

First Round Capital found startups with female founders outperformed those without by 63%—suggesting potential bias in their selection process.

Important Limitations

This technique fails if selectors use different criteria for different groups.

My Takeaway

If members of a group outperform after selection, that group may face bias in the selection process. Measure outcomes to detect bias.