Society evaluates young people at critical moments in their lives. College admissions. Scholarship applications. The first job interview.
These evaluations are profoundly broken in ways that hurt students and hinder global progress. The measures behind them – GPA, standardized tests, a list of extracurriculars, the personal essay, the interview – fail to create a picture of someone's potential. Rather, they merely indicate how well a student prepared for them.
Paul Graham describes this design flaw as hackability. Hackable measures and tests don't just fail to identify promising candidates. They teach the pernicious idea that the way to win is to hack the test. And so when future tests in life arise – opportunities for promotion, funding, election – our first instinct is not to simply demonstrate competence but to look for tricks, find loopholes, make the whole thing more byzantine than it ought to be.
In this way, test-takers are in an arms-race of trickery with test-makers.
Of course all this leads to a great deal of bias – bias toward people intimate with the ins and outs of the system; bias toward people with the spare time to prepare for the test; bias toward people with direct access to the decision-makers.
But bias is just a symptom of hackability. Trying to remove it is playing whack-a-mole. It won't actually fix the problem.
So how do we fix the problem? How do we stop failing the millions of outstanding young people in the world who miss opportunity because of design flaws in the ways we evaluate them?
A few guiding principles emerge from the wreckage of the status quo:
Minimize prep time. The amount of time required to prepare for a test corresponds to its bias and hackability. The ideal test makes dedicated prep time impossible. It's difficult to hack the interview at a company that makes a hiring decision after an 8-week work trial.
Make daily life visible. A test shouldn't demand more of someone's time, but it should shed light into how someone spends their time. This is how we find out that someone is taking ukulele lessons. This is how we find out about someone's side hustle. This is how we find out that someone takes care of their sick brother. Daily life gives us a sense of someone's hard-to-measure character traits like curiosity, courage, and stamina – traits that go completely unmeasured in most evaluations.
Make creative work visible. In an era where almost every young person is a filmmaker, it's astonishing how little of their output transfers beyond YouTube/TikTok/Instagram/Snapchat and onto portfolios, transcripts, resumes. Creative work isn't simply “art" – it's anything that someone creates. A highlight reel says more than: Captain of the wrestling team. Source code with a revision history says more than: A in Computer Science. Wherever students can show their work, we should make the effort to see it.
Distribute authority. Hiring managers, admissions officers, and other decision-makers do not scale with a growing, global application pool. They want to select the best candidates, but they don't have the bandwidth to review every candidate's life story, or evaluate their creative work. Peer networks can support decision-makers with rich qualitative feedback that bubbles up into meaningful signal. Networks also dismantle whatever meaning an authority tries to project onto a particular test. As Paul Graham writes, "Tests that aren't imposed by authorities are inherently unhackable, in the sense that no one is claiming they're tests of anything more than they actually test."
We don't need to wait for institutions to change to start transforming evaluations. We can help young people develop and showcase their potential in digestible ways that open them up to opportunities. New opportunity classes can be invented; we can create new connections between promising young people and money, mentorship, and each other.
This is what Chalon Bridges and I are working on now. We've started a non-profit whose mission is to be a community that finds and discovers the potential in everyone. We're developing a platform that can identify, support and connect future leaders around the world and spread new ways of understanding their unique potential. If our mission resonates with you, and if you've got ideas for helping us fulfill it, we'd love to connect with you.
To reply you need to sign in.