"Colleges that ditched admissions tests have to actually holistically evaluate applicants rather than reduce them to two numbers"
No doubt that it's more subjective and more difficult; human beings are complex, multi-dimensional entities and I find it sad that in many of these selection processes, they are reduced to a few numbers for the sake of low effort filtering.
With the 10s of thousands of applications top universities get, there isn't enough time to evaluate the "complex, multi-dimensional entities" behind each application.
At University of Washington, applications are not reviewed by "admissions people". They hire temporary workers (grad students, retirees etc.) to review applications, and give them 8 minutes per application [1]. And this is typical of most universities.
But at highly competitive schools you still have several times more students with 1500 SATs than you have available spots. So you eventually need to use holistic judgement anyways.
So no cutoff, just an SAT score? And then a cutoff based on that? What about GPA? Weighted GPA? AP tests matter? Creating a cure for COVID matter?
I think for the very top schools they care about the intangibles. For mid-tier schools the SAT cutoff lottery is fine. But the cut off is probably like 1150, and these arguments aren’t about those schools.
That’s an interesting race to the bottom for those schools. They’d end up with kids who basically ditched studying advanced math, physics, research, etc and just a lot of kids who prepped for the SAT for four years.
The “advanced math, physics, research, etc” in high school is all just college admissions prep too. When you major in math you have to take an analysis sequence starting at the beginning because AP calculus isn’t actually math. It’s just memorizing and learning to execute some math related algorithms.
It’s all just signaling. Let’s be open about that instead of hiding the ball.
Again there’s levels to this. Harvard Math 55 is filled with exceptionally strong math students. I’m a 1500 SAT student, but not a Math 55 student. Them not being able to differentiate these classes of students seems problematic.
Problematic is such a vague word. Connect the dots for me.
The top N schools in the nation, collectively representing enough seats to account for 5% of SAT takers, all change their admissions policies to select randomly among applicants in the top 5% of SAT scorers.
After ten years of these policies being in place we will likely see the following negative effects:
Here are some of the connection. The SAT is a basic test where strategy can come into great affect. And honestly, it's surprising how little some of the strategy is talked about -- but I suspect amongst the wealthy it's well known.
But back to the topic. A 1550 on the SAT says very little about your mathematical ability. The test just doesn't measure your ability to do more than basic algebra and has no real measure of writing ability. And honestly probably has a somewhat inverse relationship on your ability to read nuance into text (which is one of the first strategies I teach -- this isn't English class -- there's no nuance in anything you read on the SAT).
On advanced college math -- kids who are actually good at math likely have a high math SAT score, but their verbal score can vary a fair bit. I regularly see kids with 1400-1450 SATs, with perfect math scores who are Math Olympiad stars.
The likely end game of this though is that schools REALLY start teaching to the test. Why even offer AP tests if they don't help admission? Why even teach biology or physics at all? Simply teach SAT math, reading comprehension (which is comprehension of the worst kind) and what the SAT calls writing. So no more calculus or statistics. Even most of what is taught in algebra II is thrown out the window.
Today the 95%ile of SATs is around 1440. It would probably move up to around 1550, once schools teach to that test year around for four years. And any actually advanced undergrad courses are tossed because they can't get enough students to fill them.
And other countries will realize that the US left a huge gap -- the ability get students who strong in like areas and train them. They'll target the best math students, the best programmers, the best artists, etc...
Interestingly, the best meritocracy we have in society is sports. And in sports they recruit holistically -- that is they don't exclusively use HS stats (these are generally worthless) or combine numbers (these have some value, but limited). They use scouting services. And the very best athletes tend to congregate in programs that will develop them the best. Many hold up sports as a great example of meritocracy, yet it actually doesn't rely solely on measurables (in fact they are moving further and further from it).
What sports has though is a good objective function. What is the objective function of school? Answering that question will greatly help determine what the best path forward is.
It's not that it's not possible but rather we've simply been so lazy that we don't have the right heuristics.
No; I don't have the answer. I'm not an economist nor an admissions expert. But I've qualified and hired software engineers over the years and my process never focuses on a score or outcome of an assessment.
There is plenty of evidence to doubt the whole "holistic evaluation" line of thought, which is often a tool for subtle discrimination. And pretty much all of these schools already did "holistically evaluate" students; test scores were just one part of that evaluation.
Besides, at the end of the day everything boils down to a single number - applicants are ranked and those that make the cutoff get in, those that are below the cutoff don't.
Holistic evaluation in practice really means “present your best self in your application whatever that might be.” It allows for a range of impressive feats outside “was good at the arbitrary selection of school subjects and extracurriculars.”
I don’t really know what you do to combat this kind of thing because what would you do as an admissions officer trying to create a student body with a wide range of skills and experiences but you have thousands of applicants who are “focused on having perfect grades to the detriment of everything else and participated in the ‘look how smart I am clubs’.”
And I say this with full self-awareness that this was me. It’s probably the thing I regret most about my high school experience because of how unnecessary it was and how much it set me back on all the things that mattered.
I consider myself extremely lucky that any university took a chance on me because my application was 100% “I have no real interests except playing the game of school and winning by min-maxing the point system” which looking back is so obviously the worst applicant. I for sure lost out on my dream school because of it.
1. Have some minimum objective criteria (test scores, grades, class rank, etc.)
2. Accept students who meet that minimum with a lottery.
Of course, that would never happen at highly selective colleges, because it would shatter the notion that "only the best and brightest" got in and would ruin colleges' role as a maintainer of social hierarchy in the US.
I say this as a person who went to one of the most selective colleges in the world: the idea that colleges set up their application systems to build a "diverse" student body is complete and total bullshit. The reason being that some of the most important diversity comes from interacting with people of widely varying economic levels and intelligence levels. The idea that you can have a socially diverse student body, but only made up of people who got 4.0 GPAs and above, is bullshit on its face.
Exactly. Of course having numerical values you can directly compare is going to be the easiest way to select students, but it's only "fair" to the degree those numbers accurately reflect the relative academic performance and potential of the students.
I think the reason people so vigorously defend the use of standardized test scores in admissions is that we all have a built in bias towards viewing quantifiable comparisons as inherently more objective because it's easy to justify how you arrived at a decision. After all, who can argue with a decision based on one number being larger than the other?
The problem though is that this approach doesn't really remove the subjective, complex elements at all, it just moves them from the decision step to the measurement step. Sure, an SAT score is just a simple number. But how the test is constructed to arrive at that number is much more complicated. And the value of a simple score comparison is only as good as the process of constructing the test, leaving you with the illusion of having made the decision simpler and more objective when you really just move the complexity behind a curtain.
In tech terms, I think the analogy is project managers trying endless different ways to measure software development performance. Number of tickets closed, "velocity", commits, releases, etc, (lines of code if you're very unlucky), all give you nice simple numbers to look at. And while they're not entirely useless, it's easy to become overly reliant on them and assume they are more objective measures than they actually are.
Removing SAT scores doesn't remove the subjective and complex elements, but it does give a measure that took more than 8 minutes to arrive at.
As a previous commenter mentioned, UW spends about 8 minutes looking at applications at first. This isn't nearly enough time to make a "holistic" decision, especially when the application is largely subjective.
SAT scores, on the other hand, take longer than 8 minutes to arrive at. The subjective evaluations that go into designing the test can be determined over the course of months, leading to a measure that is imperfect but likely better than 8 minutes of a grad student's time.
The larger point is that in order for "holistic" admissions to work, you need to design an admissions process that actually evaluates candidates holistically. Removing SAT scores doesn't automatically fix those issues, it just removes one more piece of data to aid the process.
> But how the test is constructed to arrive at that number is much more complicated.
Is that done per student behind closed doors though, with the people calculating the score knowing who the student is and the ability to alter the scoring based on that knowledge?
Standardized tests are bad, but they're a giant step up from the corruption that is other forms of selection.
> Sure, an SAT score is just a simple number. But how the test is constructed to arrive at that number is much more complicated.
But you’re ignoring the other side of that number. That test score provides a hell of a lot of information about the individual beyond just doing well on a single test.
On the contrary, test scoring is the most objective thing we had to evaluate students based on the diligent work of a company whose entire business is maintaining that perception of objectivity. Sure, it's "the perception of", but if you think there aren't smart people analyzing college board exam scores across the country for signs of skewed testing so they can write an expose on it, you're far from reality.
Which is the first time that's been considered, because in the current climate - working extremely diligently to try to ensure that no particular student or group of students has an advantage, some students do have an advantage. They are smarter, or have better parenting, or better education, or their parents simply have more money to throw at tutoring
You can say "that's unfair! This is about selecting the best students and locks smart people from worse off backgrounds out of university!"
It does the second thing, sometimes, as a side effect - but it is also true that those students are still better students, regardless the path dependence.
Additionally, the primary winner of the policies this line of argument promotes are white women - with the primary loser being Asian men. Asian students in particular are bringing the suit against Harvard for these practices.
So while your epistemic humility ("is this REALLY objective? How do we measure that?") is well intentioned, it's the exact sort of reasoning used to cover up a strict ideology of "more black good, more women good, less white good, less men good" in universities, and the actual behavior of universities shoe that.
You’re glossing over the core issue which is that universities (and employers when hiring juniors for that matter) need to measure potential and many of the objective measures do correlate but do so weakly. It’s why we don’t hire based on Leetcode, whiteboard algorithms problems or GPA. Doing better on those is a positive sign but I can get a much much stronger signal through a conversation, probing to find out their strengths and passions and then setting them up to show off.
And look, race and gender ends up being a component but not in the way you’re describing. I’m not being like “woman good” it’s that a woman sitting across the table describing her technical achievements is made more impressive because I know first hand what women have to go in CS programs — you don’t put up with all the bullshit unless you’re really passionate. An example based on real life, a woman leading her capstone project group is unfathomably impressive because of how decisively strong her technical and communication skills have to be to earn the respect of the men in her class.
Men perform (slightly) etter on technical interviews when modulated as women, and women perform the same.
The idea that women are "putting up with so much more bullshit" when we have every major university and large tech employer bending over backwards to find more women who are interested, sky high salaries, and that we can show notable although weak bias in favor of women in settings where gender is obscured is out of date at a minimum.
> when we have every major university and large tech employer bending over backwards to find more women who are interested
I swear this is always at the core of it. Trust me it’s not as glamorous as you make it out to be. Yes tech companies are bending over backwards to find and hire competent women because we’re rare (and why!) but that statement is not at all the same as women have lower standards. You have to understand that literally no team ever is going to burn their hire on a perceived less qualified woman because she’s a woman. The desire to have the best person because the team is lazy and wants someone who will make their lives easier always wins. And not to put too fine a point on it but your average team of tech folks is pretty anti-woke liberal-tarian meritocracy so you’re not getting a whole lot social justice bias there. I guess this could happen if you have HR hiring in a vacuum but I’ve never seen a place where the team doesn’t have the final say.
> putting up with so much more bullshit
The bullshit is the culture. Look I wish this wasn’t true but it’s the unfortunate reality of being a woman in a CS major. Your male classmates do not see you as an equal. there is an in-built presumption that you don’t know what you’re doing; any normal mistake you make that guys make too just cements it. When you’re doing group work nobody listens to you and you get the “fun” experience of waiting for them to catch up and then having no recognition at all that you had already told them. Getting study groups together is awful one because guys will not let you join because they think you’ll be a drain, or they will let you join and then hit on you. Do you know how many fucking times I got told to go make me a sandwich? In more than one class I just asked the professor to do group projects alone. And that’s before dealing with how isolating it is to be the only girl in class and having the shock and head turns the first time you speak.
It’s not perfect by any stretch of imagination but I would have been driven out of tech had I not by chance landed at a few companies with older teams and realized that maturity makes a huge difference.
> and that we can show notable although weak bias in favor of women
Well yeah, of course you expect that. I just talked about a situation where a woman saying something is more impressive than a man saying the same thing. Context matters.
How likely is it that this holistic evaluation will end up preferring individuals conforming to the current Zeitgeist (e.g. using all the momentarily popular buzzwords in their applications) rather than truly talented candidates? Thus contributing to further ossification of current elites?
An academic institution shouldn't only be about picking truly talented candidates; that's the exact opposite of holistic.
My take is that part of the function of an academic institution is to help individuals discover and develop talents.
A good teacher isn't one that helps A students get A's but one that can develop any student into an A student.
To merely select and propel those who already have the talent rather than select for the capacity to grow and develop talent is precisely why score based systems are biased in the first place.
Yeah, I almost felt like flagging this submission. It's taking something that on its own could be interpreted in many ways and giving it a strong political spin.
For the sake of argument, let's imagine a college doing the opposite: admitting students solely on the basis of a standardized test score. You submit your SAT score, and everyone above a certain percentile is admitted. People below the threshold are not. Many of the comments here seem to think this is optimal.
In that scenario, there wouldn't be any complaints from admissions committees at all about them having to put effort into the situation, because they wouldn't be involved at all. So is it being difficult a sign that reality is more complex than suggested by a test score, or that they've lost an objective indicator?
Most of the statistics thrown around in the article could also be turned on their head. For instance, just hypothetically, if getting rid of the test isn't changing the student body, what's the point of the test?
Having an objective number is useless on its own. Even if it's roughly correlated with the outcomes you desire, if it's that roughly correlated with outcomes, and subject to biases, there's no point in having it.
I feel like these tests are kind of like buying a house based on nominal square footage alone. Sure, it's probably correlated with satisfaction and resell value, it's an objective indicator that involves some measurement, and so forth, but no one would buy a house that way. It's subject to all sorts of manipulation and fuzziness, and doesn't tell you anything about the architecture, lot, or anything else. And yet that's kind of how standardized tests are used.
I'm fine with using standardized tests in theory, but they're far too abused and misused. It's not so much the tests I have problems with, as it is how it seems once you introduce them, it seems they can't be interpreted rationally in making any sorts of decisions about human beings. It's like people take something that has a sliver of information value with all sorts of limitations, and then treat it like an infallible signal just because it's easy to use.
Its ease of use could be seen as a problem, not something desirable.
> I feel like these tests are kind of like buying a house based on nominal square footage alone
Great analogy.
Buying a house is about far, far more than just the square footage. How are the neighbors? How's the walkability? Noise levels? Schools? Access to transport routes? Access to jobs? Property taxes? HOA fees? Access to activities of interest?
There are so many factors to buying a house that it would be ridiculous to look at any single dimension alone and make a determinate based on that one dimension.
> There are so many factors to buying a house that it would be ridiculous to look at any single dimension alone and make a determinate based on that one dimension.
But that's not what they've been doing. Standardized tests played a role, but were not the only factor.
To continue the house analogy: look at everything _but_ square footage before you make an offer.
