"We find that the same minority applicants who faced lower approval rates on auto loans do not face lower approval rates on credit cards, even during the same year. This finding suggests that the human element of auto lending, rather than differences in hard information, leads to the lower approval rates for minorities."
I'm not seeing any evidence that the ML approach doesn't have the same or similar biases. I'm sure the results are different but we can't say they're better just because a computer generates them. In fact, we've seen many times that ML may often tend to give implicit biases even more emphasis, because in an attempt to create "objective" measures against which to judge a dataset, ML creators inject their own biases, or rely on data which is itself biased in its collection or categorization. (So, for example, if zip code is a major indicator in the ultimate ML algorithm, it may be completely "objective", while still reinforcing biases against minority applicants, simply because of how wildly self-segregated many US cities are, or because of self-reinforcing effects of systemic bias in other, human-controlled parts of the economy.)
I am truly curious of what the business model of denying minorities a car loan is?
I mean, long ago, I guess I can at least process the ugly mindset of "we don't want 'them' as our neighbor", but I can not come up with any logical or illogical reasonable reason why a business person would intentionally lower their income just to be racist.
Literally no one would know that you made a loan to a minority, so that can't be it.
Is the insinuation that the loan officer might "get a feeling" that a person might not repay a loan based on the race?
If that were the case, at least in the financial sector, it would seem that another financing institution would have noticed this inefficiency long before 2019 and gobbled up all of that free money on the table.
I also wonder if the higher interest rates of credit cards accounts for higher approval rates.
You seem to assume racism is always an active choice, rather than largely (these days) a set of systemically biased policies as well as implicit and often unconscious prejudice. It's also not all about rejecting loans. Rather, it's that the terms offered are different for every buyer, and the groups that are being discriminated against are charged higher rates, and thus pay more, for the same product.
There's plenty of data out there proving that, for whatever reason, African American loan applicants with identical credit scores and income are routinely charged higher interest rates for auto and home loans than their white counterparts. The next question to ask is "why?" but we can't pretend the effect doesn't exist.
> I am truly curious of what the business model of denying minorities a car loan is?
Why does it have to be a business model? Humans are not rational actors. The segregation era almost certainly lost tremendous amounts of money for retail businesses as a whole, but their management still went along with it. The remaining implicit bias in many businesses still loses tremendous amounts of money for retail businesses as a whole (see the various "wealthy black celebrity treated like a shoplifter" news stories, for example), but people still do it.
> Why does it have to be a business model? Humans are not rational actors.
True! I guess my real point was why no arbitrageurs eating the free money up?
> The segregation era almost certainly lost tremendous amounts of money for retail businesses as a whole, but their management still went along with it.
I don't think I agree with that. If the larger society was racist and would not go to a business that was segregated, then the business owner made a "rational" business decision to keep the wealthier customers ... at least that could be true.
> The remaining implicit bias in many businesses still loses tremendous amounts of money for retail businesses as a whole (see the various "wealthy black celebrity treated like a shoplifter" news stories, for example), but people still do it.
I have seen some of those anecdotal stories, but it remains that there certainly are (and have been) other businesses very happy to take money from "wealthy black celebrity" customers.
I get your angle, but it still seems like something is missing, especially now in non-segregated times.
If you’d been neglected and abused for your whole childhood, then finally managed to run away and escape, you’d be out of your bad situation right? How long do you think it would take for you to catch up to the adults who weren’t raised that way? Evidence suggests that you never would, unless you were so gifted that you could compensate for your past. Most people won’t be that gifted, and won’t thrive.
What I’m saying is that the missing ingredient is context. The market is a competition in many ways, and black people only got the chance to compete on a nominally fair playing field over the last few decades. Remember the Tuskeegee syphilis experiment only ended in 1972. Rodney king was nearly beaten to death in 1991. Families that were impoverished, segregated, and denied an education and voting rights take time to recover. A history of aggressive policing of the “war on drugs” hasn’t helped either.
So at what point does it occur to you that the missing ingredient is history that only people like you and me get to forget?
And nominally fair is correct. It's not fair now, but people say that it is.
Sure there is stuff left over from past racist policies that impact people. For example Black families on average have 1/10th the wealth of a white family, in large part because of redline mortage practices and other policies that undermined home ownership for black people.
But it's also wrong to suggest that everything is fair now, and black people just haven't caught up because of historical injustices.
There's still strong evidence of systemic racial bias against Blacks (and Latinx people) in traffic stops, criminal sentencing, resume responses (including controlled experiments with identical resumes save for different names), hiring rates etc. And this is not just statistics in RL, but studies show that people are biased, even if they don't think they are.
It's sadly quite easy, if you are white, to not see the issue, and if you don't see it it's easy to think it's not a problem anymore. It's still a huge problem.
I get all of that, but it in no way explains the question that I asked.
How do the terrible atrocities of the past cause auto lenders of today to deny minorities loans based on racism alone and not business sense?
That is my question and that is what this post is about.
TFA has evidence that people are going out of their way now to deny car loans to people that would otherwise be eligible for those loans except for their race.
I am not doubting that this is happening, I am trying to determine why it would be and how no start-up has recognized the "market inefficiency" and opened a "we specialize in loans to black and hispanic people" and made billions and billions of free dollars currently being neglected.
> I am trying to determine why it would be and how no start-up has recognized the "market inefficiency" and opened a "we specialize in loans to black and hispanic people" and made billions and billions of free dollars currently being neglected.
Because money service licenses are expensive and require capital from the same group that would balk at this business plan
Secondly auto loan commissions are GREAT, the pie from people who arent getting denied is still large enough to make enough free dollars
Third, minorities that are denied still have a variety of institution to go to that will more likely lend, such as credit unions who will lend on more favorable terms anyway (so your startup MAY exist?). We dont have data on the whole lifecycle of peoples user experience.
Sorry that people were so distracted by your question that you couldnt get an answer.
I think the third one seems most likely to answer my question ... minorities are actually getting the loans that they are qualified for, but have to apply more times than an equally qualified white person due to racism.
It is still baffling to me that a bank officer, or car dealership salesperson would purposely decrease their own income, forgo their own bonuses, and risk the ire of their supervisors and investors to purposely not give a loan to a black or hispanic person ... all with no "rational" reason, even a warped rational reason, they just would do it because ... well no reason! This would have to show up to the management during analytics.
I know plenty of racist people in the south, but none of them would risk their livelihoods or income just to be racist, they would just shut the hell up, "take it", then bitch about it back at home or around their friends while getting drunk.
And then I think of large corporations that are literally willing to fuck over babies, children, women, hell the whole planet, for money, and to think that they would be willing to lose money to be racist is hard to square.
I guess I have an impression that greed is a bigger driver in people's lives than racism ... but perhaps not, we do have this data (which I haven't seen the raw data, but don't doubt it) that seems to indicate otherwise.
The irony of it all is that if these people really do hate minorities, there is no legal way to have more power over them, or to fuck them over, I guess, then to keep them in debt!
> minorities are actually getting the loans that they are qualified for, but have to apply more times than an equally qualified white person due to racism.
We dont actually know that either. We know of some institutions that create this user experience, and we know that this is therefore a possible ux that minorities should be aware of
How do the terrible atrocities of the past cause auto lenders of today to deny minorities loans based on racism alone and not business sense?
Because attitudes take time to change, and if we didn’t know that before the resurgence of facism in places like Brazil, Italy, Austria, Hungary and other countries should reach us that. The same people who thought it was acceptable to redline, or even kill people over their race still do, they’ve just changed their language. Some of their children doubtless feel differently, others almost certainly inherent their attitudes from their parents and communities. The older generation with its concentration of wealth and influence also haven’t changed.
I understand, and agree with the "stickiness" of both racism and poverty, but like I mentioned above, home loans are one thing because maybe those racist people thought that "the blacks" where going to bring down their property values or make eyes at their daughters if they lived in the neighborhood. So even though it was terrible and misguided, it was rational from their warped perspective.
But car loans? In 2019?
It is just so baffling to me. Both why they would do it, and also why no one else at all is stepping up to get that money.
Two things can be equally true - systemic racial bias is a thing and has wide-ranging effects, and it doesn’t really make sense to discriminate on issuing auto loans in 2019. I mean, regardless of your personal feelings toward someone, you’re just losing out on commission if you deny the loan. Also, it probably doesn’t matter if you think they can pay the loan back as it will just get packaged and sold several times to other institutions.
There are probably other factors which weigh in more heavily than the loan processor’s person biases, e.g. the credit score system, minorities may be less willing to challenge the initial rate offered, people perhaps negotiating from a weaker position if they don’t have multiple credit options available (bank, credit union, heloc), etc.
> If that were the case, at least in the financial sector, it would seem that another financing institution would have noticed this inefficiency long before 2019 and gobbled up all of that free money on the table.
Of course. They'll just only hire people who aren't subconsciously biased. No problem. They'll just put "No racists please," in their job advertisements. Simple as that.
LOL!
You might reasonably apply by mail for a credit card from an out-of-state bank, but a lot of auto lending revolves around a purchase made in a nearby dealer.
If the dealers in minority-dominant communities tend to be more aggressive about lending qualifications, that could explain it all neatly.
Though, my blind guess would be that you'd see more "buy-here/pay-here" and "no-credit-required-but-the-loan-is-at-34-percent-interest" dealers in minority neighbourhoods (skewing poorer), so approval rates would likely be at par or higher. Or are they qualifying "approval rates" with "at sane market prices?"
This is where things get complex with ML. Any dimension that is correlated with the protected categories can potentially get you into a biased and discriminatory outcome. Because embedding are not necessarily transparent, it’s hard to evaluate what the model attaches to.
The most accessible example is anything to do with fine grained location, such as zip+4. Because people tend to cluster, zip+4 has been identified as a close proxy for race and other protected classes.
When we are talking ML-derived decisions and the nuanced approach, we have to be cognizant of the fact that certain repayment patterns and behaviors could also be strongly correlated to a protected class.
What if the correct predictor really is the zip code, though? Let's say there's a city that's predominantly black, and it has a statistically significantly higher default rate than other cities. However, the default rate in that city is high for residents of all races who live there. In this scenario, the 'true' predictor really is the city, but it also correlated with race. Is simple correlation to a protected class sufficient to nix a variable?
The protected classes are not protected because the data is somehow statistically not significant or cannot be a predictor. The classes are protected because as a society, we have decided that greater social good is achieved by sacrificing these predictors.
The question is then what is it that you are after with this prediction? Max profit or max social good with enough profit.
My point is that lots of things correlate with race. Almost any conceivable variable you could devise to predict loan defaults correlates with race. Gender and age are maybe the only two things I can think of that almost certainly don't correlate with race (although African Americans do tend to die earlier so...age might be out). Given that, what standard can you realistically use here?
How do auto loans and credit card approvals differ? Surely both have some human element to it still?
Maybe the sums involved are different, so giving loans is more risky than approving credit cards?
Also, incentives might be different. Maybe credit card companies want "bad" borrowers, because they make the most money if people go into high debts on credit cards? On the other hand for normal loans there is no such incentive, because the interest rates are fixed and much lower.
most people get auto loans st the dealership with a salesperson (the human element), but get credit cards via online or paper forms (no human involved).
If you make bad decisions by offering loans at too low a rate, you will lose money on average and eventually go bankrupt.
If you make bad decisions by offering loans at too high a rate because you have high overheads or you’re cautious or your actuaries are wrong or you’re racist then people won’t take out your loans because they get a better deal elsewhere or the loan is too expensive so you lose money by not profiting from making the loan.
Now there’s a balance here: you want to be competitively priced so people will take out your loans but you also want higher margins so you can make more money for investors or expand or make less money in other places to attract customers.
I think in this case “make better decisions” means either morally better (ie less racist), or better in the sense of “make better deals to win more customers to make more money”
What kind of function ML is optimizing? Who specified that function? Lenders made their decision from the dawn of time, so we have some data to compare. Now (say, last year) they made decisions with help of ML - what changed? Who got a better deal?
Makes sense. Just don't use race, gender, or ethnicity as a feature in the model and those factors _per_se_ won't be a consideration. Their correlates might still be a consideration, however, so it's not an entirely "fair" system, in the social-justice sense. Bias generated in meatspace will still affect it indirectly.
wouldnt a system that is "blind" in the sense that it has no prejudice be free to use those (anything) as features? isnt the whole point of the NN to get loans repaid?
Yes, but then you end up in a morally fraught territory where someone can claim that you ended up rejecting the loan because someone is a woman or gay or whatever. If those factors _directly_ aren't a consideration, your position is a lot more defensible.
Story time: Google ad serving system used to think (and probably still does) that if someone is interested in engineering topics per their browsing patterns, they're a male, much to the consternation of female Google engineers.
If gender wasn't a feature in the system at all (an unrealistic proposition in an ad targeting system), this problem would not arise as there would be no way for a system to determine if someone is male or female due to the absence of the corresponding feature, and therefore descriptive vocabulary. More defensible, at least, if not "fair".
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
" 'Thou shalt not make a machine in the likeness of a man's mind,' " Paul quoted.
"Right out of the Butlerian Jihad and the Orange Catholic Bible," she said. "But what the O.C. Bible should've said is: 'Thou shalt not make a machine to counterfeit a human mind.'
The prescription against thinking machines, along with similar plot devices (shields promoting hand-to-hand combat, the defacto ban on atomic weapons) allowed FH to construct a quasi-medieval SF setting.
I wouldn’t take mystical pronouncements in fictional settings intended to advance a plot too seriously.
After a quick scan, I only saw this used once. Does it really bother you that much?
That said, the ellipsis construction is used in an attempt to make text feel more like casual conversation — in this case, something like drawing out the preceding word in a questioning manner or a maybe a pregnant pause. This is a very standard device used in the register that Matt Levine uses for his column, and I imagine that most people find that these casual devices he uses makes his column more accessible for a wider audience.
Three times. There's nothing wrong with writing informally for a wide audience, but in this context it just comes across as "How do you do, fellow kids" rather than making the content any more accessible.
I’m not sure you know who Matt Levine is. He’s very much not a kid. He is a Harvard grad, a former lawyer, a former investment banker at Goldman, and a current writer with Bloomberg who is often described as one of the best writers at explaining complex aspects of finance in an accurate, understandable, and entertaining way. He has many fans on HN.
I encourage you to subscribe to his Money Stuff article/newsletter. It’s free... and good.
Plus ML can be more biased than humans (or more biased than we will admit to) https://www.bbc.co.uk/news/technology-45809919