in Reed's "No Rules Rules" book, they discuss how this contest was really a way to recruit top-quality engineering talent, which is a key assumption in how they ran their culture (highly paid small teams with lots of freedom, trusted to know what's best from the ground-up rather than top-down).
They didn't really need an algorithm in the first place, they just brainstormed what would sound the coolest to developers.
A little secret is that the content rights holders negotiated draconian streaming contracts that are outright prohibitive to Netflix. Netflix is trying to do everything possible to hide the fact that their streaming library is small and shrinking. In the old hey-day of Netflix mail-in red envelopes with DVDs you were hard pressed to find a movie they did not have. These days your search returns mostly crap.
> A little secret is that the content rights holders negotiated draconian streaming contracts that are outright prohibitive to Netflix. Netflix is trying to do everything possible to hide the fact that their streaming library is small and shrinking.
I think that's fairly well-known at this point, though I agree it should be more widely known.
I think the real secret is that Netflix discovered that their users aren't trying to watch the best movies, they're just trying to kill time. Apparently you can substitute good films for generic junk and people will keep their subscriptions.
Really, the business model for Netflix has some weird perverse incentives. They make money from monthly subscriptions. They lose money when actually streaming content to users. So the ideal user from their standpoint is one who keeps paying but never watches anything.
Thus, the company is incentivized to put out just enough good stuff that you can't miss that you don't cancel, but otherwise produce garbage you don't want.
I genuinely don't think Netflix discovered or believes that. I think if they could they'd have the highest quality and most extensive library possible. I think other media companies got stupid business school level FOMO and tried to copy Netflix when they didn't realize that Neflix is the Steam of videos/shows and as a user I don't want N streaming apps I want one or two. So the market is selecting for a few high quality streaming implementations because the market doesn't want 20 separate cable bills all tallying more than a single cable bill with content licensing. Netflix will be the streaming service I purchase again once their library comes back online, unless Apple TV+ is competitive at that point.
For us Netflix has been replaced, we don’t keep subscribed to it as everything they have that we want is available elsewhere and with better deals. So personally I would say it could be the Steam of streaming; if Steam was split into many more platforms which all provided a large chunk of desired content while Steam wallowed with the scraps for revenue via price-gouging and account limitations.
oh, that would be horrible, I canceled "max" after the discovery takeover, I'd hate to cancel paramount+ as well, they have a good catalog that discovery would eviscerate.
I pay $16.55 per month for Netflix. I generally watch movies with my spouse. That cost is less than the cost for both of us to go to a movie theater. If Netflix has one item per month of roughly movie length or longer that we want to watch, the price seems worth it just for that.
In recent years, this has still not been a sure bet month to month, but over a year, 12 scores are pretty much guaranteed.
With the theater you're buying a very different experience than Netflix, though. A more apt comparison would be to movie rentals, which you consume on the same screens and with the same people.
Amazon will rent you most movies for <$4, so 4 movies per month is the number for Netflix to beat, not 1.
Movies yes, but often a single episode of a show is say $2. That means if you watch a season in a month, you're up to $16-20 pretty quickly with 8-10 episodes.
You can rent movies on many services like google, itunes, youtube, etc... But for shows, you only have the option to buy each episode, which makes no sense.
This is always going to be a very subjective assessment. I don't doubt that the comparison makes sense for you, but it's not the way my spouse or I see it.
Also, most movies I might go to a theater to see cost substantially more than $4 on amzn.
Paying monthly to Netflix vs. paying per use with Amazon isn't a subjective difference. If you are only watching 2 movies a month of average, then Amazon is objectively a better deal than Netflix as you not only pay less, but have more options. (Amazon movies are generally $4 or $6 to rent.)
Given that we do both, the actual question is whether Netflix manages to offer us one exclusive offering per month that ticks the box. This particular January, it is setting itself quite nicely for the year :)
I rent individual movies for $3-4 apiece, and that turns out to be a big win over Netflix because I typically only watch 1-2 movies per month, and the rental service has a bigger library.
But I think that I'm a very unusual consumer in that respect. If I liked to watch TV series more then I'd be bingeing those, and that's where I think Netflix really attracts consumers, not movies. I don't think there's an economical way to do a la carte viewing of TV series on a regular basis.
Someone I know who was a fairly early-on high level technical person at Netflix (and later AWS) once told me that people came to Netflix initially for the movies and stayed for the TV shows.
And, yes, on an hourly basis they're probably comparable but renting a film now and then feels reasonable to me whereas paying per TV episode feels expensive.
Netflix isn't just competing against movie theaters though, it's competing against 10 other similarly priced streaming services. If they let their quality degrade too much people are going to start choosing other services as their movie theatre alternative (likely already happening).
Buying a DVD on Amazon isn't much different in price and paying for an a la carte streaming rental if available is probably cheaper.
With the fragmentation, streaming services start to make more sense if you're someone who likes to have the TV on as background. They make less sense if you watch a few hours of deliberately selected video a week.
That is me with mubi! I keep paying but I rarely watch any films. I find that I need to be in the right headspace to watch any serious stuff but when I do, I do watch a couple of them and I don't really mind paying. I like to joke that I'm their ideal customer haha.
Unlike netflix where I downgraded my plan to the lowest tier, there's literally nothing on it and they also keep on cancelling good tv shows. I think they should focus on serious stuff and work with more indie directors. But then again I can't blame them because they're just following the trend.
Guess this also explains why they cancel good shows after a few seasons but not sure how well the hypothesis extends to reality.
Netflix rarely does good shows that are created exclusively by them. My go to example is the Witcher series. What they are amazing at though is picking up projects from small scale studios and helping them produce it. Nimona, Klaus and Arcane are some of the stuff I remember on top of my head. Wish they stick on to that, helping small scale studios reach a wider audience
The ideal customer of any company is someone who just gives you money, of course, but if that actually happened those customers would eliminate themselves eventually by running out of money.
(This is part of what economics means by "markets are rational" - the irrational people go broke.)
This is why I'm still surprised they separated out (and eventually killed) their disc-by-mail service. I understand the visceral feeling that discs are an antiquated relic—but you just can't beat that catalog.
Combining discs and streaming in one subscription provides customers with the best of both worlds. Need something to watch right now? No problem, we have an excellent streaming catalog and you're going to love Stranger Things! Want a specific movie or show, maybe Friends or The Lion King? We've got that too, it'll be in your mailbox in a couple days.
And if some customers choose to forgo either the digital or physical side of the package, it doesn't cost Netflix anything.
RedBox is mostly useful for current blockbusters. The selection is (has to be) very limited so it's not great for random older films.
You can always buy most films. It's just relatively expensive (and not instant) for a one-time watch. Of course many are available for a la carte buying/rental via streaming as well.
this afternoon and saw 5 DVD or Blu-Ray players that cost about $20. They even had a funky VCR with a USB port (to digitize movies on your computer) for $12.
That assumes you have some kind of TV set. I'd think a lot of people get their entertainment on a laptop or phone, VR headset or something like that and don't have a TV.
I see more and more DVD collections on TikTok lately. I’m not sure how widespread it is but I think DVDs are coming back. It can be hard to find specific movie or shows streaming anymore even when willing to buy a digital copy and hope if doesn’t get taken away when you cancel prime or whatever.
Yeah, I'm buying DVD's again cause I can't stream a lot of my old faves any more, I'm not alone. If you want something older they're just disappearing, or you have to subscribe to watch one show.
Even many relatively older people I know look at you weird if you talk about playing a DVD. Netflix didn't drive this change. It just happened--Netflix isn't that powerful.
Right? I have a half-dozen old devices around my house that could play a DVD, but I've never owned anything that can play bluRay, much less 4K disks (I've never even seen one of those IRL). And DVDs are no longer worth bothering with mostly because of the aspect ratio hassle if nothing else.
Many people own an Xbox One S/X, Xbox Series X, or PS5, which can all play 4K Blu-ray. Many more people own a PS3, PS4, or regular Xbox One, which can all play regular Blu-ray.
Xbox One S/X doesn't have a disc drive, Xbox Series X doesn't have a disc drive, PS5 needs the disc drive bought separately and paired over the internet.
Only the original Xbox One had a disc drive integrated.
> Xbox One S/X doesn't have a disc drive, Xbox Series X doesn't have a disc drive, PS5 needs the disc drive bought separately and paired over the internet.
Only the original Xbox One had a disc drive integrated.
It’s always fascinating that people say things that are verifiably untrue. Like, really?
I meant One S and One X, plus Series X. The One S and One X are refreshes of the One, with the One S getting 4K HDMI and Blu-ray support and the One X getting both of those plus more power and 4K gaming support. There was a version of the One S that didn't have an optical drive, but it was a bit of a flop. The Series X is one of the two successors to the One line (along with the Series S), with the Series X having a 4K Blu-ray drive and the Series S not having an optical drive.
The Xbox One X certainly does have a disc drive, it's a little out-of-the-way though. Check on the left side of the front of the console, under the overhang.
Not to dump on LoganDark, but I find it absolutely fascinating that you owned an Xbox and legitimately only today discovered that it has an optical drive!
EDIT: Definitely shows the shift to online game delivery in a way that I have not seen before.
I grew up with a 360, and knew about the drive on the original Xbox One, but I really could not tell that my Xbox One X had a disc drive. I don't know how I missed it.
I checked and you are right, apparently it does. It's weird that I was never able to find it before, because I did look under the overhang, but was unable to get any discs in there. Oops.
Ironically one of the reasons I bought a PS3 was to play Blu Ray movies. But during Covid I ripped all my physical media and got rid of them, I only use the disc model of the PS5 to play my old PS4 games.
The regular One S and the One X both have 4K Blu-Ray drives (the One S All-Digital Edition doesn’t). The Series X does have a 4K Blu-Ray drive (the Series S doesn’t).
Isn't there a slightly wider hole the left (other side than the Xbox log) between the top part and the black bottom part? You should be able to fit a disk there even if it's pretty hard to notice...
The "base" one didn't exist when the console first released, there was only digital. The "base" version bundles the disc drive that is normally purchased separately.
They launched with one with a built-in disc drive and one without. For their mid-cycle refresh they changed it to a single model with a disc drive add-on but sell a SKU that includes the drive in the box.
My pet peeve is that I wanted to buy a couple blu-rays a few years ago to watch on my PS4. But I either I had to buy the DVD, or the Blu-Ray+DVD. There was no blu-ray only option, and I was like... what I'm gonna do with the dvd?
DVDs aspect ratio is 16:9, just like the vast majority of TVs and PC Monitors.
Were you referring to resolution instead? The picture quality of a DVD (even properly digitized) would be atrocious by modern standards (high bitrate 1080p)
NTSC DVD native max resolution is 720x480 (ignore more depth like overscan) that isn't 16:9. So most 16:9 titles uses non-1:1 PAR to display 16:9 that isn't ideal for quality. Though anyway any DVD quality is now considered poor regardless of aspect ratio.
480p LCD's do exist (or did), iirc they were marketed as "Enhanced Definition" as opposed to the later "High Definition" displays. However, they were 4:3.
I guess either my DVDs or my player is garbage because even though it's hooked up by HDMI I often find DVDs with 4:3 menus and then double-letterboxed (so, like, framed) video within.
This is HN in 2024, and people are complaining about aspect ratios? For about 15 years now I have been feeding physical DVDs & BluRays into < $100 USB DVD/BR players, filtered through makemkv, and then through mpv or whatever which understands AC3 pass through to at first a NAD pre-amp and currently now that my NAD died, Denon receiver, which does the down mix to 3.0. Looks and sounds glorious, aspect ratios are not a problem at all, and a backup drops out, huh.
I loved greencine and physical netflix but those killed themselves, seppuku, it seemed like, a long time ago.
I'm sure they just ran the numbers and realized that being both a data streaming software company and a physical package logistics company just didn't add up. Every dollar they spent on the latter would have net them more profit if they instead invested it in the former.
Maybe--I of course don't have their numbers--but remember that Netflix kept offering discs up until last year, just under a separate subscription. So they were still investing some money.
I think Netflix got cocky. When Netflix separated the plans, streaming rights were cheap and Netflix's streaming catalog was growing quickly. They didn't foresee the market becoming so competitive.
Continuing to offer discs would have allowed Netflix to hedge their bets and avoid relying on rights-holders. The original content push did that too, but at a much higher cost. If keeping DVDs meant Netflix could produce a few less original shows per year, that seems worth the trade.
The transition from not needing a license (mailing discs) to renting content to owning it was intrinsic in their business plan all along.
They saw that streaming was the future but that mailing discs would let them build a big brand early on, but they took the disc market very seriously and optimized it.
Once you're streaming the cost structure is like this: Fox or Disney or someone like that will charge them $X /month/sub but it costs $Y to create a piece of content, independent of how many subscribers you have. There is a crossover point where you have enough subscribers that it makes sense to own.
The DVD business was ~1% of total revenues by the time they shuttered it. At that point, it was no longer really cost effective for them to retain such a large physical infrastructure for so little ROI.
Funny enough it has been so long that we have come full circle on this. Studios that were pulling their content from Netflix to launch competing streaming services are now willing to license it again because their services are failing, leaving them bleeding for cash. You can even find the newest Disney and Warner Bros movies and shows on Netflix now.
Is it region locked? In the UK it's basically cobwebs. 99% of new stuff is just unwatchable filler. Template stories with dialogues probably written by ChatGPT.
I live in Australia and have noted the same phenomenon on Netflix. There is an absolute dearth of new, quality content coming through. I've seen no indication that formerly exclusive content from other platforms is making its way to Netflix in my country.
I don't believe the person you're replying to is overstating at all, and certainly not 'considerably'.
The general consensus in the media industry is that Netflix basically won the streaming wars this year. Every other service is contracting, has a fraction of the viewership, is running at a loss. Netflix is running in the black, has by far the highest viewership (like an order of magnitude over many of them), dominates over seas. The other studios have started to license content again to get every last bit of cash flow they can.
I can't help but feel that being in the black (or the degree to which they are) is a short term effect of the password sharing crackdown and the price increases. Both of those are likely to have a drop-off in impact over the long run, especially as the cost of living crisis develops.
It makes sense to me that the first thing to go is the other streaming services, as Netflix has such a strong mindshare, but their lack of content is more noticeable all the time, and I think there will be a swing back to cheaper services with at least different content, if not better.
> I can't help but feel that being in the black (or the degree to which they are) is a short term effect of the password sharing crackdown and the price increases.
The numbers disagree. Net income and profit margin was doing great even before password sharing crackdown.
For me Netflix has the best foreign content selection hands down, look at the impact of RRR (India), Dark (Germany), or Squid Game (Korea). They’ve become _the _ platform for South Korean content when South Korea is ascendant on the global pop culture stage. Somebody on Netflix has been using the ol noggin
My favorite part about this is that a lot of the rights holders are losing their shirts on their alternative streaming services, because it turns out making content is different than running a profitable streaming service. Peacock and Disney+ are losing huge money, Amazon is introducing commercials or an upcharge to their service because they've been spending on their own content with abandon (I can't imagine why WoT costs per episode what I read it costs), and I don't know how appletv+ is doing in 2023 but in 2022 and 2021 they were also losing huge sums of money. Is netflix the only major streaming service that makes money at this point?
Everyone decided "I can cut out the middleman and keep the money in my pocket" and they all lost hugely on that strategy.
Funny to think back on all those HN/Reddit stock threads parroting “anyone can put a video on a CDN Netflix is doomed lmao” as if being responsible for a double digit percentage of the entire internet’s bandwidth was some intern project.
apple has been in the content licensing and user distribution business for a long time with books, music, etc. Its a logical extension of their business. But they have more money than god and if their studio spending far outweighs its income they can absorb that with prophets from other divisions.
Also, apple has the problem where they have more money than they know what do with - if they can figure out a profitable movie and tv studio then it would be an honest to god diversification.
The real "Netflix Optimization Problem" is them figuring out how to minimize the cost of licensing or creating content while staying just good enough that you renew your subscription.
I'm still surprised Netflix hasn't bought a bunch of media companies with big back catalogs. I feel like it'd help with people complaining about lack of content and give them better negotiating power.
> Netflix is trying to do everything possible to hide the fact that their streaming library is small and shrinking.
And it appears to be working. I was telling this girl the other day about why I canceled Netflix years ago, because they frequently didn't have whatever I searched for. She insisted that "no, Netflix has everything!" So we proceeded to search for various things with her account, including the final straw that made me cancel. Netflix proceeded to not have a single thing we searched for.
There are a couple movies/TV shows that were unobtainable via Netflix DVDs, and which seem unstreamable. Some very old DVDs float around at high prices.
Examples are: Exit to Eden, The Others, and Baccano!
How does this happen? Why do rights holders want movies killed?
That is annoying as a user. I guess they remove content because of the contracts they have.
Wouldn't it be nicer to have all content on all platforms, and then when content is watched the creators receive a money payment? Not sure if that is feasible though, storing content also has a cost. To make it simpler they could switch from subscription fees to a fee based on views.
All the top talent they've acquired and money spent still can't save them from their complete garbage lineup of content they push out, all while limiting account access and raising prices.
Neflix revenue went from 4.3b in 2013 to 31b in 2022. Astounding success by any measure. That means they are doing something right, including their talent strategy. "garbage" is subjective. Neflix employees are likely paid bonuses on views / subscriptions generated, which from their revenues looks like they know how to hit.
Netflix obviously since enjoyed a period of success. You may need to reconcile that with your overly negative portrayal of the situation which honestly seems more about your personal feelings about Netflix rather than more objective measures of success.
You call it garbage content, but every other person is somehow watching them. I’m probably not their target audience, but looking at my friend circles, those shows are definitely being consumed.
Not only being watched, but most desired. Like, people threatened to cancel netflix if they implemented password sharing but inevery country they implemented it, they actually grew on subscriber numbers..
Unlike the old days of broadcast TV when the networks had to produce shows that appealed to the broadest possible audience, Netflix can, and does, produce shows that appeal to a narrow demographic. This results in a sliver of shows that are highly appealing amidst a lot of stuff perceived as "garbage." But what a teenage boy thinks is garbage differs widely from what a retired grandma things is garbage. To each their own.
I kind of agree with you. I feel like there's a narrow set of content there worth watching and when you get through it all it's just like: okay, now what? There's nothing else worth watching. I haven't been able to use my netflix account for months now. I just go to youtube now and subscribe to channels I like. Seems much more interesting than netflix.
Currently following some interesting permaculture projects. I find the type of work these people do so useful and clever really.
It's funny, all the top talent, all the blog posts, yet sorry to say as everyone has long learned, streaming platforms are utter commodity and were not an unique advantage at all. All made up. They were always just a nice wrapping paper around their content & licensing.
Wait, so they knew _in advance_ that they weren't going to use the result? Before they even started the competition? Or is this ret-conning once they decided not to use it. I feel like I can't really trust Reed's word on it, since he has a very strong incentive to paint himself as a mastermind.
In these kind of competitions (e.g. Kaggle), the algos never really get used because they are too complex/optimized to implement into production (i.e. 3 layers of 10 ensembled XGB + 1000 highly engineered features + etc, etc).
Mainly they are used for recruiting, new ideas (simpler versions get implemented), and proof of concept (if we did this really really well, what would it look like, should we spend 2 engineering years on it?)
Everything that Netflix says or does externally wrt to tech appears to be for this purpose, including open-sourcing some things. It is all catnip for the HN crowd to come work for them.
Netflix pays in the same ballpark as other FAANG companies but the product is so much clearer (stream people movies and shows, maybe recommend them something to watch) and the complete lack of grey ethics like in AdTech or a Monopoly like Microsoft is the real catnip.
I think it’s worth noting that actually, they pay the most of the FAANG companies, and they pay all cash, no RSUs. They also have some extremely generous benefits - one that comes to mind is 6mo fully-paid parental leave.
The HN take is highly sensationalized. It’s a great place to work with fantastic work / life balance, great pay, and friendly coworkers. I was there for nearly a decade and would never describe it as toxic.
If you knew you were going to stay there 9 years and make a tractor trailer load of money and love the culture you’d take the offer.
If you’re not sure which is more likely and you really like the maximum upside then you should take the offer and get out in 9 months if it’s not for you.
In this case we do know on average you are out in 9 months. Do you gamble you can somehow make an important friend or you will out political influence some of the 10 year vets; maybe you take the gamble. For the average person you would be better offer taking the other offer.
Well we know that in this case because you just made it up and added it to the story lol.
And yes, if I knew that the average new hire at Netflix lasted 9 months, I’d still probably take it. I think I’m much better than average and I’d want to test myself at a place that is challenging to work at. If I’m good enough to get a Netflix offer I can probably get another offer in the future.
The average person should do what the average person wants but the upside of a long career at Netflix seems pretty darn high and might make an average person an above average person.
I think that context makes a reasonable difference for some people.
(The following is a tangent.)
To put it another way, using a slightly different kind of situation, I don't want BigCorp's AI watching everything I do to "serve me better", but I'm very interested in the idea of a local AI that watches everything I do and helps me be N times more efficient, while I maintain full control. I don't care if the local AI uses the same data-gathering, data analysis, etc. concepts as BigCorp's AI to accomplish the goals I want it to accomplish.
Caveats include:
- The local AI is Skinner-boxing you or similar. This isn't about the AI having conscious malicious intent; it could be a design defect. Also, a local, offline, open source video game can still be addictive.
- The risk of the system being compromised / manipulated, and all that data on you being exfiltrated.
Hmm. I can't think of any time I've thought the app was really missing something, or didn't work. In fact, I always thought the app was pretty smooth and "just worked".
By comparison, several years ago, I was regularly streaming Netflix content without any trouble. One day I streamed a show on the HBO app on mobile data and it ate up almost all of my month's data usage. I guess it's behind the scenes work, and not so exciting, to e.g. stream a smaller resolution for a mobile phone, but in moments like that you realise there are a substantial amount of things to think about when making such an app.
If you live in the US there is a good chance that was your phone company throttling Netflix and the app automatically switching to 480p to adjust to the reduced bandwidth. They probably do that more broadly now but when they first started it was only Netflix.
I don't live in the US, but how would this work? Isn't the video sent over SSL? In that case how would the phone company know what to send back?
I just assumed that Netflix had different video formats on its' servers, ready to go, and when the app requested the video it also sent the screen size/resolution to get the smallest file size, that would also look good on my phone. If that was 480p then all well and good, it was fine to watch.
It was a while ago now, but IIRC the amount HBO used was pretty horrendous, like a GB for 30 mins or so of video, so I guess they just downloaded a big file and let the player on the phone handle that.
Like a say, a few years ago now so the situation may well be better with HBO now.
Yeah, it's not rocket science to build a streaming service using AWS or GCloud lego blocks in 2024, but remember that Netflix started streaming in 2007.
So the entire premise by which they were “recruiting” was dishonest? No one sees any problems with lying to people? Dishonesty has apparently become the norm, not the exception. Honestly and trust is a bedrock principle of society. It’s no wonder our society is in the state that it’s in today.
You really cannot trust anything anyone says, especially large companies. They basically always lie and manipulate and are at a constant advantage to those of us who find dishonesty morally wrong.
Eh, I don't think anyone participated because they just really wanted Netflix to have a better recommendation system. Teams put serious effort into it for the million dollar prize, which wasn't a lie. And hackers and tinkerers enjoyed the novelty of the challenge and the cool dataset to work with, also not a lie.
I don't think they ever said they would use it necessarily either? I feel like it was obvious that it wasn't going to capture all of their actual requirements - your algorithm could be slow or expensive or difficult to adapt to new requirements or difficult to adapt to new data etc etc.
Ironically, Netflix's recommendation algorithm is now notoriously bad. In my experience, it seems to heavily push whatever "original" they just dumped $100 million into as well as what's most popular on the platform at the time. But it makes sense, since people increasingly rely on social media for deciding what to watch. A dollar Netflix spends improving their recommendation algorithm simply won't stack up to the dollar TikTok/Instagram/etc. spends improving theirs, because social media apps have so much more data to work with. It's probably more economical from their POV to let broad social media trends dictate what people watch.
I suspect their algorithm has other success metrics than "did user enjoy movie". For example if the movie is a Netfix Original or exclusive movie then it is more valuable to Netflix because they may discuss or recommend it, which leads to more signups. Similarly they may prefer newer shows that are more likely to be discussed and raise hype than older movies that even if the user loves them may be less likely to advertise to others.
This 100%. I promise you that I’m not secretly very interested in Dragon Rider episodes without my kids present. They have their own accounts, but it is hard to share family movies or when they just use the wrong account.
I also think half of my viewing is with my spouse. We are pretty aligned in some shows but we are different enough where “My List” really means when I’m alone, and we are missing an “Our List” for when we are together.
As a sibling pointed out. This already exists. Profiles don't just let you do "John" they let you do "John and Sally". Maybe they should be more explicit that people can be creative ith their profiles. Would be good marketing for people who are stuck like this.
I'm pretty happy with how the accounts work: my children watch Octonauts in a Kids account (sometimes I watch with them), my wife and I have separate accounts, so she doesn't get recommendations for war movies and psychological thrillers, and I don't get recommendations for Queer Eye for the Straight Guy.
My daughter isn't allowed to watch Home Alone by herself, but she's allowed to watch it with either my wife or I. Parental Guidance.
But neither my wife nor I have any interest in Home Alone 2 being in our personal profile Suggestion lists.
And when my wife is trying to bring up Home Alone 2 to watch with my daughter, she doesn't need my daughter seeing Naked European Romance Drama showing up in the Continue Watching.
The point being that when I watch something with my 7yo, she's not gonna watch any of the crap I watch and I'm not gonna watch any of the crap she watches. We're gonna find something in-between.
I think they’re making the case that viewing profiles are an antiquated concept and what’s more interesting is shared viewership profiles, or some algorithm that takes multiple profiles into account at once.
in my experience from the outside it's difficult to separate their rec algorithm quality vs. the quality of their inventory. you're making the assumption they have a ton of great stuff that they're just not showing you, but they may not.
I wonder if the real goal is the secondary effect. If you watch a show on Netflix and enjoy it, that’s good for customer satisfaction. If you watch their exclusive original content, not only are you satisfied but now you are telling all your friends about something only Netflix can provide.
As someone who works in search relevance, having just a great algorithm isn't worth much.
You need all the team and know-how that has the maturity to maintain such an algorithm. Not just the ML skills. But all the bazillion ops, data quality, and many other things that go around it.
I've worked with a lot of teams that have one smart person building stuff off to the side, in R or a Notebook, and then nobody knows how to productionize it. They try to throw the algorithm over the fence. Even if the team somehow succeeds in getting it into an A/B test, it eventually falls by the wayside, unless they can build the team and workflow around that person / algorithm / methodology
> I've worked with a lot of teams that have one smart person building stuff off to the side, in R or a Notebook, and then nobody knows how to productionize it. They try to throw the algorithm over the fence. Even if the team somehow succeeds in getting it into an A/B test, it eventually falls by the wayside, unless they can build a team and workflow around that person / algorithm / methodology.
In my view, this marks a cultural failure - and certainly not on the part of the 'one smart person.'
Using a recommendation algorithm doesn't make any sense for Netflix' new business model where they produce their own content. It should be easy. They greenlight or buy the rights certain content for different demographics to maximize the coverage of their user base that has their needs satisfied enough to stay subscribed. If you're a 18-35 male, they've surely got people there working daily to make sure there's a content pipeline coming just for you. They shouldn't need AI to tell me that, they just need to identify when I'm in the demographic of one of their shows coming, and tell me about it. Maybe like a list. Still, they seem like they're doing a bad job at it, as I never really know what new shows Netflix is making for me. Sometimes I find out about them years later.
For example, compare this to Disney+. I vaguely know about every Marvel and Star Wars thing 2 years in advance, and usually vaguely know what order they're coming out, and I don't even know how I know this, somehow I just know. Okay, that's easy because it's basically 2 IPs. But ultimately Netflix is doing something similar behind the scenes, why haven't they succeeded at making me aware of what content I'm supposed to be hyped for?
To be honest, I would be happy if their 20+ people UX team would create an experience where I can easily find what I was watching instead of shoving stuff down my throat that I have no interest in.
Then again, as long as people pay for that experience, it will continue to be as unbearable as it is.
Right? Like can we please get a user story on some agile board somewhere that says: "As a user, if the last 10 times I opened netflix I watched the same show, I should be shown the option to resume that show very prominently"
The number of times I've had to scroll or even search to find the thing that I very obviously want seems intentionally maddening.
Same reason IKEA has no story: "As a custom, if I know what I want to buy, I would like to be able to go the shortest way from entrance to product to exit."
And yet IKEAs have signs notifying you of shortcuts. If you know what to get in an IKEA and know which department to find it in you can get in and out with some real quickness if you pay a little attention.
It's clearly not their main target, but it's also obviously still a target.
I am old enough to know a time when these signs did not exist and the shortcuts were unlabeled doors looking like the typical emergency exit. Regardless of the signs, IKEA makes it deliberately hard to just get what you want without being exposed to products you probably don't want. I find that apps that send you on a detour is pretty much the same tactic.
This is the problem across the board, regardless of service almost. Everything is some crappy recommendation system, instead of enabling me to find what I want.
In addition to sucking in general, recommendation systems have this great property of radicalizing people politically.
This feels like a disingenuous argument. “Young to middle-aged man likes Marvel and Star Wars” is such an obvious pick that it just about invites parody. As someone with at this point zero interest in Star Wars / Marvel content I a) wouldn’t retain the information that you did and b) wouldn’t find it to be that impressive that D+ was shoving it into my face.
To not understand the usefulness of recommendation algorithms almost feels intentionally contrarian.
Not picking on you at all, just going on a tangent because I've been thinking recently about the good ideas that end up on the cutting floor because their impact, while positive, is intractable or completely impractical to quantify.
Some ideas (like this Netflix one!) are "obvious" winners, because they have a diffused, positive impact in many important dimensions – each of which is almost impossible to measure, but the integral of which is almost certainly greater than the idea's cost, probably by one or two OOM. If there's high conviction in an idea being a 10x idea, and attempting it costs quite little, it's better to do it now and maybe consider measuring it later. Who cares if it's 9x or 12x ROI, the point is it has a big margin of safety and large expected returns on capital.
But in a "we can't greenlight this project unless we can directly attribute it to positive motion in our KPI" org, these flavors of idea are dead on arrival. The double-kicker is that the cost of trying to measure these ideas is often greater than the cost of the idea.
For some ideas, especially when it's a rounding error on the company's annual budget, it should be OK if we don't invest much effort into quantifying the results. It's one of those rare set of ideas where it's anathema to the professional/investor culture of SV of the last decade, while simultaneously being how plucky Seed/Series A companies can punch above their weight.
Yup. $1m is nothing. We live in a world where movies have $100m marketing budgets, game companies invest hundreds of millions into esports events and other material, and tiny companies are bought out "just in case" for similar amounts.
$1m to generate some publicity and attract talent? Sure. They're probably paying headhunters similar amounts in comissions each year.
Movies have very real returns that are trivial to see from box office results and merchandizing. Not to mention they have famously perverse incentives to spend as much of the profit from a movie as they can, due to how a lot of the royalties are structured.
More to the point, though, this is a bit of a tangent/deflection from my question. How do you assert the return on this $1m? This line simply says that no return was needed for it to be spent. It shows nothing to say how you calculate what is made from it.
I would push back on this specific case, though. What makes this an obvious winner? Netflix was already clearly on the upward swing. At the time of this contest, I'm not even sure I remember who their competition was.
Now, I think it is fully fair to say that this was very cheap for them to do. In which case, why not do it? But I think you would be hard pressed to give me a counter factual world that is believable where not doing this had a meaningful impact on Netflix's future.
> I think you would be hard pressed to give me a counter factual world that is believable where not doing this had a meaningful impact on Netflix's future.
Some scale for perspective: if this had a 10x ROI, they'd get $10mm for $1mm. At EOY 2009 (same year this competition was paid out), Netflix had a twelve-month-trailing Revenue of $1.671B. In 2006 they took in ~$1Bn. So it's 0.06% of annual revenue at payout time, 0.1% of annual revenue at announcement/green-light time.
The idea had a large margin of safety: in the worst case, nobody pays attention and you've generated some IP which would have cost ~$1mm to generate anyways, and which would theoretically increase retention (a top 3 KPI) by X%. And it had big upside potential, in the form of additional marginal hires, press/exposure of the core product in the news cycle leading to marginal subscribers.
Re hiring: I think it's totally plausible that it contributed to ~5% of 500 peoples' employment decisions since 2009.
Re subscribers: it's plausible to think it contributed ~1% to 10k peoples' decisions to subscribe, most of whom have retained to-date. It's classic "Brand Advertising" – and although it's difficult to quantify, it works.
Those numbers could be way higher, but it doesn't really matter if they were or weren't, the margin of safety, uncapped upside of the brand advertising angle, and low relative cost made it a bona fide good idea.
Edit: FWIW, that we're still discussing this idea in 2024 is more evidence that it was a good idea. This whole thread is full of Netflix's target demographic. We're all talking about how well-compensated and smart Netflix engineers are... You would never be able to purchase that kind of long-burn, compounding, high-expected-value attention for $1mm from any advertising network, in 2006-2009 or now.
Right, I think this is a good analysis. But, I challenge that they actually got these returns. Specifically from this effort. It is popular among the type of crowd that posts here in HN. I would be surprised if any of my non-tech family even knows it happened.
I think I would push on the ~5% of 500 people's employment decisions. With the sizes of the teams that competed, I'd not be too surprised that the teams involved were largely the extent of who was directly impacted by this contest. Everyone else was as happy as those of us that read any of the blogs/articles that they would publish about their infrastructure. Which is not nothing, but probably hard to spot the difference. (Ironically, I would argue that the raised morale of the people that publish the articles are probably more measurable? Which, I confess I don't think is a consistent position from me.)
And I stress that I agree it was a safe thing to do. I am even glad that they did it. I just don't know that you could really convince me they would not have had the revenue that they had without doing this contest.
Their competition was the old outdated business model. As easy as it was to say that streaming was the future, they still had to convince millions of people to learn about and sign up for it.
This implies that a large number of subscribers are here because of this. I would hazard most subscribers don't know this even happened. Much less did it have any impact on a choice to sign up. That was driven much more directly by high speed internet access.
I wouldn't fully disagree, but I would point out that they had a ton of press at the time for virtue of who they were. Indeed, a large part of why we still talk of them doing this contest, is because of who they were and are.
More directly, though, if you were to plot subscriptions to Netflix over time across an unlabeled chart, would you be able to see this contest?
If we are only referring to the cost of any media placement, I can't really disagree. My assertion is fully on ROI of this.
Take my challenge. Plot their subscribers and revenue over time, hiding the year/month. See if you can spot any sort of inflection where this happened.
It /could/ be that they would have seen declining numbers absent this. Such that this prevented a bad inflection on the chart. I don't know that I find that compelling.
To an extent I can't but agree. I can't help but speculate you are in the media placement game, though? :D
Like, I get that you can't measure everything. There are plenty of reasons to believe that.
I just can't bring myself to accept the odd logic that "popular tech company got press coverage which was instrumental in their growth." Seems there is a circular logic at play there that makes this such that, even if it is true, it is not actionable for anyone else.
Put another way, popular things get press coverage by virtue of them being popular things. At large, as long as they don't implode in very terrible coverage, they don't have to do much to get more coverage. Just doing anything will do that for them. To that end, odds are high we would still have discussed this even if they only offered something silly like a lifetime subscription for the winning team.
As I said in another reply, I don't remember them having trouble hiring in the first place?
I can certainly mostly agree with this. I just also think them producing their own shows was likely more impactful, and had basically nothing to do with this.
Don't get me wrong, plenty of moves in life are "win more." Such that I could see this one fitting that description. But a lot of "win more" choices are surprisingly low on the ROI side, when you control for everything else in that situation.
Coca-Cola doesn't have trouble selling soda, but they still have commercials.
Of course making their own shows was more impactful in a bunch of ways. But, the investment is also so much more. A season of a show is in the tens to hundreds of millions.
I think that is a bit of an apples to oranges comparison. Coke does brand management. They are not necessarily educating folks to their existence.
But, the real difference is that Coke is, in fact, selling their own brand and products. For Netflix, they are largely selling access to other people's products. In a very real way, Netflix found an arbitrage opportunity for viewing movies. With how poorly the content owners calculated things, they got good deals at the beginning so that they could do well on their investment in the CDN infrastructure so that they could deliver a good experience to users with minimal unit costs.
This is, in a large sense, why Netflix is having to raise prices now. The content owners are starting to demand more money for that access, driving up their unit costs. Combined with a draught of original content, expect that they will lose access to their own productions in time. You can already see this with the likes of Daredevil.
I'm talking about hiring. I believe this project had positive ROI for Netflix (where I used to work) because of how it helped bring attention to Netflix's ML problems. Maybe you disagree with this, totally fine.
My comment about Coca-Cola was to draw the analogy that just because they don't have trouble hiring (which you mentioned) doesn't mean it can't be helped.
Ah, that makes sense. I was thinking you were comparing to general advertising and the impact on drinker/user base.
If you saw a measured rise in hiring, I will trust you on that. My assertion here is largely on the trajectory that Netflix seemed to already have and how there is no clear "inflection point" in any of their numbers post this contest that I have been presented with. And probably more cynicism than makes sense for how little impact ML seems to have on any choices that Netflix seems to show.
You can think, how much would their marketing team have to spend to get the same results that the algorithm contract gave. I'm not a marketing expert, but I'm sure they have metrics like consumer sentiment, name recognition, number of users visiting the site, google search trends, etc. There could also be benefits in recruitment, and that can be estimated based on how much you'd have to pay an external recruiter to bring in candidates the applied, or other things like that.
It was in the news a lot, and was discussed on a lot of tech sites. Plus it gets people talking about their recomendation algorithm, and makes people thing Netflix subscription is more valuable becasue it recommends good shows. It wouldn't be cheap to get the amount of media that they got through more traditional marketing.
Right, but that is a lot of what I'm asking. What makes you think those are all better due to this contest? If I recall, a lot of why it was making the news was because Netflix was already popular in the industry. They certainly weren't that new of a name.
The main one I don't think I would have doubts about is the recruitment. But, I don't recall them being a place that needed recruitment help, even at that time.
To be clear, I found the thing fun to consider. I certainly am not upset that they did it. I do harbor a gut feeling that its ROI is greatly overstated.
at least for the internal goal of hiring top ML talent, you can probably just calculate how long it took to hiring the good ones, and then how much money that team made.
Are they objectively better at ML for movie recommendations than they were beforehand? I confess I've never been too impacted by their recommendations. Outside of new releases, but that doesn't really use any ML.
The Netflix Prize competition (2006, completed in 2009) was a Kaggle competition before Kaggle competitions (2010), with the same business-side incentives.
Given the meteoric rise of ML/AI in the past few years, I'm surprised that Kaggle doesn't come up more often. It was all the rage 2013-2018...then most I heard about it was that it allowed free access to TPUs.
I feel like kaggle is a copycat wasteland for most projects now. You don't quite get top talent, rather you get a bunch of people xgboosting their way up your leaderboard.
"...rather you get a bunch of people xgboosting their way up your leaderboard" -->
I don't see that as a universally bad thing. I feel this is actually very representative of the majority of ML projects that most organizations encounter. Most organizations don't have petabytes of data and a huge compute budget to train a DNN. They typically have megabytes to gigabytes of somewhat crappy data and need something that can be developed and deployed relatively quickly for low cost.
There have been quite a few interesting Kaggle competitions in recent years, as well as other interesting ML/data science competitions on other platforms.
Platforms like Kaggle, DrivenData, Zindi, AIcrowd, CodaLab and others are running dozens-hundreds of competitions a year in total, including ones linked to top academic conferences. One interesting recent one is this one on LLM efficiency - trying to see to what extent people can fine-tune an LLM with just 1GPU and 24h: https://llm-efficiency-challenge.github.io/
Or the Makridakis series of challenges, running since the 80s, which are a great testbed for time-series models (the 6th one finished just last year): https://mofc.unic.ac.cy/the-m6-competition/
Interesting that the answer is basically "because of akrasia". I wish more of these recommendation algorithms were based more off of who we'd like to be rather than how we behave.
Anyone who has experience with ML wouldn’t be surprised by that. Oftentimes ML competitions are about combining dozens of models together to juice the extra 0.01%—something that isn’t viable in a production environment as the quote in the article confirms.
AFAIK, modern ML challenges try to combat that by moving from answer-only submissions to code submissions and putting constraints on compute.
Hang on a sec, it looks like BelKor's Pragmatic Chaos and The Ensemble had the exact same Best Test Score and % Improvement. Did BelKor win because they submitted their solution 20 minutes earlier than The Ensemble? Or is that Best Submit Time something else?
Yes - from what I remember the team that came second lost out due to something like 6 decimal places and the time diff in submission. It was pretty close!
I might be mis-remembering this part - but I believe napoleon dynamite had its own algorithm in the ensemable because it was a tricky one to predict the rating for.
Dr. David Belanger (RIP) was a part of advising this team (BelKor)and knew all about this, but never ever bragged about until one day I stumbled upon it and asked him about his involvement.
Also, they definitely used code from the first two competitions in the company.
"The first year of the competition, in 2006 and 2007, the technical advancements that were made by us and the other big teams I think were really significant in the field of recommender systems," says Volinsky. He thinks the idea that Netflix didn't use the results is a misconception. "We gave them our code. They definitely did implement and use those breakthroughs that we made in the first year."
For more that are interested , I can speak about it to the best of my ability. https://ieeexplore.ieee.org/author/38180399800 ( that is some things about him) Essentially he would say that they initially shelved it over time as it was not needed, but they definitely used. Belanger tragically died November 18, 2022.
Streaming gives them much better information than a user's voluntary stars. Did you actually watch it? All of it? How soon? All at once?
User ratings are fraught. The things a user actually does are more likely to be sincere. And it doesn't put any burden on the user. I have a feeling that even the simple thumb up/down means less than whether you actually finished watching the movie.
To bad they did'nt use any of that information they have on me to recommend me things I actually wanted to watch. They just kept pushing the same shit I never once displayed intention in watching over, and over and over again.
Sounds like everybody is winning other than me. Though given I cancelled and never looked back, maybe I am winning afterall.
The problem is that they just don't have enough content you want. I think they'd give it to you if they had it. They're not hiding it. They just have so much catalog and when you've watched the good parts, cancel.
Most companies that size (in evolving industries at least) should have 10-100 of those moonshot/R&D/marketing projects on the go. And I'm speaking to the choir here - exactly: surely most readers of thenextweb would be thinking the same here.
does anybody here know a lot about recommendation algorithms? The one I'm interested in seems obvious to me, but it's never implemented so maybe it doesn't work, or it serves outliers and not the mainstream so why bother. The data is there, but I don't get to mine it. (heheh "mine" it)
I don't want to know what's popular, I want to know what other people who share my past tastes are watching currently. Doesn't matter the size of my niche, why can't i find my niche, and be shown my cohort?
I just don't feel it's an impossible task to find, for example, a set of people who enjoyed Deadpool 1, Guardians of the Galaxy 1, but think the entire rest of the superhero cavalcade is utter garbage. ok, I'll allow Kickass also, and a grudging nod to Iron Man 1. (as an illustration, I just searched to look up the name of Iron Man, I could only remember Tony Stark. Top search results were "a fictional character", "played by R Downey Jr", "Marvel cinematic universe"... You see, none of those is Iron Man 1. I don't care about the horde of people who care about a supposed cinematic universe. We (in my cohort) liked the first one, and what we liked about it wasn't continued in the 2nd.
How about preferences that show I like a few early films with say George Clooney, but after that, no. Julia Roberts, same. I remember when Meryl Streep was a new actress, but now she's a red flag. I'm sure my tastes aren't shared by everybody, but I'm also sure, since I can articulate reasons, there must be others who in general share them, people who are willing to watch what's new and trendy in some way, but then not beat the dead horse. Oh, Spielberg, a few winners, but overall "nope nope nope", the way he aims at sentiment is a zero for me.
I originally had this idea back in the 80's when Consumer Reports had their monthly mail-in bingo card for what current movies you liked, then they would tabulate what was popular among their readers. That type of "best of" list is entirely missing co-variance, which I think is where I live.
That's called collaborative filtering (https://en.wikipedia.org/wiki/Collaborative_filtering) and is perhaps the most battle-hardened and most effective approaches in recommender systems. Even now, novel deep learning approaches implement the concept, but simple naive approaches are still as/more effective. The first paper published on it specifically in the field was in the 90s but the seeds of it go back to the 70s. It would have made a good thesis in the 80s :)
Part of why it's so effective are for the reasons you outline, in that it can find items that you'll probably like, that aren't similar to items you already like, based on people that have similar tastes to you.
perfect info, yes, it's a way of discovering things that bore no relationship to what you had already seen, except that your cohort also liked them.
You triggered me to remember additional elements of my idea. the 80's saw the arrival of the CD-ROM, and I thought "Consumer Reports can't publish a detailed chart of everybody's taste, but I'll bet their movie voting database would fit on a CD-ROM, and a piece of statistical software could figure out what movies each person would like." But I didn't think people would pay enough for the service to get a new CD-ROM in the mail every month, and if it didn't stay up to date with the latest VHS releases :) it wouldn't be useful, and Consumers Union was such a ... purist/hairshirt type organization how could anybody even work with them.
I remember seeing the leaderboard rankings before the prize was awarded. If I recall correctly the top two teams were very close to the coveted goal. They ended merging to become one team and spit the prize once they achieved the goal.
Engineers called it "Not worth my valuable time" kind of engineering effort. An early version of the 10x engineer (who refuses to do lots of meaningless work).
This prize/competition has become a meme in itself and inspired hordes and hordes of people to dabble with recommendation systems.
The advance of recsys is in big part thanks for Netflix because it attracted a lot of people to the field, all companies started developing inhouse RecSys just like today you hear about inhouse LLM/chatgpt wrappers etc
> “We have adapted our personalization algorithms to this new scenario in such a way that now 75% of what people watch is from some sort of recommendation,” says Netflix
I am always sceptical of these claims. Today algorithms have become ai models. Sorting algorithms are recommendation engines.
I remember meeting someone at a networking event who believed they could win the competition and get > 90% accuracy.
I tried to explain that you can't read minds; you can't account for the fact that I'm going to watch a movie that I saw in a random internet post that I didn't know that I liked...
Which was my real beef with the contest: Computers can't read minds. They can't analyze the content of the movie and have the emotional experience that a person can.
They probably could. Apple photos is already capable of judging the quality of your photos, so it’s not too much of a leap to think that a model could be trained to recognise good from bad movies.
On the other hand there are far fewer movies than photos for training on so it’s hard to say how good the results would be.
Netflix became so user-hostile and it is truly baffling to me. They act as if they are paid by amount of time you spend watching movies and have all of these dark patterns shoving the user all over the place and disorienting them. I remember when they had tools to see what your friends were watching and to discover hidden gems which was always such a fun experience.
Does Netflix have such content nowadays? I really wouldn't know. They optimize the experience for binge-watching and nothing else. I wish so badly that they'd do a letterboxd thing and allow for curation of their catalog but it all goes back to the ephemeral nature of their content and how they have to hide that fact.
They care so little about that approach now it's no wonder they never used this algorithm.
One thing I dislike about Netflix is that there's a category for 'top ten in [your region]'. This doesn't really show the aggregate top ten pieces of media for a region, it is actually personalised to you. If I compare across accounts, the top ten are are different, sometimes overlapping but usually in a different order. This dark pattern exploits the 'join the crowd' bias (https://en.wikipedia.org/wiki/Argumentum_ad_populum).
Also, constant price increases and laying off large numbers of employees...
I agree. Netflix died the day they switched to thumbs-up and thumbs-down versus 5-star rating content. My recommended list was so solid for so long and all of that went down the drain.
IIRC, they pushed pretty hard on the idea that the thumbs up/down system was objectively better.
I was never convinced. Partly because I wanted to see the actual number of stars when choosing a video, and partly because their motives at the time were highly suspect (one of their own produced videos was getting bombed with 1-star reviews, IIRC).
The proof was(n't) in the pudding. Movielens has given me superb recommendations and led me to watch content that became some of my all-time favorite content. Netflix has never offered a useful recommendation at all.
Their death was presaged when their executives decided to "become HBO before HBO can become us" and when they spun out Roku for lack of appetite to be a proper tech company. Reed Hastings wasn't capable of running that business and so their vision shrank to match.
Netflix sold a lot of people a lie, their customers, their employees and their investors. We jumped on board thinking Netflix was serious about the living room experience in a wholistic way.
how was netflix ever going to continue being a "tech company." Video streaming is a commodity at this point, they are not longer doing anything technologically revolutionary.
Once every other media company could do what they were doing they realized they have to become an entertainment company. It was shrewd foresight on their end, otherwise they wouldnt exist anymore. Unless they tried to license out their infrastructure or something
Netflix's technology is incredibly advanced. The term 'revolutionary' is somewhat subjective, but what isn't subjective are achievements like streaming video at 400GB/s with a single server[1], no, make that 800GB/s[2], and writing a FUSE-based filesystem that runs at gigabit speeds despite storing data on AWS[3].
You can say streaming is commodity and uninteresting now but back in 2012 when these decisions were being made, they had the ability to pivot into making proprietary, higher-end TVs and set top boxes that improved streaming significantly to where it is today or further. I understand they didn't want to become TiVO, but this is where Netflix had the ability to change things by publishing exclusive content directly to their customers without relying on cable/satellite providers. I can't say it would be easy -- and that is what I mean about Reed Hastings not being capable.
Strategically, Netflix is beholden to its manufacturing partners and never had a wholistic story for the living room the way they claimed to want because they never had control. They never had control because they didn't come up with a compelling prototype. This would necessarily include gaming and other content.
A simple "don't show me this thing ever again" button would make Netflix 100x more usable, but it would quickly demonstrate just how small their catalog is.
> They act as if they are paid by amount of time you spend watching movies
At least up until 2015 (last published paper that I'm aware of) this actually is how they optimised their recommender:
"However, we have observed that improving engagement—the time that our members
spend viewing Netflix content—is strongly correlated with improving retention. Accordingly, we design randomized, controlled experiments, often called A/B tests, to compare the medium-term engagement with Netflix along with member cancellation rates across algorithm variants"
Gomez-Uribe, C. A., & Hunt, N. (2015). The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS), 6(4), 1-19.
It's probably a reasonable approach and has clearly been successful
What's wild is that it's not like they're matching industry norms. Most of the other big streaming platforms are far less chaotic and confusing. Even the shitty new "Max" interface is better than Netflix. It's the only one I will not use to just browse, I only open it with a purpose because the UI is so unpleasant.
> Even the shitty new "Max" interface is better than Netflix
I have to disagree. I don't love the Netflix interface, but Max is truly terrible. It is egregiously slow and it's difficult to locate the things that I want. And when the thing I want happens to be highlighted at the top of the homepage, it's always a dumb "just play more of this" instead of a link to the show info page, which almost always plays the end credits of the last episode I watched, and it takes 60 seconds to load that, then another 60 seconds to start the correct episode when I find it. If I get a take-out burrito and sit down to watch TV and eat it, I always think to myself "I wonder if I can actually find and start a show before I finish my entire burrito," which is an exaggeration, but not by a lot. All the accumulated UI "wait for something to load" moments add up to minutes. Netflix, on the other hand, is almost always more or less instantaneous (2-3 seconds maximum per UI navigation, 5ish seconds to start streaming).
Not surprising. I'd been using movielens for recommendations prior to the prize, and even today Netflix still does not provide high quality content recommendations.
Worse yet it seems to think I want to watch the three things that are most popular on the platform.
This indicates that the Netflix UX business is being significantly mismanaged.
I suspect the Netflix recommendations system would be excellent if they had a catalog to back it. But the Netflix catalog is of such poor quality that even a perfect recommendation system ends up recommending garbage because it's all they have.
Isn't their algorithm also very intendedly only catering to Netflix originals by default?
I find its algo absolutely rubbish and most of my initial excitement to go watch something with my wife on the couch instantly fades as soon as we face the doomscrolling of finding content to watch.
For me it's the missing quality markers. Netflix provides no way at all to see whether a show or movie is good. So when one starts scrolling and does not find immediately something one likes, there is no reason to ever find anything good - because nothing seems good, and all alike.
Since I added a userscript to see IMDB ratings directly next to the netflix shows it's way easier to find something promising.
I always hear people say this about the Netflix catalog, but I never understand what metric or criteria people are using. I find there is a ton of great content on Netflix, and I have way more things I want to watch on there than time to watch it.
I always wonder if people who say this either have very particular interests, or are trying to find specific tv shows/movies and not finding them on Netflix. If you are not set on a particular thing, it seems to me that there is a ton of great content.
I don't stream a lot. Every few months I'll just want to zonk out for a while though. The past few times I've wanted to watch "nothing in particular" it's gone something like:
1. Sit down, open Netflix.
2. Scroll through all the categories and suggestions on the home screen, finding nothing interesting.
3. Search for a few things I thought of while scrolling. Search returns things I can watch instead since what I searched for isn't on Netflix.
3. Open Prime Video, see something I'm interested in suggested pretty quickly, pay $ to rent or buy.
I second this. I also experience this with Youtube. If Youtube monetized the part where you're scrolling looking for a video they would 10x their ad revenue.
That used to my experience many years ago before I started using the "not interested" button aggressively. Now every video on my YouTube front page is 10/10
I have really broad interests in film—it blows my mind when people complain that not enough good or original films come out in a given year, I can't even make it through half the probably-good ones each year, I don't know what they're looking at or what their tastes must be like to come to that conclusion—but definitely find Netflix's catalog to be notably weak compared to Hulu or (HBO)Max, and it's been that way for years. I'd class it with second-tier services like Paramount, but from what I've seen on there lately, even that may be better.
It's got Apple TV beat, I guess, but I wouldn't call that a major streaming service.
[EDIT] Oh and I'd say it's overall worse than D+, for that matter, even though that's basically an umbrella for a few niche services. Like if I could only have one of those two, I'd take D+. And D+ kinda sucks.
> it blows my mind when people complain that not enough good or original films come out in a given year
I have a weird take on this.
Before I started working on my startup, I had a lot of time to watch and even make movies. I typically would watch about ~50-100 films a year. When I was younger, it was even more. I was voracious.
I tend to like indie and art house films, but I'll also enjoy popular fare. I'm not a snob. As long as the choices of the writers, directors, actors, and editors make sense, I'm typically able to assign a rating and slot the film into some kind of tier list.
But I'm always left unsated by the decision envelope.
The film will make its choices, but I always find myself wanting to go in other directions. Sometimes wildly different ones. I feel more like an active participant, critic, or perhaps even script doctor or producer.
When the story beats of a film change, my imagination keeps predicting outcomes along various trajectories. Sometimes I'm a perfect predictor. In these instances, films tend to be full of lazy tropes and are completely unsatisfying. Other times, my predictor is way off. If the dissonance is huge, sometimes I'm also left upset by the departure from expectations. It depends on whether the rest of the work makes sense as a whole.
I can't turn this off. It's always running.
I only see a handful of films a decade that will sit with me for days or weeks. The themes rhyme and have an ethereal logic that transcends me. These are the films that play my predictor like an instrument. Their choices are better than mine. Surprisingly and delightfully so. They reach my soul and change who I am as a person. These are films that I wouldn't change in the slightest. That I wish I could forget and watch again, and again, and again with fresh eyes.
I enjoy film, but I can't stop my overactive analysis. As I've gotten older, this attitude has only deepened. It makes it hard to call watching film entirely passive or enjoyable.
I enjoy the activity, but not always the film. The feelings are fleeting, and I'm left wanting for more.
You've thought about this a lot, and I understand it's a deeply personal thing, but I'll ask:
What are a handful of recent films that were noteworthy by your criteria?
And a handful of all-time noteworthies?
Watching time-filler crowd-pleaser nothing-to-say movies that seem to be made simply because the world is full of non-discriminating consumers and someone needs to take their money ... leaves me feeling pretty bored.
There's a time and a place for it all, I guess. But life (free time) is too short (scarce) to waste.
What sort of thing are you into? What are some movies you really like? Would help to tune any suggestions.
[edit] Meanwhile, some shots in the dark:
- The Green Knight — Based on the story by the Pearl Poet and expanded and adapted with a combo of other Arthurian tales and the filmmakers’ own twists. Basically, a lonely, somewhat psychedelic (but not in a bright-kaleidoscope-colors sort of way…) dark fantasy journey with tasteful hints of dozens of influences adorning it. A great Winter watch, too. Benefits from the best screen and system you can get in front of.
- You Won’t Be Alone — Broadly horror-themed but not in a Hollywood monster movie or Halloween popcorn muncher sort of way. It’s not really aiming for scares, exactly. Comically similar themes and message to a major widely-lauded off-kilter film from the same year (2022) but approached from a very different angle. Macedonian language. You may worry in the first twenty minutes that you’ve stumbled into the Easter European Indie Movie Doldrums of Mediocrity, but persevere, it starts to come together and develop purpose before long. Hard recommend for fans of The Witch who liked the “historically accurate” grossness & mythology of that film, but it’s not as firmly in the horror genre as that was.
On the lighter (and not quite as good, good) side:
- The Kid Detective — Semi-comedy. Imagine the stereotypical middle-aged guy who’s never topped their time as a high school quarterback. Now imagine if instead of playing football, they were a kid detective (you know, Nancy Drew sorts of stuff) and that is the thing they peaked at in high school. That’s our protagonist. Film sags at times but the premise is so fun and relatively-well-used that it’s still a good time, if that blurb above piqued your interest at all. Certainly a sort of noir, with all that entails, so if Your Hero getting his ass kicked more than doing the ass kicking bothers you, avoid (as with lots of noirs)
- Spontaneous — Dark high school romance-comedy. Also, people, all over the place, have started occasionally suddenly exploding for no evident reason. That’s the movie’s deal. It took me embarrassingly long to realize what it was really about (or, at least a major one of the things it’s about) and it punched me in the damn gut when I did. Oof.
I'm open to genre. Mostly I'd like to spend a couple hours watching something that is either an impressive work of some kind (but not one that requires a film degree to appreciate), or has something mutilayered and interesting to say about people or the world. The latter seemed to be where GP was going, and I was curious to learn what films meet that criteria for them.
I reliably enjoy films from Miranda July and Greta Gerwig. I think I'd enjoy Wes Anderson more if he was a bit less campy (but I get it, that's a big part of the point, so I guess I appreciate the other things he's good at).
Thank you for the suggestions -- I have never heard of any of them. They are far afield from my known preferences, but sound like the better versions of the options that Netflix wants me to watch, which is great! I will check them out.
With those preferences stated (I almost listed Lady Bird!) I'd toss on:
- Nomadland — Lonely older woman clinging to the bottom rung of the economic ladder is pushed into van-living and finds community. Basically positive-but-bittersweet vibes by the end, and with a similar strong reliance on setting and mood to something like a Wes Anderson work (though without the precise placement of every single element in frame), but definitely not "cute" like his are, and not about comfortable rich people. Includes many real people living in vans, blurring the lines between fiction and reality (and deftly almost-entirely avoiding the usual problems that come with including non-actors in a film). Lots of great outdoor shooting and lighting work.
- Red Rocket — Struggling, middle-aged male porn star moves back to his small, blue-collar hometown in Texas. We learn about his still-evolving relationships with the people he left behind when he moved away to LA, and see him make some... choices. Probably the most into archetype and symbolism of anything I've listed except maybe The Green Knight. By the end, is very much about its protagonist as a kind of daemon, if you will, of a part of the American experience and spirit (and also a representation of one or more sorts of real person with whom one may be familiar from one's own experience). Great attention to detail for a certain kind of ordinary setting, in the way that Anderson or maybe Miyazaki pay attention to that kind of everyday detail, but employed to obtain perfectly-trashy environs that aren't at all pretty but are also entirely true-to-life, not caricature. If you've spent time in such places, the clutter, the way trash has accumulated, the way in which the miniblinds are broken just so, and the way the people exist in that space, will feel very familiar.
- Eighth Grade — Bo Burnham's only film (I think?). Akin to something like Gerwig's Lady Bird, I'd say, but less heightened and more "real".
Hey, maybe you should consider writing fanfiction to satiate these urges and analyses. Not joking. I don't write fanfiction, but I read it, and whenever I read the author stating their motivation for writing, it's something like this, just for the story of some particular series, books, or game, rather than it being noted as a trend.
That's especially true since I've noticed stuff max yanked has popped up on netflix.
If netflix wants to lean in even further, they should pick up rights to produce additional seasons of most of the stuff Warner Brothers Discovery cancelled last year (they're the parent company of HBO and have subpar management, even by Hollywood exec standards).
depends on which country you live in, and what you've already watched so far.
I'm not one of the people complaining, but I've definitely seen my self scrolling a lot with Netflix recommending I re-watch a lot of the stuff I liked before only to end up giving up and switching to something else.
This raises an interesting point. Streaming has become so fragmented that I wonder if there enough space for a "$1M algorithm" which spans across all of the streaming platforms.
I am not sure how that data would be gathered, maybe via an intermediary player like Plex, Roku, Android TV, etc..?
They want to present to you what they want to serve you. To save money, they want to serve everyone as much of the same content as possible. Their algorithm isn't optimizing for you, it's optimizing for them.
How does their licensing structure work? Do they pay a flat fee for unlimited license to some/all shows? Or do they pay a base + a micropayment for each view? If the latter, then of course they dont want to optimize for you, they want to find the optimization point between maximizing subscriptions vs cheapest content. Similar to the recurring conversations about the economics of buffets: you want a lot of people to come in, but not the ones who just eat all the expensive things.
Unfortunately, if they didn't go heavily into content creation, they'd be dead as a business, as all the rights holders are no longer selling them streaming rights for a song.
That’s by design. They used to have a perfectly reasonable system where you could search by category and sort by popularity and rating. They killed that specifically because otherwise no one would watch the buckets of random trash they have on there.
Those contests are never about the actual code/algorithms they produce, it's always about recruiting the people who write them.
For the price of $1mil (plus a bit of engineering time) they got a list of talented engineers who have too much time, and also everyone now knows that they are the company with the really challenging problems. Forward the list to recruiting, and you got a bunch of new hires, much more effective than paying for job ads.
I don't get the obsession of some companies to recommend things. I'm happy not only for not paying Netflix but also for not wasting time watching crap. I have my own manual recommender: browse IMDB or Rotten Tomatoes of past directors, cast, etc that I've enjoyed. Also from a good old friend that from time to time recommends me films as we have very similar preferences. Randomly browsing genres worked very well too. That has become a near perfect movie recommender over many years.
They push very aggressively for whatever new TV series / movie they just released that is "top 1 in your country".
Spotify is the same with whatever crappy playlists and podcasts they want you to listen.
YouTube doesn't even bother showing results that you searched past the 10 or so results, it's "content you might like" which is crap and unrelated to anything I've watched, 100% of the time.
Is this a serious comment? Netflix does not and should not base its strategy on how you specifically choose to consume media. Surely you understand that most Netflix users don’t use IMDb/RT at all, let alone as a recommendation ‘engine’.
Netflix, especially Netflix back then, is highly reliant upon in-product recommendations because if people sought out specific pieces of media they’d often find that Netflix didn’t have it.
I dunno I think when they work recommendations are actually quite good. YouTube's recommendation system is very dumb but even so it's quite good (sorry Googler's; I'm sure you have optimised the wrong metric extremely well). Amazon's "people also looked at" is very useful - more useful than the actual search in my experience.
I think it really falls down on Netflix because it doesn't have anything good to recommend to you 90% of the time. Doesn't matter how good your recommendation algorithm is if your catalogue is 90% dross and the user has already watched the remaining 10%.
This is fine (although not particularly in harmony with Netflix's business goals), but I think it's also a recipe for stagnation.
My question is: how do you discover new things that you might like even more?
I think about this more in terms of music than movies, but maybe it applies to many media.
Some people will listen to Aerosmith and The Eagles until they die. They do not need media discovery. But how do new artists find an audience and how does that audience find new artists?
Recommendation engines suck, but is this a failure of current implementations, or something more fundamental?
I do use the IMDB recommender but is rarely my first choice. I'm not entirely disconnected from recommendations, but I do take my time to pick what to see (by reading some critics you can also get related content). If it's something interesting and maybe not so similar to something that I've recently seen I consider watching it. I guess that at some point yes, you need to have some source where to pick things from. With music sometimes can get easier, you listen a band then learn that maybe a guitarist or bassist is now part of another project (or has been in a past band). Going to Wikipedia and starting with a band you're interested in you can get a ton of new music this way. Other times simply if you're listening something you like you can use Shazam.
They didn't really need an algorithm in the first place, they just brainstormed what would sound the coolest to developers.