One important metric is precision for the first result or "P@1" which was something Google originally excelled at.
I was working at a search vendor where I had the chance to debrief high-level defectors from Google and Bing and found that they (like us, with our patent search) struggled to get P@1 above 70%. They saw personalization as a way forward but also told us that personalization at both of those vendors was limited to a few cheap tricks and their were structural reasons why neither vendor would do it in a deep way.
The trouble is that you can at best make an informed guess about what somebody searching for "testing" wants. You ultimately have to get them into a dialogue which might be "I gotta search for 'nondestructive testing'" or could be based on some diversity ranking so that 'covid testing', 'software testing', 'psychological testing', etc. are represented in the top few results.
Right - this was called result diversity. But in markets like the US, where avg revenue per user is high, polarization is also high and there is some belief that big software companies have a role in changing societal wrongthink, you get these types of results. For this type of a query, I'd think P@1 would be 0% for some 30+% of the society.
Perhaps that'll be readjusted in a year, when covid is not top of mind for a majority of the users.
For a mass market product the emotional response that people have to the results is pretty important.
It doesn't seem crazy to me that people are seeking out COVID-19 testing now. On the other hand I don't have any question in my mind about where I would go for COVID-19 testing because both Cornell and Cayuga Medical Center run testing sites that are well oiled machines. I am entitled to use one for free because I am staff, students are required to get testing once a week. If I am getting a medical procedures done I am required to get tested by CMC. For people in Tompkins County there is very much a "right answer".
Is testing, particularly personal interested in testing really politicized? My wife was required to get tested once a week when she was helping out at a nursing home and she found that pretty annoying. If somebody wants to get tested personally though what could be wrong with that?
That makes it a social network optimized for engagement.
Someone posted a link to you.com in another sub thread. The results for the same query over there is what I'd expect.
I'm thinking that living in a deep blue geo is what's coloring my experience. Perhaps others can post what they see from an incognito browser.
Ultimately this is where big data analysis should be used. Log incognito results from a geo-diverse set of IPs to understand (a) the ranking model (b) the consequences of the model.
I was amazed to search for "social media behavioral sink" and has Google turn up a recent comment on HN I wrote.
One cheap personalization trick that works is to raise the rank of links that you click on preferentially. It drives SEO and other people concerned with their rank up the wall because they see it changing and can only speculate about what other people see.
One important metric is precision for the first result or "P@1" which was something Google originally excelled at.
I was working at a search vendor where I had the chance to debrief high-level defectors from Google and Bing and found that they (like us, with our patent search) struggled to get P@1 above 70%. They saw personalization as a way forward but also told us that personalization at both of those vendors was limited to a few cheap tricks and their were structural reasons why neither vendor would do it in a deep way.
The trouble is that you can at best make an informed guess about what somebody searching for "testing" wants. You ultimately have to get them into a dialogue which might be "I gotta search for 'nondestructive testing'" or could be based on some diversity ranking so that 'covid testing', 'software testing', 'psychological testing', etc. are represented in the top few results.