To make this research more interesting I'd like to see:
1) repeated queries from the same user. Do the results stay constant over time or do they change?
2) comparisons to the same experiment run against e.g. Bing or DuckDuckGo.
It seems to me that some variation in results is to be expected because of users hitting different backends which might be at different stages of index rollouts. Similarly, response times of different backends matter. If for example the video results don't come back in time you'll end up not having them in the result set.
Lastly, the insinuation of the article is that "unbiased" search results are clearly preferable. I'm not convinced. I for one like that STD for me is associated with the C++ standard namespace (which I search for all the time) rather than sexually transmitted diseases (which I luckily don't have to care about as much).
> Lastly, the insinuation of the article is that "unbiased" search results are clearly preferable.
the insinuation is that you should know if they are biased or that you should be able to get unbiased result if you so wish.
It also raises suspicions on how much google tracks each user.
From this point of view what would be interesting would be a local study, to see in 100 people all in the same neighbourhood with different browsing habits have different results. this would eliminate the "non-tracking" part of the personalization.
> Lastly, the insinuation of the article is that "unbiased" search results are clearly preferable. I'm not convinced. I for one like that STD for me is associated with the C++ standard namespace (which I search for all the time) rather than sexually transmitted diseases (which I luckily don't have to care about as much).
On the other hand, authors could find better names for their libraries ...
Further, there are different solutions, where the user has full control over the context of their search. For instance by maintaining a fully user-controlled list of keywords that is remembered by a cookie (which can be deleted as well).
1) repeated queries from the same user. Do the results stay constant over time or do they change?
2) comparisons to the same experiment run against e.g. Bing or DuckDuckGo.
It seems to me that some variation in results is to be expected because of users hitting different backends which might be at different stages of index rollouts. Similarly, response times of different backends matter. If for example the video results don't come back in time you'll end up not having them in the result set.
Lastly, the insinuation of the article is that "unbiased" search results are clearly preferable. I'm not convinced. I for one like that STD for me is associated with the C++ standard namespace (which I search for all the time) rather than sexually transmitted diseases (which I luckily don't have to care about as much).