[Disclaimer: Former Amazon employee and not involved with Go since 2016.]
I worked on the first iteration of Amazon Go in 2015/16 and can provide some context on the human oversight aspects.
The system incorporated human review in two primary capacities:
1. Low-confidence event resolution: A subset of customer interactions resulted in low-confidence classifications that were routed to human reviewers for verification. These events typically involved edge cases that were challenging for the automated systems to resolve definitively. The proportion of these events was expected to decrease over time as the models improved. This was my experience during my time with Go.
2. Training data generation: Human annotators played a significant role in labeling interactions for model training-- particularly when introducing new store fixtures or customer behaviors. For instance, when new equipment like coffee machines were added, the system would initially flag all related interactions for human annotation to build training datasets for those specific use cases. Of course, that results in a surge of humans needed for annotation while the data is collected.
Scaling from smaller grab-and-go formats to larger retail environments (Fresh, Whole Foods) would require expanded annotation efforts due to the increased complexity and variety of customer interactions in those settings.
This approach represents a fairly standard machine learning deployment pattern where human oversight serves both quality assurance and continuous improvement.
The news story is entertaining but it implies there was no working tech behind Amazon Go which just isn't true.
That's some fascinating background, thanks! Probably explains why they keep operating it in stadiums but not grocery stores. Works pretty well with a small handful of items, does not scale up reliably to shopping carts full of stuff.
I worked on the first iteration of Amazon Go in 2015/16 and can provide some context on the human oversight aspects.
The system incorporated human review in two primary capacities:
1. Low-confidence event resolution: A subset of customer interactions resulted in low-confidence classifications that were routed to human reviewers for verification. These events typically involved edge cases that were challenging for the automated systems to resolve definitively. The proportion of these events was expected to decrease over time as the models improved. This was my experience during my time with Go.
2. Training data generation: Human annotators played a significant role in labeling interactions for model training-- particularly when introducing new store fixtures or customer behaviors. For instance, when new equipment like coffee machines were added, the system would initially flag all related interactions for human annotation to build training datasets for those specific use cases. Of course, that results in a surge of humans needed for annotation while the data is collected.
Scaling from smaller grab-and-go formats to larger retail environments (Fresh, Whole Foods) would require expanded annotation efforts due to the increased complexity and variety of customer interactions in those settings.
This approach represents a fairly standard machine learning deployment pattern where human oversight serves both quality assurance and continuous improvement.
The news story is entertaining but it implies there was no working tech behind Amazon Go which just isn't true.