An offshore workforce is training Amazon’s warehouse-monitoring algorithms

This report is published in partnership with the Bureau of Investigative Journalism.

Inside a vast Amazon warehouse in Beaumont, California, squat blue robots carrying eight-foot yellow shelving units perform a jerky, mechanized dance around each other as they make their way to human workers. 

Amari* works 42 hours a week there as a stower, placing products on the shelves robots bring to him. “Cameras are trained on your station at all times,” he said. “It’s kind of demeaning to have someone watching over your shoulder at every second.”

But it’s not just Amari’s managers who are watching. An AI camera system also monitors the stowers’ movements — and if it fails, a video is sent to someone thousands of miles away whose input helps to improve Amazon’s machine learning tools.  

The videos are reviewed by workers like Viraj in Bengaluru, India. “It is very hectic work,” he said. “We shouldn’t blink our eyes while reviewing a video because our accuracy will go less. We have to be on screen at least eight hours — which is kind of painful.”

Amari and Viraj may work in different countries doing different jobs. But both perform mind-numbingly repetitive tasks whose output is strictly monitored, all of it serving to fine-tune the very system used by Amazon to closely monitor its own workers — and create the seamless experience enjoyed by its customers.

Reviewers like Viraj get through up to 8,000 videos a day, with their output ranked against that of their colleagues. The relentless pace of their work can take a serious physical toll. They are paid as little as $252 a month.

Video reviewers interviewed by the Bureau of Investigative Journalism reported physical problems, including headaches, eye pain, and even deterioration in their eyesight. They said they were made to hit punishing targets, with tracking software logging any periods of inactivity outside of designated break times. While one expert recommended that people doing video annotation should take screen breaks every half an hour, some said their targets did not allow for this.

The Bureau interviewed 33 current and former Amazon employees, including 21 video reviewers, to shed light on a little-known outpost of Amazon’s sprawling global operations.

Amazon spokesperson Steve Kelly disputed a number of the allegations in this story, saying the Bureau and Verge had “selected a handful of anecdotes to paint a misleading picture, and we do not believe they represent the vast majority of our team.”

Amazon has developed an extensive range of applications that use computer vision — a branch of machine learning in which computers process large numbers of images and learn to recognize patterns.

The cameras trained on Amari’s station use computer vision to automatically register the location of products in its inventory and flag errors he makes. This technology has also been deployed inside Amazon Go stores and to monitor compliance with social distancing guidelines by warehouse workers.

Amazon says the system’s algorithm is 95 percent accurate; the remainder of cases require manual checks. That means that every day, millions of images and videos are sent to workers based in India and Costa Rica, who decide whether a product was successfully stowed and indicate where on the shelving unit it is located. 

Video reviewers said their primary role was stock management — but they can also record errors made by their colleagues overseas: two former workers said reviewers could raise “stow etiquette” issues if they saw stowers breaking Amazon’s rules on camera. 

Most crucially, though, their nonstop manual work helps to improve the computer vision system, which learns from their responses and becomes more and more accurate. But the people teaching Amazon’s computers to see said their own eyes have been damaged by the work. 

“We will not be able to even blink our eyes as we need to keep a watch on the videos,” said Prisha, a former video reviewer based in Hyderabad, India. “That impacted my health a lot. It makes the eyes really dry because you constantly stare at that screen.”

The videos are anywhere between two seconds and two minutes, and reviewers said they can watch thousands in a day. Shifts are usually eight to nine hours long, though they can be increased to 11 hours during busy periods, such as in the run-up to Christmas or Black Friday. Reviewers get around one and a half hours’ break time, with any periods of inactivity outside of that instantly logged by tracking software.

Indian interviewees said they earned 25,000 rupees ($306) per month on average, while the Costa Rican average was 514,000 colones ($843). 

The humans behind Amazon’s all-seeing computer vision are themselves tightly monitored while they work. Mateo, a former reviewer in Costa Rica, usually spent his shifts checking that workers in US warehouses were observing covid protocols. But on one occasion, he saw something unsettling in his feed.

It was an Amazon break room, with chairs set out, very similar to the one in his own building. It gave him an uncanny sense of being observed. “Probably someone else, somewhere else, was watching me at the moment I was watching them,” he said.

Managers keep track of reviewers’ performance with real-time analytics, and they must maintain a high accuracy rate, between 95 and 99.5 percent. Nitara, who took a job with Amazon in Bengaluru after university, failed to pass her probation period due to missing her accuracy targets. “We were not allowed to make mistakes,” she said. “For me, that was quite difficult to handle. I’m human, I’m not a robot.”

While they decide how to categorize a video, a timer on their screen counts how long they take. If they linger too long, their “TAKT” time — the average time to get through a video — will increase, and they may be subject to retraining, disciplinary processes, or even face losing their job.

“You can’t move or do anything,” said Prisha. “If you even give a little gap, your TAKT will increase, and you may land at the bottom.”

A document passed to the Bureau by a former reviewer in India shows the TAKT ranking system for 25 employees, with the bottom four names highlighted in red. The top performer has a TAKT time of 5.7 seconds; the bottom-ranked person has 13 seconds.

Jiyan, another former reviewer based in India, said that while the targets were manageable, the work was still “stressful.” What bothered him most, though, was the monotony. “It’s a very boring job,” he said. “The entire day, for seven and a half hours, you’re doing the same thing over and over again. There is nothing new.”

In Amazon’s peak period of October to December, reviewers said their work increased significantly, with less time between videos, and one said bathroom breaks were more difficult to take. Another India-based worker said he couldn’t take time off for the Hindu festival of Diwali.

Amazon said that workers in India had the option to take Diwali off, and the company’s spokesperson Kelly said workers in India and Costa Rica were “encouraged by the software they use to take short breaks throughout their shifts.”

In the summer of 2020, Amazon’s warehouse in Bolton, England, was rolling out a new stowing system — known internally as “Nike” — that relied on computer vision and manual checks from workers in India and Costa Rica.

Workstations had been remodeled to include three new cameras trained on stowing stations, which would register the location of a product, largely eliminating the need for handheld scanners. Amazon said this would shave crucial seconds off the stowing of each item. 

But Naomi, who worked at the warehouse at the time, found it difficult to adjust. “It was just quite nitpicky — the way you had to stand, the way you had to move,” she said. “You couldn’t really have your own freedom in the way you did things.”

To maximize the computer’s chance of success, stowers were told to ensure they were in view of the camera and to use “clean and straight movements” when putting an item away. In cases where the system failed, footage was sent to video reviewers to verify.

The new system was also rolled out at the California facility, where Jade, a former supervisor, would often start her day by reviewing error reports from previous shifts. “There were like 30 or 40 different rules of how things had to be specifically stowed,” she said. 

Many of these reports, containing photos of the stowers’ violations, were generated by the new camera system. If enough of these errors accumulate, it could lead to a disciplinary process known as a “write-up.”

Amazon disputes that the automated system was used to discipline workers. “Stowing metrics are shared with employees and managers to identify strengths and opportunities for growth,” said Kelly. “We don’t use the Nike system information to coach associates on ‘stow etiquette violations’. Nike cameras are programmed only for inventory purposes … Their focus is product placement.”

Jade would also check that workers were performing well in other key metrics: “rate”, or the number of units they stowed per hour, and “time off task”, or how long they were inactive outside of break times. 

Isaac, a former stower at a Michigan warehouse, received a write-up after accumulating around four minutes of time off task. He was feeling unwell and went to get some medication and use the bathroom at the end of his break. Despite explaining this to a manager, he still received a written warning.

(Amazon insists Isaac’s experience isn’t representative. “Employees are reminded to take short breaks throughout the day,” Kelly told us, “in addition to regularly scheduled longer breaks during each shift.”)

Jade would also check that workers were performing well in other key metrics: “rate,” or the number of units they stowed per hour, and “time off task,” or how long they were inactive outside of break times.

She said she felt the productivity targets were set at a reasonable level, and people who tried could hit them. However, she also described the work as “mind-numbing.”

“Your brain just kind of dies slowly while you’re doing it,” she said. “Even though the people in Costa Rica aren’t doing the physical side, they’re doing the mind-numbing brain side. You’re treated like a robot.”

During 2020, when covid protocols were in place across the world, the company said it would retool its machine learning systems to help enforce social distancing in its warehouses — a program it called Proxemics. 

Workers in US warehouses saw themselves displayed on large screens with a six-foot green circle around their feet. If they got too close to a colleague, the ring would turn red. In cases where the computer was unsure how close they were, images were sent abroad for additional checks.

Thiago, who worked on the Proxemics team in Costa Rica, told the Bureau: “It was a difficult job. I think it could be the worst one I ever did in my entire life.”

Like Prisha, he felt it was hard to look away from the screen if you wanted to maintain high metrics. “At the beginning, my eyes would cry,” he said. “Every six seconds or less, you get another image. It was crushing. 

“It was difficult — to the point that you can’t look to the side because if you look to the side, you’ve already had 10 seconds on your screen.” 

Sudip Bhattacharya, assistant professor at the All India Institute of Medical Sciences, Deoghar, said video reviewers are at risk of digital eye strain, symptoms of which include dry eyes, impaired vision, and headaches. “If the resolution is low,” he said, “there is a risk of permanent eye damage.”

He also recommended screen breaks every 20 to 30 minutes. But some workers interviewed by the Bureau felt it was difficult or impossible to take breaks outside of their allotted times.

Thiago said he was given around six minutes each day as a designated “eye refresher,” with an additional 10 minutes to use the bathroom outside of scheduled break times.

When he was promoted to team leader, he was given access to Amazon’s employee monitoring software, which tracks workers’ periods of inactivity. “They could tell how much time you didn’t touch something on the screen or you moved your mouse,” he said.

Reviewers have also worked on footage from Amazon Go grocery stores, which use computer vision to detect what a customer has purchased, billing their card automatically without the need to scan items through a checkout. 

Amazon Go’s marketing trades heavily on the futuristic novelty of an unstaffed store. But in reality, the role of shop assistant has simply been outsourced to video reviewers in India.

Ishan, who worked on the Amazon Go team, said he got regular headaches from the work and was given just four minutes per day to use the bathroom outside of his scheduled breaks. “Some who are lucky will survive, and the rest will have to leave,” he said. “An employee is a replaceable asset for them.”

 “[It was] very minimal pay for the continuous high-level stress of a never-ending workload.”

Amazon said that employees are free to use the bathroom as needed and the accounts given to the Bureau do not represent the vast majority of those working on the team.

The workers being filmed by Amazon’s cameras know little about the people watching them from the other side of the world. Of the nine stow workers interviewed, just one said they were aware footage from their station could be sent to other countries for manual review. 

And for their part, some workers in India and Costa Rica said they were unsure exactly how Amazon used the fruits of their labor. “We had no idea where this particular data was going,” one worker said. “We were never given such knowledge [of] what exactly is happening in the backend.”

In California, a new law aims to address the conditions endured by Amazon workers. Assembly Bill 701, which came into effect at the start of this year, prohibits performance targets from being set at a level that could pose a safety risk or prevent sufficient bathroom or meal breaks. It also gives warehouse workers the right to request three months’ worth of their own productivity data.

Lorena Gonzalez wrote the bill when she was a state representative and now heads the California Labor Federation. She was concerned about productivity quotas causing injuries in the workplace and believed that giving workers access to their own data would be the first step in resisting management by algorithm.

“We wanted to make sure that they had the right to that information, especially if they felt like it violated their basic labor rights to health and safety,” Gonzalez said. 

Amari, the stower in California, said the law had made a tangible difference in the warehouse and managers were no longer regularly reprimanding people for falling behind on their productivity rates — although workers could still be disciplined for other violations.

The work of a stower in California is very different from that of a video reviewer in India, but both are vital cogs in Amazon’s optimization machine. As they work, they are constantly generating new data points to refine the algorithmic tools that monitor and discipline them.

“The only way to fight back against [algorithmic management] is to refuse to go quicker,” Gonzalez said. “But an individual can’t do that alone.”

“Until workers come together and organize as a collective against the kind of speed-up that happens with computer management, then they’re just going to … require people to go faster and faster.”