Matt, an American with a Ph.D. in Communications, recently embarked on a freelance career with Scale AI, working from home training AI models.
But is Matt now an esteemed AI programmer? Not quite. His job is rather mundane: he works within Scale AI's system, reviewing AI responses from a user's perspective to ensure accuracy and providing feedback. For instance, he might verify which responses from ChatGPT could potentially receive negative user feedback when training AI for Google to book flights.
Scale AI itself doesn't produce large models; instead, it collaborates with major AI producers like Alphabet, OpenAI, Meta, and others, providing "human oversight."
However, the inexpensive labor from places like Africa, India, and the Philippines is no longer sufficient to meet client demands. Consequently, Scale AI has begun scaling back its overseas operations and is now employing hundreds of thousands of workers domestically in the United States, including highly educated individuals like Matt.
Founded eight years ago, this large-scale data annotation company recently completed a $1 billion Series F funding round in May, led by Accel with participation from global giants such as Amazon, Intel, AMD, Cisco, Meta, and Tiger Global. The company projects its revenue to exceed $1 billion this year, positioning it as one of the top revenue-generating companies in the generative AI sector.
Its latest valuation has soared to $13.8 billion, marking a significant achievement among AI startups and surpassing Silicon Valley standout Hugging Face, which achieved a $4.5 billion valuation following its funding round last August. The valuation now places it closer to Elon Musk's xAI, which reached a valuation of $18 billion in its latest funding round.
A
Scale AI, the company that employs humans to do grunt work for AI, has become a critical arsenal in today's AI race.
When we talk about "large-scale model training," we envision thousands of advanced chips driving large models to analyze hundreds of billions of bytes of text. However, this is just the first step-pre-training.
But relying solely on this isn't enough to ensure systems like Anthropic's Claude, OpenAI's ChatGPT, Meta's Llama, and Google's Bard provide correct answers in a human-like style.
To achieve this, a second step is necessary: fine-tuning. This involves a significant amount of human effort, either employed internally by AI manufacturers or sourced from companies like Scale, Surge AI, Labelbox, Telus International, and others. These companies provide large teams to write ideal responses for customer chatbots, hand-holding the bots to deliver more "perfect" answers.
Companies providing data annotation services for AI models are not entirely new; the last wave that lifted such companies was in autonomous driving.
Scale AI was founded in 2016. In fact, Scale AI has deep ties with OpenAI from its inception, incubated in Y Combinator's startup racehorse program, garnering support from YC even before the program ended. At that time, YC's*was Sam Altman, who later co-founded OpenAI.
Back then, before the "Battle of Thousand Models" began, Scale AI first caught the wave of the autonomous driving technology frenzy sweeping through Silicon Valley. Achieving autonomous driving required training AI algorithms, and there were no other outsourcing companies capable of annotating the 3D images generated by radar and sensors of autonomous vehicles at that time.
Initially, Scale AI engineers spent several months developing a 3D annotation product for the autonomous delivery startup Nuro. Soon, Alphabet's Waymo, General Motors' Cruise, and even Apple became Scale AI's clients.
By late 2017, Scale AI had hired over 1,000 annotators, primarily in the Philippines. On average, these contract workers earned $1.5 per hour and worked 10 hours per week.
By 2019, OpenAI had also been established for several years, focusing primarily on developing AI large models, subsequently becoming a customer of Scale AI. However, at that time, AI large models were not a key source of revenue for Scale AI.
As the hype around autonomous driving technology gradually subsided and the market returned to rationality, Scale AI faced a crisis. By 2022, Scale AI's revenue growth had declined by 50%, disappointing investors.
However, by the end of 2022, OpenAI released ChatGPT, and Scale AI experienced a sudden resurgence.
In addition to OpenAI, Scale AI also partnered with Meta and Alphabet, Google's parent company, on large model initiatives. The company's revenue surged from $227 million in 2022 to $680 million in 2023.
Riding the crest of this wave, Scale AI set a target of 206% revenue growth for 2024 and aims to achieve profitability.
B
At this juncture, Scale AI has begun making changes. Cheap labor from overseas can only handle basic tasks, but products driven by large models in writing, programming, and specialized knowledge are booming. Scale AI needs to upgrade its workforce.
In an investor presentation, Scale stated it is building critical AI infrastructure, aiming to become an "AI data foundry," reminiscent of semiconductor companies. The company's founder has also publicly emphasized the contributions of individuals with doctoral degrees, doctors, lawyers, and others in training AI systems: "We need the best and brightest minds to contribute data."
According to Rest of World's report, Scale AI recently shut down contractor sites in Kenya, Nigeria, and Pakistan. The company is now focusing on recruiting high-skilled individuals within the US to assist in training large models.
Around 300,000 people are waiting for tasks through Scale AI's subsidiary Outlier's operational workgroups.
Scale AI's "mercenaries" in the US are not cheap, with average hourly wages reaching $40. However, the job still carries a sense of "grunt work."
Melissa Quashie from Massachusetts earns $40 per hour as a freelancer and editor at Scale AI. Her tasks include evaluating various responses generated by large models, assessing how models answer questions, and rating the quality of responses.
For Quashie, working at Scale AI feels like "the dumbest video game I've ever played." She once spent two hours writing a "three-day meal plan" just to improve answers for a chatbot.
Moreover, with Scale AI accumulating a large workforce, the supply-demand balance has started to tilt. Often, the tasks distributed by Scale AI are insufficient to meet the demand from its "mercenaries." Many find that while the job offers flexible hours and attractive pay, there are long periods with nothing to do. According to interviews by The Information with 10 Scale AI contractors, most share similar complaints.
Perhaps the company's rapid expansion in the AI wave has led to these issues. Scale AI seems more focused on pleasing clients than on the work experience of its labor force. In addition to complaints about insufficient training and frequent system crashes, workers also criticize the company for not providing enough tasks.
Another vexing issue is the compensation system. Even highly skilled individuals providing labor to Scale AI in the US have little say. One contractor, mentioned earlier, a PhD named Matt, was inexplicably kicked off the platform.
Compensation is based not on the amount of work but on quality assessments, and Scale AI retains ultimate control. Even when payment is due, it can be delayed due to clients taking time to confirm.
C
Based on quality rather than quantity for payouts helps Scale AI control costs, which is a critical factor for the company at this stage.
As Scale AI shifts its focus from offshore markets providing cheap labor to the U.S., cost control becomes more challenging. According to financial data obtained by The Information, Scale AI's gross margin (including costs paid to human laborers) dropped from 59% in 2022 to 49% in 2023.
Simultaneously, Scale AI has informed investors that it is working on reducing costs. The company forecasts a 5-percentage-point increase in gross margin this year, aiming for 60% by 2025.
Scale AI has stated it aims to cut costs by using internal tools to automate identification of "efficient experts," reducing the cost of manually training models, and relying on computer-generated data to enhance human labor efficiency.
Another cost-saving measure includes downsizing internal staff (distinct from the "mercenaries," referring to formal employees at Scale AI). In February 2023, Scale AI seized an opportunity amid the AI boom and Silicon Valley layoffs, slashing its workforce by 20%.
In addition to cost-cutting efforts, Scale AI is expanding its business.
Despite employee opposition, Scale AI has discarded its commitment to not working with governments. Recently, co-founder Alexander Wang appeared alongside a U.S. Army general in Washington, indicating the company earns over $100 million annually from government contracts. Wang also visited Qatar for closed-door meetings with officials eager to develop their own large language models.
Beyond providing abundant human resources for AI manufacturers, Scale AI offers AI-generated synthetic datasets—data generated by AI to train AI, meeting the increasing demands in AI model training.
Currently, "high-quality humans" remain Scale AI's crucial "resource," and the company is taking steps to nurture these top performers.
In Texas's Austin and Jacksonville, Florida, Scale AI hosted several-day workshops inviting dozens of "elite workers" to participate.
One attendee from the Austin workshop mentioned around 50 trainers involved in a project reportedly related to Alphabet's Bard chatbot. They discussed responses written for various prompts and even enjoyed karaoke together in the evenings.
In Jacksonville, Quasi encountered university professors, PhD students, writers, and podcast hosts. "We worked continuously for six hours and then enjoyed a glass of wine."
"Everyone is excited about improving large language models. But no one talks about who might lose their jobs because of the work we're doing."
Ironically, hundreds of thousands of humans are working for AI to enhance its performance. Yet, when AI becomes proficient enough, these workers might be among the first to be replaced. After all, if AI can become self-sufficient, why rely on 'high-intellect labor' costing $40 per hour?
