LLM Gateways How They Enable Frontier Model Abuse
LLM Gateways: How They Enable Frontier Model Abuse
Team Cymru has uncovered 10,000+ hidden gateway servers masking malicious activity originating in China that bypassed AI providers’ region bans and potentially siphoned proprietary model outputs. 🚨
Relay infrastructure, known as “transfer stations”, weakens controls, acting as a proxy by separating the apparent requestor from the party actually consuming the service. What we have uncovered is an entire ecosystem designed explicitly to break the frontier model providers’ T&Cs, enabling fraud and illicit activity. One of the most consequential forms of abuse is model distillation. Rather than independently creating the research, data, and compute required to build a frontier model, an actor can query a stronger “teacher” model at scale, collect its outputs, and use them to improve a cheaper “student” model. This allows portions of the teacher model’s capabilities to be transferred at a substantially lower cost than developing them independently, which is why major frontier-model providers prohibit using their outputs to train competing models.
Claude Relay Service (CRS 1.x) and its successor sub2api (CRS v2.0), are published on GitHub by author Wei-Shaw in Simplified Mandarin and are toolkits that have been widely adopted as proxies to access the frontier models. The latest version (sub2api) includes user management, per-user billing, subscription-to-API conversion layer, and a prompt-audit subsystem. While sub2api has more capabilities, both enable an actor to turn one set of AI accounts into a shared gateway. The transfer station holds a pool of LLM credentials to be used upstream and re-exposes them behind keys of its own. The provider sees credentials and IP addresses of the transfer station and never sees the actual source IP of the user. The sub2api codebase has been forked over eight thousand times and the Telegram channel for the project has almost seven thousand subscribers, showing the suite is mature and has high adoption. 📈
By analyzing our scanning data over an eight-day window, we identified almost eleven thousand confirmed transfer stations. When we analyzed telemetry collected from a transfer station cluster hosted on a few VPS providers in the U.S., we clearly identified Chinese IP addresses initiating connections to the transfer station cluster en masse, which subsequently connected to multiple frontier models. In its entirety, over four thousand Chinese and Hong Kong addresses connected to 304 transfer stations. Over just eight days in late August, the Chinese IP addresses transferred approximately 14 TB up to the transfer station cluster and received over 7 TB down from the transfer station cluster. The concentration and the uniform volume of traffic on a per-relay basis is more consistent with a coordinated deployment than with independent users.
We divided the AI endpoints into two clusters: Chinese AI companies and Western Frontier LLM providers. Endpoint Cluster 1 is comprised of domestic Chinese AI providers. Endpoint Cluster 2 is significantly larger and comprises multiple U.S.-based frontier models (OpenAI, Anthropic, xAI, and Google endpoints). Traffic to this cluster was very upload-heavy, with a large amount of traffic directed to api.anthropic.com. During the eight days, six of the seventeen transfer stations reached the Anthropic endpoint consistently. For every byte these transfer stations downloaded from Anthropic, they uploaded roughly 58 bytes to Anthropic. We found this extremely interesting. Because of the nature of the data we are using for this analysis, we cannot see the actual prompts to gauge if this activity is a form of distillation or other type of abuse. If the 81GB uploaded to api.anthropic.com is text-based context, the upstream carries an estimated 16-23 billion input tokens over the eight days, with 140-200 million output tokens returned. This is just seventeen transfer stations of a 10,867-host population!