A closer look at the AI Gateway
Cloudflare Inc. announced the launch of a new identity-aware tool for managing artificial intelligence requests on Tuesday. This service adds a verified identity to each AI prompt, making it easier to determine who sent it. IT and security teams benefit by being able to monitor employee and automated system behavior more clearly. They can also set spending limits and automatically flag potentially risky content.
The service is designed to integrate smoothly with existing infrastructure. It does not require the installation of additional software or systems. Instead, it enhances current tools to track who is using AI models. As AI adoption grows, companies often face unexpected spikes in costs. Employees and automated bots tend to generate more requests, which can lead to sudden bill increases. The Identity-Aware AI Gateway helps prevent such surprises by providing detailed visibility into AI usage patterns.
How it solves AI spending issues
Cloudflare introduced a complementary feature named AI Spend to address cost challenges. This system learns what normal AI usage looks like for each individual or automated system within a network. If the usage patterns deviate, the system sends alerts to IT teams. For example, if an employee begins submitting significantly more requests than usual, the system detects and reports it. This helps businesses avoid unexpected financial surprises.
Enterprise AI usage is increasing rapidly, but current tools struggle to keep up. While they can track expenses at the account level, they often lack the ability to identify specific individuals responsible for costs. Additionally, the content of the requests is not always visible in logs, leaving IT teams to address issues after the fact. These limitations can slow operations, as overly complex security tools create friction for employees using AI systems.
Security and identity in action
The AI Gateway works in conjunction with Cloudflare Access, the company's zero-trust security product. This integration allows businesses to replace shared application programming interface (API) keys, which are difficult to trace. Every AI request is verified using identity providers and zero-trust network access frameworks before it reaches the model provider.
Before any AI request is processed, the system confirms the identity and device status of the user. This verified identity is then logged, making tracking more straightforward and secure. The system also acts as a filter, removing sensitive data such as employee names and passwords before they leave the company network. This helps prevent accidental leaks to external AI providers.
Max Baumgarten, a security engineer at Flexport Inc., described shared API keys as making it nearly impossible to identify who is using AI services. After implementing Cloudflare Access with AI Gateway, Flexport can now apply its pre-existing identity policies directly at the gateway. This change gives the company greater control over which individuals or systems can access AI tools.
The Identity-Aware AI Gateway has roots in a system introduced in September 2023. However, the new security controls represent a more recent enhancement. Cloudflare began integrating these features into its zero-trust platform back in August 2023. These additions aim to help companies manage AI use more effectively, balancing user freedom with necessary oversight from IT teams.
The system's benefits include caching repeated requests to reduce costs, implementing rate limits to prevent excessive usage, and filtering out sensitive data before it leaves the network. Additionally, AI Spend helps identify cases where more complex, expensive models are used unnecessarily for simple tasks. These insights allow companies to optimize their AI usage efficiently.
Chief Technology Officer Dane Knecht emphasized the importance of streamlining security to support AI adoption. He explained that companies are now prioritizing tighter AI access controls due to concerns about sudden bill increases and accidental data leaks. By linking access controls directly to the AI Gateway, Cloudflare helps teams use any AI model they choose without compromising visibility or control for IT departments.
Cloudflare's approach is designed to empower users while maintaining the necessary oversight for security and cost management. The company’s integration of identity verification and zero-trust principles into AI workflows aims to resolve the key challenges enterprises face with AI adoption today.