Cloudflare’s new AI tools – MS Office Store

News
September 27, 2023

Cloudflare: Easily Deploy and Run Models with Cloudflare's New AI Tools

Are you looking for a more cost-effective and simpler way to deploy and run AI models? Look no further than Cloudflare’s new AI tools. Cloudflare, the cloud services provider, has launched a collection of products and apps aimed at helping customers build, deploy, and run AI models at the network edge. These new offerings, including Workers AI, Vectorize, and AI Gateway, provide a seamless and efficient solution for managing AI applications. In a market filled with complexity and high costs, Cloudflare’s AI tools offer a simplified and affordable option for software developers. Read on to learn more about how Cloudflare is revolutionizing the AI landscape.

Cloudflare, the cloud services provider, is seizing the AI trend with the introduction of a fresh array of products and apps. These innovations are tailored to assist customers in constructing, deploying, and operating AI models directly at the network edge.

Among these new offerings is Workers AI, a platform that allows customers to leverage GPUs hosted by Cloudflare partners in close proximity. This provides the flexibility of running AI models on a pay-as-you-go basis. Additionally, Vectorize is now available, offering a robust vector database to store vector embeddings – mathematical representations of data – generated by models from Workers AI. Lastly, AI Gateway is designed to furnish metrics that empower customers to efficiently manage the expenses associated with running AI applications.

Cloudflare’s CEO, Matthew Prince, shared that the introduction of this new AI-centric product suite stemmed from a clear demand among Cloudflare’s customers. They were seeking a more streamlined and user-friendly AI management solution, especially one that emphasizes cost-effectiveness.

Matthew Prince explained in an email interview, “The existing offerings in the market are still overly complex, often necessitating the integration of numerous vendors, which can quickly escalate costs. Additionally, there is currently a scarcity of insights into how AI budgets are being allocated; maintaining visibility is a significant challenge as AI spending rises. We aim to simplify all these dimensions for developers.”

In pursuit of this objective, Workers AI ensures that AI inference takes place on GPUs located in close proximity to users, ensuring a low-latency, AI-enhanced end-user experience. By harnessing ONNX, the Microsoft-backed intermediary machine learning toolkit designed for seamless conversion between diverse AI frameworks, Workers AI empowers AI models to operate where it’s most optimal in terms of factors such as bandwidth, latency, connectivity, processing capabilities, and localization requirements.

Users of Workers AI have the flexibility to select models from a curated catalog, spanning a range of options including large language models (LLMs) like Meta’s Llama 2, automatic speech recognition models, image classifiers, and sentiment analysis models. Importantly, Workers AI maintains data within the server region of its origin, ensuring data integrity and privacy. Furthermore, any data utilized for inference, such as prompts provided to an LLM or input for image generation, is not employed in the training of present or future AI models.

“Ideally, inference should occur in close proximity to the user to ensure a low-latency user experience. However, devices don’t always possess the computational capacity or battery power needed to run resource-intensive models like LLMs,” explained Prince. “Conversely, traditional centralized clouds are frequently situated far from the end user geographically. Moreover, these centralized clouds are primarily located in the U.S., creating complexity for global businesses that either choose not to or are legally prohibited from transmitting data outside their home country. Cloudflare offers the optimal solution to address both of these challenges.”

Workers AI has already formed a significant partnership with AI startup Hugging Face. In this collaboration, Hugging Face will optimize generative AI models for seamless integration with Workers AI, marking Cloudflare as the pioneering serverless GPU partner for deploying Hugging Face models.

Another notable collaboration is with Databricks, which aims to bring AI inference capabilities to Workers AI through MLflow, the open-source platform for managing machine learning workflows. Additionally, Databricks will make these capabilities accessible through its marketplace for software. To further enhance this partnership, Cloudflare will actively contribute to the MLflow project, and Databricks will extend MLflow functionalities to developers actively working on the Workers AI platform.

Vectorize serves a distinct customer base—those requiring a database to store vector embeddings essential for AI models. Vector embeddings, the fundamental components of machine learning algorithms used across applications, including search engines and AI assistants, offer a more concise representation of training data while retaining its essential characteristics.

Within Workers AI, models can generate embeddings, which can subsequently be stored in Vectorize. Alternatively, customers have the flexibility to preserve embeddings created by third-party models from providers like OpenAI and Cohere. This versatility ensures efficient management of vector embeddings for a wide range of AI applications.

Now, vector databases are hardly new. Startups like Pinecone host them, as do public cloud incumbents like AWS, Azure and Google Cloud. Prince firmly asserts that Vectorize gains a significant advantage from Cloudflare’s extensive global network. This network proximity facilitates database queries occurring closer to users, resulting in notably reduced latency and inference time.

Prince emphasized, “For developers, entering the AI arena today often entails navigating complex and often inaccessible infrastructure. We are here to simplify this experience right from the outset… We seamlessly integrate this technology into our existing network, enabling us to leverage our established infrastructure, ultimately delivering enhanced performance and cost efficiency.”

The final component of the AI suite, AI Gateway, plays a crucial role in enhancing observability for AI traffic. AI Gateway provides comprehensive insights into various aspects of AI usage, including the volume of model inferencing requests, the duration of these requests, the number of users interacting with a model, and the overall expenditure associated with running an AI application.

Furthermore, AI Gateway offers cost-reduction features, such as caching and rate limiting. Caching enables customers to store responses generated by large language models (LLMs) for frequently asked questions, thus reducing the necessity for LLMs to generate fresh responses continuously. Rate limiting provides better control over application scaling by effectively managing malicious activities and heavy traffic.

Prince claims that, through AI Gateway, Cloudflare distinguishes itself as one of the few providers of its size that enables developers and companies to exclusively pay for the compute resources they utilize. While it’s worth noting that third-party tools like GPTCache can replicate AI Gateway’s caching functionality on other providers, and services like Vercel offer rate limiting, Prince contends that Cloudflare’s approach is more streamlined compared to competitors. However, whether Cloudflare’s assertion holds true remains to be seen.

Prince elaborated, “Presently, customers incur expenses for idle compute resources in the form of virtual machines and GPUs that remain unused. We identify an opportunity to simplify and abstract much of the complexity typically associated with machine learning operations, thereby providing developers with a comprehensive solution for their machine learning workflows.”

Scenario 1: Increased Adoption of AI at the Network Edge

Cloudflare’s introduction of AI tools, especially Workers AI, has the potential to significantly impact the AI industry. In this scenario, we envision a substantial increase in the adoption of AI at the network edge. Developers and businesses seeking cost-effective and simplified AI solutions will turn to Cloudflare’s offerings. As a result, more AI applications and models will run at the network edge, closer to end-users, leading to reduced latency and improved user experiences.

Outcome: The AI industry witnesses a surge in the deployment of AI models at the network edge. Cloudflare becomes a leading provider in this space, and other cloud service providers may follow suit by offering similar solutions. This shift results in faster and more responsive AI applications, benefiting various sectors like e-commerce, IoT, and content delivery.

Scenario 2: Competition Spurs Innovation in AI Infrastructure

Cloudflare’s entry into the AI infrastructure market with a focus on simplicity and cost-efficiency could stimulate innovation and competition in the AI infrastructure sector. Other cloud service providers may respond by enhancing their AI offerings, reducing complexity, and adjusting pricing models to stay competitive. This scenario could lead to a positive cycle of innovation, ultimately benefiting developers and organizations working with AI technologies.

Outcome: Cloud service providers invest in improving their AI infrastructure services, leading to more accessible and affordable AI solutions. Developers gain access to a wider range of tools and platforms, driving innovation in AI application development. This increased competition results in better performance, lower costs, and improved support for AI workloads across various industries.

In both scenarios, Cloudflare’s AI tools are a catalyst for change in the AI industry, making AI more accessible, efficient, and cost-effective for developers and businesses.

Conclusion

In conclusion, Cloudflare’s new AI-focused product suite aims to simplify the process of building, deploying, and running AI models at the network edge. With offerings such as Workers AI, Vectorize, and AI Gateway, Cloudflare addresses the complexity and cost challenges that developers and businesses face when working with AI infrastructure. By providing physically nearby GPUs, a vector database, and observability features, Cloudflare empowers developers to access and manage AI resources more efficiently. With its streamlined approach and focus on cost savings, Cloudflare aims to make AI more accessible and user-friendly for software developers.

How do you think Cloudflare’s AI tools will impact the adoption of AI at the network edge? In your opinion, what challenges do developers and businesses currently face when working with AI infrastructure, and can Cloudflare’s streamlined approach address these challenges effectively? Share your insights below.