Workers AI 推理服务实测：在 CDN 边缘运行大语言模型
Workers AI Inference Service Review: Running LLMs at the CDN Edge
| iDev Tech | 2026-07-15T18:36:16
Cloudflare Workers AI 允许开发者直接在 CDN 边缘运行机器学习推理，支持 Llama、Mistral 等开源大模型。本文实测推理延迟、吞吐量和准确性，分析其与集中式 GPU 推理的适用场景差异。
Cloudflare Workers AI enables developers to run ML inference directly at CDN edge nodes, supporting open-source LLMs like Llama and Mistral. This article benchmarks inference latency, throughput, and accuracy, comparing use cases with centralized GPU inference.
AI 推理走向边缘Cloudflare Workers AI 是一项运行在 Cloudflare 全球网络上的 AI 推理服务。开发者无需管理 GPU 基础设施，只需通过 API 调用即可在离用户最近的边缘节点执行机器学习推理。目前支持超过30个开源模型。支持的模型类别文本生成：Llama 3.1 70B、Mistral 7B、Gemma 2 等文本嵌入：bge-base、all-MiniLM 等向量化模型图像分类：ResNet-50、EfficientNet 等语音转文字：Whisper large-v3文生图：Stable Diffusion XL Turbo性能实测数据我们在亚太地区测试了三个核心场景。文本生成（Llama 3.1 8B，256 token 输出）：首 token 延迟约180毫秒，生成速度约45 token/s。文本嵌入（bge-base，512 token 输入）：单次请求平均28毫秒。图像分类（ResNet-50）：单张图片约35毫秒。与集中式 GPU 推理（如 AWS Inferentia）相比，Workers AI 的延迟更低但吞吐量受限。适用场景分析Workers AI 最适合对延迟敏感、单次推理量不大的场景，如：实时内容审核、个性化推荐打分、搜索查询理解。不适合批量推理、模型微调和需要高吞吐的训练类任务。定价按推理次数计费，小规模使用极具性价比。
AI Inference Goes to the EdgeCloudflare Workers AI is an inference service running on Cloudflare's global network. Developers can execute ML inference at the nearest edge node via API calls without managing GPU infrastructure. It currently supports over 30 open-source models.Supported Model CategoriesText Generation: Llama 3.1 70B, Mistral 7B, Gemma 2, and moreText Embeddings: bge-base, all-MiniLM, and other vectorization modelsImage Classification: ResNet-50, EfficientNet, and othersSpeech-to-Text: Whisper large-v3Text-to-Image: Stable Diffusion XL TurboPerformance Benchmark DataWe tested three core scenarios in the Asia-Pacific region. Text generation (Llama 3.1 8B, 256 token output): time to first token ~180ms, generation speed ~45 tokens/s. Text embeddings (bge-base, 512 token input): single request averaging 28ms. Image classification (ResNet-50): single image ~35ms. Compared to centralized GPU inference (e.g., AWS Inferentia), Workers AI has lower latency but limited throughput.Use Case AnalysisWorkers AI is best suited for latency-sensitive, low-per-request-volume scenarios such as real-time content moderation, personalized recommendation scoring, and search query understanding. It is not suitable for batch inference, model fine-tuning, or high-throughput training tasks. Pricing is per-inference, making it extremely cost-effective at small scale.