Fluid compute (updated)

Fluid compute is Vercel's hybrid execution model that reduces cold starts by keeping high-performance “server-like” instances warm while preserving serverless scaling and billing. Key facts from Vercel's docs:

• Fluid compute is enabled by default for new projects (since April 23, 2025).
• It supports optimized concurrency (multiple invocations per instance), background post-response work via waitUntil, automatic bytecode caching, and automatic pre-warming to reduce cold-start tail latencies. (Vercel Docs, last updated Jan 29, 2026)
• Fluid compute is designed to improve performance for I/O-bound and AI/streaming workloads while keeping billing tied to actual compute usage.

Operational guidance:

• Prefer Fluid compute for APIs, streaming endpoints, and AI inference paths where cold-start tail latency matters.
• Keep module initialization small; Fluid compute reduces container provisioning but does not eliminate expensive module init work.
• Use waitUntil for post-response background work when you want the request to return quickly but still complete logging, telemetry, or non-critical work.

(References: Vercel Docs — Fluid compute, vercel.com/docs/fluid-compute, Jan 29, 2026)

Cold start anatomy and mitigation (refined)

Phases and mitigations:

• Container provision: mitigated by Fluid compute and pre-warming
• Runtime init: prefer Edge runtimes where feasible (V8 isolates) for minimal runtime init
• Code/module init: minimize heavy top-level work; lazy-load DB clients and SDKs
• Handler execution: keep hot paths efficient; external calls determine much of observed latency

Additional mitigations:

• Acknowledge webhooks quickly and offload work to Vercel Queues
• Use Edge functions for auth/geo-routing checks to avoid a full container path
• Monitor P95/P99 cold-start telemetry and focus optimization on the highest-impact functions

Function types and durations (clarified)

• Default maxDuration is enforced; configure per-function via export const maxDuration = N for Node.js and supported runtimes. (Vercel Docs — Configuring Maximum Duration, updated Feb 27, 2026)
• Typical defaults: Hobby 300s; Pro/Enterprise can configure higher limits depending on plan and deployment. Always verify plan limits in the dashboard and docs.
• Streaming and background work still count toward wall-clock duration; use waitUntil for short post-response work when appropriate.

(References: Vercel Docs — Configuring Maximum Duration, Feb 27, 2026)

Workflow

Step 1 — Create an API route

(Example unchanged; use standard Next.js app router API routes and keep handlers small and focused.)

Step 2 — Handle webhooks with verification and idempotency

• Verify signatures, persist an idempotency key to your datastore and use it to deduplicate retries.
• If webhook latency requirements are strict, return 200 quickly and enqueue heavy work to Vercel Queues.

Step 3 — Configure cron jobs

• Configure cron entries using vercel.json or the dashboard cron settings. Protect cron endpoints with a secret header and rotate it periodically.

Step 4 — Streaming responses for AI

• Streaming endpoints are supported but count toward maxDuration. If you need longer-lived streaming, assess Workflow or a dedicated service.
• For AI streaming, prefer Fluid compute for improved tail latencies on cold starts; still limit module init costs.

Optimize cold starts (practical patterns)

• Lazy-load heavy dependencies only when needed (Stripe, DB SDKs, ML models).
• Use shared/global variables for pooled clients (DB, HTTP client) — they persist across warm invocations but must not hold request-scoped state.
• For latency-sensitive endpoints, prefer Edge runtime for the authentication/validation path.
• For webhooks, respond immediately then enqueue to Queues; for critical synchronous work, ensure your maxDuration is sufficient and allocate extra time.

Vercel Queues (updated)

Vercel Queues is a durable, append-only event streaming system for asynchronous work. Key updates:

• Queues are available in Beta on all plans and support topic-based fan-out with independent consumer groups (Vercel Docs, Mar 3, 2026).
• Message retention is configurable; recent updates allow a maximum TTL up to 7 days. Defaults are shorter — check the Queues docs for current defaults and pricing considerations. (Vercel Changelog / Docs)
• Delivery is at-least-once; consumers must implement idempotency. Queues support push and poll consumption modes, automatic retries, visibility timeouts, and observability dashboards.
• Use Queues to absorb spikes, schedule delayed work (within retention limits), and decouple request handling from heavy processing.
• Workflow is a higher-level abstraction built on Queues for multi-step stateful processes — prefer Workflow for complex orchestrations.

(Reference: Vercel Docs — Queues, Mar 3, 2026; Vercel Changelog — Queues 7-day TTL)

Examples

• Keep webhook handlers idempotent and enqueue heavy work to Queues with a deduplication key.
• Use Promise.allSettled for fan-out notification patterns and surface partial failures for retry logic.

DDoS and edge protection (updated)

• For typical Vercel-hosted APIs, combine CDN-level protections (rate-limiting, bot management, WAF) with application-level defenses (idempotency, per-endpoint quotas, Queues to absorb spikes).
• Cloudflare’s recent edge improvements (Programmable Flow Protection and Gen 13 servers) give network-level options for dropping malicious traffic earlier and delivering higher edge compute throughput. If you place Cloudflare or another provider in front of Vercel, consider edge-level filtering to reduce origin load. (Cloudflare Blog — Programmable Flow Protection, Mar 31, 2026; Gen 13 servers, Mar 23, 2026)

Decision tree

What kind of compute do you need? ├── Request-response API → Vercel Serverless Function ├── Low-latency, global → Vercel Edge Function ├── Scheduled task → Vercel Cron + Serverless Function ├── Background job > function maxDuration → Vercel Queues or Workflow ├── Real-time bidirectional → WebSocket service (not serverless) └── Heavy compute (ML, video) → Dedicated instance or GPU

Cold start a problem? ├── P99 latency > 1s → Profile: module init vs container init │ ├── Module init → Lazy-load heavy deps, reduce bundle │ └── Container init → Use Edge runtime or Fluid compute ├── Webhook must respond < 3s → Acknowledge immediately, process async via Queues └── Cold start acceptable → No action needed

Edge cases and gotchas (condensed)

• Function timeout is wall-clock, not CPU time: waiting on I/O counts toward maxDuration.
• Default maxDuration is 300s for many plans; Pro/Enterprise projects can configure higher per the docs. Always confirm plan limits in the dashboard.
• Fluid compute reduces but does not eliminate cold starts; module init remains important.
• Global variables persist across warm invocations — fine for pooled clients, not for request-scoped state.
• Streaming/long-lived responses consume duration; use waitUntil for short post-response work where appropriate.
• Webhook retries cause duplicates — implement idempotency.

Evaluation criteria

• Cold start P95: < 500 ms (target; measure per-service)
• Warm invocation P95: < 100 ms (excluding external calls)
• Error rate: < 0.1% for API routes
• Timeout rate: < 0.01% (functions completing within limit)
• Bundle size: < 5 MB per function where possible
• Idempotency coverage: 100% for webhook and payment handlers