Cloudflare

​

Why Use Cloudflare Workers?

​

Benefits

• Capture AI crawler traffic that JavaScript-based analytics miss
• Zero latency impact - Workers run asynchronously without blocking requests
• Edge-level tracking - Logs collected at Cloudflare’s edge before reaching your server
• Simple setup - Deploy in minutes with no code changes to your site

​

Prerequisites

• A Cloudflare account (free tier works)
• Your domain configured with Cloudflare (using Cloudflare DNS)
• A ChatFeatured account with Agent Analytics enabled

​

Setup Instructions

​

Step 1: Get Your API Key

1. Log in to ChatFeatured and navigate to your brand
2. Go to Agent Analytics > Settings
3. Click Create API Key and give it a name (e.g., “Cloudflare Worker”)
4. Copy the generated API key - you’ll need it in Step 4

​

Step 2: Create the Worker in Cloudflare

1. Log in to your Cloudflare Dashboard
2. Click Workers & Pages in the sidebar
3. Click Create Application
4. Select Start with Hello World!
5. Click Deploy
6. At the top right, click Edit Code
7. Clear the contents of the editor
8. Copy and paste the worker code from Step 3
9. Click Deploy

​

Step 3: Worker Code

/**
 * Cloudflare Worker – stream HTML page views to ChatFeatured
 *
 * Excludes common static-asset extensions.
 * It runs as a middleware and doesn't interfere with the actual request handling.
 */

const IGNORE_EXT =
  /\.(?:png|jpe?g|gif|svg|webp|ico|mp4|webm|mp3|wav|ogg|css|js|mjs|json|map|txt|xml|pdf|docx?|xlsx?|pptx?|zip|rar|7z|tar|gz|woff2?|ttf|eot)$/i;

export default {
  async fetch(request, env, ctx) {
    // Get the original response
    const response = await fetch(request);

    // Clone the response for processing
    const responseClone = response.clone();

    // NOTE: Does NOT block / wait
    ctx.waitUntil(handleRequest(request, responseClone, env));
    return response;
  },
};

async function handleRequest(request, response, env) {
  const requestUrl = new URL(request.url);

  // Skip obvious static assets early
  if (IGNORE_EXT.test(requestUrl.pathname)) {
    return;
  }

  // Only log HTML pages (and OPTIONAL: status < 500)
  const contentType = response.headers.get("content-type") || "";
  if (!contentType.startsWith("text/html") || response.status >= 500) {
    return;
  }

  const logData = {
    timestamp: new Date().toISOString(),
    host: requestUrl.hostname,
    method: request.method,
    pathname: requestUrl.pathname,
    query_params: requestUrl.search.startsWith("?")
      ? requestUrl.search.slice(1)
      : "",
    ip: request.headers.get("cf-connecting-ip") || "",
    userAgent: request.headers.get("user-agent") || "",
    referer: request.headers.get("referer") || "",
    status: response.status,
  };

  await fetch(env.ANALYTICS_ENDPOINT, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": env.ANALYTICS_KEY,
    },
    body: JSON.stringify(logData),
  }).catch((error) => console.error("Failed to send logs:", error));
}

​

Step 4: Configure Environment Variables

1. In your Worker’s settings, click Variables and Secrets
2. Add the following variables:

3. Click Deploy to save the changes

​

Step 5: Set Up Worker Routes

1. Navigate to your domain in Cloudflare
2. Click Worker Routes in the sidebar
3. Click Add Route
4. Enter your route pattern: *yourdomain.com/*
5. Select your worker from the dropdown
6. Click Save

​

Step 6: Verify Setup

​

How It Works

1. Request arrives at Cloudflare’s edge
2. Worker executes and captures request metadata (URL, user agent, IP, etc.)
3. Original request continues to your origin server without delay
4. Logs are sent asynchronously to ChatFeatured’s ingestion endpoint
5. AI crawlers identified and displayed in your analytics dashboard

​

What Gets Tracked

• Host - The domain being accessed
• Path - The URL path being requested
• User Agent - Used to identify AI crawlers
• IP Address - For geographic insights
• Referer - Where the request came from
• Status Code - The HTTP response status
• Timestamp - When the request occurred

​

Advanced Configuration

​

Excluding Routes

1. Go to Worker Routes for your domain
2. Click Add Route
3. Enter the pattern to exclude (e.g., *yourdomain.com/admin/*)
4. Set Worker to None
5. Click Save

​

Multi-Domain Setup

1. Create separate Workers for each domain
2. Or use a single Worker with conditional routing based on hostname
3. Point each domain’s Worker Routes to the appropriate Worker

​

Performance Optimization

1. Add sampling - Only log a percentage of requests in the Worker code:

// Log only 50% of requests
if (Math.random() > 0.5) return;

2. Filter user agents - Skip logging for known bots:

const userAgent = request.headers.get("user-agent") || "";
if (userAgent.includes("Slack-ImgProxy")) return;

3. Exclude paths - Skip logging for specific path patterns:

if (requestUrl.pathname.startsWith("/api/")) return;

​

Custom User Identification

const logData = {
  // ... existing fields
  userId: request.headers.get("x-user-id") || "",
  sessionId: getCookieValue(request, "session_id") || "",
};

​

Pricing & Cost Estimation

​

Troubleshooting

​

Security Best Practices

​

API Key Management

• Store API key only in Cloudflare’s environment variables (encrypted at rest)
• Create a dedicated API key for Cloudflare (don’t reuse keys across integrations)
• Rotate keys periodically from the ChatFeatured dashboard
• Revoke keys immediately if you suspect they’ve been compromised

​

Data Privacy

• Only request metadata is logged (no page content or user data)
• IP addresses are used for geographic aggregation only
• All data is transmitted over HTTPS
• ChatFeatured is GDPR compliant

​

Next Steps

1. Monitor your dashboard - Check Agent Analytics to see AI crawler activity
2. Track citations - See which pages AI models cite in their responses
3. Optimize content - Use insights to improve AI visibility for high-traffic pages

​

See Also

• Agent Analytics with Webflow — Setup for Webflow-hosted sites (requires O2O)
• Vercel Integration — Alternative for Vercel projects
• Agent Analytics — Understanding your data