> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chatfeatured.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloudflare

Track AI bot activity on your website using Cloudflare Workers

**Status:** Available | **Complexity:** Advanced | **Setup time:** 15-20 minutes | **Cost:** Free tier available, \$5/month recommended

The Cloudflare integration enables server-side tracking of AI crawler activity on your website using Cloudflare Workers. This allows you to monitor when AI bots like GPTBot, ClaudeBot, and PerplexityBot access your content.

## Why Use Cloudflare Workers?

Traditional analytics tools rely on JavaScript that runs in the browser. AI crawlers don't execute JavaScript, so they're invisible to tools like Google Analytics.

Cloudflare Workers run at the edge (server-side), capturing **all** traffic including AI scrapers that traditional analytics miss. This gives you complete visibility into how AI platforms discover and index your content.

### Benefits

* Capture AI crawler traffic that JavaScript-based analytics miss
* Zero latency impact - Workers run asynchronously without blocking requests
* Edge-level tracking - Logs collected at Cloudflare's edge before reaching your server
* Simple setup - Deploy in minutes with no code changes to your site

***

## Prerequisites

Before you begin, you'll need:

* A Cloudflare account (free tier works)
* Your domain configured with Cloudflare (using Cloudflare DNS)
* A ChatFeatured account with Agent Analytics enabled

<Warning>
  While the free tier of Cloudflare Workers is compatible, we **highly recommend the paid plan (\$5/month)** for reliable analytics tracking. The free tier has request limits that may be insufficient for production sites. Review the [Cloudflare Workers pricing page](https://www.cloudflare.com/plans/developer-platform/) for details.
</Warning>

***

## Setup Instructions

### Step 1: Get Your API Key

1. Log in to ChatFeatured and navigate to your brand
2. Go to **Agent Analytics** > **Settings**
3. Click **Create API Key** and give it a name (e.g., "Cloudflare Worker")
4. Copy the generated API key - you'll need it in Step 4

<Tip>
  Create a dedicated API key for each integration. This makes it easier to revoke access if needed.
</Tip>

### Step 2: Create the Worker in Cloudflare

1. Log in to your [Cloudflare Dashboard](https://dash.cloudflare.com)
2. Click **Workers & Pages** in the sidebar
3. Click **Create Application**
4. Select **Start with Hello World!**
5. Click **Deploy**
6. At the top right, click **Edit Code**
7. Clear the contents of the editor
8. Copy and paste the worker code from Step 3
9. Click **Deploy**

### Step 3: Worker Code

Copy this code and paste it into the Cloudflare Worker editor:

```javascript theme={null}
/**
 * Cloudflare Worker – stream HTML page views to ChatFeatured
 *
 * Excludes common static-asset extensions.
 * It runs as a middleware and doesn't interfere with the actual request handling.
 */

const IGNORE_EXT =
  /\.(?:png|jpe?g|gif|svg|webp|ico|mp4|webm|mp3|wav|ogg|css|js|mjs|json|map|txt|xml|pdf|docx?|xlsx?|pptx?|zip|rar|7z|tar|gz|woff2?|ttf|eot)$/i;

export default {
  async fetch(request, env, ctx) {
    // Get the original response
    const response = await fetch(request);

    // Clone the response for processing
    const responseClone = response.clone();

    // NOTE: Does NOT block / wait
    ctx.waitUntil(handleRequest(request, responseClone, env));
    return response;
  },
};

async function handleRequest(request, response, env) {
  const requestUrl = new URL(request.url);

  // Skip obvious static assets early
  if (IGNORE_EXT.test(requestUrl.pathname)) {
    return;
  }

  // Only log HTML pages (and OPTIONAL: status < 500)
  const contentType = response.headers.get("content-type") || "";
  if (!contentType.startsWith("text/html") || response.status >= 500) {
    return;
  }

  const logData = {
    timestamp: new Date().toISOString(),
    host: requestUrl.hostname,
    method: request.method,
    pathname: requestUrl.pathname,
    query_params: requestUrl.search.startsWith("?")
      ? requestUrl.search.slice(1)
      : "",
    ip: request.headers.get("cf-connecting-ip") || "",
    userAgent: request.headers.get("user-agent") || "",
    referer: request.headers.get("referer") || "",
    status: response.status,
  };

  await fetch(env.ANALYTICS_ENDPOINT, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": env.ANALYTICS_KEY,
    },
    body: JSON.stringify(logData),
  }).catch((error) => console.error("Failed to send logs:", error));
}
```

### Step 4: Configure Environment Variables

After deploying, configure the required environment variables:

1. In your Worker's settings, click **Variables and Secrets**
2. Add the following variables:

| Variable Name        | Value                                     |
| -------------------- | ----------------------------------------- |
| `ANALYTICS_ENDPOINT` | `https://ingest.chatfeatured.com/v1/logs` |
| `ANALYTICS_KEY`      | Your API key from Step 1                  |

3. Click **Deploy** to save the changes

<Warning>
  Treat `ANALYTICS_KEY` like a password. Never share it or hardcode it in version control.
</Warning>

### Step 5: Set Up Worker Routes

Connect the Worker to your domain:

1. Navigate to your domain in Cloudflare
2. Click **Worker Routes** in the sidebar
3. Click **Add Route**
4. Enter your route pattern: `*yourdomain.com/*`
5. Select your worker from the dropdown
6. Click **Save**

<Info>
  Replace `yourdomain.com` with your actual domain. The `*` wildcards ensure all subdomains and paths are tracked.
</Info>

### Step 6: Verify Setup

Once deployed, AI crawler traffic will appear in your Agent Analytics dashboard within a few minutes. Visit your site in a browser to verify the Worker is running, then check your dashboard for incoming data.

***

## How It Works

The Cloudflare Worker runs on every request to your site:

1. **Request arrives** at Cloudflare's edge
2. **Worker executes** and captures request metadata (URL, user agent, IP, etc.)
3. **Original request continues** to your origin server without delay
4. **Logs are sent** asynchronously to ChatFeatured's ingestion endpoint
5. **AI crawlers identified** and displayed in your analytics dashboard

The Worker only logs HTML page requests. Static assets (images, CSS, JS, fonts) are automatically excluded to reduce noise.

## What Gets Tracked

The Worker captures the following data for each request:

* **Host** - The domain being accessed
* **Path** - The URL path being requested
* **User Agent** - Used to identify AI crawlers
* **IP Address** - For geographic insights
* **Referer** - Where the request came from
* **Status Code** - The HTTP response status
* **Timestamp** - When the request occurred

ChatFeatured's backend automatically identifies AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Bingbot, etc.) and filters out regular user traffic.

***

## Advanced Configuration

### Excluding Routes

To exclude specific routes from tracking (like admin pages):

1. Go to **Worker Routes** for your domain
2. Click **Add Route**
3. Enter the pattern to exclude (e.g., `*yourdomain.com/admin/*`)
4. Set Worker to **None**
5. Click **Save**

Routes are evaluated in order, so add exclusion routes after your main tracking route.

### Multi-Domain Setup

If you manage multiple domains, you can:

1. Create separate Workers for each domain
2. Or use a single Worker with conditional routing based on hostname
3. Point each domain's Worker Routes to the appropriate Worker

### Performance Optimization

To reduce log volume and costs:

1. **Add sampling** - Only log a percentage of requests in the Worker code:

```javascript theme={null}
// Log only 50% of requests
if (Math.random() > 0.5) return;
```

2. **Filter user agents** - Skip logging for known bots:

```javascript theme={null}
const userAgent = request.headers.get("user-agent") || "";
if (userAgent.includes("Slack-ImgProxy")) return;
```

3. **Exclude paths** - Skip logging for specific path patterns:

```javascript theme={null}
if (requestUrl.pathname.startsWith("/api/")) return;
```

### Custom User Identification

To track which users interact with your content, modify the Worker to capture user identifiers from cookies or headers:

```javascript theme={null}
const logData = {
  // ... existing fields
  userId: request.headers.get("x-user-id") || "",
  sessionId: getCookieValue(request, "session_id") || "",
};
```

***

## Pricing & Cost Estimation

Cloudflare Workers pricing:

| Tier | Requests         | Cost      |
| ---- | ---------------- | --------- |
| Free | 100,000/day      | \$0       |
| Paid | 10 million/month | \$5/month |

**Estimated costs for different traffic volumes:**

| Monthly Requests | Estimated Cost  |
| ---------------- | --------------- |
| 1M               | \$0 (free tier) |
| 10M              | \$5/month       |
| 100M             | \$50/month      |
| 1B               | \$500/month     |

<Tip>
  Start with the free tier and upgrade when you need more capacity. Monitor your usage in the Cloudflare dashboard.
</Tip>

***

## Troubleshooting

<Accordion title="No data appearing in dashboard">
  1. **Verify the Worker is deployed** - Check Workers & Pages in Cloudflare
  2. **Check environment variables** - Ensure both `ANALYTICS_ENDPOINT` and `ANALYTICS_KEY` are set correctly
  3. **Verify route configuration** - Ensure your domain's Worker Routes include your site
  4. **Test the API key** - Try creating a new key if the current one isn't working
  5. **Check Cloudflare logs** - Visit your Worker's Real-time Logs tab for errors
</Accordion>

<Accordion title="Worker errors in Cloudflare logs">
  1. Check the Worker's **Real-time Logs** in Cloudflare
  2. Verify the `ANALYTICS_ENDPOINT` URL is correct: `https://ingest.chatfeatured.com/v1/logs`
  3. Ensure your API key is valid and active
  4. Check for syntax errors in the Worker code
</Accordion>

<Accordion title="Some pages not being tracked">
  The Worker only tracks HTML pages (content-type: `text/html`). API endpoints, JSON responses, and static files are intentionally excluded.

  If you need to track other content types, modify the Worker code to remove the content-type check.
</Accordion>

<Accordion title="High request volume or approaching free tier limits">
  If you're approaching Cloudflare's free tier limits:

  1. **Consider upgrading** to the paid Workers plan (\$5/month)
  2. **Add exclusion routes** for high-traffic non-essential paths
  3. **Implement sampling** in the Worker code to reduce log volume
  4. Contact ChatFeatured support for volume optimization tips
</Accordion>

<Accordion title="API key not working (401 Unauthorized)">
  1. Verify the API key hasn't expired
  2. Generate a new API key in ChatFeatured
  3. Update the `ANALYTICS_KEY` environment variable in your Worker
  4. Click **Deploy** to activate the change
  5. Wait a few minutes for the update to propagate
</Accordion>

***

## Security Best Practices

### API Key Management

* Store API key only in Cloudflare's environment variables (encrypted at rest)
* Create a dedicated API key for Cloudflare (don't reuse keys across integrations)
* Rotate keys periodically from the ChatFeatured dashboard
* Revoke keys immediately if you suspect they've been compromised

### Data Privacy

* Only request metadata is logged (no page content or user data)
* IP addresses are used for geographic aggregation only
* All data is transmitted over HTTPS
* ChatFeatured is GDPR compliant

***

## Next Steps

Once your Cloudflare integration is set up:

1. **Monitor your dashboard** - Check Agent Analytics to see AI crawler activity
2. **Track citations** - See which pages AI models cite in their responses
3. **Optimize content** - Use insights to improve AI visibility for high-traffic pages

For more information on using Agent Analytics, see [Setting up Agent Analytics](/using-chatfeatured/agent-analytics).

***

## See Also

* [Agent Analytics with Webflow](/integrations/webflow-agent-analytics) — Setup for Webflow-hosted sites (requires O2O)
* [Vercel Integration](/integrations/vercel) — Alternative for Vercel projects
* [Agent Analytics](/using-chatfeatured/agent-analytics) — Understanding your data