gr.AI
Question explained · gr.AI

Does the Cloudflare change on 15 September affect me?

by Cătălin Popa · updated 2026-09-10

Most likely not. And if you read otherwise in the press, let us sort out where the confusion comes from, because it can be checked in ten seconds.

Diagram: three requests approach a barrier. The one marked Search, in cyan, passes through it and continues towards a three-row record. The ones marked Agent and Training, in red, are stopped by the barrier and each ends in an X. The record shows the default setting for each category.
The three categories and the default for each. Search passes, Agent and Training are stopped on pages that carry ads. The trap is not visible in the diagram: crawlers that do both fall together with Training.

What changes on 15 September

Cloudflare splits automated traffic into three categories and gives each a different default. The blocking applies to pages that display ads, the logic being that ads mark a page built for humans.

  • Search

    Stays allowed

    Crawlers that index the page so they can answer questions about it later. In exchange you expect traffic back.

  • Agent

    Blocked on pages with ads

    Real-time automated activity on behalf of a person. The bot that opens the page because someone just asked something, or an agent browsing.

  • Training

    Blocked on pages with ads

    Crawlers that take the content to train or fine-tune a model.

Who it actually applies to

This is where what was written differs from what the source says. We read two Cloudflare documents, the blog post and the developer changelog. They say the same thing.

The new defaults apply to domains that come onto Cloudflare from 15 September 2026 onwards. Sites already on the platform are not changed automatically, whatever the plan. Anyone who wants the new settings can adopt them voluntarily, at any time.

The version circulating in the press was that the change also catches existing customers on the free plan. We did not find that claim supported in any Cloudflare document. If you have a site that works today, on 15 September it will work the same.

The trap, which concerns everyone

The part that can actually do damage has nothing to do with 15 September and nothing to do with new domains.

Some crawlers do two things at once. They index for search and collect for training, with the same bot. Googlebot, Applebot and Bingbot are exactly in that position.

Cloudflare writes explicitly that these multi-purpose crawlers will be blocked for customers who choose to block the Training category.

Translated: someone who ticks “I do not want my site used to train models”, convinced they are stopping only training, is cutting off their Google indexing at the same time. It is not a platform bug, it is the consequence of one bot serving both purposes. But it is a consequence you find out about afterwards, if nobody tells you first.

Cloudflare leaves a valve open: in the security settings you can explicitly confirm that you want no change for crawlers that also index. Worth ticking before, not after.

The robots.txt signals, and what they cannot do

Separately from the dashboard settings, Cloudflare is pushing a format for declaring your preferences directly in robots.txt. Three signals, each with two possible values:

  • search

    building a search index and displaying results, meaning links and short snippets

  • ai-input

    feeding the content into a model, for answers generated in real time

  • ai-train

    training or fine-tuning a model

They go on one line, separated by commas. A site that wants to be indexed and cited, but not used for training, writes exactly this:

User-Agent: *
Content-Signal: search=yes, ai-train=no
Allow: /

A signal you do not write means neither yes nor no. The policy text says plainly that the absence of a signal neither grants nor restricts permission for that purpose.

And the part Cloudflare writes itself, to its credit: the signals express preferences, they are not technical measures. Some companies can simply ignore them.

They are worth writing, because a publicly declared refusal carries weight if you ever end up in a conversation about how your content is used. But whoever adds them believing they have closed a door has declared an intention, not locked anything. The same distinction as with llms.txt, about which we have written that it blocks nothing and permits nothing.

How to check in ten seconds whether this concerns you

Cloudflare leaves its trace in the server response. If you see the cf-ray header, you go through Cloudflare. If not, this discussion does not concern you at all.

curl -sI https://site-ul-tau.ro | grep -iE "^server:|^cf-ray:"

We ran the command on our own site and on the three client sites we write about in the other pages. None of them is on Cloudflare, all of them answer with Server: Vercel and none of them has cf-ray. So the change touches none of them, and that is all the result means: a check that finishes quickly, not a recommendation against Cloudflare.

If cf-ray does appear, there is one thing left to do: go into the security settings and see whether you have ticked the block on the Training category. If you have, check that you really want to lose Googlebot too.

The money part, briefly

In the same move, Cloudflare replaced the programme that paid publishers for crawler access with one that pays them when the content is actually used in an AI answer. It launched with two partners, Ceramic.ai and You.com.

It is a change of principle worth watching, but for an ordinary company site it means nothing practical today. Two partners are not a market. We will come back to it when there are numbers, not now.

Where the data comes from

  • The settings and the date. Cloudflare's post about AI traffic options and the developer changelog, both of which state that the new defaults apply to new domains.
  • The robots.txt signals. The Content Signals Policy text, the source both of the signal names and of the note that they are not technical measures.
  • The check. Run on our own site and on the three client sites we write about in the other pages, with the command above, on the date at the top of this page. You can repeat it any time.

Share this article

Further reading