gr.AI
Question explained · gr.AI

The mistakes that make you invisible to AI engines

The expensive mistakes are not strategic, they are mechanical: one line in a file, a page that no longer exists, a name written five different ways. None of them shows up in your site's admin panel and almost none appears in ordinary SEO reports, which is why they sit there for years.

The ones below come from real audits, ordered by how much damage they do. We describe them without names: some of these companies are only prospects, and we name publicly only the successes. The mechanism stays verifiable anyway, on your own site.

  1. 01

    You are blocking AI crawlers without knowing it

    One line in robots.txt, put there by someone two years ago or inherited from the theme template, telling AI engines to stay out. It is not a setting you see in the site's admin panel, so it can sit there for years.

    How it feels
    You appear nowhere in AI answers, yet you do reasonably well in Google. That is the signature: Googlebot and GPTBot are different bots with different rules.
    How to check
    Open yoursite.com/robots.txt and look for GPTBot, ClaudeBot, PerplexityBot and Google-Extended. Any Disallow: / against their names is the cause.

    From the field
    A charity collecting donations was blocking Google-Extended. With no access to the real site, Gemini was sending donors to a similar-looking domain that did not exist.

  2. 02

    You let them past robots.txt and stop them at the firewall

    The version that is completely invisible. robots.txt explicitly allows them, but the protection layer in front of the site returns an error, because it treats them as suspicious automated traffic.

    How it feels
    None. From a browser the site works perfectly, robots.txt looks correct, and you have no way of knowing that half the engines are getting an error.
    How to check
    Request the page with the bot's user agent, on an inner page. The home page is usually cached and answers anyone.

    From the field
    A shop with over twenty thousand products nominally allowed every AI crawler, but the firewall returned an error on any uncached page. For two engines out of four, the entire catalogue did not exist. The other two got through, which makes the problem even harder to suspect.

  3. 03

    You have structured data, but it is empty or wrong

    The structured data block exists, the platform generates it automatically, but the fields that matter are empty. Or worse, they are filled in wrongly everywhere at once.

    How it feels
    The validators say you have schema and the test passes. The engine reads an object that says nothing useful.
    How to check
    Do not check whether the block exists, read what is in it: does it have a price, availability, a real address, a real phone number?

    From the field
    A shop had a product block on all 358 of its pages, with no price and no availability in any of them. In another case the same wrong address was declared on every page of the site, and the engines learned the wrong one perfectly.

  4. 04

    Your company has five names

    The site says one thing, the Google listing another, the trade register a third version, old directories two more. Each of them is almost right.

    How it feels
    You show up sometimes, with mixed-up details: the name from the site and the phone number from a directory that shut down three years ago.
    How to check
    Search for your company and write down every version of the name, address and phone number you find. If the list runs to more than one line, you have found the problem.

    From the field
    A car repair shop appeared under five different names: on its site, on its Google listing, in a workshop network, in the trade register and in an old directory. No engine could merge them into a single business.

  5. 05

    The About page returns 404

    The link is in the menu or the footer, the page is gone. It is exactly the page an engine goes to in order to learn who you are, how long you have existed and who you represent.

    How it feels
    Engines describe you vaguely or confuse you with someone of a similar name, because they have nowhere to take your description from.
    How to check
    Click every link in the menu and the footer. It takes two minutes and you usually find more than one.

    From the field
    Two different sites in the portfolio had the footer link and a deleted page. On one of them, three of the twelve tiles on the home page also led to a 404.

  6. 06

    The demo content stayed in production

    The products, articles or pages the theme shipped with, still published after launch. They are indexable like anything else, so they get learned like anything else.

    How it feels
    Engines attribute to you products or services you have never sold.
    How to check
    Search your site for products or articles you do not recognise. Sort the catalogue by date added and look at the oldest entries.

    From the field
    A chain selling cooked Romanian food still had the theme's demo products in its catalogue, sushi among them. Asked what the company sells, an AI assistant had no way to separate the real menu from the scenery left over from installation.

The check that catches the second mistake

The first one is visible by reading robots.txt. The second is invisible from a browser, because your browser is not a bot. Request the page exactly the way the engine does, on an inner page:

curl -A "GPTBot" -I https://site-ul-tau.ro/o-pagina-interioara
curl -A "ClaudeBot" -I https://site-ul-tau.ro/o-pagina-interioara
curl -A "PerplexityBot" -I https://site-ul-tau.ro/o-pagina-interioara

You want 200 from all of them. If even one returns something else while the page opens fine in your browser, you have found the block that no SEO tool reports.

The order worth fixing them in

Not in the order you find them, but in the order they block the rest of the work. Access first: as long as the crawler cannot get in, nothing you write afterwards matters. Then identity, who you are, where you are, what you are called, the same everywhere. Only then content.

  1. 1Access. robots.txt and the firewall. Without this, the rest is work nobody reads.
  2. 2Identity. one name, one address, one phone number, declared as structured data.
  3. 3Cleanup. 404s in the menu, demo content, copy left over from another year.
  4. 4Content. the pages that answer the customer's questions, written so they can be quoted.

Further reading