> ## Documentation Index
> Fetch the complete documentation index at: https://help.kajabi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Block AI & search engine bots from scraping a Kajabi site

> Learn how to prevent AI bots and search engines from indexing or scraping your Kajabi site content.

## Permanently remove your URL from search engines

To remove content or a URL from Google Search permanently, take one of the following actions to remove the page permanently:

* Remove or update the content on your site (images, pages, directories) and make sure that your site returns a 404 (Not Found) HTTP status code. Non-HTML files (like PDFs) should be completely removed from your site. These can be indexed by bots.
* **Block access to the** content: For example, require a password.
* **Indicate that the page should not be indexed** using the `noindex` meta tag. This is less secure than the other methods. More information can be found [here](https://developers.google.com/search/docs/crawling-indexing/block-indexing).
* **Do not use robots.txt** as a blocking mechanism.

***

## Use custom code to hide your Kajabi site

This can be prevented by [using the Removals Tool to hide the URL.](https://support.google.com/webmasters/answer/9689846?hl=en\&visit_id=637254387395474418-2521364218\&rd=1)

Alternatively, if you are on an account plan that includes code editor access, like the [Pro Plan](https://kajabi.com/pricing), you have the ability to add custom code to your site to hide your Kajabi site from search engines.

<Warning>
  **Important:**
  Introducing custom code to your site can open you up to vulnerabilities and potential malware attacks. We recommend using vulnerability tools like [VirusTotal](https://www.virustotal.com/gui/home/url) or [Snyk](https://snyk.io/) to scan any custom code that you intend to use and apply certain [best practices](../../website/domains/code-based-customizations) to help reduce potential impact.

  Continue reading to learn how to add custom code to your pages on Kajabi to hide your login page from search engines.
</Warning>

***

## Add custom code to hide your Kajabi site

Prevent a page from appearing on most search engines by including a `noindex` meta tag in your page's code.

### How to include noindex meta tag in your page's code

1. Open the **Website** tab from the Dashboard.
2. Select **Modify code** from the dropdown option next to your Live site name.
3. From the **Snippets** directory, open the **global\_head.liquid** file.
4. Copy the **code snippet** below and paste between the **<head />** tags in your file.

**Code snippet:**

```liquid theme={null}
<!-- 'noindex' to hide login page from search engines === -->
{% if template == "login" %}
  <meta name="robots" content="noindex">
{% endif %}
```

5. Click **Save**:

<div className="w-full aspect-video">
  <iframe src="https://fast.wistia.net/embed/iframe/wy00ktn2vg" allow="autoplay; fullscreen" allowFullScreen className="w-full h-full" />
</div>

***

## Add custom code to the legacy template, Premier

If you are using the legacy template, Premier, you will add the same code snippet to your Premier template.

### How to add your code to your Premier template

1. Open the **Website** tab from the Dashboard.
2. Select **Modify code** from the dropdown option next to your Live site name.
3. From the **Layouts** directory, open the **theme.liquid** file.
4. Copy the **code snippet** below and paste between the **<head />** tags in your file.

**Code snippet:**

```liquid theme={null}
<!-- 'noindex' to hide login page from search engines === -->
{% if template == "login" %}
  <meta name="robots" content="noindex">
{% endif %}
```

5. Click **Save**:

<div className="w-full aspect-video">
  <iframe src="https://fast.wistia.net/embed/iframe/y7cpml6ljb" allow="autoplay; fullscreen" allowFullScreen className="w-full h-full" />
</div>

***

## Block AI bots - How Kajabi can help

Completely blocking AI bots from scraping your content can be tricky, but Kajabi can help you make it harder for them and discourage bots from scraping.

Many AI scrapers identify themselves with a unique code called a "user agent." Kajabi leverages Cloudflare to protect your sites, and can help you block scrapers with this method. Reach out to [security@kajabi.com](mailto:security@kajabi.com) and we'd be more than happy to help.

What are some web crawlers associated with AI? (this list is not exhaustive):

* User agent: anthropic-ai
* User agent: CCBot
* User agent: ChatGPT-User
* User agent: GPTBot
* User agent: OmigiliBot

It's important to consider the trade-offs between blocking access and allowing some scraping to occur. Blocking all bots can hurt your SEO since search engine crawlers are also bots.
