<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-1234567890123456" crossorigin="anonymous"></script>
<!-- Homepage Leaderboard -->
<ins class="adsbygoogle"
style="display:inline-block;width:728px;height:90px"
data-ad-client="ca-pub-1234567890123456"
data-ad-slot="1234567890"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</div>
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-1234567890123456" crossorigin="anonymous"></script>
<!-- Homepage Leaderboard -->
<ins class="adsbygoogle"
style="display:inline-block;width:728px;height:90px"
data-ad-client="ca-pub-1234567890123456"
data-ad-slot="1234567890"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</div>
Skip to main content
Help Center
Community
Announcements
Code implementation guide
Where to place ad unit code in your HTML
Next: Connect to AdSense and show ads on your WordPress site
This article describes where to add ad unit code in your HTML to serve AdSense ads.
If you are not familiar with working with HTML, you should first see the instructions on how to paste ad code in your HTML pages, depending on the tools that you use. Before proceeding, you should also know the differences between AdSense code and ad unit code.
Tip: For help with HTML, try searching for online resources using terms like "HTML basics"
On this page
Where to add ad unit code
How to position ads on your site with HTML
Where to add ad unit code
After you get and copy your ad unit code, you need to paste it between the <body> and </body> tags of your page. If you paste the ad code outside the <body> tags it will prevent your ads from appearing correctly.
Note: This is sample code only which shows where to add ad unit code. Don't use this code exactly in your own site's code.
Example of an HTML page before ad unit code has been added
<html>
<head>
This is the head of your page.
<title>Example HTML page</title>
</head>
<body>
This is the body of your page.
</body>
</html>
Example of an HTML page after ad unit code has been added in the <body> tag
<html>
<head>
This is the head of your page.
<title>Example HTML page</title>
</head>
<body>
This is the body of your page.
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-1234567890123456" crossorigin="anonymous"></script>
<!-- Homepage Leaderboard -->
<ins class="adsbygoogle"
style="display:inline-block;width:728px;height:90px"
data-ad-client="ca-pub-1234567890123456"
data-ad-slot="1234567890"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</body>
</html>
How to position ads on your site with HTML
Just like the other elements of your site (images, paragraphs, etc.), you can change the position of your Google ads by using HTML tags (<div>, <table>, etc.). For example, if you surround your ad code with the HTML tags <div align="center"> and </div>, your ads will appear in the center of your page.
Example of an HTML <div> element used to position an ad in the <body> tag
<html>
<head>
This is the head of your page.
<title>Example HTML page</title>
</head>
<body>
This is the body of your page.
<div align="center">
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-1234567890123456" crossorigin="anonymous"></script>
<!-- Homepage Leaderboard -->
<ins class="adsbygoogle"
style="display:inline-block;width:728px;height:90px"
data-ad-client="ca-pub-1234567890123456"
data-ad-slot="1234567890"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</div>
</body>
</html>
Give feedback about this article
Next: Connect to AdSense and show ads on your WordPress site
Was this helpful?
YesNo
Need more help?
Try these next steps:
Post to the help community
Get answers from community members
Contact us
Tell us more and we’ll help you get there
Code implementation guide
About the AdSense code
Get and copy the AdSense code
How to paste the ad code in your page
Where to place AdSense code in your HTML
Where to place ad unit code in your HTML
WordPress sites
Common issues with ad code implementation
Use Chrome DevTools to troubleshoot issues with your ads not showing
Integrate the AdSense ad code with a Content Security Policy (CSP)
Unlock your revenue with personalized tips today!
Visit the
Optimization tips for AdSense page to see personalized suggestions to maximize your revenue.
Unlock now
Language
Help guide
<head>
This is the head of your page.
<title>Example HTML page</title>
</head>
<body>
This is the body of your page.
</body>
# Image Sitemaps | Google Search Central
# Image sitemaps
Image sitemaps are a way of telling Google about other images on your site, especially those that
we might not otherwise find (such as images your site reaches with JavaScript code). You can
create a separate image sitemap or add image sitemap tags to your existing sitemap; either
approach is equally fine for Google.
Image sitemaps are based on generic sitemaps so the
[general sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap#general-guidelines)
also apply to image sitemaps. We also recommend that you follow the
[general best practices for publishing images](https://developers.google.com/search/docs/appearance/google-images).
## Example image sitemap
The following example shows a regular sitemap with image sitemap extension, with two
`<url>` elements:
- `https://example.com/sample1.html`, which contains two images
- `https://example.com/sample2.html`, which contains one image
```
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:image="http://www.google.com/schemas/sitemap-image/1.1">
<url>
<loc>https://example.com/sample1.html</loc>
<image:image>
<image:loc>https://example.com/image.jpg</image:loc>
</image:image>
<image:image>
<image:loc>https://example.com/photo.jpg</image:loc>
</image:image>
</url>
<url>
<loc>https://example.com/sample2.html</loc>
<image:image>
<image:loc>https://example.com/picture.jpg</image:loc>
</image:image>
</url>
</urlset>
```
## Image sitemap reference
The `image` tags are defined in the Image Sitemaps namespace:
[`http://www.google.com/schemas/sitemap-image/1.1`](http://www.google.com/schemas/sitemap-image/1.1)
To make sure Google can use your image sitemap, you must use the following required tags:
| Required tags ||
|---|---|
| `<image:image>` | Encloses all information about a single image. Each `<url>` tag can contain up to 1,000 `<image:image>` tags. |
| `<image:loc>` | The URL of the image. In some cases, the image URL may not be on the same domain as your main site. This is fine, as long as you verify both domains in Search Console. If, for example, you use a content delivery network such as Google Sites to host your images, make sure that the hosting site is verified in Search Console. In addition, make sure that your [robots.txt](https://developers.google.com/search/docs/crawling-indexing/robots/intro) file doesn't disallow the crawling of any content you want indexed. |
### Deprecated tags and attributes
We removed the following tags and attributes from our documentation:
`<image:caption>`, `<image:geo_location>`,
`<image:title>`, `<image:license>`. See the
[deprecation announcement](https://developers.google.com/search/blog/2022/05/spring-cleaning-sitemap-extensions)
for more information.
## Troubleshooting sitemaps
If you're having trouble with your sitemap, you can investigate the errors with Google Search Console.
See Search Console's
[sitemaps troubleshooting guide](https://support.google.com/webmasters/answer/7451001#errors)
for help.
## Additional resources
Want to learn more? Check out the following resources:
- [Submit your sitemap to Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap#addsitemap)
- [Learn how to combine sitemap extensions](https://developers.google.com/search/docs/crawling-indexing/sitemaps/combine-sitemap-extensions)
# How to specify a canonical URL with rel="canonical" and other methods
To specify a [canonical URL](https://developers.google.com/search/docs/crawling-indexing/canonicalization) for
duplicate or very similar pages to Google Search, you can indicate your preference using a
number of methods. These are, in order of how strongly they can influence canonicalization:
- [**Redirects**](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#redirects-method): A strong signal that the target of the redirect should become canonical.
- [**`rel="canonical"` `link` annotations**](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#rel-canonical-link-method): A strong signal that the specified URL should become canonical.
- [**Sitemap inclusion**](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#sitemap-method): A weak signal that helps the URLs that are included in a sitemap become canonical.
Keep in mind that these methods can stack and thus become more effective when combined.
This means that when you use two or more of the methods, that will increase the chance
of your preferred canonical URL appearing in search results.
While we encourage you to use these methods, none of them are required; your
site will likely do just fine without specifying a canonical preference. That's because
if you don't specify a canonical URL,
[Google will identify which version of the URL is objectively the best version to show to users in Search](https://developers.google.com/search/docs/crawling-indexing/canonicalization#canonical-how).
> [!IMPORTANT]
> **If you use a CMS, such as WordPress, Wix, or Blogger** , you might not be able to edit your HTML directly. Instead, your CMS might have a search engine settings page or some other mechanism to tell search engines about the canonical URL. Search for instructions about modifying the `<head>` of your page on your CMS (for example, search for "wordpress set the canonical element").
## Reasons to specify a canonical URL
[Video](https://www.youtube.com/watch?v=gEWkYTPSEjs)
While it's generally not critical to specify a canonical preference for your URLs, there
are a number of reasons why you would want to explicitly tell Google about a canonical
page in a set of duplicate or similar pages:
- **To specify which URL that you want people to see in search results.** You might prefer people to reach your green dress product page through `https://www.example.com/dresses/green/green-dress.html` rather than `https://example.com/dresses/cocktail?gclid=ABCD`.
- **To consolidate signals for similar or duplicate pages.** It helps search engines to be able to consolidate the signals they have for the individual URLs (such as links to them) into a single, preferred URL. This means that signals from other sites to `https://example.com/dresses/cocktail?gclid=ABCD` get consolidated with links to `https://www.example.com/dresses/green/green-dress.html` if the latter becomes canonical.
- **To simplify tracking metrics for a piece of content.** With a variety of URLs, it can be more challenging for you to get consolidated metrics for a specific piece of content.
- **To avoid spending crawling time on duplicate pages.** You may want Googlebot to get the most out of your site, so it's better for it to spend time crawling new (or updated) pages on your site, rather than crawling duplicate versions of the same content.
## Best practices
For all canonicalization methods, follow these best practices:
- **Don't** use the robots.txt file for canonicalization purposes. Google may still index URLs that are disallowed in robots.txt without their content.
- **Don't** use the URL removal tool for canonicalization. It hides *all* versions of a URL from Search.
- **Don't** specify different URLs as canonical for the same page using different canonicalization techniques (for example, don't specify one URL in a sitemap, but a different URL for that same page using `rel="canonical"`).
- **Don't** specify a URL fragment as canonical, as [Google generally doesn't support URL fragments](https://developers.google.com/search/docs/crawling-indexing/url-structure#fragments).
- **Do** include a `rel="canonical"` link on the canonical page itself (also known as a self-referential canonical).
- **We don't recommend** using [`noindex`](https://developers.google.com/search/docs/crawling-indexing/block-indexing) to prevent selection of a canonical page within a single site, because it will completely block the page from Search. `rel="canonical"` `link` annotations are the preferred solution.
- If you're using [`hreflang` elements](https://developers.google.com/search/docs/specialty/international/localized-versions), make sure to specify a canonical page in the same language, or the best possible substitute language if a canonical page doesn't exist for the same language.
- When linking within your site, link to the canonical URL rather than a duplicate URL. Linking consistently to the URL that you consider to be canonical helps Google understand your preference.
- If you're using client-side rendering with JavaScript, it's important to make sure that the information about the canonical URL is as clear as possible. The best way to do this is to specify the canonical URL in the HTML source code and make sure that JavaScript doesn't change the canonical link element. If you can't set the canonical URL in the HTML source code, leave it out and only set it with JavaScript. This ensures that the information about the canonical URL is as clear as possible.
## Comparison of canonicalization methods
The following table compares the different canonicalization methods, highlighting their
strengths and weaknesses when it comes to maintenance and efficacy in different scenarios.
| Method and description ||
|---|---|
| [`rel="canonical" link` element](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#rel-canonical-link-method) | Add a `<link>` element in the code for all duplicate pages, pointing to the canonical page. |---|---| | **Pros:** - Can map an infinite number of duplicate pages. | **Cons:** - Can be complex to maintain the mapping on larger sites, or sites where the URLs change often. - Only works for HTML pages, not for files such as PDF. In such cases, you can use the `rel="canonical"` HTTP header. | |
| [`rel="canonical"` HTTP header](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#rel-canonical-header-method) | Send a `rel="canonical"` header in your page response. |---|---| | **Pros:** - Doesn't increase page size. - Can map an infinite number of duplicate pages. | **Cons:** - Can be complex to maintain the mapping on larger sites, or sites where the URLs change often. | |
| [Sitemap](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#sitemap-method) | Specify your canonical pages in a sitemap. |---|---| | **Pros:** - Simple to implement and maintain, especially on large sites. | **Cons:** - Google must still determine the associated duplicate for any canonicals that you declare in the sitemap. - Less powerful signal to Google than the `rel="canonical"` mapping technique. | |
| [Redirects](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#redirects-method) | Use permanent redirects to tell Google that a redirected URL is a worse version than the URL it redirects to. Use this only when deprecating a duplicate page. |
| [AMP variant](https://developers.google.com/search/docs/crawling-indexing/amp) | If one of your variants is an AMP page, follow the AMP guidelines to indicate the canonical page and AMP variant. |
## Use `rel="canonical"` `link` annotations
Google supports explicit `rel` canonical `link`
annotations as described in
[RFC 6596](https://www.rfc-editor.org/rfc/rfc6596).
`rel="canonical"` annotations that suggest alternate versions of a page are ignored;
specifically, `rel="canonical"` annotations with `hreflang`,
`lang`, `media`, and `type` attributes are not used for
canonicalization. Instead, use the appropriate `link` annotations to specify alternate
versions of a page; for example, `link` `rel="alternate"`
[`hreflang`](https://developers.google.com/search/docs/specialty/international/localized-versions) for
language and country annotations.
You can provide the `rel="canonical"` `link` annotations in two ways:
- [The `rel="canonical"` `link` element in the HTML](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#rel-canonical-link-method)
- [The `rel="canonical"` `link` HTTP header](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls#rel-canonical-header-method)
We recommend that you choose one of these and go with that; while supported, using
both methods at the same time is more error prone (for example, you might provide
one URL in the HTTP header, and another URL in the `rel="canonical"`
`link` element).
### The `rel="canonical"` `link` element
A `rel="canonical"` `link` element (also known as a
*canonical element* ) is an element used in the `head` section of HTML
to indicate that another page is representative of the content on the page.
Suppose you want `https://example.com/dresses/green-dresses` to be the
canonical URL, even though a variety of URLs can access this content. Indicate this
URL as canonical with these steps:
1. Add a `<link>` element with the attribute `rel="canonical"` to the `<head>` section of duplicate pages, pointing to the canonical page. For example:
```
<html>
<head>
<title>Explore the world of dresses</title>
<link rel="canonical" href="https://example.com/dresses/green-dresses" />
<!-- other elements -->
</head>
<!-- rest of the HTML -->
```
> [!NOTE]
> We recommend adding this same self-referential `rel="canonical"` link element to the canonical page itself as well.
2. If the canonical page has a mobile variant on a separate URL, add a `rel="alternate"` `link` element to it, pointing to the mobile version of the page:
```
<html>
<head>
<title>Explore the world of dresses</title>
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.example.com/dresses/green-dresses">
<link rel="canonical" href="https://example.com/dresses/green-dresses" />
<!-- other elements -->
</head>
<!-- rest of the HTML -->
```
3. Add any [`hreflang`](https://developers.google.com/search/docs/specialty/international/localized-versions) or other elements that are appropriate for the page.
Use absolute paths rather than relative paths with the `rel="canonical"`
`link` element. Even though relative paths are supported by Google, they
can cause problems in the long run (for example, if you unintentionally allow your
testing site to be crawled) and thus we don't recommend them.
**Good example** :
`https://www.example.com/dresses/green/green-dress.html`
**Bad example** :
`/dresses/green/green-dress.html`
The `rel="canonical"` `link element` is only accepted if it
appears in the `<head>` section of the HTML, so make sure at least the
[`<head>` section is valid HTML](https://developers.google.com/search/docs/crawling-indexing/valid-page-metadata).
If you use JavaScript to add the `rel="canonical"` `link` element,
make sure to
[inject the canonical link element properly](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics#properly-inject-canonical-links).
### The `rel="canonical"` HTTP header
If you can change the configuration of your server, you can use a `link`
[HTTP response header](https://en.wikipedia.org/wiki/List_of_HTTP_header_fields)
with a `rel="canonical"` target attribute as defined by
[RFC5988](https://www.rfc-editor.org/rfc/rfc5988.html#section-5.1)
rather than an HTML element to indicate the canonical URL for a document supported
by Search, including non-HTML documents such as PDF files.
Google supports this method for web search results only.
If you publish content in many file formats, such as PDF or Microsoft Word, each on
their own URL, you can return a `rel="canonical"` HTTP header to tell
Googlebot what is the canonical URL for the non-HTML files. For example, to indicate
that the PDF version of the `.docx` version should be canonical, you might
add this HTTP header for the `.docx` version of the content:
```
HTTP/1.1 200 OK
Content-Length: 19
...
Link: <https://www.example.com/downloads/white-paper.pdf>; rel="canonical"
...
```
As with the `rel="canonical"` `link` element, use absolute URLs
in the `rel="canonical"` HTTP header.
## Use a sitemap
Pick a canonical URL for each of your pages and submit them in a
[sitemap](https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview). All pages
listed in a sitemap are suggested as canonicals; Google will decide which pages
(if any) are duplicates, based on similarity of content.
Supplying the preferred canonical URLs in the sitemaps is a straightforward way of defining
canonicals for a large site, and sitemaps are a useful way to tell Google which pages
you consider most important on your site.
## Use redirects
Use this method when you want to get rid of existing duplicate pages. All
[permanent redirection methods](https://developers.google.com/search/docs/crawling-indexing/301-redirects)
have the same effect on Google Search, however the time it takes for search engines to notice the
different redirect methods may differ.
For the quickest effect, use HTTP (also known as *server-side*) redirects.
Suppose your page can be reached in multiple ways:
- `https://example.com/home`
- `https://home.example.com`
- `https://www.example.com`
Pick one of those URLs as your canonical URL, and use redirects to send traffic from
the other URLs to your preferred URL.
## Other signals
Apart from explicitly provided methods, Google also uses a set of canonicalization signals
that are generally based on site setup: preferring HTTPS over HTTP, and URLs in
`hreflang` clusters.
### Prefer HTTPS over HTTP for canonical URLs
Google prefers HTTPS pages over equivalent HTTP pages as canonical, except when there
are issues or conflicting signals such as the following:
- The HTTPS page has an invalid SSL certificate.
- The HTTPS page contains insecure dependencies (other than images).
- The HTTPS page redirects users to or through an HTTP page.
- The HTTPS page has a `rel="canonical"` `link` to the HTTP page.
Although our systems prefer HTTPS pages over HTTP pages by default, you can ensure this
behavior by taking any of the following actions:
- Add redirects from the HTTP page to the HTTPS page.
- Add a `rel="canonical"` `link` from the HTTP page to the HTTPS page.
- Implement [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security).
To prevent Google from incorrectly making the HTTP page canonical, **avoid**
the following practices:
- Avoid bad TLS/SSL certificates and HTTPS-to-HTTP redirects because they cause Google to prefer HTTP very strongly. Implementing HSTS cannot override this strong preference.
- Don't include the HTTP version of your pages in your sitemap or [`hreflang` annotations](https://developers.google.com/search/docs/specialty/international/localized-versions) rather than the HTTPS version.
- Avoid implementing your SSL/TLS certificate for the wrong host-variant. For example, `example.com` serving the certificate for `subdomain.example.com`. The certificate must match your complete site URL, or be a wildcard certificate that can be used for multiple subdomains on a domain.
### Prefer URLs in `hreflang` clusters
To help with sites' localization efforts, for canonicalization purposes Google prefers
URLs that are part of `hreflang` clusters. For example, if
`https://example.com/de-de/cats` and
`https://example.com/de-ch/cats` reciprocally point to each other with
`hreflang` annotations, but not to
`https://example.com/de-at/cats`, the pages for
`de-de` and `de-ch` will be preferred as canonicals instead
of the `/de-at/` page that doesn't appear in the
`hreflang` cluster.
Read more about
[troubleshooting and fixing canonicalization issues](https://developers.google.com/search/docs/crawling-indexing/canonicalization-troubleshooting).
# What Is Googlebot | Google Search Central
# Googlebot
Googlebot is the generic name for two types of
[web crawlers](https://developers.google.com/search/docs/fundamentals/how-search-works) used by Google Search:
- [**Googlebot Smartphone**](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers#googlebot-smartphone): a mobile crawler that simulates a user on a mobile device.
- [**Googlebot Desktop**](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers#googlebot-desktop): a desktop crawler that simulates a user on desktop.
You can identify the subtype of Googlebot by looking at the
[HTTP `user-agent` request header](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers)
in the request. However, both crawler types obey the same product token (user agent token) in
robots.txt, and so you cannot selectively target either Googlebot Smartphone or Googlebot
Desktop using robots.txt.
For most sites Google Search primarily
[indexes the mobile version](https://developers.google.com/search/docs/crawling-indexing/mobile/mobile-sites-mobile-first-indexing)
of the content. As such the majority of Googlebot crawl requests will be made using the mobile
crawler, and a minority using the desktop crawler.
## How Googlebot accesses your site
For most sites, Googlebot shouldn't access your site more than once every few seconds on
average. However, due to delays it's possible that the rate will appear to be slightly higher
over short periods. If your site is having trouble keeping up with Google's crawling requests, you
can [reduce the crawl rate.](https://developers.google.com/search/docs/crawling-indexing/reduce-crawl-rate)
When crawling for Google Search, Googlebot crawls the first 2MB of a
[supported file type](https://developers.google.com/search/docs/crawling-indexing/indexable-file-types), and
the first 64MB of a PDF file. From a rendering perspective, each resource referenced in the HTML
(such as CSS and JavaScript) is fetched separately, and each resource fetch is bound by the same
file size limit that applies to other files (except PDF files).
Once the cutoff limit is reached, Googlebot stops the fetch and only sends the already downloaded
part of the file for indexing consideration. The file size limit is applied on the uncompressed
data.
Other Google crawlers, for example Googlebot Video and Googlebot Image, may have
[different limits](https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers#file-size-limits).
When crawling from IP addresses in the US, the timezone of Googlebot is
[Pacific Time](https://g.co/kgs/WSf8oR).
Other
[technical properties of Googlebot](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers#crawl-technical-props)
are described in the overview of Google's crawlers.
## Blocking Googlebot from visiting your site
Googlebot discovers new URLs to crawl primarily from links embedded in previously crawled pages.
It's almost impossible to keep a site secret by not publishing links to it. For example, as soon
as someone clicks a link from your "secret" site to another site, your "secret" site URL may
appear in the referrer tag and can be stored and published by the other site in its referrer log.
If you want to prevent Googlebot from crawling content on your site, you have a
[number of options](https://developers.google.com/search/docs/crawling-indexing/control-what-you-share). Remember
there's a difference between *crawling* and *indexing*; blocking Googlebot from crawling
a page doesn't prevent the URL of the page from appearing in search results:
- **Prevent Googlebot from crawling a page?** Use a [robots.txt file](https://developers.google.com/search/docs/crawling-indexing/robots/intro).
- **Don't want Google to index a page?** Use [`noindex`](https://developers.google.com/search/docs/crawling-indexing/block-indexing).
- **Prevent a page from being accessible at all by both crawlers or users?** Use [another method, such as password protection](https://developers.google.com/search/docs/crawling-indexing/control-what-you-share).
Blocking Googlebot affects Google Search (including Discover and all Google Search features), as
well as other products such as Google Images, Google Video, and Google News.
## Verifying Googlebot
Before you decide to block Googlebot, be aware that the HTTP `user-agent` request
header used by Googlebot is often spoofed by other crawlers. It's important to verify that a
problematic request actually comes from Google. The best way to verify that a request actually
comes from Googlebot is to
[use a reverse DNS lookup](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot#manual)
on the source IP of the request, or to match the source IP against the
[Googlebot IP ranges](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot#use-automatic-solutions).
# Robots.txt Introduction and Guide | Google Search Central
# Introduction to robots.txt
[Video](https://www.youtube.com/watch?v=IXNEVt9rZG8)
A robots.txt file tells search engine crawlers which URLs the crawler can access on your site.
This is used mainly to avoid overloading your site with requests; **it is not a
mechanism for keeping a web page out of Google** . To keep a web page out of Google,
[block indexing with `noindex`](https://developers.google.com/search/docs/crawling-indexing/block-indexing)
or password-protect the page.
> [!IMPORTANT]
>
> **If you use a CMS, such as Wix or Blogger**, you might not need to (or
> be able to) edit your robots.txt file directly. Instead, your CMS might expose a search
> settings page or some other mechanism to tell search engines whether or not to crawl your
> page.
>
>
> If you want to hide or unhide one of your pages from search engines, search for instructions
> about modifying your page visibility in search engines on your CMS (for example,
> search for "wix hide page from search engines").
## What is a robots.txt file used for?
A robots.txt file is used primarily to manage crawler traffic to your site, and
*usually* to keep a file off Google, depending on the file type:
| robots.txt effect on different file types ||
|---|---|
| Web page | You can use a robots.txt file for web pages (HTML, PDF, or other [non-media formats that Google can read](https://developers.google.com/search/docs/crawling-indexing/indexable-file-types)), to manage crawling traffic if you think your server will be overwhelmed by requests from Google's crawler, or to avoid crawling unimportant or similar pages on your site. > [!WARNING] > **Warning**: Don't use a robots.txt file as a means to hide your web pages > (including PDFs and other text-based formats supported by Google) from Google Search > results. > If other pages point to your page with descriptive text, Google could still index > the URL without visiting the page. If you want to block your page from search > results, use another method such as password protection or > [`noindex`](https://developers.google.com/search/docs/crawling-indexing/block-indexing). **If your web page is blocked with a robots.txt file** , its URL can still appear in search results, but the search result [won't have a description](https://support.google.com/webmasters/answer/7489871). Image files, video files, PDFs, and other non-HTML files embedded in the blocked page will be excluded from crawling, too, unless they're referenced by other pages that are allowed for crawling. If you see this search result for your page and want to fix it, remove the robots.txt entry blocking the page. If you want to hide the page completely from Search, use [another method](https://developers.google.com/search/docs/crawling-indexing/remove-information#i-control-the-web-page). |
| Media file | Use a robots.txt file to manage crawl traffic, and also to prevent image, video, and audio files from appearing in Google Search results. This won't prevent other pages or users from linking to your image, video, or audio file. - [Read more about preventing images from appearing on Google.](https://developers.google.com/search/docs/crawling-indexing/prevent-images-on-your-page) - [Read more about how to remove or restrict your video files from appearing on Google.](https://developers.google.com/search/docs/appearance/video#remove) |
| Resource file | You can use a robots.txt file to block resource files such as unimportant image, script, or style files, **if you think that pages loaded without these resources won't be significantly affected by the loss**. However, if the absence of these resources make the page harder for Google's crawler to understand the page, don't block them, or else Google won't do a good job of analyzing pages that depend on those resources. |
## Understand the limitations of a robots.txt file
Before you create or edit a robots.txt file, you should know the limits of this URL blocking
method. Depending on your goals and situation, you might want to consider other mechanisms to
ensure your URLs are not findable on the web.
- **robots.txt rules may not be supported by all search engines.**
The instructions in robots.txt files cannot enforce crawler behavior to your site; it's up to the crawler to obey them. While Googlebot and other respectable web crawlers obey the instructions in a robots.txt file, other crawlers might not. Therefore, if you want to keep information secure from web crawlers, it's better to use other blocking methods, such as [password-protecting private files on your server](https://developers.google.com/search/docs/crawling-indexing/control-what-you-share).
- **Different crawlers interpret syntax differently.**
Although respectable web crawlers follow the rules in a robots.txt file, each crawler might interpret the rules differently. You should know the [proper syntax](https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#syntax) for addressing different web crawlers as some might not understand certain instructions.
- **A page that's disallowed in robots.txt can
still be indexed if linked to from other sites.**
While Google won't crawl or index the content blocked by a robots.txt file, we might still find and index a disallowed URL if it is linked from other places on the web. As a result, the URL address and, potentially, other publicly available information such as anchor text in links to the page can still appear in Google Search results. To properly prevent your URL from appearing in Google Search results, [password-protect the files on your server](https://developers.google.com/search/docs/crawling-indexing/control-what-you-share), [use the `noindex` `meta` tag or response header](https://developers.google.com/search/docs/crawling-indexing/block-indexing), or remove the page entirely.
> [!CAUTION]
> **Caution** : Combining multiple crawling and indexing rules might cause some rules to counteract other rules. Learn how to [combine crawling with indexing and serving rules](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag#combining).
## Create or update a robots.txt file
If you decided that you need one, learn how to
[create a robots.txt file](https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt). Or if
you already have one, learn how to
[update it](https://developers.google.com/search/docs/crawling-indexing/robots/submit-updated-robots-txt).
Want to learn more? Check out the following resources:
- [How to write and submit a robots.txt file](https://developers.google.com/crawling/docs/robots-txt/create-robots-txt)
- [Update your robots.txt file](https://developers.google.com/crawling/docs/robots-txt/submit-updated-robots-txt)
- [How Google interprets the robots.txt specification](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec)
# Video sitemaps and alternatives
A video sitemap is a [sitemap](https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview)
with additional information about videos hosted on your pages. Creating a video sitemap is a
good way to help Google find and understand the video content on your site, especially
content that was recently added or that we might not otherwise discover with our usual
crawling mechanisms.
Google recommends using video sitemaps, however we also support [mRSS feeds](https://developers.google.com/search/docs/crawling-indexing/sitemaps/video-sitemaps#sitemap_alternatives).
## Video sitemap best practices
Video sitemaps are based on generic sitemaps, so the
[general sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap#general-guidelines)
also apply to video sitemaps. You can create a separate sitemap or mRSS feed just for video,
or you can add video sitemap tags within an existing
[sitemap](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap#xml), whichever is more
convenient for you.
Additionally, the following requirements apply to video sitemaps specifically:
- Don't list videos that are unrelated to the content of the host page. For example, a video that is a small addendum to the page, or unrelated to the main text content.
- All files referenced in the video sitemap must be accessible to Googlebot. This means that all URLs in the video sitemap:
- must not be disallowed for crawling by [robots.txt](https://developers.google.com/search/docs/crawling-indexing/robots/intro) rules,
- must be accessible without metafiles and without logging in,
- must not be blocked by firewalls or similar mechanism,
- and must be accessible on a supported protocol: HTTP and FTP (streaming protocols are not supported).
If you want to prevent spammers from accessing your video content at the
`<player_loc>` or `<content_loc>` URLs,
[verify that any bots accessing your server are really Googlebot](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot).
For more tips about videos in Google Search, see our
[video best practices](https://developers.google.com/search/docs/appearance/video).
## Example video sitemap
The following example shows a regular sitemap with video extension. It includes two video
entries nested in the single `<url>` tag. The first
`<video>` entry includes all the tags that Google can use while the second
only the required tags.
```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://www.example.com/videos/some_video_landing_page.html</loc>
<video:video>
<video:thumbnail_loc>https://www.example.com/thumbs/123.jpg</video:thumbnail_loc>
<video:title>Grilling steaks for summer</video:title>
<video:description>
Alkis shows you how to get perfectly done steaks every time
</video:description>
<video:content_loc>
http://streamserver.example.com/video123.mp4
</video:content_loc>
<video:player_loc>
https://www.example.com/videoplayer.php?video=123
</video:player_loc>
<video:duration>600</video:duration>
<video:expiration_date>2021-11-05T19:20:30+08:00</video:expiration_date>
<video:rating>4.2</video:rating>
<video:view_count>12345</video:view_count>
<video:publication_date>2007-11-05T19:20:30+08:00</video:publication_date>
<video:family_friendly>yes</video:family_friendly>
<video:restriction relationship="allow">IE GB US CA</video:restriction>
<video:platform relationship="allow">web tv</video:platform>
<video:requires_subscription>yes</video:requires_subscription>
<video:uploader
info="https://www.example.com/users/grillymcgrillerson">GrillyMcGrillerson
</video:uploader>
<video:live>no</video:live>
<video:tag>steak</video:tag>
<video:tag>meat</video:tag>
<video:tag>summer</video:tag>
</video:video>
<video:video>
<video:thumbnail_loc>https://www.example.com/thumbs/345.jpg</video:thumbnail_loc>
<video:title>Grilling steaks for winter</video:title>
<video:description>
In the freezing cold, Roman shows you how to get perfectly done steaks every time.
</video:description>
<video:content_loc>
http://streamserver.example.com/video345.mp4
</video:content_loc>
<video:player_loc>
https://www.example.com/videoplayer.php?video=345
</video:player_loc>
</video:video>
</url>
</urlset>
```
#### More examples
The following example demonstrates how to add a Vimeo video embed to a video sitemap:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://www.example.com/videos/some_video_landing_page.html</loc>
<video:video>
<video:thumbnail_loc>https://www.example.com/thumbs/123.jpg</video:thumbnail_loc>
<video:title>Lizzi is painting the wall</video:title>
<video:description>
Gary is watching the paint dry on the wall Lizzi painted.
</video:description>
<video:player_loc>
https://player.vimeo.com/video/987654321
</video:player_loc>
</video:video>
</url>
</urlset>
```
The following example demonstrates how to add a YouTube video embed to a video sitemap:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://www.example.com/videos/some_video_landing_page.html</loc>
<video:video>
<video:thumbnail_loc>https://www.example.com/thumbs/345.jpg</video:thumbnail_loc>
<video:title>John teaches cheese</video:title>
<video:description>
John explains the differences between a banana and cheese.
</video:description>
<video:player_loc>
https://www.youtube.com/embed/1a2b3c4d
</video:player_loc>
</video:video>
</url>
</urlset>
```
## Video sitemap reference
The `video` tags are defined in the video sitemaps namespace:
[`http://www.google.com/schemas/sitemap-video/1.1`](http://www.google.com/schemas/sitemap-video/1.1).
Each tag can only be added one time per video, unless otherwise specified.
To make sure Google can use your video sitemap, you must use the following required tags:
| Required tags ||
|---|---|
| `<video:video>` | The parent element for all information about a single video on the page specified by the `<loc>` tag. You can include multiple `<video:video>` tags nested in the `<loc>` tag, one for each video on the hosting page. |
| `<video:thumbnail_loc>` | A URL pointing to the video thumbnail image file. Follow the [video thumbnail requirements.](https://developers.google.com/search/docs/appearance/video#thumbnails) |
| `<video:title>` | The title of the video. All HTML entities must be escaped or wrapped in a [`CDATA` block](http://wikipedia.org/wiki/CDATA). We recommend that this match the video title displayed on the web page where the video is embedded. |
| `<video:description>` | A description of the video. Maximum 2048 characters. All HTML entities must be escaped or wrapped in a [`CDATA` block](http://wikipedia.org/wiki/CDATA). It must match the description displayed on the web page where the video is embedded, but it doesn't need to be a word-for-word match. |
| `<video:content_loc>` | A URL pointing to the actual video media file. The file must be one of the [supported formats.](https://developers.google.com/search/docs/appearance/video#file-types) > [!NOTE] > It's required to provide either a `<video:content_loc>` or `<video:player_loc>` tag. We recommend that your provide the `<video:content_loc>` tag, if possible. This is the most effective way for Google to fetch your video content files. If `<video:content_loc>` isn't available, provide `<video:player_loc>` as an alternative. **Additional guidelines** - HTML and Flash aren't supported formats. - Must not be the same as the URL in the parent `<loc>` tag. - This is the equivalent of `https://developers.google.com/search/docs/appearance/structured-data/video` in structured data. - **Best practice:** If you want to restrict access to your content but still have it crawled, ensure that Googlebot can access your content by using [verifying Googlebot](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot). |
| `<video:player_loc>` | A URL pointing to a player for a **specific** video. Usually this is the information in the `src` attribute of an `<embed>` tag. > [!NOTE] > It's required to provide either a `<video:content_loc>` or `<video:player_loc>` tag. We recommend that your provide the `<video:content_loc>` tag, if possible. This is the most effective way for Google to fetch your video content files. If `<video:content_loc>` isn't available, provide `<video:player_loc>` as an alternative. **Additional guidelines** - Must not be the same as the `<loc>` URL. - For Vimeo, YouTube, and other video hosting platforms that allow embedding videos through `iframe` videos, this value is used rather than `video:content_loc`. This is the equivalent of `https://developers.google.com/search/docs/appearance/structured-data/video` in structured data. - **Best practice:** If you want to restrict access to your content but still have it crawled, ensure that Googlebot can access your content by using [verifying Googlebot](https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot). |
Additionally, the following optional tags may help Google better understand your videos and
its properties:
| Optional tags ||
|---|---|
| `<video:duration>` | The duration of the video, in seconds. Value must be from `1` to `28800` (8 hours). |
| `<video:expiration_date>` | The date after which the video is no longer be available, in [W3C format](http://www.w3.org/TR/NOTE-datetime). Omit this tag if your video doesn't expire. If present, Google Search won't show your video after this date. For recurring videos at the same URL, update the expiration date to the new expiration date. Supported values are complete date (`YYYY-MM-DD`), or complete date plus hours, minutes and seconds, and timezone (`YYYY-MM-DDThh:mm:ss+TZD`). **Example:** `2012-07-16T19:20:30+08:00`. |
| `<video:rating>` | The rating of the video. Supported values are float numbers in the range `0.0` (low) to `5.0` (high). |
| `<video:view_count>` | The number of times the video has been viewed. |
| `<video:publication_date>` | The date the video was first published, in [W3C format](http://www.w3.org/TR/NOTE-datetime). Supported values are complete date (`YYYY-MM-DD`) or complete date plus hours, minutes and seconds, and timezone (`YYYY-MM-DDThh:mm:ss+TZD`). **Example:** `2007-07-16T19:20:30+08:00` |
| `<video:family_friendly>` | Whether the video is available with [SafeSearch](https://developers.google.com/search/docs/crawling-indexing/safesearch). If you omit this tag, the video is available when SafeSearch is turned on. **Supported values**: - `yes`: The video is available when SafeSearch is turned on. - `no`: The video is only available when SafeSearch is turned off. |
| `<video:restriction>` | Whether to show or hide your video in search results from specific countries. Specify a space-delimited list of country codes in [ISO 3166 format](http://wikipedia.org/wiki/ISO_3166). If there's no `<video:restriction>` tag, Google assumes that the video can be shown in all locations. *Note that this tag only affects search results; it doesn't prevent a user from finding or playing your video in a restricted location through other means.* [Learn more about applying country restrictions.](https://developers.google.com/search/docs/appearance/video#restrict_by_country) **Attributes:** If the parent tag `<video:restriction>` is used, the following attribute is required: - `relationship`: Whether the video is allowed or denied in search results in the specified countries. Supported values are: - `allow`: The listed countries are allowed and unlisted countries are denied. - `deny`: The listed countries are denied and unlisted countries are allowed. **Example:** This example allows the video search result to be shown only in Canada and Mexico: `<video:restriction relationship="allow">CA MX</video:restriction>` |
| `<video:platform>` | Whether to show or hide your video in search results on specified platform types. This is a list of space-delimited platform types. *Note that this only affects search results on the specified device types; it doesn't prevent a user from playing your video on a restricted platform.* If there's no `<video:platform>` tag, Google assumes that the video can be played on all platforms. [Learn more about applying platform restrictions.](https://developers.google.com/search/docs/appearance/video#restrict_by_platform) **Supported values**: - `web`: Computer browsers on desktops and laptops. - `mobile`: Mobile browsers, such as those on cellular phones or tablets. - `tv`: TV browsers, such as those available through Google TV devices and game consoles. **Attributes:** If the parent tag `<video:platform>` is used, the following attributes are required: - `relationship`: Specifies whether the video is restricted or permitted for the specified platforms. Supported values are: - `allow`: Any omitted platforms will be denied. - `deny`: Any omitted platforms will be allowed. **Example:** The following example allows users on web or TV, but not mobile devices: `<video:platform relationship="allow">web tv</video:platform>` |
| `<video:requires_subscription>` | Indicates whether a subscription (either paid or free) is required to view the video. Supported values are: - `yes`: Subscription is required. - `no`: Subscription is not required. |
| `<video:uploader>` | The video uploader's name. The string value can be a maximum of 255 characters. **Attributes:** - `info` \[*Optional*\]: Specifies the URL of a web page with additional information about this uploader. This URL must be in the same domain as the `<loc>` tag. |
| `<video:live>` | Indicates whether the video is a livestream. Supported values are: - `yes`: The video is a livestream. - `no`: The video is not a livestream. |
| `<video:tag>` | An arbitrary string tag describing the video. Tags are generally very short descriptions of key concepts associated with a video or piece of content. A single video could have several tags, although it might belong to only one category. For example, a video about grilling food may belong in the "grilling" category, but could be tagged "steak", "meat", "summer", and "outdoor". Create a new `<video:tag>` element for each tag associated with a video. A maximum of 32 tags is permitted per video. |
### Deprecated tags and attributes
We removed the following tags and attributes from our documentation:
`<video:category>`, `<video:gallery_loc>`, the
`autoplay` and `allow_embed` attributes of the
`<video:player_loc>` tag, the `<video:price>` tag and its
attributes, and the `<video:tvshow>` tag and its attributes. See the
[deprecation announcement](https://developers.google.com/search/blog/2022/05/spring-cleaning-sitemap-extensions)
for more information.
## Sitemap alternative: mRSS
While Google recommends using video sitemaps, we also support mRSS feeds.
Google supports [mRSS](http://www.rssboard.org/media-rss),
an RSS module that supplements the element capabilities of
[RSS 2.0](http://cyber.law.harvard.edu/rss/rss.html).
mRSS feeds are very similar to video sitemaps and can be tested, submitted, and updated just
like sitemaps.
For more information about media feeds, see the
[official media RSS documentation](http://www.rssboard.org/media-rss).
> [!NOTE]
> **RSS vs mRSS**: mRSS is a RSS extension used for syndicating multimedia files. It allows for a much more detailed description of the content than the RSS standard.
### mRSS Example
Here's an example of an mRSS entry that provides all the tags that Google uses.
```xml
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/" xmlns:dcterms="http://purl.org/dc/terms/">
<channel>
<title>Example MRSS</title>
<link>https://www.example.com/examples/mrss/</link>
<description>MRSS Example</description>
<item xmlns:media="http://search.yahoo.com/mrss/" xmlns:dcterms="http://purl.org/dc/terms/">
<link>https://www.example.com/examples/mrss/example.html</link>
<media:content url="https://www.example.com/examples/mrss/example.flv" fileSize="405321"
type="video/x-flv" height="240" width="320" duration="120" medium="video" isDefault="true">
<media:player url="https://www.example.com/shows/example/video.swf?flash_params" />
<media:title>Grilling Steaks for Summer</media:title>
<media:description>Get perfectly done steaks every time</media:description>
<media:thumbnail url="https://www.example.com/examples/mrss/example.png" height="120" width="160"/>
<media:price price="19.99" currency="EUR" />
<media:price type="subscription" />
</media:content>
<media:restriction relationship="allow" type="country">us ca</media:restriction>
<dcterms:valid xmlns:dcterms="http://purl.org/dc/terms/">end=2020-10-15T00:00+01:00; scheme=W3C-DTF</dcterms:valid>
<dcterms:type>live-video</dcterms:type>
</item>
</channel>
</rss>
```
### mRSS reference
The
[full mRSS specification](http://www.rssboard.org/media-rss)
contains more optional tags, best practices, and examples.
To make sure Google can use your mRSS feed, you must use the following required tags:
| Required tags ||
|---|---|
| `<media:content>` | Encloses information about the video. Attributes: - `medium`: Type of content. Set to `video`. - `url`: The direct URL to the raw video content. **If this isn't specified, you must specify the `<media:player>` tag.** - `duration` \[*Optional but recommended*\]: Length of the video in seconds. For all of the other optional attributes and child fields of the `<media:content>` tag, see the [mRSS specification](http://www.rssboard.org/media-rss). |
| `<media:player>` | **You must specify at least one of `<media:player>` or the `url` attribute in `<media:content>`.** A URL pointing to a player for a **specific** video. Usually this is the information in the `src` attribute of an `<embed>` tag and must not be the same as the content of the `<loc>` tag. It can't be the same URL as the `<link>` tag. The `<link>` tag points to the URL of the page hosting the video, while this tag points to a player. |
| `<media:title>` | The title of the video. Maximum 100 characters. All HTML entities must be escaped or wrapped in a [CDATA bock](http://wikipedia.org/wiki/CDATA). |
| `<media:description>` | The description of the video. Maximum 2048 characters. All HTML entities must be escaped or wrapped in a [CDATA block](http://wikipedia.org/wiki/CDATA). |
| `<media:thumbnail>` | A URL pointing to a preview thumbnail. Follow the [Video thumbnail requirements.](https://developers.google.com/search/docs/appearance/video#thumbnails) |
Additionally, the following optional tags may help Google better understand your videos and
its properties:
| Optional tags ||
|---|---|
| `<dcterms:valid>` | The publication and expiration date of the video. Here's the [Full specification of the `dcterms:valid`](https://www.dublincore.org/specifications/dublin-core/dcmi-terms/terms/valid/) tag. **Example:** ```xml <dcterms:valid> start=2002-10-13T09:00+01:00; end=2002-10-17T17:00+01:00; scheme=W3C-DTF <dcterms:valid> ``` |
| `<media:restriction>` | A space-delimited list of countries /www.dublincore.org/specifications/dublin-core/dcmi-terms/terms/valid/) tag. **Example:** ```xml <dcterms:valid> start=2002-10-13T09:00+01:00; end=2002-10-17T17:00+01:00; scheme=W3C-DTF <dcterms:valid> ``` |
| `<media:restriction>` | A space-delimited list of countries
