Glossary

What is Crawl budget?

The number of URLs a search engine is willing and able to crawl on a site in a given period.

Crawl budget mostly matters for large sites with tens of thousands of URLs. Small sites are usually crawled fully.

Duplicate URLs, endless filter combinations, and slow servers waste it.

Why it matters when you launch

For most new products, crawl budget isn't the bottleneck; getting linked and submitting a sitemap is.

Related terms