Crawling

Crawling is how search engines find pages: bots follow links from page to page, reading what they find. If a page cannot be crawled, because it is blocked, orphaned, or buried too deep, it usually will not be indexed or ranked. Clear internal linking and a clean sitemap make crawling efficient, which matters most on large sites.