• Gabrielle Woolley

SEO Starting Points

Crawling and indexing are the two first steps of the SEO process, and they basically dictate everything that goes on in the digital world.

Understanding the processes of crawling and indexing is an essential foundation to comprehending how and why SEO operates in certain ways.


Crawling is when a search engine bot reads and processes a web page.

Pages can be crawled if they've been linked from another source, have a spike in traffic, or have a URL thats been searched for. Crawlers will look at pages and process their information, including any links that the pages have on them. They'll crawl from link to link and gather data about the pages for search engine servers.

It's an important first step in the process of SEO. It's the essential step of discovery, when a web bot is introduced to new content and gathers data. For any page to be ranked for SEO, it has to be crawled first.


Indexing is when a page gets stored in the search engine's database.

If the crawler bot liked the page, it will be catalogued into the larger collection of pages and ranked. Search engines usually index fresh, relevant content. If a page makes the cut of being indexed, the search engine will determine the value and rank of its keywords, the value of its content, and where it will rank on search results.

Google Search Console is a great tool that checks on pages that have been indexed. It also allows users to Google that they'd like to be indexed. It's a unique, rare opportunity for web designers and content creators to communicate with Google.

Crawling vs. Indexing

Not all pages that are crawled will be indexed. If search engines crawl a page and like its content, they will index it and determine how it will come up in searches.

What came first, the index or the crawl? Indexing must always come after crawling. Crawling is when search engine bots discover new content and analyze it. In indexing, online algorithms determine how the new content meets different user intents.

Why It Matters

These two processes are the basis of all SEO practices. Understanding the importance of having easily craw-able content is essential for being index-able.

All pages that are internally or externally linked will be crawled. This means they have the potential to be indexed. If pages have the honor of being indexed, they're able to rank in the search engine. If pages rank high, they'll be more competitive in attracting new visitors, which will ultimately boost SEO and organic traffic.

Search engine algorithms are continuously being updated, and new content is being produced at a pace that's hard to keep up with. It's hard to make it out there in the digital world, and impress the crawl-y bots enough to rank on index. With a stronger knowledge of the processes underlying the search engine system, it'll be easier to implement practices that lead to higher rankings and more easily searchable content. There's a lot to consider, but understanding the two initial building blocks of how search engines operate will set your content up for SEO success.

  • White Facebook Icon
  • White Twitter Icon
  • White Instagram Icon

© 2019 by Bantam Social Media

  • Black Facebook Icon
  • Black Twitter Icon
  • Black Instagram Icon