Join in the conversation on Twitter by using hashtag #TheBoosterSeat for your comments and questions

Tweet This

The Booster Seat Season 2 Episode 6

Hey there boosters, my name is Andrew Eagar and welcome to another episode of the Booster Seat. Today I’m going to answer some common SEO Questions. Our first question is “How does Google find the information they display in their Results page?”

This is a great question. Google uses a series of robots or spiders to crawl the web to gather information about given websites. These spiders are very busy as they crawl every website that is created.

When they crawl a website they gather all important information about the website such as content, design, meta tags and other bits of code. When the information is gathered they send the data back to the Search Engine for it to index.

As a side note, Google uses the data collected for your website to compare against other websites to determine who ranks or not. Now, don’t worry about whether your website is being crawled by Google or not. This is a scenario that if you build it, they will come. In this case, if you have a website, chances are Google already knows that you exist and is already crawling your website.

If you want to see if Google has crawled and subsequently indexed your website simply use the “site colon” search in the search bar, as seen here. If your website shows up in the search results then Google has crawled and indexed your site. If your website does not show up in the search results, don’t panic. Just install or check your Google Webmaster Tools to see if there is anything wrong with your website that is stopping Google from crawling or indexing it.

This leads us to our next question. That is, “How often is Google Crawling my Website?”

What a great question.

First off, Google wants the most up to date information possible. This is after all why we all use Google so much right? To get the most up to date information Google may be crawling your website multiple times a day. It really depends on how often you update your website.

For one of my own websites as an example, Google only crawls it once every 3-5 days. This is likely due to the fact that I only update my website but maybe once a year. For a news website like cnn.com Google likely crawls that website multiple times every minute. This is due to the fact that news website update constantly with timely information.

So, how often is Google crawling your website? Easy, install Google webmaster tools to your website and you can see EXACTLY how often Google crawls your website.

Click on this link below for an article written by our own Kylie Brooks to get more details on Google crawling.

Well, that covers it for today. Stay Cool!

Share:

subscriber

9 Comments

  • Andrew Williams, March 24, 2015 @ 11:38 am

    This is very helpful because this question comes up a lot. Something else that can be done is that if you search your website in the Google Search Bar that a small upside down green arrow appears next to the URL of your website and you can click this and click cached and Google will tell you the last time they crawled your site. This way you know if they have seen changes and updates to the site.

  • Maria Williams, March 24, 2015 @ 11:47 am

    Good job Andrew. This #Booster #seat episode is very informative. I also like that you mentioned that you mentioned Google Webmasters tool. It’s one of my favorite tools. Also I noticed that it says”Click on this link below for an article written by our own Kylie Brooks to get more details on Google crawling.” but I didn’t see any link..

  • Ammon Mailo, March 25, 2015 @ 3:38 pm

    One key takeaway is the colon site search. I had no clue that it was that easy in finding out if Google crawled and subsequently indexed a website. This is definitely being shared with my clients.

  • Josh, March 26, 2015 @ 9:09 am

    @Andrew – cool tip! I just did that and found that the home page on one of my blogs was last indexed 3 days ago. Good to know! I think that the frequency will increase as I increase the rate at which I add fresh content, as Andrew describes. It has been a couple of weeks since my last post.

  • M Andrew Eagar, March 26, 2015 @ 4:49 pm

    I loved filming this episode. To be clear, I am not a big fan of the creepy spiders. I think it is important for webmasters to know how often Google is crawling their site AND that they can somewhat control or increase how often Google crawls the site.

  • Sean R Parker, March 27, 2015 @ 8:10 am

    I think this has motivated me to add a blog to my portfolio site.

  • Reliable Carpet & Upholstery C, March 28, 2015 @ 10:10 am

    I suspect that Google does not necessarily crawl 100% of your site at the same time. My reason for thinking this is that when we recently created a mobile rendering of only the key pages of our site, a subsequent notice from Google alerted us to the absence of some of the pages NOT in the mobile rendering – some but not all of those that were missing. I also learned that in due course I will no doubt be notified of all the missing pages. While it doesn’t particularly impact my ranking, it’s still something to be aware of.

  • Caz*, March 29, 2015 @ 4:33 pm

    You’re right that Google isn’t crawling 100% of the time. In most instances, it won’t crawl your site at all without a submitted XML sitemap for starters. It would just be too much to expect Google to randomly find a site and keep track of every pages every update. Top performing pages will get crawled more often. If you’re still in a new page, creeping up the page rank, then resubmitting a sitemap is a good practice to ensure that major changes to your site are crawled more quickly.

  • Caz*, March 29, 2015 @ 4:33 pm

    Some great insight from Reliable Carpet & Upholstery Company below! Sounds like they have some great questions or at least a discussion at bay about Google crawling.

Leave a Reply

Your email address will not be published.