10 Jul Fetch and Render in Search Console
Search Console is a valuable yet underutilized marketing tool. I use the word underutilized because a lot of website owners install Google Analytics and Search Console without exploring, learning about, or tapping into all the resources that these services offer.
One such tool is the Fetch and Render option. In its most basic sense, this option gets Google to index your page, shows you how Google indexes a page, and alerts you if there are issues with indexing your page.
Why Is It Important?
“Google indexes the page naturally anyway, right?”
Because your time and money are valuable, wouldn’t you rather ensure everything is running correctly than hope for it? Below is a list of the six most common reasons I have seen to use Search Console’s Fetch and Render.
1. A page hasn’t been indexed
Sometimes a targeted page is deeper within the navigation of a website. Google might have indexed other pages on your website but not the one you are targeting. Google could take a while to naturally index your page, or it could never happen at all.
You can see check whether Google has indexed a specific page by typing in “Site:” followed by the URL of your page right into Google. If the page is in Google’s index, it will be listed in the results. If the page has not yet been indexed, no pages will be listed in the search results.
(Side note: You can input your root domain to get a full list of your website’s indexed pages.)
It doesn’t hurt your website to use the Fetch and Render tool on pages that have already been indexed. If you wanted, you could go straight to Search Console and ensure the page has been indexed (though the Fetch and Render tool).
2. Your website is new, or you have made changes to it
If you don’t want to wait for Google to index or re-index your website, you can take control into your own hands and let Google know about your new website or website changes.
3. You aren’t sure whether your pages redirect correctly
Google will only look at the exact page you Fetch and Render, but it can tell you where the page redirects and what kind of redirection code it used. If you are unsure if you set up a redirection correctly, you can use this tool to confirm and test.
Google will only follow the path of the exact URL, and it will not Fetch and Render the page it redirects to.
4. You suspect server issues
Most of the time, people use Fetch and Render because it provides everything the Fetch option does and more. If you are testing your server or think there might be a problem, you can use this tool to test the connection.
5. You suspect hacked pages or security issues
I once worked on a website that was hacked and had malicious code put onto the website. I could still visit the website, Google said it was safe, and the reconsideration request was approved. However, anytime I tried to Fetch and Render the website or pages, there was an error. When I dug in deeper, I realized the webpage still had a couple of lines of bad code, which Google was refusing to index.
6. You want to see how Google looks at the page
Fetch and Render also allows you to see how Google views your website. Sometimes the coding on a website impedes and limits the information Google can see. This Fetch and Render function eliminates any guesswork or questions about how Google sees your website.
How Do I Fetch and Render?
Under the Crawl tab on the left-hand side of Google’s Search Console navigation menu, click on “Fetch as Google.”
Once there, you will be prompted to input a URL. To look at your home page, leave the box blank, and click the Fetch and Render button. For all other URLs or pages, just put the URL minus the root domain into the empty box.
For example, if I wanted to index the page https://www.boostability.com/online-marketing-company/about-us/, I would type in /online-marketing-company/about-us/ in the empty box.
Make sure the root domain you put into Search Console is the preferred and live version of your website. For Boostability, we include the https (instead of http) and the www (instead of the non-www version) when setting up the website in Search Console.
If you run into problems while fetching and rendering, first check that the preferred domain is set up correctly in Search Console. Most of you will not have any problems with this policy, but be aware that there is a 500 fetch limit per month.
When fetching a page, you can select different file types. For most purposes, “Desktop” works, but if you want to see how the page looks on mobile, you can also select “Mobile: Smartphone.”
What Do Different Statuses Mean?
After you input the page, you can check the status of the fetch. Search Console uses several different statuses, which you need to know, as they dictate what action, if any, you need to take.
Google was able to successfully make contact with your website and reach all referenced resources. Ideally, each page’s status would pull as a Complete.
This status will show up with a green checkmark, which is a good sign. Google was able to contact and fetch your website. However, something in the robots.txt file blocked a website resource referenced by the page. If you click on the Partial status, it will expand and let you know what resource has been blocked.
When you expand, Google will also let you know the severity of the resource being blocked. Most of the time, you can unblock these files, and the Partial status will turn into a Complete when you fix your robots.txt file.
Google only checks the exact URL you input. If the URL redirects to another page (even a page on the website), you will receive a Redirected response. If you click on the status, it will expand and let you know where the page points to.
Search Console could generate this status for several reasons, most commonly a 404 error (the page does not exist). I have also seen this status when dealing with a hacked website. The page was live, but Google didn’t want to index it.
If you thought the page was live, and Google is unable to fetch it, you will want to investigate and fix the error.
The next resource fetch errors are possible but not as common.
The page is restricting access, possibly through a password. In a normal browser, this issue results in a 403 error (forbidden or no access).
DNS Not Found
This message means the domain couldn’t be found, likely because of a typo. However, if everything looks correct, you will want to contact your hosting company and make sure the website is live.
Google is unable to reach the host of the resource for the robots.txt file. Luckily, Search Console includes a place where you can test your robots.txt file.
This message most likely comes from a timeout error with the host or a situation where the host refused the fetch request from Google. Make sure your server is up and working.
This message could also result from a timeout error or too many fetch requests to the specific server.
This message shouldn’t come up a lot, but this message results from an unspecified error. This error happens so little that, if it happens more than once, Google suggests posting to their help forum.
Submit Page for Indexing
After you have fetched and rendered the page, you can submit it to be indexed. The Fetch tool pulls the page and reads the page, but it does not add the page to Google’s database. All you need to do is click “Submit to Index.”
There is a delay between submitting the page to index and the page actually getting indexed. Be patient (wait a day) before trying to fetch and index the page again.
Fetch and Render might sound complicated, but this function is pretty simple once you dive in because Google provides resources, tips, and additional information within the tool. This is one of the tools I use more often within Search Console. Make sure Google can find everything in the way you want it to be found by using and analyzing the data from Fetch and Render.