Understand to how Google indexes JavaScript based websites

Creative Website Design and Development Case Study
January 31, 2016
Increase your brand awareness through Creative Website
March 29, 2016
Show all

Understand to how Google indexes JavaScript based websites

Google’s John Mueller written an awesome post on Google+ regarding JavaScript sites / Progressive Web Apps in Google Search.

We occasionally see questions about what JS-based sites can do and still be visible in search, so here’s a brief summary for today’s state:

#1 Don’t cloak to Googlebot

  • Use “feature detection” & “progressive enhancement” techniques to make your content available to all users.
  • Avoid redirecting to an “unsupported browser” page. Consider using a polyfill or other safe fallback where needed.
  • The features Googlebot currently doesn’t support include Service Workers, the Fetch API, Promises, and requestAnimationFrame.

#2 Use rel=canonical attribute when serving content from multiple URLs is required

#3 Avoid the AJAX-Crawling scheme on new sites

Consider migrating old sites that use this scheme soon. Remember to remove “meta fragment” tags when migrating. Don’t use a “meta fragment” tag if the “escaped fragment” URL doesn’t serve fully rendered content.

#4 Avoid using “#” in URLs (outside of “#!”)

Googlebot rarely indexes URLs with “#” in them. Use “normal” URLs with path/filename/query-parameters instead, consider using the History API for navigation. Use Search Console’s Fetch and Render tool to test how Googlebot sees your pages. Note that this tool doesn’t support “#!” or “#” URLs.

Understand to how Google indexes JavaScript based websites Click To Tweet

#5 Check your robots.txt file

Ensure that all required resources (including JavaScript files / frameworks, server responses, 3rd-party APIs, etc) aren’t blocked by robots.txt. The Fetch and Render tool will list blocked resources discovered. If resources are uncontrollably blocked by robots.txt (e.g., 3rd-party APIs) or otherwise temporarily unavailable, ensure that your client-side code fails gracefully.

#6 Limit the number of embedded resources

In particular the number of JavaScript files and server responses required to render your page. A high number of required URLs can result in timeouts & rendering without these resources being available (e.g., some JavaScript files might not be loaded). Use reasonable HTTP caching directives.

#7 Google supports the use of JavaScript to provide titles, description & robots meta tags, structured data, and other meta-data.

When using AMP, the AMP HTML page must be static as required by the spec, but the associated web page can be built using JS/PWA techniques. Remember to use a sitemap file with correct “lastmod” dates for signaling changes on your website.

#8 Other search engines might not support JavaScript at all

Finally, keep in mind that other search engines and web services accessing your content might not support JavaScript at all, or might support a different subset.


Some useful links:

[1] Progressive Web Apps : https://developers.google.com/web/progressive-web-apps
[2] Progressive enhancement:https://en.wikipedia.org/wiki/Progressive_enhancement
[3] rel=canonical: https://support.google.com/webmasters/answer/139066
[4] AJAX Crawling scheme: https://developers.google.com/webmasters/ajax-crawling/docs/specification

Understand to how Google indexes JavaScript based websites
Article Name
Understand to how Google indexes JavaScript based websites
We occasionally see questions about what JS-based sites can do and still be visible in search, so here's a brief summary for today's state:
It's only fair to share...Share on Facebook0Share on Google+0Tweet about this on Twitter0Share on LinkedIn0
Amit Patriwala

I’m a Programmer (.Net, PHP & Coldfusion), Consultant (Application Architecture Design, Web Portal Design, and Database design), Blogger (Digital Marketing, Website Design, Website Development, and Website Optimization & Latest Technology Stuff) and Strategist (Brand Design & Development). I work with companies, agencies, and top management executives to optimize their online brand presence with digital strategy, website design & development strategy, content marketing strategy and social media marketing.

My blog is about all the things to do with “Branding”, “Digital Marketing” and “Website Optimization”.

I’m passionate about the “Brand” since my college timing, I pursued my bachelor’s degree in information technology (B.E.I.T) from renowned engineering college “Nirma Institute of Technology”.