Profile cover photo
Profile photo
Samar Panda
271 followers
271 followers
About
Samar's interests
View all
Samar's posts

Post has shared content
An update (March 2016) on the current state & recommendations for JavaScript sites / Progressive Web Apps [1] in Google Search. We occasionally see questions about what JS-based sites can do and still be visible in search, so here's a brief summary for today's state:

# Don't cloak to Googlebot. Use "feature detection" & "progressive enhancement" [2] techniques to make your content available to all users. Avoid redirecting to an "unsupported browser" page. Consider using a polyfill or other safe fallback where needed. The features Googlebot currently doesn't support include Service Workers, the Fetch API, Promises, and requestAnimationFrame.

# Use rel=canonical [3] when serving content from multiple URLs is required.

# Avoid the AJAX-Crawling scheme on new sites. Consider migrating old sites that use this scheme soon. Remember to remove "meta fragment" tags when migrating. Don't use a "meta fragment" tag if the "escaped fragment" URL doesn't serve fully rendered content. [4]

# Avoid using "#" in URLs (outside of "#!"). Googlebot rarely indexes URLs with "#" in them. Use "normal" URLs with path/filename/query-parameters instead, consider using the History API for navigation.

# Use Search Console's Fetch and Render tool [5] to test how Googlebot sees your pages. Note that this tool doesn't support "#!" or "#" URLs.

# Ensure that all required resources (including JavaScript files / frameworks, server responses, 3rd-party APIs, etc) aren't blocked by robots.txt. The Fetch and Render tool will list blocked resources discovered. If resources are uncontrollably blocked by robots.txt (e.g., 3rd-party APIs) or otherwise temporarily unavailable, ensure that your client-side code fails gracefully.

# Limit the number of embedded resources, in particular the number of JavaScript files and server responses required to render your page. A high number of required URLs can result in timeouts & rendering without these resources being available (e.g., some JavaScript files might not be loaded). Use reasonable HTTP caching directives.

# Google supports the use of JavaScript to provide titles, description & robots meta tags, structured data, and other meta-data. When using AMP, the AMP HTML page must be static as required by the spec, but the associated web page can be built using JS/PWA techniques. Remember to use a sitemap file with correct "lastmod" dates for signaling changes on your website.

# Finally, keep in mind that other search engines and web services accessing your content might not support JavaScript at all, or might support a different subset.

Looking at this list, none of these recommendations are completely new & limited to today -- and they'll continue to be valid for foreseeable future. Working with modern JavaScript frameworks for search can be a bit intimidating at first, but they open up some really neat possibilities to make fast & awesome sites!

I hope this was useful! Let me know if I missed anything, or if you need clarifications for any part.

-- update 2016-11-23 --

Here are some more resources that give some more insight into how we're crawling, rendering, and indexing modern sites:

https://webmasters.googleblog.com/2016/11/building-indexable-progressive-web-apps.html
https://www.youtube.com/watch?v=JlP5rBynK3E


Links:
[1] PWA: https://developers.google.com/web/progressive-web-apps
[2] Progressive enhancement: https://en.wikipedia.org/wiki/Progressive_enhancement
[3] rel=canonical: https://support.google.com/webmasters/answer/139066
[4] AJAX Crawling scheme: https://developers.google.com/webmasters/ajax-crawling/docs/specification
[5] https://support.google.com/webmasters/answer/6066468
Photo

Post has attachment
Symmetrical language support for sync & async languages. Async Generators, CancelToken, Observable & Composability features of future JS.

Post has attachment
Alexa skill using Amazon Tap. Fantastic hacknight experience. with +Hirak Chatterjee & +Amit Jotwani Can't ask for more. #seqhack #AmazonAlexa #js

Post has attachment
Built Alexa skill using Amazon tap. We managed to pull up a voice medical assistance. Things that can be built with this is pretty amazing. Team: +Hirak Chatterjee & +Samar Panda #seqhack #AmazonAlexa #bangalore #seqhack2016 

Post has attachment

Post has attachment

Post has attachment
'contain:strict' css property can define boundaries for style, layout & paint work. Performance micro tip. #performance   #web  

Post has shared content
Great tutorial on tracking JavaScript errors with Google Analytics: http://bit.ly/1wkDEgh - tip: use realtime view for instant feedback!
Animated Photo

Post has attachment
A year old video but i still like it. Most of the things still stand as the best practice even till today. Even with fast changing technologies & perf recommendations.

Post has shared content
Planning on moving to HTTPS? Here are 13 FAQs! What's missing? Let me know in the comments and I'll expand this over time, perhaps it's even worth a blog post or help center article. Note that these are specific to moving an existing site from HTTP to HTTPS on the same hostname. Also remember to check out our help center at https://support.google.com/webmasters/answer/6073543

# Do I need to set something in Search Console? No, just add the HTTPS site there. The change-of-address setting doesn't apply for HTTP -> HTTPS moves.

# How can we do an A/B test? Don't cloak to Googlebot specifically, use 302 redirects + rel=canonical to HTTP if you want to test HTTPS but not have it indexed. Don't block via robots.txt . More about A/B testing at https://googlewebmastercentral.blogspot.ch/2012/08/website-testing-google-search.html (302 redirects aren't cached.)

# Will the rel=canonical guarantee that the HTTP URL is indexed? No, but it's a very strong signal when picking the indexed URL.

# What's the next step after testing? Follow our site-move documentation ( https://support.google.com/webmasters/answer/6033049 ). Use 301 redirects from HTTP to HTTPS, confirm the new version by adding a rel=canonical on the HTTPS page, pointing to itself, and submit sitemaps including both HTTP & HTTPS URLs with new change-dates (in the long run, just keep the HTTPS sitemap).

# What about the robots.txt file? The HTTPS site uses the HTTPS robots.txt file. Check that it's reachable or serves a 404 result code, and check that your HTTP URLs aren't blocked by the HTTP robots.txt file.

# Is it OK to have just some pages on HTTPS? Yes, no problem! Start with a part, test it, add more.

# Should I move everything together, or is it fine to do sections? Moving in sections is fine.

# Will I see a drop in search? Fluctuations can happen with any bigger site change. We can't make any guarantees, but our systems are usually good with HTTP -> HTTPS moves.

# Which certificate do I need? For Google Search, any modern certificate that's accepted by modern browsers is acceptable.

# Do I lose "link juice" from the redirects? No, for 301 or 302 redirects from HTTP to HTTPS no PageRank is lost.

# Will we see search keywords in Google Analytics when we're on HTTPS? This won't change with HTTPS, you can see the search queries in Search Console.

# How can I test how many pages were indexed? Verify HTTP / HTTPS separately in Search Console, use Index Status for a broad look, or the sitemaps indexed counts for sitemap URLs.

# How long will a move from HTTP to HTTPS take? There are no fixed crawl frequencies, it depends on the size of your site, and the speed of crawling that's possible. The move takes place on a per-URL basis.


Hope this helps clarify some of the open questions! Let me know if there's anything missing.

Photo
Wait while more posts are being loaded