SEO & Progressive Web Apps: Looking to the Future

Practitioners of SEO have all the time been mistrustful of JavaScript.

This is partly primarily based on expertise; the capacity of search engines like google and yahoo to uncover, crawl, and precisely index content material which is closely reliant on JavaScript has traditionally been poor. But it’s additionally ordinary, born of a normal wariness in direction of JavaScript in all its varieties that isn’t primarily based on understanding or expertise. This manifests itself as dependence on conventional SEO strategies that haven’t been related for years, and a conviction that to be good at technical SEO doesn’t require an understanding of contemporary net improvement.

As Mike King wrote in his publish The Technical SEO Renaissance, these attitudes are contributing to “an ever-growing technical knowledge gap within SEO as a marketing field, making it difficult for many SEOs to solve our new problems”. They additionally put SEO practitioners prone to being left behind, since too many people refuse to discover – not to mention embrace – applied sciences akin to Progressive Web Apps (PWAs), fashionable JavaScript frameworks, and different such developments that are more and more being seen as the way forward for the net.

In this text, I’ll be taking a contemporary have a look at PWAs. As properly as exploring implications for each SEO and usefulness, I’ll be showcasing some fashionable frameworks and construct instruments which you’ll not have heard of, and suggesting methods during which we want to adapt if we’re to put ourselves at the technological forefront of the net.

1. Recap: PWAs, SPAs, and repair employees

Progressive Web Apps are primarily web sites which offer a person expertise akin to that of a local app. Features like push notifications allow simple re-engagement along with your viewers, whereas customers can add their favourite websites to their residence display screen with out the complication of app shops. PWAs can proceed to operate offline or on low-quality networks, and so they enable a top-level, full-screen expertise on cellular gadgets which is nearer to that supplied by native iOS and Android apps.

Best of all, PWAs do that whereas retaining – and even enhancing – the essentially open and accessible nature of the net. As advised by the identify they’re progressive and responsive, designed to operate for each person no matter their alternative of browser or system. They may also be stored up-to-date robotically and — as we will see — are discoverable and linkable like conventional web sites. Finally, it’s not all or nothing: present web sites can deploy a restricted subset of those applied sciences (utilizing a easy service employee) and begin reaping the advantages instantly.

The spec remains to be pretty younger, and naturally, there are areas which want work, however that doesn’t cease them from being one in every of the greatest developments in the capabilities of the net in a decade. Adoption of PWAs is rising quickly, and organizations are discovering the myriad of real-world enterprise objectives they will impression.

You can learn extra about the options and necessities of PWAs over on Google Developers, however two of the key applied sciences which make PWAs attainable are:

  • App Shell Architecture: Commonly achieved utilizing a JavaScript framework like React or Angular, this refers to a manner of constructing single web page apps (SPAs) which separates logic from the precise content material. Think of the app shell as the minimal HTML, CSS, and JS your app wants to operate; a skeleton of your UI which may be cached.
  • Service Workers: A particular script that your browser runs in the background, separate out of your web page. It primarily acts as a proxy, intercepting and dealing with community requests out of your web page programmatically.

Note that these applied sciences are usually not mutually unique; the single web page app mannequin (introduced to maturity with AngularJS in 2010) clearly predates service employees and PWAs by a while. As we will see, it’s additionally completely attainable to create a PWA which isn’t constructed as a single web page app. For the functions of this text, nonetheless, we’re going to be specializing in the ‘typical’ strategy to creating fashionable PWAs, exploring the SEO implications — and alternatives — confronted by groups that select to be a part of the rapidly-growing variety of organizations that make use of the two applied sciences described above.

We’ll begin with the app shell structure and the rendering implications of the single web page app mannequin.

2. The app shell structure


In a nutshell, the app shell structure entails aggressively caching static property (the naked minimal of UI and performance) after which loading the precise content material dynamically, utilizing JavaScript. Most fashionable JavaScript SPA frameworks encourage one thing resembling this strategy, and the separation of logic and content material on this manner advantages each pace and usefulness. Interactions really feel instantaneous, very like these on a local app, and knowledge utilization may be extremely economical.

Credit to

As I alluded to in the introduction, a heavy reliance on client-side JavaScript is an issue for SEO. Historically, many of those points centered round the undeniable fact that whereas search crawlers require distinctive URLs to uncover and index content material, single web page apps don’t want to change the URL for every state of the utility or web site (therefore the phrase ‘single page’). The reliance on fragment identifiers — which aren’t despatched as a part of an HTTP request — to dynamically manipulate content material with out reloading the web page was a significant headache for SEO. Legacy options concerned changing the hash with a so-called hashbang (#!) and the _escaped_fragment_ parameter, a hack which has long-since been deprecated and which we gained’t be exploring immediately.

Thanks to the HTML5 historical past API and pushState technique, we now have a greater answer. The browser’s URL bar may be modified utilizing JavaScript with out reloading the web page, thereby holding it in sync with the state of your utility or website and permitting the person to make efficient use of the browser’s ‘back’ button. While this answer isn’t a magic bullet — your server should be configured to reply to requests for these deep URLs by loading the app in its right preliminary state — it does present us with the instruments to resolve the downside of URLs in SPAs.

// Run this in your console to modify the URL in your 
// browser - observe that the web page does not really reload. 
historical past.pushState(null, "Page 2", "/page2.html");

The greater downside dealing with SEO immediately is definitely a lot simpler to perceive: rendering content material, specifically when and how it will get carried out.

Rendering content material

Note that once I refer to rendering right here, I’m referring to the technique of establishing the HTML. We’re specializing in how the precise content material will get to the browser, not the technique of drawing pixels to the display screen.

In the early days of the net, issues had been easier on this entrance. The server would sometimes return all the HTML that was vital to render a web page. Nowadays, nonetheless, many websites which make the most of a single web page app framework ship solely minimal HTML from the server and delegate the heavy lifting to the shopper (be person or a bot). Given the scale of the net this requires rather a lot of time and computational useful resource, and as Google made clear at its I/O convention in 2018, this poses a significant downside for search engines like google and yahoo:

“The rendering of JavaScript-powered websites in Google Search is deferred until Googlebot has resources available to process that content.”

On bigger websites, this second wave of indexation can generally be delayed for a number of days. On prime of this, you’re doubtless to encounter a myriad of issues with essential info like canonical tags and metadata being missed fully. I’d extremely suggest watching the video of Google’s glorious discuss on this topic for a rundown of a few of the challenges confronted by fashionable search crawlers.

Google is one in every of the only a few search engines like google and yahoo that renders JavaScript in any respect. What’s extra, it does so utilizing an internet rendering service that till very not too long ago was primarily based on Chrome 41 (launched in 2015). Obviously, this has implications outdoors of simply single web page apps, and the wider topic of JavaScript SEO is an interesting space proper now. Rachel Costello’s latest white paper on JavaScript SEO is the finest useful resource I’ve learn on the topic, and it contains contributions from different specialists like Bartosz Góralewicz, Alexis Sanders, Addy Osmani, and an incredible many extra.

For the functions of this text, the key takeaway right here is that in 2019 you can’t depend on search engines like google and yahoo to precisely crawl and render your JavaScript-dependent net app. If your content material is rendered client-side, it will likely be resource-intensive for Google to crawl, and your website will underperform in search. No matter what you’ve heard to the opposite, if natural search is a priceless channel to your web site, you want to make provisions for server-side rendering.

But server-side rendering is an idea which is continuously misunderstood…

“Implement server-side rendering”

This is a typical SEO audit advice which I usually hear thrown round as if it had been a self-contained, easily-actioned answer. At finest it’s an oversimplification of an unlimited technical enterprise, and at worst it’s a misunderstanding of what’s attainable/vital/useful for the web site in query. Server-side rendering is an final result of many attainable setups and may be achieved in many alternative methods; in the end, although, we’re involved with getting our server to return static HTML.

So, what are our choices? Let’s break down the idea of server-side rendered content material a bit and discover our choices. These are the high-level approaches which Google outlined at the aforementioned I/O convention:

  • Dynamic Rendering — Here, regular browsers get the ‘standard’ net app which requires client-side rendering whereas bots (akin to Googlebot and social media providers) are served with static snapshots. This entails including an extra step onto your server infrastructure, specifically a service which fetches your net app, renders the content material, then returns that static HTML to bots primarily based on their person agent (i.e. UA sniffing). Historically this was carried out with a service like PhantomJS (now deprecated and not developed), whereas immediately Puppeteer (headless Chrome) can carry out an identical operate. The principal benefit is that it might probably usually be bolted into your present infrastructure.

  • Hybrid Rendering — This is Google’s long-term advice, and it’s completely the manner to go for newer website builds. In quick, everybody — bots and people — get the preliminary view served as fully-rendered static HTML. Crawlers can proceed to request URLs on this manner and can get static content material every time, whereas on regular browsers, JavaScript takes over after the preliminary web page load. This is a superb answer in idea, and comes with many different benefits for pace and usefulness too; extra on that quickly.

The latter is cleaner, doesn’t contain UA sniffing, and is Google’s long-term advice. It’s additionally price clarifying that ‘hybrid rendering’ will not be a single answer — it’s an final result of many attainable approaches to making static prerendered content material out there server-side. Let’s break down how a few methods such an final result may be achieved.

Isomorphic/common apps

This is a technique during which you may obtain a ‘hybrid rendering’ setup. Isomorphic functions use JavaScript which runs on each the server and the shopper. This is made attainable thanks to the creation of Node.js, which – amongst many different issues – permits builders to write code which may run on the backend in addition to in the browser.

Typically you’ll configure your framework (React, Angular Universal, no matter) to run on a Node server, prerendering some or all of the HTML earlier than it’s despatched to the shopper. Your server should, subsequently, be configured to reply to deep URLs by rendering HTML for the applicable web page. In regular browsers, that is the level at which the client-side utility will seamlessly take over. The server-rendered static HTML for the preliminary view is ‘rehydrated’ (good time period) by the browser, turning it again right into a single web page app and executing subsequent navigation occasions with JavaScript.

Done properly, this setup may be improbable because it gives the usability advantages of client-side rendering, the SEO benefits of server-side rendering, and a speedy first paint (even when Time to Interactive is commonly negatively impacted by the rehydration as JS kicks in). For worry of oversimplifying the process, I gained’t go into an excessive amount of extra element right here, however the key level is that whereas isomorphic JavaScript / true server-side rendering is usually a highly effective answer, it’s usually enormously complicated to arrange.

So, what different choices are there? If you’ll be able to’t justify the time or expense of a full isomorphic setup, or if it is merely overkill for what you’re attempting to obtain, are there every other methods you’ll be able to reap the advantages of the single web page app mannequin — and hybrid rendering setup — with out sabotaging your SEO?


Having rendered content material out there server-side doesn’t essentially imply that the rendering course of itself wants to occur on the server. All we want is for rendered HTML to be there, prepared to serve to the shopper; the rendering course of itself can occur wherever you want. With a JAMstack strategy, rendering of your content material into HTML occurs as a part of your construct course of.

I’ve written about the JAMstack strategy earlier than. By manner of a fast primer, the time period stands for JavaScript, APIs, and markup, and it describes a manner of constructing complicated web sites with out server-side software program. The technique of assembling a website from front-end part elements — a process a conventional website may obtain with WordPress and PHP — is executed as a part of the construct course of, whereas interactivity is dealt with client-side utilizing JavaScript and APIs.

Think of it this manner: every little thing lives in your Git repository. Your content material is saved as plain textual content markdown information (editable by way of a headless CMS or different API-based answer) and your web page templates and meeting logic are written in Go, JavaScript, Ruby, or no matter language your most popular website generator occurs to use. Your website may be constructed into static HTML on any laptop with the applicable set of command line instruments earlier than it’s hosted wherever. The ensuing set of easily-cached static information can usually be securely hosted on a CDN for subsequent to nothing.

I actually assume static website mills – or reasonably the rules and applied sciences which underpin them — are the future. There’s each probability I’m fallacious about this, however the energy and suppleness of the strategy must be clear to anybody who’s used fashionable npm-based automation software program like Gulp or Webpack to writer their CSS or JavaScript. I’d problem anybody to check the deep Git integration supplied by specialist webhost Netlify in a real-world venture and nonetheless assume that the JAMstack strategy is a fad.

The recognition of static website mills on GitHub, generated utilizing

The significance of a JAMstack setup to our dialogue of single web page apps and prerendering must be pretty apparent. If our static website generator can assemble HTML primarily based on templates written in Liquid or Handlebars, why can’t it do the similar with JavaScript?

There is a brand new breed of static website generator which does simply this. Frequently powered by React or Vue.js, these packages enable builders to construct web sites utilizing cutting-edge JavaScript frameworks and might simply be configured to output SEO-friendly, static HTML for every web page (or ‘route’). Each of those HTML information is absolutely rendered content material, prepared for consumption by people and bots, and serves as an entry level into an entire client-side utility (i.e. a single web page app). This is an ideal execution of what Google termed “hybrid rendering”, although the exact nature of the pre-rendering course of units it fairly aside from an isomorphic setup.

An awesome instance is GatsbyJS, which is inbuilt React and GraphQL. I gained’t go into an excessive amount of element, however I’d encourage everybody who’s learn this far to try their homepage and glorious documentation. It’s a well-supported instrument with an inexpensive studying curve, an energetic group (a feature-packed was launched in September), an extensible plugin-based structure, wealthy integrations with many CMSs, and it permits builders to make the most of fashionable frameworks like React with out sabotaging their SEO. There’s additionally Gridsome, primarily based on VueJS, and React Static which — you guessed it — makes use of React.

Nike’s latest Just Do It marketing campaign, which utilized the React-powered static website generator GatsbyJS and is hosted on Netlify.

Enterprise-level adoption of those platforms seems to be set to develop; GatsbyJS was utilized by Nike for his or her Just Do It marketing campaign, Airbnb for his or her engineering website, and Braun have even used it to energy a significant e-commerce website. Finally, our buddies at SEOmonitor used it to energy their new web site.

But that’s sufficient about single web page apps and JavaScript rendering for now. It’s time we explored the second of our two key applied sciences underpinning PWAs. Promise you’ll stick with me to the finish (haha, nerd joke), as a result of it’s time to discover Service Workers.

three. Service Workers

First of all, I ought to make clear that the two applied sciences we’re exploring — SPAs and repair employees — are not mutually unique. Together they underpin what we generally refer to as a Progressive Web App, sure, but it surely’s additionally attainable to have a PWA which isn’t an SPA. You might additionally combine a service employee into a conventional static web site (i.e. one with none client-side rendered content material), which is one thing I imagine we’ll see occurring much more in the close to future. Finally, service employees function in tandem with different applied sciences like the Web App Manifest, one thing that my colleague Maria not too long ago explored in additional element in her glorious information to PWAs and SEO.

Ultimately, although, it’s service employees which make the most enjoyable options of PWAs attainable. They’re one in every of the most important modifications to the net platform in its historical past, and everybody whose job entails constructing, sustaining, or auditing an internet site wants to concentrate on this highly effective new set of applied sciences. If, like me, you’ve been eagerly checking Jake Archibald’s Is Service Worker Ready web page for the final couple of years and watching as adoption by browser distributors has grown, you’ll know that the time to begin constructing with service employees is now.

We’re going to discover what they’re, what they will do, how to implement them, and what the implications are for SEO.

What can service employees do?

A service employee is a particular sort of JavaScript file which runs outdoors of the principal browser thread. It sits in-between the browser and the community, and its powers embrace:

  • Intercepting community requests and deciding what to do with them programmatically. The employee may go to community as regular, or it’d rely solely on the cache. It might even fabricate a wholly new response from quite a lot of sources. That contains establishing HTML.
  • Preloading information throughout service employee set up. For SPAs this generally contains the ‘app shell’ we mentioned earlier, whereas easy static web sites may choose to preload all HTML, CSS, and JavaScript, guaranteeing primary performance is maintained whereas offline.
  • Handling push notifications, comparable to a local app. This means web sites can get permission from customers to ship notifications, then depend on the service employee to obtain messages and execute them even when the browser is closed.
  • Executing background sync, deferring community operations till connectivity has improved. This may be an ‘outbox’ for a webmail service or a photograph add facility. No extra “request failed, please try again later” – the service employee will deal with it for you at an applicable time.

The advantages of those sorts of options transcend the apparent usability perks. As properly as driving adoption of HTTPS throughout the net (all the main browsers will solely register service employees on the safe protocol), service employees are transformative when it comes to pace and efficiency. They underpin new approaches and concepts like Google’s PRPL Pattern, since we are able to maximize caching effectivity and reduce reliance on the community. In this manner, service employees will play a key position in making the net quick and accessible for the subsequent billion net customers.

So yeah, they’re an absolute powerhouse.

Implementing a service employee

Rather than doing a foul job of writing a primary tutorial right here, I’m as an alternative going to hyperlink to some key assets. After all, you’re in the finest place to know the way deep your understanding of service employees wants to be.

The MDN Docs are a great place to study extra about service employees and their capabilities. If you’re already assured with the necessities of net improvement and revel in a learn-by-doing strategy, I’d extremely suggest finishing Google’s PWA coaching course. It features a entire sensible train on service employees, which is an effective way to familiarize your self with the fundamentals. If ES6 and guarantees aren’t but part of your JavaScript repertoire, put together for a baptism of fireside.

They key factor to perceive — and which you’ll notice in a short time when you begin experimenting — is that service employees hand over an unimaginable degree of management to builders. Unlike earlier makes an attempt to resolve the connectivity conundrum (akin to the ill-fated AppCache), service employees don’t implement any particular patterns in your work; they’re a set of instruments for you to write your personal options to the issues you’re dealing with.

One consequence of that is that they are often very complicated. Registering and putting in a service employee will not be a easy train, and any makes an attempt to cobble one collectively by copy-pasting from StackExchange are doomed to failure (severely, don’t do that). There’s no such factor as a ready-made service employee to your website — when you’re to writer an acceptable employee, you want to perceive the infrastructure, structure, and utilization patterns of your web site. Uncle Ben, ever the net improvement guru, mentioned it finest: with nice energy comes nice accountability.

One final thing: you’ll in all probability be stunned what number of websites you go to are already utilizing a service employee. Head to chrome://serviceworker-internals/ in Chrome or about:debugging#employees in Firefox to see a listing.

Service employees and SEO

In phrases of SEO implications, the most related factor about service employees might be their capacity to hijack requests and modify or fabricate responses utilizing the Fetch API. What you see in ‘View Source’ and even on the Network tab will not be essentially a illustration of what was returned from the server. It may be a cached response or one thing constructed by the service employee from quite a lot of completely different sources.


Here’s a sensible instance:

  • Head to the GatsbyJS homepage
  • Hit the hyperlink to the ‘Docs’ web page.
  • Right-click – View Source

No content material, proper? Just some inline scripts and kinds and empty HTML components — a traditional client-side JavaScript app inbuilt React. Even when you open the Network tab and refresh the web page, the Preview and Response tabs will inform the similar story. The precise content material solely seems in the Element inspector, as a result of the DOM is being assembled with JavaScript.

Now run a curl request for the similar URL (, or fetch the web page utilizing Screaming Frog. All the content material is there, together with correct title tags, canonicals, and every little thing else you may count on from a web page rendered server-side. This is what a crawler like Googlebot will see too.

This is as a result of the web site makes use of hybrid rendering and a service employee — put in in your browser — is dealing with subsequent navigation occasions. There isn’t any want for it to fetch the uncooked HTML for the Docs web page from the server as a result of the client-side utility is already up-and-running – thus, View Source reveals you what the service employee returned to the utility, not what the community returned. Additionally, these pages may be reloaded when you’re offline thanks to the service employee’s efficient use of the cache.

You can simply spot which responses got here from the service employee utilizing the Network tab — observe the ‘from ServiceWorker’ line under.

On the Application tab, you’ll be able to see the service employee which is operating on the present web page together with the varied caches it has created. You can disable or bypass the employee and check any of the extra superior performance it may be utilizing. Learning how to use these instruments is a particularly priceless train; I gained’t go into particulars right here, however I’d suggest learning Google’s Web Fundamentals tutorial on debugging service employees.

I’ve made a acutely aware effort to hold code snippets to a naked minimal on this article, however grant me this one. I’ve put collectively an instance which illustrates how a easy service employee may use the Fetch API to deal with requests and the diploma of management which we’re afforded:

The consequence:

I hope that this (massively simplified and non-production prepared) instance illustrates a key level, specifically that we have now extraordinarily granular management over how useful resource requests are dealt with. In the instance above we’ve opted for a easy try-cache-first, fall-back-to-network, fall-back-to-custom-page sample, however the prospects are countless. Developers are free to dictate how requests must be dealt with primarily based on hostnames, directories, file varieties, request strategies, cache freshness, and hundreds extra. Responses – together with whole pages – may be fabricated by the service employee. Jake Archibald explores some frequent strategies and approaches in his Offline Cookbook.

The time to study the capabilities of service employees is now. The skillset required for contemporary technical SEO has a good diploma of overlap with that of an internet developer, and immediately, a deep understanding of the dev instruments in all main browsers – together with service employee debugging – must be considered a prerequisite.

four. Wrapping Up

SEOs want to adapt

Until not too long ago, it’s been too simple to get away with not understanding the penalties and alternatives posed by PWAs and repair employees.

These had been cutting-edge options which sat on the periphery of what was related to search advertising, and the aforementioned wariness of many SEOs in direction of JavaScript did nothing to encourage experimentation. But PWAs are quickly on their manner to turning into a norm, and it’ll quickly be unattainable to do an efficient job with out understanding the mechanics of how they operate. To keep related as a technical SEO (or SEO Engineer, to borrow one other time period from Mike King), it is best to put your self at the forefront of those sorts of paradigm-shifting developments. The technical SEO who’s illiterate in net improvement is already an anachronism, and I imagine that additional divergence between the technical and content-driven elements of search advertising isn’t any dangerous factor. Specialize!

Upon studying improvement crew is adopting a brand new JavaScript framework for a brand new website construct, it’s not unusual for SEOs to react with a level of cynicism. I’m actually responsible of joking about builders being attracted to the newest shiny expertise or framework, and at how quickly the world of JavaScript improvement appears to evolve, layer upon layer of abstraction and automation being added to what — from the outdoors — can usually appear to be a leaning tower of a improvement stack. But it’s price taking the time to perceive why frameworks are chosen, when applied sciences are doubtless to begin being utilized in manufacturing, and how these selections will impression SEO.

Instead of criticizing 404 dealing with or inner linking of a single web page app framework, for instance, it will be much better to give you the chance to supply significant suggestions that are grounded in an understanding of how they really work. As Jono Alderson noticed in his discuss on the Democratization of SEO, contributions to open supply initiatives are extra priceless in spreading appreciation and consciousness of SEO than repeatedly fixing the similar issues on an ad-hoc foundation.

Beyond SEO

One final thing I’d like to point out: PWAs are such a transformative set of applied sciences that they clearly have penalties which attain far past simply SEO. Other areas of digital advertising are instantly impacted too, and from my standpoint, one in every of the most attention-grabbing is analytics.

If your web site is partially or absolutely practical whereas offline, have you ever tailored your analytics setup to account for this? If push notification subscriptions are a KPI to your web site, are you monitoring this as a purpose? Remembering that service employees wouldn’t have entry to the Window object, monitoring these occasions will not be attainable with ‘normal’ monitoring code. Instead, it’s vital to configure your service employee to construct hits utilizing the Measurement Protocol, queue them if vital, and ship them instantly to the Google Analytics servers.

This is an interesting space that I’ve been exploring rather a lot currently, and you may learn the first publish in my sequence of articles on PWA analytics over on the Builtvisible weblog.

That’s all from me for now! Thanks for studying. If you will have any questions or feedback, please go away a message under or drop me a line on Twitter @tomcbennet.

Many thanks to Oliver Mason and Will Nye for his or her suggestions on an early draft of this text.

Source hyperlink Internet Marketing