Using Vaadin webcomponents in Gatsby

Flow
1

Does anyone have a sample or example of using vaadin webcomponents in Gatsby?

Gatsby is built on top of react and I can use a component which shows up in development mode. But when I try to build the Gatsby site, it fails. For example, the following snippet in your src/pages/index.js page shows me the date-picker component in development mode:

import React from "react"
import '@vaadin/vaadin-date-picker';

export default () => (
    <div>
        <div>Hello world!</div>
        <vaadin-date-picker label="When were you born?"></vaadin-date-picker>
    </div>
    
)

But, when I try to build the site, I get following error message:


See our docs page for more info on this error: https://gatsby.dev/debug-html


> 1 | class Lumo extends HTMLElement {
    | ^
  2 |   static get version() {
  3 |     return '1.5.0';
  4 |   }


  WebpackError: ReferenceError: HTMLElement is not defined
  
  - version.js:1 Module../node_modules/@vaadin/vaadin-lumo-styles/version.js
    node_modules/@vaadin/vaadin-lumo-styles/version.js:1:1
  
  - sizing.js:1 Module../node_modules/@vaadin/vaadin-lumo-styles/sizing.js
    node_modules/@vaadin/vaadin-lumo-styles/sizing.js:1:1
  
  - vaadin-date-picker-overlay-styles.js:1 Module../node_modules/@vaadin/vaadin-date-picker/theme/lumo/vaadin-date-picker-overlay-styl    es.js
    node_modules/@vaadin/vaadin-date-picker/theme/lumo/vaadin-date-picker-overlay-styles.js:1:1
  
  - vaadin-date-picker.js:1 Module../node_modules/@vaadin/vaadin-date-picker/theme/lumo/vaadin-date-picker.js
    node_modules/@vaadin/vaadin-date-picker/theme/lumo/vaadin-date-picker.js:1:1
  
  - vaadin-date-picker.js:1 Module../node_modules/@vaadin/vaadin-date-picker/vaadin-date-picker.js
    node_modules/@vaadin/vaadin-date-picker/vaadin-date-picker.js:1:1
  
  - index.js:1 Module../src/pages/index.js
    src/pages/index.js:1:1

I’m not even trying to add any kind of pollyfils for older browser support. Simply installed the component by running yarn add @vaadin/vaadin-date-picker and then imported it into the React component. I’m able to do the same and build it on a project with create-react-app but not sure what is missing on the Gatsby side.

2

If you look at how Gatsby [works internally]
(https://www.gatsbyjs.org/docs/webpack-and-ssr/), it appears that is is using SSR engine, which effectively means “render the resulting component [to a string]
(https://www.gatsbyjs.org/docs/html-generation/#7-render-final-html-document)”. That is done in Node.js, not in browser, where only a subset of DOM is present.

The error of HTMLElement is not defined is indicating exactly that. There are more errors likely to happen, especially related to the fact that Shadow DOM can not be serialized at the moment (there is no “declarative Shadow DOM”).

Personally, I recommend using static site generators which do not perform SSR. If you look at [web.dev]
(https://github.com/GoogleChrome/web.dev) by Google, you will notice they are using web components and a static site generator - namely, [11ty]
(https://www.11ty.io). You can also use Jekyll and what not.

One simple rule: any static site generator which produces vanilla HTML is good for using Vaadin components (and web components in general). Any Gatsby-like generator, forcing you to use SSR, is a trap in terms of implications it has.

3

I assume that the issue is that whatever library Gatsby uses to pre-render the React app doesn’t support custom elements. I did see a mention in a Gatsby GitHub issue that they support web components to the same extent as React, but it seems that might not be the case.

4

I’m not familiar with Gatsby but decided to go down the rabbit hole. HTMLElement is a standard part of the browser so it shouldn’t need anything extra. This entails to me that “basic browser things” are not available. Gatsby seems to be heavily a Server-Side Rendering (SSR) solution so that may lead to some problems. Gatsby compiles a lot of the content to plain HTML on the server. By googling this, I ended to this GitHub issue: [Document is not defined]
(https://github.com/gatsbyjs/gatsby/issues/11885). Document is another basic browser thing since the dawn of world wide web, so another general browser thing missing.

According to the thread, the charting library in use is not SSR friendly and they are recommending to use another charting library. I think it might be something similar in play here with Gatsby and Vaadin.

SSR is not my realm of expertise. There might be ways to fix it by adding some explicit dependencies or changes in code, but sadly I’m not sure what it would be. Hopefully this helps you forward or someone else can step in to give pointers on how to fix it.

EDIT: Apparently two others wrote their own answers while I was writing mine, so it says quite a bit of the same things. :slight_smile:

5

Thanks everyone for your responses. I came to similar conclusion in my research regarding SSR being the root cause. The reason for choosing Gatsby was because of the large community and flexibility of pulling contents from any data sources regardless of where they live.

I think there are some benefits to using SSR based generators, for example seo of the site. I wonder if there are any existing library that can be used with the underlying SSR. I do understand that this might be something that has to be taken care of on the Gatsby (or the SSR) side. I can’t imagine not being able to take advantage of webcomponents in static site generators that uses SSR :slight_smile:

I’m relatively new to both webcomponents and gatsby in general. If you happen to come across any resources that can help with this, please let me know. Thanks a lot!!

6

At Vaadin, we’ve used [Rendertron]
(https://github.com/GoogleChrome/rendertron) (+ Vaadin workarounds) to be able to render Vaadin apps for SEO purposes, so it is possible to do rendering in some environments. I think you could open up a question towards Gatsby and ask if anyone there have experiences with web components / HTMLElement.

7

For SEO purposes when using/creating Web Components the main thing you should try to optimize is to avoid using Shadow DOM for all actual content (which is relevant to search engines) when possible. You can still benefit from Web Components if they’re done in a way where you can put most content in Light DOM (so that the content is visible from document scope just by using document.querySelector() directly) and only hide the implementation details of the component in Shadow DOM. Some Web Components might not even use Shadow DOM though.

This is just a generic guideline for when working with Web Components. There might be additional technical limitation when you’re using some specific frameworks like some SSR. But if you can make it output the web component HTML without any special server side rendering (except for the Light DOM part if needed) it can work out ok.

If you need to have important content in Shadow DOM, then you might need to use a solution like Rendertron to serve a separate rendered version of your site to search engines to make sure they can index all the content easily. Google can nowadays index content within Shadow DOM too (they execute the JavaScript), but they might first index only the Light DOM content and then queue a second indexing with JavaScript execution so it might take much longer for content within Shadow DOM to be indexed so it’s still much more reliable for good SEO to have everything SEO relevant in Light DOM.

8

I think the main issue of this thread is that webcomponents cannot be used in Gatsby natively (due to the SSR being used). I also tested a very simple webcomponent I created with some contents in the shadow dom (not worried about seo yet). Even though the webcomponent works in development mode (which uses the browser), I couldn’t build the site. Kept getting the following error:

"window" is not available during server side rendering.

See our docs page for more info on this error: https://gatsby.dev/debug-html


  37 |      * True if the custom elements polyfill is in use.
  38 |      */
> 39 |     const isCEPolyfill = window.customElements !== undefined && window.customElements.polyfillWrapFlushCallback !== undefined;
     | ^
  40 |     /**
  41 |      * Removes nodes, starting from `start` (inclusive) to `end` (exclusive), from
  42 |      * `container`.

I wrote the webcomponent using lit-element and as you can see, Gatsby doesn’t have an instance of window when it’s trying to build the site. I’m not an expert but as far as I know, customElements.define is a basic command we need to use to register a webcomponent in the dom.

I opened an issue in Gatsby already (https://github.com/gatsbyjs/gatsby/issues/16815), but I’m starting to wonder if it’s even a Gatsby issue. Sounds like the rabbit hole is even deeper since the root issue stems from dom or other basic elements needed by webcomponents not being available in SSRs.

Gatsby does have the following pages in their docs that talks about this, but I couldn’t use the work-arounds successfully.

Hopefully someone in this community can show me an working example.

9

There is indeed a rabbit hole, as you mentioned. Let me expand my previous explanation a bit.

The root problem is that, even though Custom Elements are possible to emulate on Node.js using JSDOM or any other implementation (which is what Gatsby is most likely using), you can’t do the same for Shadow DOM.

Shadow DOM used by LitElement and Polymer creates a DOM sub-tree, which means that DOM is no longer flat, and not something that can be serialized to the string. This prevents us from using Shadow DOM with SSR.

Furthermore, the proposal to add “declarative” Shadow DOM in the current shape was declined by implementors:
https://github.com/whatwg/dom/issues/510

The only SSR implementation I’m aware of which has kind of workaround is the one used by Stencil. I haven’t checked it but I’m afraid it would be hard to make it “portable” and usable outside of Stencil apps.

So, I think with Gatsby there is no way to use Vaadin components at this point, except that trying to “bypass” SSR and rendering them lazily, so that they are only instantiated on the client side somehow.

10

What if you turn of Shadow DOM and put everything in Light DOM?

11

I’m not familiar with how Gatsby works but I think the issue is that it’s trying to run/render code, which only works in the browser, in the server. I would imagine you can somehow configure it so that these parts that need to run in the client are not part of the server side rendering step.

https://www.gatsbyjs.org/docs/using-client-side-only-packages/

That link looks like the right docs to enable this. But when using web components which have their own dependencies (like Polymer or lit etc.) you would probably need to configure a separate build just for the web components (e.g. using “polymer build”, webpack or similar) so that you can produce a bundle file which you can then add as a “script” tag in Gatsby.

Basically it’s not possible to server side render web components with Shadow DOM but I would imagine it should be possible to just include the web component JS and HTML markup needed to use a web component even in a page generated by Gatsby.

I think it’s a generic issue of including client side dependent JS in a Gatsby build, not specifically restricted to web components.

12

What if you turn of Shadow DOM and put everything in Light DOM?

That’s possible when creating own LitElement components with overriding createRenderRoot.
However, you can’t do the same for Vaadin components as they will break.

I do recommend taking a look at client side only packages mentioned by Kari in the above comment.

13

I had a bit of luck yesterday with a simple webcomponent I wrote using lit-element but I think it’s not complete fix/solution. I just finished trying out same fixes (they were suggestions in the two Gatsby docs pages I mentioned before including the client side package Kari mentioned). Although they worked for the simple webcomponent I wrote using lit-element, it didn’t work for the vaadin component.

Here’s one branch where I configured webconfig to ignore vaadi components during server side rendering (https://github.com/amimas/webcomponent-gatsby/tree/fix-using-webpack-config). It’s suggested in this Gatsby doc (https://www.gatsbyjs.org/docs/debugging-html-builds/#fixing-third-party-modules)

The above again works in develop mode which uses the browser, but doesn’t work after building the site. Good news is that build does not fail anymore. Bad news is that when you serve the generated static site, the vaadin component does not load.

Here’s another branch where I’m using React Helmet + Vendor Copy packages together to load the webcomponents during runtime (https://github.com/amimas/webcomponent-gatsby/tree/fix-using-react-helmet). This is mentioned in the client side packaging guide Kari mentioned. Unfortunately this didn’t work either. It doesn’t work in develop mode and also doesn’t work after building the site, although build doesn’t fail anymore. I think it’s because the main vaadin component is trying to load other modules but can’t find it. I see the following error in the console:

Uncaught TypeError: Failed to resolve module specifier "@vaadin/vaadin-lumo-styles/sizing.js". Relative references must start with either "/", "./", or "../".

I think it requires some additional work in terms of bundling these components; not sure.

I’ll keep investigating as far as I can. I see so much potential in webcomponents; didn’t expect to run into this sort of issues so soon :slight_smile:

14

Browser-only dynamic import of the component (and custom-elements-es5-adapter) works for this:

if (global.window) {
  import("@webcomponents/webcomponentsjs/custom-elements-es5-adapter.js");
  import("@vaadin/vaadin-date-picker/vaadin-date-picker.js");
}