Migrating from vaadin 13 to 14 in gradle built workspace

I am trying to migrate a working vaadin 13 web application to vaadin 14 and I can’t quite seem to get it correct. Our application does do some custom theme work which is causing some of the problems and I am also trying to figure out how to properly generate the frontend bundle for a production jar. In our build we use an embedded tomcat server launched from IDEA that points to our Vaadin 14 web application war file, built by a gradle build system. Simply updating our dependencies to Vaadin 14 and launching I saw an error suggesting the node application be installed.

  • Failed to determine ‘node’ tool.

Ideally I would like to use the gradle node plugin which can automatically install this for our development teams but for now I simply installed NPM manually. This allow the ‘dev-mode’ logic in the vaadin-server to auto-generate the frontend bundle and allow the application to start. From the start I noticed a layout issue on our login page (see screenshot comparison). The background and layout of the text fields has changed.

After investigating I see that we use some custom css as applied to two classes in our application.

   @HtmlImport("frontend://styles/shared-styles.html")
   @StyleSheet("styles/shared-styles.css")
   @Viewport("width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes")
   @Theme(value = Lumo.class)
   public class WebUI_UI extends UI {...

and

   @HtmlImport("frontend://styles/shared-styles.html")
   @Viewport("width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes")
   @Theme(value = Lumo.class)
   @Push
   public class MainWebLayout extends VerticalLayout
           implements RouterLayout, PageConfigurator, BeforeEnterObserver { ...

Where the shared-styles.css and shared-styles.html are in src/main/webapps/frontend/styles. After reading the migration guide for themes (https://vaadin.com/docs/v14/flow/theme/migrate-p2-to-p3.html) I tried to convert the items in shared-styles.html from

   <dom-module id="my-grid-styles" theme-for="vaadin-grid">
       <template>
           <style>
               [part~="body-cell"]
 {
                   font-size: 12px;
               }
           </style>
       </template>
    </dom-module>

to a my-grid-styles.css file in src/main/webapp/frontend/styles

   [part~="body-cell"]
 {
       font-size: 12px;
   }

Then applying the annotation @CssImport(value = “styles/my-grid-styles.css”, themeFor = “vaadin-grid”) to the WebUI_UI class. After applying this for every dom-module in shared-styles.css and running again I do not see a difference in the login page. I also tried removing the @HtmlImport and @StyleSheet annotations and adding @ImportCss as described in the vaadin docs but this made the layout worse. Restoring just @StyleSheet put the back to where it originally was for my first Vaadin 14 test so I don’t think the @HtmlImport is having any impact on the layout but the @StyleSheet CSS is used and needed.

I looked deeper into the @CssImport annotation and based on the java doc I believe the shared-styles.css must be included in the frontend bundle. The way the web application is started does not align with how the vaadin dev-mode expects to find the project dir in the ‘user.dir’ of the running application and then find src/main/webapp/frontend/… To correct this I added these settings to the JVM.

   -Dcom.vaadin.flow.server.project.basedir=PATH_TO_PROJECT
   -Dvaadin.frontend.generated.folder=PATH_TO_PROJECT/build
   -Dvaadin.frontend.frontend.folder=PATH_TO_PROJECT/frontend

With this change my css files must be be processed now because I ran into this node related error.

  • ERROR dev-webpack - FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

To fix this I added the following to my environment so node will have more memory.

  • NODE_OPTIONS=–max-old-space-size=8192

With this change the node error is gone but now I see this error from the webapp which I believe is the result of node not creating the ‘stats.json’ file.

  • com.vaadin.flow.server.BootstrapException: Unable to read webpack stats file.

I believe the root of the problems is getting the frontend bundle to be generated correctly but I am finding this is very challenging to get correct. Is there a tool other than the maven plugin that helps with this? I tried the gradle-vaadin-flow gradle plugin but I could not get it to generate the bundle into my war file. The docs indicate to set the ‘productionMode’ flag then use the ‘war’ task, but this task did not work until I added the gradle ‘war’ plugin and then the war did not have the frontend bundle in it.

I looked at the source of the vaadin maven plugin and the vaadin ‘dev-mode’ logic, and there is a class in vaadin flow-server.jar NodeTasks.builder that I could call from gradle. Is this how the frontend bundle should be built if not using ‘dev-mode’ or the maven plugin?

Once the bundle is created correctly are there any files that should be versioned in source control because they may need to be manually edited as the project grows (webpack.config.js, package.json, …)?

17837701.png

I have had some success building the frontend bundle via gradle by looking at the maven plugin and vaadin devmode logic to create the bundle. However building the frontend bundle with the CSS conversions recommended did not fix the layout issue.

For history and if anyone else has similar problems, here are the steps I used to create the vaadin frontend bundle for our gradle build.

  1. Created a gradle task to dump the runtime classpath for the web application to a file.
  2. Created a java application to read the classpath elements from the file and create a URLClasspath so that the vaadin annotated classes can be found and passed to the Vaadin ClassFinder needed by NodeTasks.Builder.
  3. In the Java application make calls to NodeTasks.Builder to update the node_modules with the proper vaadin files.
  4. Used the node gradle plugin to install node and run webpack.
  5. Generated flow-build-info.json from a gradle task.

Step 1 can be done by capturing the classpath in a gradle configuration and using groovy to write it to a file.

task generateRuntimeClassList() {
    inputs.files(configurations.bundleSources)
    outputs.file(classListFile)
    doLast {
        StringBuilder classList = new StringBuilder()
        configurations.bundleSources.each { File f ->
            classList.append(f.getCanonicalPath()).append("\n")
        }
        classListFile.write(classList.toString())
        logger.quiet("Created frontend runtime classlist in $buildDir/runtime-class-list.txt")
    }
}

In step 2 involves finding the classes annotated with Route.class, NpmPackage.class, NpmPackage.Container.class, WebComponentExporter.class, UIInitListener.class, and VaadinServiceInitListener.class and passing them into ClassFinder.DefaultClassFinder so it can be used with NodeTasks.Builder. I did this by scanning jar files and directories on the classpath for these classes and using the URLClassloader to load them and determine if they had the proper annotations.

The basic outline of the core of step 3 is the following code.

// Custom class to find the annotated classes to pass into the class finder by processing classpath dumped by gradle task
// I used a builder type approach here
classBuilder.addClassPathList(classListFiles)
    // Vaadin only processes classes with these annotations
    .addAnnotationClassesRestriction(annotatedClasses)
	.addScanPackages(scannedPackages)
	.build();
// Create a 'DefaultClassFinder' - trivial extended class	
ClassFinder finder = new BundlerClassFinder(classBuilder)

// In my case I used separate dirs for npm install and webpack output
// but to make this work you need additional parameters when running webpack
// first execution is used mainly for copying resources into node_modules
Builder b = new Builder(finder, npmDir, outputDir, inputDir)
b.runNpmInstall(true)
		.enablePackagesUpdate(true)
		.copyResources(classBuilder.getFrontendResources())
		// Make sure to point to where your application's resources are stored here
		.copyLocalResources(localResourceDir)
		// Run separately so gets installed correctly, running here puts in wrong directory
		.enableImportsUpdate(false)
		.withEmbeddableWebComponents(true)
		// Now done via gradle node plugin
		.runNpmInstall(false)
		.build()
		.execute();
// Running the 'enableInportsUpdate' creates the build/generated-flow-imports.js file in the wrong dir
// Run the Builder again with specifying the correct output dir.
File outputBuildDir = new File(npmDir, "build");
outputBuildDir.mkdirs();
b = new Builder(finder, npmDir, outputBuildDir, inputDir);
b.enableImportsUpdate(true)
		.runNpmInstall(false)
		.enablePackagesUpdate(false)
		.build()
		.execute();

In step 4 using the node gradle plugin lets you delegate install of node to the plugin with this config.

node {
    version = '10.16.3'
    npmVersion = '6.9.0'
    download = true
    nodeModulesDir npmDir
}

The trick part of webpack generation is to be sure to run from the NPM install dir but set the output to your output dir. This is not done in the maven plugin, but it does nicely isolate the npm files and configuration from the generated bundle.

task runWebpack(type: NodeTask) {
    dependsOn(setupWebpack)
    script = new File(npmDir, "node_modules/webpack/bin/webpack.js")
    workingDir = npmDir
    // Write the 'bundle' to the defined bundle directory
    // in the proper directory structure needed for war deployment
    args = [ "--output-path", new File(bundleDir, warBundlePath).getCanonicalPath() ]

}

Once the bundle is generated you will need to create the flow-build-info.json file, which is pretty easy with gradle/groovy code.

task createFlowBuildInfo() {
    doLast {
        String flowBuildInfo = JsonOutput.toJson([
                "productionMode": true,
                "compatibilityMode": false,
                "enableDevServer": false,
                "npmFolder": npmDir.getCanonicalPath(),
                "generatedFolder": new File(bundleDir, warBundlePath).getCanonicalPath(),
                "frontendFolder": projectDir.getCanonicalPath(),
        ])

        File flowBuildInfoDir = new File(bundleDir, warBundlePath+"/config")
        flowBuildInfoDir.mkdirs()
        File flowBuildInfoFile = new File(flowBuildInfoDir, "flow-build-info.json")
        flowBuildInfoFile.write(flowBuildInfo)

        logger.info("Created $flowBuildInfoFile")
    }
}

You may also need to copy config/stats.json from the npm install dir if you did not disable stats.

The files package.json, webpack.generated.js package-lock.json, webpack.config.js, bulid/package.json were generated once then committed to revision control. These files belong in the npmDir. The code below from NodeTasks.Builder can be used to generate these files.

b.createMissingPackageJson(true)
     .withWebpack(npmDir, FrontendUtils.WEBPACK_CONFIG,
                 FrontendUtils.WEBPACK_GENERATED)
     .enableImportsUpdate(false)
     .enablePackagesUpdate(false)
     .runNpmInstall(false)
     .build()
     .execute();

After trial and error and a little expert help I was able to get the bundle generated correctly with CSS and use the @CssImport annotation as described in the migration guide. The key seems to be in how the node directory is setup so that webpack can find everything it needs an annotating the @Route annotated view class with the @CssImport annotations. In my case this resulted in something like the following after migrating the css out of ‘shared-styles.html’ as delivered in Vaadin 13 build.

@CssImport("styles/shared-styles.css")
@CssImport(value = "styles/my-grid-styles.css", themeFor = "vaadin-grid")
@CssImport(value = "styles/appbar-context-menu.css", themeFor = "vaadin-context-menu-overlay")
@CssImport(value = "styles/appbar-context-menu-item.css", themeFor = "vaadin-context-menu-item")
@CssImport(value = "styles/SplitViewer-splitter.css", themeFor = "vaadin-split-layout")
@CssImport(value = "styles/vaadin-button-hover.css", themeFor = "vaadin-button")
@CssImport(value = "styles/vaadin-ripple-text-field.css", themeFor = "vaadin-text-field")
@CssImport(value = "styles/vaadin-ripple-password-field.css", themeFor = "vaadin-password-field")
@CssImport(value = "styles/custom-tabs.css", themeFor = "vaadin-tabs")
@CssImport(value = "styles/custom-tab.css", themeFor = "vaadin-tab")
@CssImport(value = "styles/custom-checkbox.css", themeFor = "vaadin-checkbox")
@CssImport(value = "styles/table-sorter.css", themeFor = "vaadin-grid-sorter")
@Route(value = "")
@Push//(transport = Transport.LONG_POLLING)
public class WebUiPrimaryStage extends Div implements HasUrlParameter<String>, BeforeEnterObserver

The second part is to be sure to populate the ‘frontend’ directory where you placed the node build files for webpack with all the CSS and other static resources you need at runtime (In my case the files shared-sytles.css, my-grid-styles.css, appbar-context-menu.css, …). When the webpack script completes you will be able to find the content of your CSS files embedded in the bundle output in the file named like vaadin-bundle-*.cache.js.

The layout I used is summarized below. The gradle build logic in my previous post added one new task to copy the frontend CSS files from ‘src/main/frontend’ where I stored them in revision control into the node directory so the webpack script could find them.

build
  |
  +- node    # <- Node setup for running webpack script
  |   |
  |   +-package.json
  |   |
  |   +-package-lock.json
  |   |
  |   +-webpack.config.js
  |   |
  |   +-webpack.generated.js
  |   |
  |   +-build
  |   |   |
  |   |   +-generated-flow-imports.js
  |   |   |
  |   |   +-package.json
  |   |
  |   +-config
  |   |   |
  |   |   +-stats.json
  |   |
  |   +-node_modules
  |   |   |
  |   |   +-@vaadin/...
  |   |   ...
  |   |
  |   +-frontend
  |       |
  |       +-styles
  |           |
  |           +-shared-styles.css
  |           |
  |           +-my-grid-styles.css
  |           ...
  |
  +-bundle   # <- Generated bundle (no node files)
      |
      +-WEB-INF/classes/META-INF/VAADIN
          |
          +-build
          |   |
          |   +-vaadin-bundle.es5-9b92ddac0048e68d44ba.cache.js
          |   |
          |   +-vaadin-bundle-ccb78c27faafd1e8f2dc.cache.js
          |   |
          |   +-webcomponentsjs/...
          |
          +-config
              |
              +-flow-build-info.json

I used the gradle ‘build’ directory where all build output goes to construct the intermediate node configuration dir structure and the bundle output directory. This way on a clean build a fresh bundle is created and on re-build the existing node data is re-used.

Hi Ted Dennler:
I found this article really informative and now i am starting to understand how things are in the vaadin 14 world. 5 stars for that

However, I would really appreciate if you could make working example like a “HelloWold” project with gradle and used plugins etc, and publish it to github. This would really make thing simpler to understand.

Thanks in advance,

I am also interested in this article.
Anyone got any progress on this issue?

FYI: There’s now an official Vaadin gradle plugin: https://github.com/vaadin/vaadin-gradle-plugin

Includes a task to install node

Grate news!
Unfortunately a week later. I was writing a plugin too.
Thanks!