Application run and build hang in different points

Flow
1

Hi,
In starter application to run the app it takes 10 minutes for us. The following processes stand too much time:

10:40:16 [INFO]
— vaadin-maven-plugin:14.0.3:prepare-frontend (default) @ frontend —

// It takes 3-4 minutes

10:44:11.980 [INFO ]
.springframework.web.context.ContextLoader::prepareWebApplicationContext:284 - Root WebApplicationContext: initialization completed in 6801 ms
10:45:19.379 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:294 - Search for classes with annotations took 65 seconds
10:46:16.551 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:301 - Search for WebComponentExporter took 57 seconds
10:47:05.712 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:308 - Search for UIInitListener took 49 seconds
10:47:59.533 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:316 - Search for VaadinServiceInitListener took 53 seconds
10:47:59.533 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:320 - Search for subclasses took 160 seconds
10:47:59.533 [INFO ]
m.vaadin.flow.spring.VaadinServletContextInitializer::contextInitialized:323 - Search for all classes took 225 seconds

// It takes 3-4 minutes

10:48:01.283 [INFO ]
dev-updater::updateMainJsFile:467 - No js modules to update
10:50:22.962 [INFO ]
dev-webpack:::152 - Starting webpack-dev-server, port: 56067 dir: C:\sources\folder\src\frontend

// It takes 2 minutes

We use: Vaadin 14.0.3, NodeJs 10.16.2, Springboot 2.1.7.
Has anyone any Idee what can cause them? What do we make wrong?

2

Hi. First, which starter are you referring to and how are you building it (what command) ? What type of system or CI server is it ? Is the second run as slow ? That is indeed insanely very slow and I would like to get more details so I might better figure out what is going on.

We are currently working on two issues related to build time:

  1. Whitelisting packages to scan in Spring Boot apps triggers an extra npm install: https://github.com/vaadin/spring/issues/489
  2. running Maven clean causes unnecessary deletion of package.json https://github.com/vaadin/flow/issues/6151

Both of these are unfortunately hindering the build time, which is unbearably slow for some Spring Boot projects at the moment.
This is why we have recommended whitelisting the application packages for all Spring Boot apps: https://vaadin.com/docs/v14/flow/spring/tutorial-spring-configuration.html#special-configuration-parameters

But we have acknowledged that it is still not good enough, and thus we are currently planning further actions to improve the situation. We are looking at two things: the amount of things that are downloaded and making sure the frontend build (or any related scanning) is not triggered when it is not really needed. It should not take several minutes to start the application. There will be further details about these actions later on in our blog.

3

And apparently there is possibly yet another regression, maybe in addition to those two above, in 14.0.3 https://github.com/vaadin/flow/issues/6416

So it would be great if you could report the build time using 14.0.2 also here, thanks.

4

Hello Pekka,

we are collage with Gergo. I have the same issue with build process. The starter project is from Vaadin Getting Started site, with Spring Boot.
Here is the pom.xml of the project https://pastebin.com/xCHDCEwy
We build it with ‘mvn install’ or run it with ‘mvn spring-boot:run’ is no matter. The first run takes more time is ok, but for us from the second, third, etc build takes too long.
It is Windows 10 and/or Windows Server 2012, we use this mashines back of corporate proxy. We already set the proxy settings in npmrc. Maybe some timeout problem?!?

I ‘paste’ the log of build process You can see in the timestamp the ‘gaps’ https://pastebin.com/YgFewG0m
(The first long time is needed by ‘prepare-frontend’ goal and special in ‘>>> Running prepare-frontend in projecttwo project’
I make a test with V14.0.2 too.

5

Pekka Hyvönen:
And apparently there is possibly yet another regression, maybe in addition to those two above, in 14.0.3 https://github.com/vaadin/flow/issues/6416

So it would be great if you could report the build time using 14.0.2 also here, thanks.

Hi

Vaadin 14.0.2 seems buggy, can’t find npm.cmd, but this is installed globally by NodeJS: https://pastebin.com/wLgafFzf

6

Hi Attila.

Thank you for the detailed logs, and the crucial information that you have a proxy in place. We’ve had also other reports of the prepare-frontend goal hanging in similar scenarios. When you are taking one of our starters and you have Node.js already installed, it should be safe for you to remove the plugin completely, since the prepare-frontend basically just verifies that Node.js is installed (with proper version) and it generates some necessary frontend files (webpack.config.js, webpack.generated.js and package.json). Once you have those in place, you don’t need the goal anymore.

So can try by removing that goal prepare-frontend ? Actually then you will not need the Vaadin Maven plugin at all for development mode (still needed for production build). Another option is that you put it behind a Maven profile, so if someone needs to generate any of the above files, they may rerun the goal.

7

Hello Pekka,

I’ve moved Vaadin Maven plugin into a maven profile and updated Vaadin to 14.0.4, now is the build-run process is about 1 minute. https://pastebin.com/JQUhqB4N

But I don’t understand the hole build-run process of an Spring Boot Vaadin 10+ project, specially ‘frontend compiling, preparing’. Is there any docu or can You describe shortly the process (phases, goals)?

What do You mean that ‘prepare-frontend’ is only needed for production build? We develop on local mashines (we have own ‘dev’ maven profile), but we have Jenkins on test and production server, what must do else as on development mashine?

When must I regenerate this files: webpack.config.js, webpack.generated.js and package.json?

And what I must do (maven goals) if I modify:

  • a Java backend source
  • a Java frontend (Vaadin) source
  • a resource file
  • etc

Do I need different stepts on build process to avoid unnecassary build process?

Thanks
Bye

Pekka Hyvönen:
Hi Attila.

Thank you for the detailed logs, and the crucial information that you have a proxy in place. We’ve had also other reports of the prepare-frontend goal hanging in similar scenarios. When you are taking one of our starters and you have Node.js already installed, it should be safe for you to remove the plugin completely, since the prepare-frontend basically just verifies that Node.js is installed (with proper version) and it generates some necessary frontend files (webpack.config.js, webpack.generated.js and package.json). Once you have those in place, you don’t need the goal anymore.

So can try by removing that goal prepare-frontend ? Actually then you will not need the Vaadin Maven plugin at all for development mode (still needed for production build). Another option is that you put it behind a Maven profile, so if someone needs to generate any of the above files, they may rerun the goal.

8

To make it clear, there are two different goals I’m talking about:

The build-frontend goal is only needed for making a production build, which is what you will do when you deploy your application to staging or a production server. This will remove certain debug features and compress, minify etc. the frontend resources to a single bundle file with only the used frontend resources. This is what you want to run on your:

but we have Jenkins on test and production server

The prepare-frontend goal prepares the project so that it is possible to run it in development mode. As I said, it creates checks that Node.js (and npm) is installed on the system and then it creates necessary files so that it is possible to run the project in development mode. In development mode the framework will run npm install when needed and start Webpack dev-server which will handle serving of the frontend resources during application development.
You might have to keep this goal also for Jenkins, but it depends a bit on what files does it clear when you do the build.

Once the application is build using build-frontend and deployed there will be no frontend tooling run anymore (npm / webpack) but everything is served from the bundle done before deployment. You can do this locally too if you want to test things locally.

And what I must do (maven goals) if I modify:

  • a Java backend source

If you have setup automatic reloading with Spring dev tools and are running the server, you don’t need to rerun anything to get these changes to be picked up. Currently due to a bug, adding a new route (view) will require restarting the server

  • a Java frontend (Vaadin) source

Just refresh the page and Webpack dev server will pick up the changes. There have been some reports that this doesn’t work https://github.com/vaadin/flow/issues/6420 but we’re not able to reproduce it at the moment

  • a resource file
    Depending on the resource file type (and mostly location), it should be either of the above. Anything in /frontend and static resource locations should be picked up I think with just a browser refresh.

EDIT: Great to hear that the performance is improving. I checked the logs and the startup seems to be 50 seconds instead of a minute ?
Out of that I checked that 22 seconds is related to the frontend tooling:

  1. c. 5s scanning for used frontend resources, no npm install
  2. c. 17s of starting and running webpack

We did some fixes that will land to 14.0.5 but actually based on the build logs from above, those should not improve the situation for your case as those improve the scanning (65%) which was only 5 seconds in your logs. Also it reduces unnecessary npm installs when running clean but that is not also happening in your logs.

Thank you for the reports thus far. I’m not sure when or if we can soon improve things for the 22 seconds in your case above. The biggest impact would basically be to not have to start webpack at all which is possible, but it is not as straight forward to implement it without additional hindrances: when you edit a frontend file, you want to see changes in browser after a refresh of the page.

9

Hi Pekka,

Thanks for detailed answers.
maybe the optimism was too early: starting from friday the build process is now standing on other place

**07:48**:40.956 [INFO ]
 dev-updater::updateMainJsFile:467                                            - [] No js modules to update
**07:55**:05.274 [INFO ]
 dev-webpack::<init>:152                                                      - [] Starting webpack-dev-server, port: 57880 dir: C:\Sources\inss\appl\inss_apps\src\frontend
   C:\Program Files\nodejs\node.exe C:\Sources\inss\appl\inss_apps\src\frontend\node_modules\webpack-dev-server\bin\webpack-dev-server.js --config C:\Sources\inss\appl\inss_apps\src\frontend\webpack.config.js --port 57880 -d --inline=false

Is the webpack server starting process that takes too long? I can’t figure out what happening…

The hole build log: https://pastebin.com/ThLHLffU

Can You say me what to do? Because if today I can’t make it runnable in “human time”, then we have to delay the hole upgrade process of the project from Vaadin 8 to 14.

Thanks
Bye

10

Out of that I checked that 22 seconds is related to the frontend tooling:

  1. c. 5s scanning for used frontend resources, no npm install
  2. c. 17s of starting and running webpack

We did some fixes that will land to 14.0.5 but actually based on the build logs from above, those should not improve the situation for your case as those improve the scanning (65%) which was only 5 seconds in your logs. Also it reduces unnecessary npm installs when running clean but that is not also happening in your logs.

Thank you for the reports thus far. I’m not sure when or if we can soon improve things for the 22 seconds in your case above. The biggest impact would basically be to not have to start webpack at all which is possible, but it is not as straight forward to implement it without additional hindrances: when you edit a frontend file, you want to see changes in browser after a refresh of the page.

It seems that I need to test the build-run process without starting webpack, how can I disable it?

What fo you mean “when you edit a frontend file, you want to see changes in browser after a refresh of the page”, what does a frontend file mean, Java, some other resource?

Thanks

11

It seems that I need to test the build-run process without starting webpack, how can I disable it?

To avoid using webpack, you can also use build-frontend for development mode profile. So basically you run mvn install and get all the frontend resources bundled once. Then you start you application server and just keep developing until you need to add or change any frontend dependency. Just to clarify, frontend dependency means changing any JS or including any component that was not previously included. Java changes are not effected, unless those bring in new JS (or CSS).

I looked at the build configuration but there was so much noise there, also another step before Vaadin seems to take really long, so I’m not entirely sure what is going on… The webpack startup taking long has happened in other projects too, and I’ve heard that running the node command once manually has helped in some case to remove the delay (!). So in your case it would be

node C:\Sources\inss\appl\inss_apps\src\frontend
   C:\Program Files\nodejs\node.exe C:\Sources\inss\appl\inss_apps\src\frontend\node_modules\webpack-dev-server\bin\webpack-dev-server.js --config C:\Sources\inss\appl\inss_apps\src\frontend\webpack.config.js --port 57880 -d --inline=false

But for now I don’t have any better suggestions, we will try to figure out a) why it is taking so long) b) improve the logging for that phase so we could understand what is causing the delay sometimes.

As you update to 14.0.6, please update here if the situation changes to the better.

12

Hello everybody!

I am running a spring boot application using Vaadin 14.0.14 in Eclipse and I am experiencing the same problem described above.
When I start the application, it get stuck at this stage:

2019-11-30 13:43:04.289  INFO 4560 --- [           main]
 dev-webpack                              : Starting webpack-dev-server, port: 63467 dir: C:\Users\C42544\Documents\DevProjects\AF-Twin\DataMicroService
   C:\Users\C42544\Documents\nodejs\node.exe C:\Users\C42544\Documents\DevProjects\AF-Twin\DataMicroService\node_modules\webpack-dev-server\bin\webpack-dev-server.js --config C:\Users\C42544\Documents\DevProjects\AF-Twin\DataMicroService\webpack.config.js --port 63467 --watchDogPort=63466 -d --inline=false

Just for info, I have a proxy activated in Eclipse.
So after making some test, I realized that by deactivating the proxy, the loading of the application is much faster.

I would like to understand what is the reason and if a fix would be possible in the next Vaadin versions?

Thanks a lot!
Servan

13

Hi all,

for me it’s the same … if I run the app in dev-mode then “npm install” hangs for minutes because of a connection problem :frowning:

I guess the cause of all this is that if vaadin runs “npm install” in the “on the fly mode” then no proxy is configured.

Where is the documentation how to use vaadin 14+ behind a company proxy?

PS: I’m using the frontend plugin to download node/npm into the project-directory

Ciao
Andreas

14

Hello, here is an example and instructions for setting up npm for Vaadin 14 behind a proxy
https://github.com/Artur-/docker-nexus-npm-registry

15

Hello

for us the problem was something with the company proxy.
After we changed the windows domain and started using our Nexus for npm repository too (http://x.y.z/repository/npmjs-org/), the problem is not coming.

But the problem with “npm behind proxy” is still there I thing. The timeouts are bad handled, or something else.

Bye

16

It sounds like Vaadin is becoming unusable because of this webpack. If build at the dev machine takes more than 10 seconds, it is useless.

Why vaadin from a nice simple framework became this huge complex thingy?