Browser reloads infinitely when using multi-compiler mode

[chmanie]

• This is a bug
• This is a modification request

Code

See full example repository for reproduction here

// webpack.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin');
const path = require('path');

module.exports = [
    {
    mode: 'development',
    entry: './temp2.js',
    output: {
      filename: 'main2.js',
      path: path.resolve(__dirname, 'dist'),
    },
        plugins: [
            new HtmlWebpackPlugin(),
        ],
    },
    {
        mode: 'development',
        entry: './temp.js',
      output: {
        filename: 'main1.js',
        path: path.resolve(__dirname, 'dist'),
      },
    },
];

Please paste the results of npx webpack-cli info here, and mention other relevant information

  System:
    OS: macOS 11.5.2
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 5.88 GB / 32.00 GB
  Binaries:
    Node: 16.8.0 - /usr/local/bin/node
    npm: 7.21.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 92.0.4515.159
    Firefox: 91.0.2
    Safari: 14.1.2
  Packages:
    html-webpack-plugin: ^5.3.2 => 5.3.2 
    webpack: ^5.52.0 => 5.52.0 
    webpack-cli: ^4.8.0 => 4.8.0 
    webpack-dev-server: ^4.1.0 => 4.1.0 

Expected Behavior

I'm expecting the page not to reload infinitely. If my configuration is not correct, I would like for webpack to tell me that in some way.

Actual Behavior

The page reloads in an infinite loop. I could extract the console output from Firefox. The following output is the only output in the console and happens before the reload is triggered:

[HMR] Cannot find update. Need to do a full reload! log.js:26:12
[HMR] (Probably because of restarting the webpack-dev-server) log.js:26:12

Important

It is important to note that this behaviour is not triggered in 100% of the cases. For me it roughly triggers in 1/3 of the cases, when using a more complex configuration (in the project I encountered this in), it's almost every time).

For Bugs; How can we reproduce the behavior?

Please see https://github.com/chmanie/webpack-hmr-bug to find information on how to reproduce the bug (it's quite easy).

For Features; What is the motivation and/or use-case for the feature?

[alexander-akait]

Most likely problem in html-webpack-plugin, we do not magic here

[chmanie]

Okay thanks, I was on the edge to post it there instead, but the HMR message brought me here. I'll try my luck over there!

[alexander-akait]

Oh, found a problem

[alexander-akait]

You can't run multiple dev server on multiple compiler with web target, you need two dev server, but it is currently not supported

[alexander-akait]

Problem due HMR logic, we track hash of build but you have two builds, so hash changed when you change something in your another build

[alexander-akait]

Why do you need one dev server on multiple compilers, it is like independent builds in your case, so should be handled independent servers

[chmanie]

Some of my entry points just need to be handled differently (different output options). But for those I don't need another webserver. Is this not a valid use case?

For example, some of my files are libraries, and the main entry point is not.

[alexander-akait]

In this case you should use dependencies https://webpack.js.org/configuration/other-options/#dependencies, also due logic for static files handling and caches, you should always compile library firstly, when build main app, also for this you need two dev server an proxy library dev server, it is independent builds

[DylanPiercey]

This error also seems to surface if you have an isomorphic setup with both a node compiler and a web compiler.

[alexander-akait]

@DylanPiercey

This error also seems to surface if you have an isomorphic setup with both a node compiler and a web compiler.

Can you provide example? It should work

[DylanPiercey]

@alexander-akait ill make a simpler reproduction, but if you want to test quickly our Marko webpack example repo has this issue.

npm init marko -- --template webpack-express. Then npm run dev and edit src/pages/index/index.marko.

I’ll be downgrading this example today, and I’ll also create a more simplified reproduction. Will share once I have

[alexander-akait]

@DylanPiercey

npm init -- --template webpack-express

npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/create---template - Not found
npm ERR! 404 
npm ERR! 404  'create---template@latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:

without --

npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/create-webpack-express - Not found
npm ERR! 404 
npm ERR! 404  'create-webpack-express@latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:

[DylanPiercey]

Sorry, replying with code snippets via mobile is hard -_-. I updated the previous comment with the correct command.

[DylanPiercey]

@alexander-akait here is a more simplified reproduction: https://github.com/DylanPiercey/webpack-dev-server-issue-infinite-reload

[alexander-akait]

I will try to fix multi compiler issues on the next week, also try to add support for multiple dev servers, a lot of tasks, sorry for delay

[alexander-akait]

For me: add test and example for #3836 and for #3831 (https://github.com/koggdal/webpack-dev-server-issue-3831-demo)

[alexander-akait]

Note for everyone:

Three solutions for multi compiler mode:

• start dev server for each configuration, i.e. multiple dev server mode (we are working on it)
• wait fix for multiple web targets support here Browser reloads infinitely when using multi-compiler mode Browser reloads infinitely when using multi-compiler mode #3803, but again, you need to set valid order of dependencies (https://webpack.js.org/configuration/other-options/#dependencies) , if you have independent configurations (for example you build library and then use builded library in your application) you should use multiple dev server and proxy one on top of the other
• migrate on multi entries configuration - it will open up more opportunities for optimization

[koggdal]

Thank you @alexander-akait! 🙏

On the "migrate on multi entries configuration" point, what does this refer to?

[alexander-akait]

@koggdal found solution for your case, regarding migrate on multi entries configuration https://webpack.js.org/concepts/entry-points/#object-syntax

[alexander-akait]

PR: #3840, I am checking it on all reported repos and when add tests cases and do release

[koggdal]

@alexander-akait I think what makes that tricky for us is that the different apps require slight tweaks to the Webpack config (one app might need one more plugin, or different apps need different instances of a plugin with different parameters). But if the multiple web targets support is coming soon, and that works for us (with dependencies set), that would be great! 👍

[alexander-akait]

@koggdal You can test your repo using changes from #3840

[alexander-akait]

@chmanie PR above fix your problem too, one note - if you want to use each build from the each compilation on the same page you need to use uniqueName, i.e.:

module.exports = [
    {
      name: 'one',
      mode: 'development',
      entry: './temp2.js',
      output: {
        uniqueName: 'one',
        filename: 'main2.js',
        path: path.resolve(__dirname, 'dist'),
      },
    },
    {
        name: 'two',
        mode: 'development',
        entry: './temp.js',
        output: {
          uniqueName: 'two',
          filename: 'main1.js',
          path: path.resolve(__dirname, 'dist'),
        },
    },
];

Why? Because we have global HMR function, and if you not set different name, they will be override each, i.e. last HMR function override first and HMR will be broken, different names give different names for HMR function and allow to use different configurations on the same page

[alexander-akait]

Found another problem, but it will be fixed late (today/tomorrow)

[DylanPiercey]

@alexander-akait I don't believe that #3840 or any of your recommendations above would solve the issue of isomorphic apps that have a server and browser compiler.

Is there a plan to support that kind of setup?

FWIW Ideally any build to a server compiler would trigger a live reload to the browser unless every module changed could hot reload in the browser compiler. Currently with webpack-dev-server@3 I don't believe it works that way, but instead just does a full page reload whenever either compiler changes (which is what was broken in webpack-dev-server@4).

[alexander-akait]

@DylanPiercey this is another problem, albeit close to what I wrote here, there are two issues:

• infinity loop for multiple web targets, fix: multi compiler mode with multiple web targets #3840
• web + node doesn't reload the application, but we need to do full reloading Browser reloads infinitely when using multi-compiler mode #3803 (comment)

[alexander-akait]

@DylanPiercey also do you use https://webpack.js.org/configuration/other-options/#dependencies? Otherwise server and client don't known about yourselves

[alexander-akait]

@DylanPiercey I want to clarify that the only problem is the lack of live reload, right?

[DylanPiercey]

@alexander-akait that's correct yes

[DylanPiercey]

After making the changes you had in #3840 it does not appear to infinite loop, but simply doesn't trigger the live reload.

[alexander-akait]

Yes, because there are two different problems, currently I fix infinity loop, becaue it happens not only for your case, the second PR will fix reload logic for your case

[DylanPiercey]

Awesome, looking forward to it! Thanks for looking into this 🙏

[DylanPiercey]

Happy to test out whatever change locally if helpful, just @ me 😄

[DylanPiercey]

@alexander-akait was just playing around with this a little bit, very possible I'm going in the wrong direction. But it seems in my case the issue is due to isInitial stating as true, even with (https://github.com/webpack/webpack-dev-server/pull/3840/files?file-filters%5B%5D=.js&hide-deleted-files=true#diff-dac0f33920a54dfaf6fd4fd53e72e1c1bc07b716a421ea4ac86ae165667ed851R18).

I was able to get things working by changing that code to be

var isInitial = status.previousHash === webpackHash;

Which I think makes sense, but not totally sure.

Another (very minor) issue I see with https://github.com/webpack/webpack-dev-server/pull/3840/files?file-filters%5B%5D=.js&hide-deleted-files=true#diff-b706bbadb3d2315d99678f05e1f4bfd78a4bb4a7a6c65f3e81f1386fee3c883aR1153 is that the compiler.hooks.invalid will fire multiple times with multi compilers (even though done will only fire once). I think perhaps it'd be better to guard this such that it only invalidates the browser when the first compiler is invalidated? Something like this:

  setupHooks() {
    let pending = true;
    this.compiler.hooks.invalid.tap("webpack-dev-server", () => {
      if (!pending && this.webSocketServer) {
        pending = true;
        this.sendMessage(this.webSocketServer.clients, "invalid");
      }
    });
    this.compiler.hooks.done.tap("webpack-dev-server", (stats) => {
      if (this.webSocketServer) {
        this.sendStats(this.webSocketServer.clients, this.getStats(stats));
      }

      this.stats = stats;
      pending = false;
    });

[DylanPiercey]

Whoops, forgot to mention I also added

status.previousHash = status.currentHash;

Before this line: https://github.com/webpack/webpack-dev-server/blob/master/client-src/index.js#L90

[alexander-akait]

@DylanPiercey pending is dirty solution, and break multi compiler mode for independent builds, I will already inform you that configuration wrong due you have depended builds, but don't say webpack about it #3803 (comment), this is what I told you several times

[alexander-akait]

If you have multiple indented builds, for example you have page-1/page-2/page-3, you break reload logic, because dev server will wait all finished builds, it is wrong and incorrect

[DylanPiercey]

@alexander-akait that's not true. With the multi compiler done hook it does already wait for all builds to finish, regardless of the dependencies option, at least in my testing. The invalid hook is the special case here.

[alexander-akait]

But if you change something in one of the configuration why we should wait? If builds will be long you will not get infromation about changes, i.e. logging

[DylanPiercey]

@alexander-akait I'm not following? The multi compiler done is batched here https://github.com/webpack/webpack/blob/main/lib/MultiCompiler.js#L96 it will already wait for all compilers in the multi compiler to complete before calling that hook.

With what I had above it'd notify the browser that a build is happening only when the first compiler goes invalid. Right now it notifies the browser multiple times for the same build. I believe even with the dependencies option it would do that.

[alexander-akait]

You can run invalidate without finish build and no logs in your case, multiple invalid events are fine

[DylanPiercey]

Are you saying the invalid hook can be called without ever firing the done hook?

[alexander-akait]

Yes

[alexander-akait]

What is the problem with multiple invalid messages?

[DylanPiercey]

Ah ok, if that's the case then what I suggested won't work for sure.

The main thing I was trying to avoid was this double log happening in the browser on every reload:

[alexander-akait]

We can fix it just adding isRecompiling check and do not output unnecessary logs

[alexander-akait]

Found a problem, why we do not reload, WIP, maybe do release today

[alexander-akait]

@DylanPiercey I got your idea, because it is valid solution, try #3841, I will test it with lazyCompilation and if all will be good, add more tests and do release

[DylanPiercey]

@alexander-akait with both #3840 and #3841 it seems all is working!

[alexander-akait]

I will add more tests and will merge

[koggdal]

@alexander-akait Sorry for not getting back earlier! I've tested both your changes (together with setting up dependencies), and they do fix the reload issue! 🎉 Thank you! 🙏

However, I get a single "Disconnected" on load, and then it reconnects and then it's fine. The test repo I made for you doesn't seem to have that issue, so something must be different in our project setup. It's really late here now, and I'm likely pretty busy tomorrow, but I will try to see if I can understand what's different in our setup.

[alexander-akait]

@koggdal I think this problem in other place, please open an new issue tomorrow with example, logic for hot/reload not connected in any way with Disconnected, maybe you browser was open too fast and server is not ready yet, so web socket server try to connect, but server is still starting

[alexander-akait]

Please try https://github.com/webpack/webpack-dev-server/releases/tag/v4.2.1

[alexander-akait]

Note - hot is true by default, so if you change something and nothing happens (only logs) mostly you faced with #3794 (fix will be latest, need small discussion)

[DylanPiercey]

@alexander-akait can confirm it's working with the new release. Thanks for getting this out so fast and for the support!