Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.2k
Closed
Labels
Description
So at@brigade our build times are pretty slow, and I can't seem to figure out why this is. We use Babel, SCSS+autoprefixer+css-loader+style-loader, react+react-hot-loader, CommonsChunkPlugin, DefinePlugin, webpack-dev-server, devtool:'eval-cheap-module-source-map', and that's pretty much it. I posted a typical version of our config at the end.
I initially started investigating the incremental build time, since I run into that most often. These are the things I tried that didn't help:
- Don't use context.
- Use
includein config to remove large directories from path (we already usedexcludein some places, so this didn't help) - Scoping react-hot-loader only to components directory.
- Remove hash from css filenames.
Here are some things thatdid help:
- Disabling source maps altogether (large effect on emit and creating chunk assets).
- Pulling more modules into the common chunk (mostly affected emit, but this was trumped by disabling source maps altogether; no compound effect).
Most of the time for incremental builds was still inbuild modules though, I could only affectcreate chunk assets andemit this way. Then I tried to tackle the initial build, assuming there must be some sort of more structural flaw in our codebase. First I reset the codebase back to the way it was before I played with incremental builds. Then my methodology was to then try something, then run webpack 5 times and save that total time, and also save the stats from one of the runs to get some insight. I didn't reset the code base in between builds, but tried to compound improvements to get the time down as much as possible. Here's the result of that investigation:
Baseline:43748ms build modules110ms seal144ms optimize244ms hashing3621ms create chunk assets12ms additional chunk assets0ms optimize chunk assets1ms optimize assets1203ms emit./profile.sh 244.24s user 10.78s system 99% cpu 4:16.90 totalRemoving load postfixes (see https://github.com/elastic/kibana/blob/c9c9e565e6f744ba3fbc959cd0ac81a7b28a476d/src/optimize/BaseOptimizer.js and https://github.com/webpack/webpack/issues/24):43254ms build modules94ms seal131ms optimize223ms hashing3310ms create chunk assets10ms additional chunk assets0ms optimize chunk assets1ms optimize assets1331ms emit./profile.sh 243.28s user 10.63s system 99% cpu 4:15.42 totalNO DIFFERENCEMore messing around with resolve params:41777ms build modules104ms seal143ms optimize243ms hashing3010ms create chunk assets12ms additional chunk assets0ms optimize chunk assets1ms optimize assets1620ms emit./profile.sh 239.57s user 10.91s system 96% cpu 4:20.16 totalNO DIFFERENCEReducing number of modulesDirectories to 5:36639ms build modules125ms seal144ms optimize255ms hashing3878ms create chunk assets17ms additional chunk assets0ms optimize chunk assets1ms optimize assets857ms emit./profile.sh 212.82s user 9.79s system 92% cpu 4:00.07 totalHELPS MAYBE?Reducing number of modulesDirectories to 3:42700ms build modules92ms seal246ms optimize174ms hashing2650ms create chunk assets14ms additional chunk assets0ms optimize chunk assets0ms optimize assets1036ms emit./profile.sh 218.80s user 9.69s system 97% cpu 3:53.51 totalHELPS MAYBE?Include components_loader.js regex in webpack config:47551ms build modules69ms seal90ms optimize209ms hashing2509ms create chunk assets8ms additional chunk assets1ms optimize chunk assets0ms optimize assets1475ms emit./profile.sh 221.16s user 9.72s system 96% cpu 3:58.70 totalNO DIFFERENCE?Disable SCSS source maps:30114ms build modules102ms seal124ms optimize205ms hashing2897ms create chunk assets14ms additional chunk assets0ms optimize chunk assets1ms optimize assets860ms emit./profile.sh 196.68s user 8.60s system 100% cpu 3:25.27 totalSOME DIFFERENCE! (same as with the incremental build)Disable source maps altogether:37089ms build modules105ms seal126ms optimize239ms hashing467ms create chunk assets13ms additional chunk assets0ms optimize chunk assets1ms optimize assets510ms emit./profile.sh 198.85s user 9.73s system 97% cpu 3:34.86 totalNO DIFFERENCE?Extracting more code into a common chunk:35174ms build modules81ms seal168ms optimize91ms hashing195ms create chunk assets16ms additional chunk assets0ms optimize chunk assets1ms optimize assets337ms emit./profile.sh 182.06s user 8.63s system 97% cpu 3:16.42 totalHELPS MAYBE?De-duplicate core-js:35300ms build modules116ms seal217ms optimize123ms hashing234ms create chunk assets34ms additional chunk assets0ms optimize chunk assets1ms optimize assets270ms emit./profile.sh 189.18s user 8.86s system 98% cpu 3:21.99 totalNO DIFFERENCE?Use React.js binary:40170ms build modules116ms seal222ms optimize122ms hashing270ms create chunk assets13ms additional chunk assets0ms optimize chunk assets1ms optimize assets302ms emit./profile.sh 185.07s user 8.53s system 99% cpu 3:15.09 totalNO DIFFERENCE?So the only new thing I tried that helped was reducing the number of modulesDirectories.
Some more information. Here is what a typical incremental build looks like (with hot loading enabled), after all the changes above:
[webpack] webpack: bundle is now INVALID.3657ms build modules 83ms seal1247ms optimize147ms hashing276ms create chunk assets156ms additional chunk assets0ms optimize chunk assets 0ms optimize assets 196ms emit[webpack] Hash: b91dce65a3b67a9d69d9[webpack] Version: webpack 1.12.2[webpack] Time: 5805ms[webpack] Asset Size Chunks Chunk Names[webpack] intl_polyfill.js 1.17 MB 4 [emitted] intl_polyfill[webpack] jquery_fallback.js 1.26 MB 5 [emitted] jquery_fallback[webpack] vendor.js 6.09 MB 8 [emitted] vendor[webpack] 8.a5cd34b7ba7171bb6548.hot-update.js 2.4 kB 8 [emitted] vendor[webpack] a5cd34b7ba7171bb6548.hot-update.json 36 bytes [emitted] [webpack] chunk {0} activity_map.js (activity_map) 188 kB {8}[webpack] + 7 hidden modules[webpack] chunk {1} admin.js (admin) 141 kB {2}[webpack] + 36 hidden modules[webpack] chunk {2} application.js (application) 69.2 kB {8}[webpack] + 24 hidden modules[webpack] chunk {3} diffux_ci.js (diffux_ci) 295 kB {8}[webpack] [1829] ./app/assets/components -diffux\.jsx$ 5.12 kB {3} [built][webpack] ... -> factory:3ms building:0ms[webpack] + 116 hidden modules[webpack] chunk {4} intl_polyfill.js (intl_polyfill) 1.12 MB [rendered][webpack] + 74 hidden modules[webpack] chunk {5} jquery_fallback.js (jquery_fallback) 1.21 MB [rendered][webpack] + 69 hidden modules[webpack] chunk {6} partners.js (partners) 543 kB {2}[webpack] + 199 hidden modules[webpack] chunk {7} specs.js (specs) 1.67 MB {8}[webpack] [2201] ./app/assets/components -test\.jsx$ 8.84 kB {7} [built][webpack] ... -> factory:9ms building:0ms[webpack] + 358 hidden modules[webpack] chunk {8} vendor.js, 8.a5cd34b7ba7171bb6548.hot-update.js (vendor) 6.08 MB [rendered][webpack] [584] ./app/assets/components/CardButtons/index.jsx 2.35 kB {8} [built][webpack] ... -> factory:1ms building:0ms dependencies:1ms[webpack] + 1768 hidden modules[webpack] webpack: bundle is now VALID.This is a simplified but representative version of webpack.config.js that I ended up with:
require('babel-core/polyfill');varwebpack=require('webpack');module.exports={entry:{activity_map:'./app/assets/_activity_map.js',admin:'./app/assets/_admin.js',application:['webpack-dev-server/client?http://localhost:8080','./app/assets/_application.js'],intl_polyfill:'./app/assets/_intl_polyfill.js',jquery_fallback:'./app/assets/_jquery_fallback.js',partners:'./app/assets/_partners.js',vendor:'./app/assets/_vendor.js',specs:['webpack-dev-server/client?http://localhost:8080','./app/assets/spec/_specs.js'],diffux_ci:'./app/assets/spec/_diffux_ci.js',},output:{path:__dirname+'/public/assets',filename:'[name].js',publicPath:'http://localhost:8080/assets'},resolve:{modulesDirectories:['app/assets','vendor/assets/bower_components','node_modules',],extensions:['.jsx','.js','',],loaderExtensions:['.js',''],loaderPostfixes:[''],unsafeCache:true,postfixes:[''],alias:{'core-js':__dirname+'/node_modules/babel-runtime/node_modules/core-js','react$':__dirname+'/node_modules/react/dist/react-with-addons.js',},},// Disable source maps for now// devtool: 'eval-cheap-module-source-map',plugins:[newwebpack.DefinePlugin({// Some definitions here}),newwebpack.optimize.CommonsChunkPlugin({name:'application',minChunks:2,chunks:['admin','partners']}),newwebpack.optimize.CommonsChunkPlugin({name:'vendor',minChunks:2,chunks:['application','diffux_ci','specs','activity_map'],}),],externals:{jquery:'jQuery',},module:{loaders:[{test:/\.css$/,loader:'style!css?localIdentName=[path][name]--[local]--[hash:base64:10]!autoprefixer',include:[__dirname+'/app/assets',__dirname+'/node_modules/normalize.css',],},{test:/\.scss$/,loader:'style!css?localIdentName=[path][name]--[local]--[hash:base64:10]!autoprefixer!sass',include:__dirname+'/app/assets',},{test:/\.jsx?$/,loaders:['babel-loader?cacheDirectory','react-hot'],exclude:[/node_modules/,/bower_components/,/support\/fixtures/],include:__dirname+'/app/assets',},{test:/app\/assets\/components\/(.*?)(?:\/index)?\.jsx/,loader:__dirname+'/app/assets/components_loader',include:__dirname+'/app/assets/components',},{test:/raven-js/,loaders:['imports?this=>window'],include:__dirname+'/node_modules/raven-js',},{test:/\.(gif|jpe?g|png|svg)$/,loader:'file-loader',include:__dirname+'/app/assets',},{test:/\.json$/,loader:'json-loader',include:__dirname+'/app/assets',}],noParse:[/acorn\/dist\/acorn\.js$/,/underscore\/underscore\.js$/,/react-with-addons\.js$/,]},profile:true,stats:{hash:true,version:true,timings:true,assets:true,chunks:true,modules:true,reasons:true,children:true,source:false,errors:true,errorDetails:true,warnings:true,publicPath:true},};
The components_loader is just something small that we added for exposing components so we can more easily include them in our style guide:
module.exports=function(content){if(this.cacheable){this.cacheable();}constmatches=this.resourcePath.match(/components\/(.*?)(?:\/index)?\.jsx/);if(!matches){returncontent;}returncontent+'\n'+'window.ReactComponents = window.ReactComponents || {};\n'+"ReactComponents['"+matches[1]+"'] = module.exports;\n";};
The full stats.json can be found here:https://gist.github.com/d8feb05513656472dbf5
I really hope someone can help me with this, I've been stuck on this issue for two days now. Maybe if we find a solution it would help others as well, because I think our React/Babel/SCSS setup is pretty common. Thanks!!