- The cost of downloading the file on the network
- The cost of parsing and compiling the uncompressed file once downloaded
- The memory cost
I see a lot of very heavy sites using them, but then, my perspective is very biased as the companies that I work with work with me precisely because they are facing performance challenges. I was curious just how common the situation is and just how much of a penalty we’re paying when we make these frameworks the default starting point.
Thanks to HTTP Archive, we can figure that out.
In total, HTTP Archive tracks 4,308,655 desktop URLs, and 5,484,239 mobile URLs. Among the many data points HTTP Archive reports for those URLs is a list of the detected technologies for a given site. That means we can pick out the thousands of sites that use various frameworks and see how much code they’re shipping, and what that costs the CPU.
I ran all the queries against March of 2020, the most recent run at the time.
I decided to compare the aggregate HTTP Archive data for all sites recorded against sites with React, Vue.js, and Angular detected1.
|FRAMEWORK||MOBILE URLS||DESKTOP URLS|
Hopes and dreams
Before we dig in, here’s what I would hope.
In an ideal world, I believe a framework should go beyond developer experience value and provide concrete value for the people using our sites. Performance is just one part of that—accessibility and security both come to mind as well—but it’s an essential part.
So in an ideal world, a framework makes it easier to perform well by either providing a better starting point or providing constraints and characteristics that make it hard to build something that doesn’t perform well.
The best of frameworks would do both: provide a better starting point and help to restrict how out of hands things can get.
Looking at the median for our data isn’t going to tell us that, and in fact leaves a ton of information out. Instead, for each stat, I pulled the following percentiles: the 10th, 25th, 50th (the median), 75th, and 90th.
The 10th and 90th percentiles are particularly interesting to me. The 10th percentile represents the best of class (or at least, reasonably close to the best of class) for a given framework. In other words, only 10% of all sites using a given framework reach that mark or better. The 90th percentile, on the other hand, is the opposite of the spectrum—it shows us how bad things can get. The 90th percentile represents the long-tail—that last 10% of sites with the highest number of bytes or largest amount of main thread time.
|Sites with jQuery||110.3kb||219.8kb||430.4kb|
|Sites with Vue.js||244.7kb||409.3kb||692.1kb|
|Sites with Angular||445.1kb||675.6kb||1,066.4kb|
|Sites with React||345.8kb||441.6kb||690.3kb|
I mentioned earlier that even if the starting point is a little off, I would hope that a framework could still provide value by limiting the upper bound in some way.
The same can’t be said of the other frameworks.
Just as with the 10th percentile, Angular and React driven sites tend to distance themselves from others at the 90th percentile, and not in a very flattering way.
It’s clear from the data that sites with these frameworks in place tend to pay a large penalty in terms of bytes. But of course, that’s just one part of the equation.
|Sites with jQuery||199.6ms||399.2ms||877.5ms|
|Sites with Vue.js||350.4ms||650.8ms||1,280.7ms|
|Sites with Angular||482.2ms||777.9ms||1,365.5ms|
|Sites with React||508.0ms||1,045.6ms||2,121.1ms|
There are some very familiar themes here.
Those percentages look much better than at the 10th percentile, but keep in mind that the bulk numbers are pretty scary: that’s 20.8s of main thread work for sites built with React at the 90th percentile on mobile devices. (What, exactly, is happening during that time is a topic for a follow-up post, I think.)
There’s one potential gotcha (thanks Jeremy for making sure I double-checked the stats from this angle)—many sites will pull in multiple libraries. In particular, I see a lot of sites pulling jQuery in alongside React or Vue.js as they’re migrating to that architecture. So, I re-ran the queries, only this time I only included URLs that included only React, jQuery, Angular or Vue.js not some combination of them.
I am sorry that this post was not useful for you!
Let me improve this post!
Tell me how i can improve this post?