There have been blog posts by others about why Google Analytics (GA) has a Weighted Sort (WS) and the Estimated True Value (ETV) algorithm that GA uses to perform Weighted Sort. Even so, there are several things that remain which, as a leading user experience and web analytics agency, we get questions about all the time, so we will address those here. Those questions are:

 

      1. What is the “English” interpretation of a list that has been ordered according to Weighted Sort?
      2. Since Google Analytics doesn’t actually show you the ETV, should one generate the ETV on one’s own, and if so, when and why?
      3. Since the Google Analytics Weighted Sort, i.e. the ETV algorithm, takes into account ALL of the data in the table to be sorted, how sensitive is the algorithm to exactly what data is in the table?

 

 

Before we get into answering the questions above, let’s do a short recap of WS and ETV.

Let’s say you’re looking at a table of device types and their respective bounce rates so that you can identify which devices may have issues viewing your site. Below is such a table, and you can see that the average bounce rate for all devices is 51.98%. Note that there are several hundred rows of data in the table, but for simplicity, we’re just showing the first ten rows here.

 

data-from-google-analytics
The next thing you would naturally do is sort the data by the bounce rate column as shown below. Which, as you can tell, is fairly useless because while it does show devices with 100% bounce rates, most of those devices have such a small number of sessions associated with them that you have no confidence that the data on those devices is statistically significant.

 

case-study-weighted-shift
Now you decide to use Google’s Weighted Sort, located just above the table as shown in the following screenshot.

 

screenshot-of-table
 

Which yields the following ordering.

 

yield-from-ga-data
 

The ordering that Google Analytics’ Weighted Sort produced is based on calculating the Estimated True Value (ETV) and using the ETV as a score on which to sort. The ETV computation is performed for each row of data as follows:

ETVrow x = (F2 / F2 max * F1) + ((1 – F2 / F2 max)) * F1 avg)

F1 = Bounce Rate for the row (i.e. the “factor” you want to weight)

F2 = Sessions for the row (i.e. the “factor” that will modify the significance of F1)

F2 max = The highest number of sessions found in any row of the table (i.e. the maximum value of F2)

F1 avg = The Bounce Rate for the table (i.e. the average of F1)

 

For some extra clarity, here is the equation, but with the variables replaced with Bounce Rate and Sessions.

 

ETVrow x = (Sessionsrow x / Sessions row with the most sessions * Bounce Raterow x) + ((1 – Sessionsrow x / Sessions row with most sessions)) * Bounce Rateentire table)

 

Question 1: What is the “English” interpretation of a list that has been ordered according to Weighted Sort?

Here are the two components to the ETV calculation.

 

The first is:  (Sessionsrow x / Sessions row with the most sessions * Bounce Raterow x)

 

And the second is: ((1 – Sessionsrow x / Sessions row with most sessions)) * Bounce Rateentire table)

 

The first part says, “Weight the Bounce Rate for this row (i.e. Bounce Raterow x) by how important this row’s Bounce Rate is as determined by the ratio of the number of sessions in this row compared to the row with the highest number of sessions (i.e. Sessionsrow x / Sessions row with the most sessions).”

The second part says to compute how heavily to weight the Bounce Rate for the entire table. Specifically, “Weight the Bounce Rate for the entire table (i.e. Bounce Rateentire table) by how important the entire table’s Bounce Rate is as determined by one minus the ratio of the number of sessions in this row compared to the row with the highest number of sessions (i.e. 1 – Sessionsrow x / Sessions row with the most sessions).”

 

The ETV calculation essentially says, “I need to figure out whether the row of data that I’m looking at is a large or a small percentage of the data in the whole table. If it’s a large percentage, then I need to take this row seriously in the sense that the data is probably meaningful. If it’s a small percentage, then I shouldn’t take this row seriously, and the reality is that I’d probably be more ‘correct’ if I just leaned towards using the average Bounce Rate for the whole table. So the ETV calculation has two parts to it. One that computes how heavily to weight the Bounce Rate that is shown in the row, and another that computes how heavily to weight the average Bounce Rate for the entire table. And, the more I weight the Bounce Rate for the row, the less I weight the Bounce Rate for the entire table; and vice versa.”

Question 2: Since Google Analytics doesn’t actually show you the ETV, should one generate the ETV on one’s own, and if so, when and why?

 

—AND—

Question 3: Since the Google Analytics Weighted Sort, i.e. the ETV algorithm, takes into account ALL of the data in the table to be sorted, how sensitive is the algorithm to exactly what data is in the table?

You may indeed benefit from generating the ETV values on your own since Google Analytics does not provide them. Let’s look at some data to see why.

 

The data we were looking at earlier had 992 rows. We’ve gone ahead and computed the ETV for each of those rows, and have shown the first 15 rows of data here sorted by ETV. Looking at these rows, you can see that the first row shows that the Estimated True Value of the Bounce Rate on the Apple iPhone is 58.59% and that the next 14 rows of data all have ETVs of around 52%. Looking at that, you would correctly note that rows 2 through 14 are essentially the same with respect to their Estimated True Bounce Rates, i.e. they range from 52.30% to 52.09%.

 

data-mobile-device-table
 

However, take a look at what happens when we remove the Apple iPhone data from that table and recalculate ETVs for rows 2 through 14. You’ll see that while the rows are close together in terms of Estimate True Bounce Rate, i.e. 48.81% to 48.52%, the estimate is now in the 48% range as opposed to the 52% range.

 

info-mobile-data
 

If you had all of the 922 rows of data in front of you and scrolled all the way down, you would see this.

 

data-ipad-ga
 

We’ll go ahead and remove that row as well and recalculate the ETV and show the top 15 rows again ranked by ETV. You would now see this and think that the Estimate True Bounce Rates are more like in the 55% to 57% range. The bottom line is that the ETV calculations are sensitive to rows that have very large numbers of sessions.

 

mobile-session-analytics
 

So, yes you may want to calculate the ETVs if for no other reason than to get a sense of how big the Estimated True Bounce Rate is because just having the rank order of importance that Google Analytics gives you doesn’t say how “relatively” important the row is. You should also be aware that large rows skew the data.