A short primer for media and politicians on healthcare.gov metrics

kitty count2The other day I had to roll my eyes again at the TV screen when yet another reporter covering the federal Affordable Care Act website (healthcare.gov) did something that drove me crazy.

Many reporters — and politicians — simply don’t know how to use the terminology of website analytics. They make understandable but crucial mistakes that only help to confuse matters. I heard one reporter say that the federal website had reported more than 100,000 “visitors” and then in the next moment a graphic went up showing the 100,000+ “visits” reported by the HHS, which is not the same thing. I’m not sure who got which information wrong — but the point still sticks. There’s a lot of confusion about common website metrics terminology.

This wasn’t t he first time I’d heard, read or seen a mistake like this in reporting on the federal ACA website.

It is perfectly understandable, however, that media and politician are not familiar with web analytics and terminology. It is not always intuitive and it can get very complex. However … when reporting on a national story, it would help if everyone knew the key definitions that matter.

What follows are some key points about website metrics (and definitions) that may help shed some light on the matter.

“Visits” do not equal “Visitors.  This is an easy mistake to make, but getting this wrong will really distort understanding of website performance. A Visit is just another way of saying a session — or the number of times a visitor comes to your site, which expires after inactivity or when leaving the site. Let’s use the convention analogy: the Visit is just the number of times visitors come through the turnstiles to enter the convention center. They may visit and look around, go to lunch, come back, go to their hotel room, and come back on day two of the convention. That’s three visits — three clicks of the turnstile. So yes, one person may do that many times, so “Visit” does not equal “Visitor” in the sense that it is measuring people.

“People” and “Unique Visitors.” So what everyone really wants to know is how many real people visit a site. The closest (though imperfect) way to count this is by counting the number of Unique Visitors. This is the count of unduplicated visitors to a website, over a selected period of time. Google calculates this, for example, by looking at cookies and their match to browsers. You can see the problem there … if the same person erases cookies or switches browsers or devices (like using your smart phone or iPad) you get multiple counts from a single “visitor.”  Some experts estimate “Unique visitor” count is artificially high and can be off by 25% to 39%. Using the convention analogy again, Unique Visitor is the number of badges you give out to approved attendees. They may get their badge scanned many times for entry to the event … but you only made one badge.

Pageviews  When trying to determine overall popularity of any website or portion of a website, you’ll want to know the Pageviews. This metric is fairly intuitive — it is literally how many web pages are served to a browser, and the count is added up over time. If you visit the home page and go to another page, then back to the home page, then to another page, then back to the home page, that’s three pageviews of the homepage. Pageview metrics are great for their overall count, but where they really count is in determining interest among all the sections and pages of content of a website. An example: Let’s say a site gets one million pageviews. “Great!” you say. Then you find out that of those one million pageviews, 600,000 were in the “Help” section and the “I don’t know where to start” page … then it’s not so good news, is it?

The convention center analogy doesn’t work so great for this metric, but you can think of every page of your site as a booth at a convention, and this counts the number of times people looked at a booth (see, I told you the analogy wasn’t that good).

Bounce Rate Maybe I missed it, but I have yet to hear or read anyone asking about the ACA website’s bounce rate — which is important. It’s important to all sites as a broad metric, and it is especially important as a “red flag” indicator. Understanding Bounce Rate is also important as a “reality check” on all metrics.

So what is Bounce Rate? It’s a single visit without any interaction. In other words “We came, we saw one page, we left.” Most people would be shocked to find out just how high bounce rates are for any website. I’ve seen sites that get extremely healthy traffic — and have seen Bounce Rates of around or close to 50% … yes, meaning half of all visits are one-page-and-gone. If a website is having bad technical problems, it should have a Bounce Rate far higher than 50%. When those problems get solved, it should go down. Again, this is a metric best observed over time. As a stand alone number, it’s not so important. The question is — what’s influencing the Bounce Rate (up or down)?

For the question of form completions: Any website can use multiple forms — form completion can go by many names, but “form completion” is the best, IMHO. I saw in the most recent data released by Health and Human Services that they referred to forms completed. That is a sterile but accurate and appropriate way to judge a website’s performance as well. What media and lawmakers should be asking is how many of those forms completed have led to actual coverage — that’s what everyone is wondering.

But from what I understand how the site work, insurance companies will ultimately be granting the coverage — so that is technically not a standard metric to be gathered from live data — it can only be compiled after input from insurance companies. Forms NOT completed is an interesting question too — which may give everyone an idea of the abandonment of effort. Just my two cents.

Are there other metrics that the media and politicians should be watching?

* Original image credit: By David Corby Edited by: Arad (Image:Kittyplya03042006.JPG) [GFDL, CC-BY-SA-3.0 or CC-BY-2.5], via Wikimedia Commons

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *