Over on Usability.com there is an article about measuring the impact of a “findability” initiative: What’s in a Metric? Well, It Depends.
Ah, yes, the UX classic non-answer – it depends. And of course it does. The business is asking for a measurement of value, what are we getting for our investment in findability? This is a question we should all expect, and as budgets continue to be tight and business run leaner and leaner, it is one that is being asked in all areas of the business.
It is an interesting article and worth a read. It covers one perspective on how to look at this problem and what to think about when planning out your measurement strategy.
- Whose perspective are you measuring?
- What kind of thing are you measuring – actions or perceptions?
- What else is happening at the same time that may confound your data?
- What are the direct and indirect impacts of the design work and other changes in your site?
All valid questions and good consideration points. It is certainly on point regarding the context of the metrics you measure and interpret, as well as the difference between attitudinal and behavioral metrics. What do people actually do, and how do they feel about it?
In term of attitudes vs behaviors, it is important to remember that both are important to a brand’s overall success – attitudes are an essential component to overall loyalty, and attitudes color the user’s perception of the efficacy of the site.
About controlling variables in the site – this means how you go about determining if your design change is the actual explanation for changes in measured behaviors.
Two ways to try and control that:
- Use A/B testing to run the new design and the old design simultaneously, that way the other variables impacts are minimized or negated.
- Take the design into a controlled usability test. It becomes a qualitative test at that point, but you can control the conditions of the test and isolate the design change.
In the real world, you may use a combination of both to assess the efficacy of your design enhancements, and as always, be mindful about how you measure and how you interpret your metrics.
We will always be asked to show proof. Is what we are doing valuable and getting results? We need to measure our work, and connect it back to what the business values – revenue, cost, reputation. Then we can stand up and say that our work is truly working for the organization. Not only is doing good design good for the users and the right thing to do – but it is good for the organization as well.
And you can take that to the bank.


