How To Set Your Pants On Fire
Notes from the panel with Finck, Seeborg, and Veen. The presentation is available online.
F: What is the difference between qualitative and quantitative design? The role of the full picture versus thin-slicing presents a number of unique questions for web design.
S: There are so many tools to measure traffic and the language associated with the data make it very hard to complete a useful analysis. We need to know why we look at the metrics.
V: With years of research moved from qualitative to quantitative. You will be fitting your design to the data. Instead get beyond that and anticipate mistake by designing for why people are already coming to your site.
This pseudo-ethnographic approach leads to brilliant sites like USDA Hay Net. The user's identity and goals are immediately evident on the home page.
The wrong side of this coin are sites that try to be portals when that isn't what they are for.
Talk to people. Write what they say. Highlight the interesting things. Transfer to sticky notes. Map stuff that we are told with stuff that we have.
Use techniques to get close to the user.
F: Although I don't use stickies, you have to learn about the user. Watch people use it and see what they do. Become a user yourself and force yourself to use it.
Architect for the purpose. Use task based navigation and take your time to get there. Suffer fools and stick with what your qualitative research teaches you.
S: Find out about the money. How is money made on a project? What is the primary objective of the design. If you don't ask what the problem is you won't solve it.
V: Marketers think that server numbers have something to do with site usage. It doesn't tell you that much.
Think about what makes a blog good? Veen's wife's seminary blog get 100 page views per month. Think about Kotke. What is the comparison?
S: Find out a baseline. Look at page views, visitors per days, etc. to get a sense of where you are.
What applies to each limited domain and content structure may not apply to other genres.
S: Genre matters!
V: Why do research? To cut down the possibilities and get to a better solution. Go back to the drawing board iteratively until you get to launch. The more research you do, the faster alternatives vanish.
Now think about cost to change your mind. It accelerates at the point where you should already be eliminating alternatives.
S: Survey monkey let's you do cool stuff.
F: You cannot get what people really think from a response tool. Pay attention to what they say they want and what they show you they want.
S: But you don't want to start analyzing trivia. Think of Gladwell. Winnow down to what really matters.
Do you care who comes to you from the Cameroon?
F: On many projects there is a gap between the need of the project and the need of the users. How do you identify disparate needs to come to what is critical?
V: To identify user needs you ask them, but you do it the right way. Pick apart the goals and find specific tasks that can go on the web.
How do you match these with what a business needs to make some money?
F: Say you have a product. How do you take data and turn it to ROI and stuff?
V: Look at the Blogger registration interface redesign. Traffic went up but registrations didn't. Find where to stop the bleeding, where are you losing people.
S: If you aren't selling products this changes things a little bit. The value assigned to a particular metric is dependent on the context. If your goals are met the project is a success and the metric you used to find out has value.
V: A/B testing doesn't necessarily tell you what to do.
S: There is no metric that solves all of your problems and these are expensive decisions that you want to be able to make well.
Go online and look at a new or current client to get a basic idea of the problem scope. You can't plan that, just watch and figure out what is happening. Use data to look for exceptions.
V: Use quick wins to feel good and to build credibility.
At HotWired, after the Lycos buyout, there was suddenly technology that allowed you to watch people's eyes. But it didn't do things to make the site better.
Use simple tools and simple process to get the 80% solution. That is probably true about most enterprise packages.
Trust your gut and if something doesn't feel right look for more data and better solutions.
S: You have a sense of your audience and trust that to tell you what you are trying to do.
[silly question about billing, irrelevant]
V: Research should not be an event, it should be the culture.
[talk about research cost]
[talk about the unreliability of RSS tracking]
Who is this guy?
Leave a comment