test hypotheses

b&a article: Don't Test Users, Test Hypotheses

When testing websites or applications, I've found that generating hypotheses about user behavior helps inform the observation process, structure data collection and analysis, and organize findings. It also keeps you honest by being explicit about what you are looking for.

The first step is to generate hypotheses. We begin with an extensive review of the site or application. We use relevant heuristics in our review, of course. We ground the review in the site or application's goals and philosophy, as defined by its designers and as documented in project literature (requirements, specifications, creative brief, etc.). This is critical because we want the review to reveal whether the site conforms to, or deviates from, the design. It's important to engage the design team in a dialogue at this point to clarify their goals and articulate their predictions about user behavior. That way, the results of the testing will answer their questions, not ours. (If we are testing our own design, then we use this opportunity to reflect critically on it and identify its key predictions and assumptions.)