Behavioural mismatches in human-computer interaction can be traced back to mismatches between the user's intention to do a dialogue step and the designer's intention to make a dialogue step executable. Mismatches are indicated by user complaints when selecting an inappropriate menu option, or being puzzled by a system's reaction, or unsuccessfully searching for an expected menu option. Mismatches, often referred to as user errors, indicate that something is wrong with the usability of the system. For measuring the usability of software products or improving a prototype, appropriate criteria are needed that allow a valid assessment.

Human-computer interaction is a behaviour sequence that can be described in terms of user actions and system reactions. A number of models and languages have been developed to provide for a complete description of interaction. Commonly known models allow empirical measurements in terms of user performance time, number of elementary user actions, number of keystrokes. Examples are GOMS [1], CLG [5] and TAG [6]. With the aid of such measurable entities one is able to compare software products regarding the key criterion of usability which is user efficiency. But there are some disadvantages. First, we get only relational quality assessments, that is, we can only say a system is better than an other one regarding the time it takes an expert to accomplish a certain task without errors. Second, the concept of usability as defined in ISO 9241-11 (1998) implies that the main indicator indicator of user efficiency is that the user is able to reach an intended goal. The absolute quality of a software product then depends on the extent to which the system's behaviour matches with the intentions of the user. In order to measure intentions, we can also interpret the system's behaviour as intentional if we take into account the designer's intention in deciding for a specific system feature of behaviour. If there is a gap or mismatch between the user's intention and the system's intention, the software reveals a usability defect.

The background of an intention-based coding of interactive behaviour is the theory of human action regulation [3]. A basic concept of this theory is the notion of "complete task", elements of which are preparatory action, exploratory action, conduct of intermediary steps, error management, assessment of intermediary or final results, etc. Starting from these rather abstract elements of a task structure one can develop a refined intention-based coding scheme for analyzing the interaction.

We are especially interested in the "critical incidents" of interaction, that is, a situation in the flow of dialogue steps, which (possibly) leads to problems regarding the user's task performance. In a field setting we want to record the user's dialogue with the system by means of an established analysis scheme, called use scenario, in order to define the appropriate test criteria [2]. In practice, we do not need to code the entire sequence of dialogue steps when using, for instance, an observational laboratory, since the focus of investigation is on critical incidents. If such an incident is indicated, an intention-based coding scheme of interactive behaviour is expected to help measure intentional mismatches. The coding scheme needs to be refined to an extent that enables valid measurements. Examples for a refined scheme of user behaviour are opportunistic performance, searching for information, carrying out procedures, etc. [7]. The designer's intentions can be inferred from the rationale of design decisions.

References

1. Card, S.K.; Moran, T.P.; Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum.
2. Dzida, W.; Freitag, R. (1998). Making use of scenarios for validating analysis and design. IEEE Transactions on Software Engineering, 24, 1182-1196.
3. Hacker, W. (1985). Activity: A fruitful concept in industrial psychology. In: M. Frese and J. Sabini (Eds.). Goal Directed Behavior: The Concept of Action in Psychology. Hillsdale, NJ: Lawrence Erlbaum.
4. ISO 9241-11 (1998). Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on Usability.
5. Moran, T.P. (1981). The command language grammar: A representation for the user interface of interactive computer systems. International Journal of Man-Machine Studies, 15, 3-50.
6. Payne, S.J. (1984). Task-action grammars. Proceedings of INTERACT '84. IFIP Conference on Human-Computer Interaction, 139-144.
7. Rosson, M.B.; Carroll, J.M. (1995). Narrowing the specification-implementation gap in scenario-based design. In: J.M. Carroll (Ed.): Scenario-Based Design. Envisioning Work and Technology in System Development, 247-278. New York: Wiley.

© 2000 Noldus Information Technology b.v.