Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

Share this Page URL

12.3 Multimodality, Cognition and Perfor... > 12.3.2 Cognitive Load and Performanc... - Pg. 238

238 PART | III Multimodal Human­Computer and Human-to-Human Interaction eye contact with our dialogue partners, who also perceive, interpret and understand the protocols of conversational dialogue [29, 30]. Although speech and gesture channels do not always convey the same information, it is always semantically and pragmatically com- patible [30]. The most prevalent and widely accepted theory on the relationship between these modes of communication sees speech and gesture production as an integrated process, generated from a common underlying mental representation. Both modes are therefore considered to be equally functional in creating communicative mean- ing [29]. In this way, human communication can be seen to exploit our natural ability to easily process and produce multimodal information. Such a principle can be directly applied to the design and implemen- tation of multimodal interfaces by allowing users to make flexible use of the entire gamut of modal productions (e.g., gaze, gesture and speech). 12.3.2 Cognitive Load and Performance The relationship between working memory and performance is explored by Sweller's Cognitive load theory. The theory is driven by empirical observations of how well people are able to learn from different stimulus materials and corresponding hypotheses based on well-established modal models of working memory architectures such as Baddeley's model [16] (see Section 12.4.1 for details). Cognitive load is a concept that attempts to describe the experience of mental demand, adding an interesting dimension to performance assessment. The theory rests on the assumption that working memory is limited in capacity [31] and duration [32]. Tasks with very high or very low levels of cognitive load can severely impact a subject's performance: if too high, the subject will not have sufficient resources to perform well, and if too low, there is a chance that the subject is not being cognitively engaged in an optimal way [33]. Hence, effective use of working memory processing is vital for achieving successful knowledge transfer. Subjects exhibiting similar levels of performance also may differ in their individual cognitive load experience. The theory describes three types of cognitive load that contribute to mental demand [34]: