User Characteristic Tests

Here we will provide a description of the tests that we often use in our various studies to evaluate the impact of individual differences on HCI tasks.

User CharacteristicDefinition, ReferenceTest SourceTest DurationImpact on visualization processing (our work)Prediction from gaze + interaction data (our work)
Perceptual Speed
(Identical Pictures Test)
Speed in comparing figures or symbols, scanning to find figures or symbols, or carrying out other very simple tasks involving visual perception.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.
Paper Test 1.5 min [2, 4, 7, 8, 9, 10, 12, 13, 14] [1, 5, 6, 11]
Visual Working Memory
Measures the quantity of visual information (e.g., shapes and colours) that can be temporarily maintained and manipulated in working memory.

Vogel, E.K., Woodman, G.F., and Luck, S.J. 2001. Storage of features, conjunctions, and objects in visual working memory. J. Exp. Psychol. Hum. Percept. Perform. 27, 1, 92–114.
Matlab Test 6 mins [2, 4, 7, 8, 9, 10, 12, 13] [1, 5, 6, 11]
Verbal Working Memory
(OSPAN)
Measures the quantity of verbal information (e.g., words) that can be temporarily maintained and manipulated in working memory.

Turner, M.L. and Engle, R.W. 1989. Is working memory capacity task dependent? Journal of Memory and Language. 28, 2, 127–154.
Online Test 10 mins [2, 4, 7, 8, 9, 10, 12, 13, 15] [1, 5, 6, 11]
Visualization Literacy
The ability to confidently use a given data visualization to translate questions specified in the data domain into visual queries in the visual domain, as well as interpreting visual patterns in the visual domain as properties in the data domain.

Boy, J. et al. 2014. A Principled Way of Assessing Visualization Literacy. IEEE Transactions on Visualization and Computer Graphics. 20, 12, 1963–1972.
Online Test 10 mins [2] [1]
Disembedding
(Hidden Patterns Test)
The ability to hold a given visual percept or configuration in mind so as to disembed it from other well defined perceptual material.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.
Paper Test 3 mins [14, 15] N/A
Spatial Memory
(Shape Memory Test)
The ability to remember the configuration, location, and orientation of figural material (e.g., multiple objects on a map).

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.
Paper Test 9 mins [2, 14] [1]
Need for Cognition
Measures the extent to which individuals are inclined towards effortful cognitive activities.

Cacioppo, J.T. et al. 1984. The Efficient Assessment of Need for Cognition. Journal of Personality Assessment. 48, 3, 306–307.
Paper Test 3 mins [2, 14] N/A
Locus of Control
Measure of the degree to which individuals perceive outcomes as either a result from their own behavior, or from forces that are external to themselves.

Rotter, Julian B. 1966. Generalized Expectancies for Internal Versus External Control of Reinforcement. Psych. Monographs: General and Applied 80.1, 1-28.
Paper Test 5 mins [2, 4, 7, 9, 10] [5]
Big-Five Personality Traits
(TIPI)
Measures personality in terms of Agreeableness, Conscientiousness, Extraversion, Emotional Stability, and Openness.

Gosling, S. D., Rentfrow, P. J., and Swann, W. B., Jr. 2003. A Very Brief Measure of the Big Five Personality Domains. Journal of Research in Personality, 37, 504-528.
Paper Test 1 min [3] N/A
Reading Proficiency
(X_Lex)
Measures vocabulary size and reading comprehension ability in English.

Paul Meara. 1992. EFL vocabulary tests, second edition 2010. Lognostics, Swansea: Wales.
Paper Test 5 min [15] coming soon
Verbal IQ
(NAART)
Overall verbal intellectual abilities that measures acquired knowledge, verbal reasoning, and attention to verbal materials.

Esther Strauss, Elisabeth M. S. Sherman, Otfried Spreen, and Otfried Spreen. 2006. A compendium of neuropsychological tests: administration, norms, and commentary. Oxford University Press, Oxford; New York.
Paper Test 5 min N/A N/A

References

Only journal papers and strictly refereed conference papers published by our group are listed.

  1. Conati, C., Lallé, S., Rahman, M.A., and Toker, D. 2017. Further Results on Predicting Cognitive Abilities for Adaptive Visualizations. Proc. of the 26th International Joint Conference on Artificial Intelligence, AAAI, 1568-1574. [link]
  2. Lallé, S., Conati, C., and Carenini, G. 2017. Impact of Individual Differences on User Experience with a Visualization Interface for Public Engagement. Proc. of the 2nd International Workshop on Human Aspects in Adaptive and Personalized Interactive Environments (in conjunction with UMAP 2017), ACM. [link]
  3. Lallé, S., Mudrick, N.V., Taub, M., Grafsgaard, J.F., Conati, C., and Azevedo, R. 2016. Impact of Individual Differences on Affective Reactions to Pedagogical Agents Scaffolding. Proc. of the 16th International Conference on Intelligent Virtual Agents, Springer, 269–282. [Best paper award - link]
  4. Conati, C., Carenini, G., Toker, D., and Lallé, S. 2015. Towards User-Adaptive Information Visualization. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 4100–4106. [link]
  5. Gingerich, M., and Conati, C. 2015. Constructing Models of User and Task Characteristics from Eye Gaze Data for User-Adaptive Information Highlighting. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 1728–1734. [link]
  6. Steichen, B., Conati, C., and Carenini, G. 2014. Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data. ACM Transactions on Interactive Intelligent Systems 4, 2, Article 11. [ACM TiiS Best Paper Award, link]
  7. Conati, C., Carenini, G., Steichen, B., and Toker, D. 2014. Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making. Computer Graphics Forum (EuroVis 2014) 33, 3, 371–380. [link]
  8. Steichen, B., Michael M.A. Wu, Toker, D., Conati, C., and Carenini, G. 2014. Te,Te,Hi,Hi: Eye Gaze Sequence Analysis for Informing User-Adaptive Information Visualizations. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 183-194. [Best paper award, link]
  9. Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J.T. 2014. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 1835–1844. [link]
  10. Toker, D., and Conati, C. 2014. Eye Tracking to Understand User Differences in Visualization Processing with Highlighting Interventions. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 219–230. [link]
  11. Steichen, B., Carenini, G., and Conati, C. 2013. User-adaptive Information Visualization: Using Eye Gaze Data to Infer Visualization Tasks and User Cognitive Abilities. Proc. of the 18th International Conference on Intelligent User Interfaces, ACM, 317–328. [Best Paper nominee, link]
  12. Toker, D., Conati, C., Steichen, B., and Carenini, G. 2013. Individual User Characteristics and Information Visualization: Connecting the Dots Through Eye Tracking. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 295–304. [link]
  13. Toker, D., Conati, C., Carenini, G., and Haraty, M. 2012. Towards Adaptive Information Visualization: On the Influence of User Characteristics. Proc. of the 20th International Conference on User Modeling, Adaptation, and Personalization, Springer, 274–285. [link]
  14. Conati, C., and Maclaren, H. 2008. Exploring the Role of Individual Differences in Information Visualization. Proc. of the Working Conference on Advanced Visual Interfaces, ACM, 199–206. [link]
  15. Toker, D., Conati, C., Carenini, G. 2018. User-adaptive Support for Processing Magazine Style Narrative Visualizations: Identifying User Characteristics that Matter. Proc. of the 23rd International Conference on Intelligent User Interfaces, ACM, (to appear). [link]