User Characteristic Tests

Here we will provide a description of the tests that we often use in our various studies to evaluate the impact of individual differences on HCI tasks.

User CharacteristicDefinition, Reference, Test SourceImpact on visualization processing (our work)Prediction from gaze + interaction data (our work)
Perceptual Speed
(Identical Pictures Test)
Speed in comparing figures or symbols, scanning to find figures or symbols, or carrying out other very simple tasks involving visual perception.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.

Test Location: http://www.cs.ubc.ca/~dtoker/phd/Ekstrom_P3.pdf
[2, 4, 7, 8, 9, 10, 12, 13, 14] [1, 5, 6, 11]
Visual Working Memory
Part of the working memory responsible for temporary storage and manipulation of visual and spatial information.

Fukuda, K. and Vogel, E.K. 2009. Human Variation in Overriding Attentional Capture. The Journal of Neuroscience. 29, 27, 8726–8733.

Test Location: http://www.cs.ubc.ca/~dtoker/phd/VisualWorkingMemoryTest.zip
[2, 4, 7, 8, 9, 10, 12, 13] [1, 5, 6, 11]
Verbal Working Memory
(OSPAN)
Part of the working memory responsible for temporary maintenance and manipulation of verbal information.

Turner, M.L. and Engle, R.W. 1989. Is working memory capacity task dependent? Journal of Memory and Language. 28, 2, 127–154.

Test Location: http://www.cs.ubc.ca/~dtoker/phd/VerbalWMTest.zip
[2, 4, 7, 8, 9, 10, 12, 13] [1, 5, 6, 11]
Visualization Literacy
The ability to confidently use a given data visualization to translate questions specified in the data domain into visual queries in the visual domain, as well as interpreting visual patterns in the visual domain as properties in the data domain.

Boy, J. et al. 2014. A Principled Way of Assessing Visualization Literacy. IEEE Transactions on Visualization and Computer Graphics. 20, 12, 1963–1972.

Test Location: http://peopleviz.gforge.inria.fr/trunk/vLiteracy/home/
[2] [1]
Disembedding
(Hidden Patterns Test)
The ability to hold a given visual percept or configuration in mind so as to disembed it from other well defined perceptual material.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.

Test Location: http://www.cs.ubc.ca/~dtoker/phd/Ekstrom_CF2.pdf
[14] N/A
Spatial Memory
(Shape Memory Test)
The ability to remember the configuration, location, and orientation of figural material.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.

Test Location: http://www.cs.ubc.ca/~dtoker/phd/Ekstrom_MV1.pdf
[2, 14] [1]
Need for Cognition
Measures the extent to which individuals are inclined towards effortful cognitive activities.

Cacioppo, J.T. et al. 1984. The Efficient Assessment of Need for Cognition. Journal of Personality Assessment. 48, 3, 306–307.

Test Location: http://www.liberalarts.wabash.edu/ncs/
[2, 14] N/A
Locus of Control
Measure of the degree to which individuals perceive outcomes as either a result from their own behavior, or from forces that are external to themselves.

Rotter, Julian B. 1966. Generalized Expectancies for Internal Versus External Control of Reinforcement. Psych. Monographs: General and Applied 80.1, 1-28.

Test Location: http://www.wilderdom.com/psychology/loc/RotterLOC29.html
[2, 4, 7, 9, 10] [5]
Five-factor personality traits
Measures personality in terms of Agreeableness, Conscientiousness, Extraversion, Emotional Stability, and Openness.

Gosling, S. D., Rentfrow, P. J., and Swann, W. B., Jr. (2003). A Very Brief Measure of the Big Five Personality Domains. Journal of Research in Personality, 37, 504-528.

Test Location: http://gosling.psy.utexas.edu/scales-weve-developed/ten-item-personality-measure-tipi/
[3] N/A

References

Only journal papers and strictly refereed conference papers published by our group are listed.

  1. Conati, C., Lallé, S., Rahman, M.A., and Toker, D. 2017. Further Results on Predicting Cognitive Abilities for Adaptive Visualizations. Proc. of the 26th International Joint Conference on Artificial Intelligence, AAAI, 1568-1574. [link]
  2. Lallé, S., Conati, C., and Carenini, G. 2017. Impact of Individual Differences on User Experience with a Visualization Interface for Public Engagement. Proc. of the 2nd International Workshop on Human Aspects in Adaptive and Personalized Interactive Environments (in conjunction with UMAP 2017), ACM. [link]
  3. Lallé, S., Mudrick, N.V., Taub, M., Grafsgaard, J.F., Conati, C., and Azevedo, R. 2016. Impact of Individual Differences on Affective Reactions to Pedagogical Agents Scaffolding. Proc. of the 16th International Conference on Intelligent Virtual Agents, Springer, 269–282. [Best paper award - link]
  4. Conati, C., Carenini, G., Toker, D., and Lallé, S. 2015. Towards User-Adaptive Information Visualization. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 4100–4106. [link]
  5. Gingerich, M., and Conati, C. 2015. Constructing Models of User and Task Characteristics from Eye Gaze Data for User-Adaptive Information Highlighting. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 1728–1734. [link]
  6. Steichen, B., Conati, C., and Carenini, G. 2014. Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data. ACM Transactions on Interactive Intelligent Systems 4, 2, Article 11. [ACM TiiS Best Paper Award, link]
  7. Conati, C., Carenini, G., Steichen, B., and Toker, D. 2014. Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making. Computer Graphics Forum (EuroVis 2014) 33, 3, 371–380. [link]
  8. Steichen, B., Michael M.A. Wu, Toker, D., Conati, C., and Carenini, G. 2014. Te,Te,Hi,Hi: Eye Gaze Sequence Analysis for Informing User-Adaptive Information Visualizations. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 183-194. [Best paper award, link]
  9. Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J.T. 2014. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 1835–1844. [link]
  10. Toker, D., and Conati, C. 2014. Eye Tracking to Understand User Differences in Visualization Processing with Highlighting Interventions. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 219–230. [link]
  11. Steichen, B., Carenini, G., and Conati, C. 2013. User-adaptive Information Visualization: Using Eye Gaze Data to Infer Visualization Tasks and User Cognitive Abilities. Proc. of the 2013 International Conference on Intelligent User Interfaces, ACM, 317–328. [Best Paper nominee, link]
  12. Toker, D., Conati, C., Steichen, B., and Carenini, G. 2013. Individual User Characteristics and Information Visualization: Connecting the Dots Through Eye Tracking. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 295–304. [link]
  13. Toker, D., Conati, C., Carenini, G., and Haraty, M. 2012. Towards Adaptive Information Visualization: On the Influence of User Characteristics. Proc. of the 20th International Conference on User Modeling, Adaptation, and Personalization, Springer, 274–285. [link]
  14. Conati, C., and Maclaren, H. 2008. Exploring the Role of Individual Differences in Information Visualization. Proc. of the Working Conference on Advanced Visual Interfaces, ACM, 199–206. [link]