Interruption Lab Study
Methodology
It is hypothesized that the effect of interruptions may interact with the age of the test-taker and the type of task being interruped. This mixed experimental design involves three factors: age, level of interruption demand, and type of main task. We will additionally rely on self-reports to acquire an understanding of task resumption strategies employed by participants in three age groups.Apparatus
Half of the experimental sessions were conducted using a Lenovo ThinkPad T400 with a 2.26Ghz Core 2 Duo processor and 1.92 GB RAM, while the remaining sessions were conducted using and an IBM ThinkPad T43 with a 2.0GHz Pentium M processor and 1GB + 512MB RAM. Both laptop computers were running the Windows XP operating system. Both computers were connected to a 17 inch diagonal display, with a resolution of 1024 by 768 pixels; the laptop displays were not used. A logitech M110 optical mouse was used with both laptops; identical mouse gains and tracking speeds were used. For the experimental tasks, the screen was positioned the screen at a comfortable viewing angle. The experimental software was an Adobe AIR application written using the Adobe Flex 4.0 SDK.Participants
We are recruiting 12 participants from three age groups, for a total of 36 participants: Young (19-54), Pre-Old (55-69), and Old (70+). The naming of these age groups is based on accepted terminology in the ageing literature. [Rimkus, A., Melinchok, M. D., McEvoy, K., & Yeager, A. K. (Eds.) (2005). Thesaurus of Aging Terminology. Eighth ed. AARP.]The justification for these groupings rests on the age-related changes that occur in cognition, notably that higher cognitive function remains relatively stable up to about age 55, after which there is a small decline, followed by a much steeper one after 70. [Craik, F. I. M., & Salthouse, T. A. (Eds.) (1992). The Handbook of Aging and Cognition. 2nd ed. Hillsdale, NJ: Erlbaum.] Participants receive $5 for each half hour of participation. We are recruiting younger participants through advertisements on campus and through word-of-mouth advertisement. Older participants were recruited through word-of-mouth advertisement and a postings in the community. To participate, all participants must be free of any diagnosed cognitive impairments or motor impairments to their right hand, and have normal or corrected-to-normal eyesight. We administered the Montreal Cognitive Assessment (MoCA) test [CITATION] to help ensure that participants are not cognitively impaired. A score of 26 / 30 or higher is considered normal. Additionally, we administered the North American Adult Reading Test (NAART) [B. Uttl. North american adult reading test: Age norms, reliability, and validity. Journal of Clinical and Experimental Neuropsychology, 24(8):1123–1137, 2002.] to help ensure participants had sufficient English fluency to follow our instructions. The NAART is a quick to administer test measuring verbal intelligence, which requires participants to read a list of 59 words increasing in difficulty. Using a somewhat arbitrary threshold, we accept only participants who get at least 25% of the words correct. Participants who do not have a normal MoCA score or those who do not meet our NAART threshold completed a shorter version of the study, but their data is not included in our analysis.Tasks
To gain a better understanding of how interruption demand and age interact with task type, two main tasks were used in this study. Both tasks were adapted from C-TOC tests: Sentence Comprehension and Square Puzzles. Sentence Comprehension (Figure X) tests verbal memory, wherein each trial is comprised of an instruction step and an execution step. The instruction screen displays 1-2 sentences instructing the user to arrange coloured geometric figures. The user clicks a 'Continue' button to advance to the execution screen, where geometric figures are to be arranged as per the instruction. Square Puzzles (Figure Y) tests non-verbal spatial reasoning, wherein each trial is comprised of a single screen. The user is instructed to move lines to create a certain number of complete squares in a specified number of moves, without leaving any incomplete squares. A participant can proceed to the next trial at any time by clicking on the 'Next' button at the bottom-right corner of the screen. Two interruption tasks were used in this study, corresponding to two levels of interruption demand. The interrupting tasks occupy the entire screen, occluding the main task. Interrupting tasks are preceded by an interruption notification, in which a red banner bearing the message 'Interruption Pending' flashes for 2 seconds at the top of the screen; during this time all interactivity is disabled. Once begun, both interruption tasks display an automated sequence of a dozen cartoon images inside a box outline in the middle of the display. There is no fixed order to the presentation of images; each image is selected semi-randomly from a bank of ten images. Each image is displayed for 1500ms; no image is displayed in the box for 100ms between successive images. The total length of the interrupting tasks is approximately 19 seconds. At the end of an interrupting task, the user is prompted to click in order to dismiss the interruption, returning to the interrupted main task. Interruption types are visually differentiated by a coloured instruction at the top of the screen. In the low-demand interrupting task, the user is prompted, in red font, to watch the sequence of images passively. The high-demand interrupting task (Figure Z) is a visual n-back memory game [CITATION], which is known to place high demand on working memory. In this task, the user is prompted, in green font, to click inside the box whenever the current image repeats what was shown 2 images prior to the current image. Feedback is displayed adjacent to the box outline in the form of a green check icon for correct responses (true positives), while a red 'x' icon is displayed for incorrect answers (false positives and false negatives). No feedback is shown for true negatives. The random sequence of images is weighted such that there is a 40% probability that any given image will repeat what was shown 2 images prior, otherwise the image is selected randomly.Design
The experiment used a mixed design with two counterbalanced main tasks and three counterbalanced levels of interruption demand: 3 (age groups: young, pre-old, and old) x 2 (main tasks: Sentence Comprehension and Square Puzzles) x 3 (interruption demand: none, low, and high). Age was the only between-subjects factor. Each participant completed 30 trials in the Sentence Comprehension task, and 24 trials in the Square Puzzles task. In both main tasks, three isomorphic trial blocks were used for the three conditions of interruption demand, comprised of trials increasing in difficulty. An equivalent level of difficulty between trials was not attempted, as the original C-TOC tests from which our tasks were adapted also increase in difficulty. For both main tasks, each participant was assigned one of the six possible trial block permutations at random. Each Sentence Comprehension trial block contained 10 trials, wherein a subset of 4 trials were interrupted. This subset of trials was determined randomly with the restriction that interruptions could not occur on 4 successive trials and that no interruption may occur on the first trial in a block of trials. The same subset of trials received interruptions in conditions of low and high interuption demand. Interruption onsets were fixed in that they would occur shortly after beginning the execution phase of a trial, with a short lag corresponding to increasing trial difficulty. Each Square Puzzles trial block contained 8 trials, wherein a subset of 3 trials were interrupted. This subset of trials was determined in the same fashion as in the Sentence Comprehension task. The same subset of trials received interruptions in conditions of low and high interuption demand. Interruption onsets were fixed in that they would occur after a 500ms lag upon the completion of the first move operation in a 2-move puzzle; in a 3-move puzzle, an interruption would occur after the completion of the first or second move operation, with equal likelihood.Procedure
The experiment was designed to fit into a single 120 minute session. All participants finished in between 60 minutes and 100 minutes. We began with the Montreal Cognitive Assessment and the North American Adult Reading Test. Participants then received practice on both types of interrupting tasks: 1 practice trial was performed of the low-demand interrupting task and two practice trials was performed of the high-demand interrupting task. A 3rd practice trial was offered if performance on the high-demand interrupting task was still poor after two practice trials. We then presented the first task (either Sentence Comprehension or Square Puzzles). An example trial was provided to familiarize the participants with the mechanics of the task. Participants were instructed to perform each task both as quickly and as accurately as possible. For the Square Puzzles task, participants were additionally instructed to not perform more than the specified number of moves, and to not leave remaining incomplete squares. Following this, a practice block of three trials was completed; during this set, the second trial was interrupted with a low-demand interrupting task, while the third trial was interrupted with a high-demand interrupting task. Participants then completed 3 blocks of trials of the first task. After each block, participants completed a short workload and fatigue survey. They then completed their second task. Upon completion of the experimental trials, participants were subject to a brief interview regarding their perceptions of task difficulty and their strategies for task resumption following an interruption.Measures
We included measures of speed and accuracy. In Sentence Comprehension, we measured trial time as the uninterrupted time elapsed during the execution step of the trial until the participant clicks to proceed to the next trial; time spent in the instruction step was recorded separately. In Square Puzzles, we measured trial time as the total uninterruped time elapsed between the start of the trials and when the participant clicks to proceed to the next trial. Additionally, the average time interval between valid move operations was determined for each trial. For interrupted trials, we also recorded the interruption dismissal lag time, the task resumption time following an interruption, and the average interval between valid move operations prior to and following an interruption. We included several measures of accuracy in both main tasks. The scoring scheme for these tasks was adapted from clinical scoring schemes and from the scoring scheme used in the C-TOC validity testing project. Accuracy scores were determined by the experimenter. A screen capture video of the entire experimental sesssion was recorded, which could be used to confirm scores. Each Sentence Comprehension trial has a possible total score based on the number of geometric figures the participant is instructed to arrange and the relative positioning of these figures upon completion. Each Square Puzzles trial has a possible total score based on the number of complete squares; points are deducted for each additional move operation and each incomplete square. For both main tasks, we additionally recorded the number of completed move operations, invalid moves, aborted moves, and clicks in each trial. Performance accuracy on the high-demand interrupting task was also recorded, including the number of true positives, true negatives, false positives, and false negatives for each trial.Hypotheses
We had the following hypotheses for this study: H1. Age will interact with the presence of interruptions; older adults will perform proportionally worse on interruped trials than on corresponding uninterrupted trials. H2. High-demand interruptions will incur worse performance than low-demand interruptions; interruption demand may interact with age such that older adults perform proportionally worse on trials interrupted with a high-demand interrupting task than on corresponding trials interrupted with a low-demand interrupting task. H3. Interruptions will incur worse performance on the Sentence Comprehension task than on the Square Puzzles task; a three-way interaction between level of interruption demand, main task type, and age is also expected. H4. Self-reports regarding the disruptive effects of interruptions are expected to reinforce H1-H3.Planned Analysis
Analysis 1: To determine the local effects of interruption disruption in terms of speed and accuracy, we plan to conduct a 3 (age) x 3 (interruption demand) x [3,4] (trial) mixed-factor ANOVA for both Sentence Comprehension and Square Puzzles. We will only consider performance data from the subset of trials that are interrupted in the two interruption conditions, along with the corresponding subset of trials in the uninterrupted condition; results from the remainder of uninterrupted trials in each condition are discarded. Therefore, each subject will generate 12 Sentence Comprehension data points (4 in each interruption condition), and 9 Square Puzzle data points (3 in each interruption condition). The main effect of trial is not of interest, as the difficulty differences between trials are known to us. Similarly, the interactions of trial with age or interruption demand are also of little interest. This analysis will address H1 and H2, to determine if there are interactions between age and level of interruption demand. Analysis 2: To determine the global effects of interruption disruption, we are to conduct a 3 (age) x 3 (interruption demand) x 2 (main task) mixed-factor ANOVA. A main effect of task is expected and is not of interest. We are, however, interested in the presence of any interaction between of task, age, and interruption demand, which will address H3. For this analysis, a normalized performance score and completion time is calculated for each block of trials, such that each participant will generate 6 data points (3 from each main task).Preliminary Results
Preliminary findings indicate that there may be a three-way interaction between user age, level of interruption demand, and the requirements of the C-TOC test undertaken by the user. Research participants have reported that interruptions are more disruptive to their performance during a verbal Sentence Comprehension task than during the non-verbal Square Puzzles task. Our analyses will examine whether test performance is proportionally worse when high-demand interruptions are present, versus low-demand interruption and control conditions. The initial findings appear to suggest that technological interventions for mitigating the impact of interruptions and for ensuring the validity of the test scores could be developed to be specific to each type of test in the C-TOC battery.Results
All completion time, resumption lag time, and inter-action interval times have been log-transformed to correct for positive skews. Normalized score results, questionnaire responses, and n-back scores, have been analyzed using the Aligned-Rank Transform for nonparametric factorial ANOVAs. An analysis with 2 age groups (split at the median age of 64) was also performed; however this did not reveal any new aspects in the data. Main effects and interactions as seen with 3 groups were largely still present with 2 groups. Outlier analysis using box-plots and cost-of-interruption scatter plots was also performed, however no significant outliers were found.- Scope of analysis:
- Local: includes the mean DV for subset of interrupted trials only; corresponding uninterrupted trials from the ID.N condition are also included; all other trials are discarded from analysis; the interrupted subset of trials is selected semi-randomly for each participant, with the condition that interruptions span a range of trial difficulty levels; since trials in each bank increase in difficulty, this makes 3 or 4 consecutive interrupted trials unlikely;
- Global: included the mean DV for all trials in a bank (interrupted and uninterrupted) (except trial 1, which is treated as a practice trial and is excluded from all analysis);
- the global results are not sensitive enough to the interactions taking place at the local and RLT scale; therefore, global completion time and score results are not reported in the within-tasks section
- Data transformations:
- all completion time, inter-action interval (IIA: average time between valid moves) time, and resumption lag time (RLT: time between interruption dismissal and subsequent valid move) results are log-transformed to correct positive skew, to lower the effects of outliers
- trial completion times do not include in the interruption
- all score results are normalized per trial (out of 1.00); results are averaged in the local and global analysis; the ART aligned rank transformation is then applied to the data, allowing for non-parametric factorial analysis using ANOVA procedures; this corrects for the score ceiling effect and negative skew; for more information, see [Wobbrock, Findlater, Gergle, and Higgins. "The Aligned Rank Transformation for Nonparametric Factorial Analyses Using only ANOVA Procedures. Proceedings of CHI 2011, 143-146, Vancouver BC].
- all questionnaire data is non-parametric and is therefore also analyzed using the ART technique;
- Notes:
- N.S. = not significant
- sig = significant (p < 0.05)
- M.sig = marginally significant (p <= 0.1)
Across Tasks Analysis (Sentence Comprehension & Square Puzzles)
- all analysis are 3 (G: age, BS) x 3 (ID: interruption demand, WS) x 2 (T: task, WS) mixed design
- age levels: G.1 = young, G.2 = pre-old, G.3 = old
- interruption demand levels: ID.N = no interruption, ID.L = low-demand interruption, ID.H = high-demand interruption
- task levels: T.SC = sentence comprehension, T.!SqP = square puzzle
Completion Time
- note: A < B = A is faster than B (A has lower log completion time)
Local
Younger adults were significantly faster than older adults. Pre-old adults were marginally faster than older adults. Sentence Comprehension was faster than Square Puzzles. There were significant interactions between age and interruption demand, between task and interruption demand, and between age, interruption demand, and task. These interactions are explored in the Within-Tasks Analysis section below.- G sig. (F(2,33) = 9.90, p < 0.01, p.eta^2 = 0.375)
- pairwise: G.1 < G.3 (p < 0.01); G.2 < G.3 (M.sig p = 0.07)
- ID N.S.
- T sig. (F(1,33) = 49.65, p < 0.01, p.eta^2 = 0.601)
- pairwise: SC < SqP (p < 0.01)
- G*ID sig. (F(4,66) = 3.25, p = 0.02, p.eta^2 = 0.165)
- G.3 are best in ID.L condition; G.1, G.2 are worse in ID.L condition
- G*T N.S.
- T*ID sig. (F(2,66) = 4.69, p = 0.01, p.eta^2 = 0.124)
- SC logT increases linearly with ID, SqP logT highest in ID.L condition
- G*ID*T sig. (F(4,66) = 2.85, p = 0.03, p.eta^2 = 0.147)
- G.3 is worse than G.1, G.2 on both tasks, but perform proportionally better (relative to themselves) in SqP ID.L condition
Global
Younger adults were significantly faster than older adults, and marginally faster than pre-old adults. Pre-old adults were significantly faster than older adults. The no-interruption condition was marginally faster than the low-demand and high-demand conditions. Sentence Comprehension was significantly faster than Square Puzzles. There was a marginal 3-way interaction between age, interruption demand, and task.- G sig. (F(2,33) = 12.96, p < 0.01, p.eta^2 = 0.440)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M. sig p = 0.05); G.2 < G.3 (p = 0.04)
- ID sig. (F(2,66) = 3.90, p = 0.03, p.eta^2 = 0.106)
- pairwise: ID.N < ID.L (M.sig p = 0.08); ID.N < ID.H (M.sig p = 0.05)
- T sig. (F(1,33) = 125.93, p < 0.01, p.eta^2 = 0.792)
- pairwise: SC < SqP (p < 0.01)
- G*ID N.S.
- G*T N.S.
- T*ID N.S.
- G*ID*T M.sig. (F(4,66) = 2.24, p = 0.07, p.eta^2 = 0.120)
- G.3 are worse than G.1, G.2 on both tasks, but perform proportionally better (relative to themselves) in SqP ID.L condition
Score
- note: A > B = A has higher normalized score rank than B (after ART)
Local
Younger adults had significantly higher scores than pre-old and older adults.;- G sig. (F(2,33) = 7.39, p = 0.02, p.eta^2 = 0.309)
- pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (p = 0.05)
- ID N.S.
- T N.S.
- G*ID N.S.
- G*T N.S.
- T*ID N.S.
- G*ID*T N.S.
Global
Younger adults have significantly higher score than younger adults and marginally higher score than pre-old adults. Scores were significantly higher in the Sentence Comprehension task than in the Square Puzzles task.- G sig. (F(2,33) = 8.22, p < 0.01, p.eta^2 = 0.333)
- pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (M. sig p = 0.05)
- ID N.S.
- T sig. F(1,33) = 14.97, p < 0.01, p.eta^2 = 0.312)
- pairwise: SC > SqP (p < 0.01)
- G*ID N.S.
- G*T N.S.
- T*ID N.S.
- G*ID*T N.S.
Task Resumption Lag Time (Local)
Times have been log-transformed to correct positive skew.- Analysis 1: RLT-L vs. RLT-H = log resumption lag times compared between ID.L and ID.H conditions
- Analysis 2: IIA-N vs. RLT-L vs. RLT-H = log inter-action interval (avg. time between valid moves) in ID.N condition compared against log resumption lag times in ID.L and ID.H conditions
- note: A < B = A has shorter log resumption lag time than B (A resumes task faster than B)
RLT-L vs. RLT-H
Younger adults have significantly shorter task resumption lags than pre-old and older adults. Lags were significantly shorter in the low-demand condition than in the high demand condition. Lags were significantly shorter in the Sentence Comprehension task than in the Square Puzzles task. There were significant interactions between age and interruption demand, between task and interruption demand, and between age, task, and interruption demand. These interactions are explored in detail in the Within-Tasks analysis section.- G sig. F(2,33) = 8.93, p < 0.01, p.eta^2 = 0.351)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
- ID sig. F(1,33) = 4.46, p = 0.04, p.eta^2 = 0.119)
- pairwise: ID.L < ID.H (p = 0.04)
- T sig. F(1,33) = 23.03, p < 0.01, p.eta^2 = 0.411)
- pairwise: SC < SqP (p < 0.01)
- G*ID sig. F(2,33) = 6.33, p < 0.01, p.eta^2 = 0.277)
- G.1 resumes faster in ID.H than in ID.L; opposite holds for G.2, G.3 - steeper ascending RLT slope for older adults;
- ID RM-ANOVA at G.1: ID N.S.
- ID RM-ANOVA at G.2: ID N.S.
- ID RM-ANOVA at G.3: ID sig. (F(1,23) = 13.41, p < 0.01, p.eta^2 = 0.368)
- pairwise: ID.L < ID.H (p < 0.01)
- G one-way ANOVA at ID.L: G N.S.
- G one-way ANOVA at ID.H: G sig. (F(2,69) = 19.20, p < 0.01, p.eta^2 = 0.358)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
- G*T N.S.
- T*ID sig. F(1,33) = 13.21, p < 0.01, p.eta^2 = 0.286)
- in SC, steep ascending slope from ID.L to ID.H; in SqP, downward slope from ID.L to ID.H
- T RM-ANOVA at ID.L T sig. (F(1,35) = 30.90, p < 0.01, p.eta^2 = 0.469)
- pairwise: SC < SqP (p < 0.01)
- T RM-ANOVA at ID.H: T N.S.
- G*ID*T sig. F(2,33) = 3.59, p = 0.04, p.eta^2 = 0.179)
- age, ID effects in SC (no interactions); age but not ID effect in SqP (G*I interaction) exists
IIA-N vs. RLT-L vs. RLT-H
Younger adults have significantly shorter task resumption lags than pre-old and older adults. Lags were significantly shorter in the Sentence Comprehension task than in the Square Puzzles task. There were significant interactions between age and interruption demand, between task and interruption demand, and between age, task, and interruption demand. These interactions are explored in detail in the Within-Tasks analysis section.- G sig. (F(2,33) = 7.66, p < 0.01, p.eta^2 = 0.410)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p = 0.01)
- ID M.sig. (F(2,33) = 2.86, p = 0.07, p.eta^2 = 0.732)
- pairwise: N.S.
- T sig. (F(1,33) = 90.34, p < 0.01, p.eta^2 = 0.080)
- pairwise: SC < SqP (p < 0.01)
- G*ID sig. (F(4,66) = 4.81, p < 0.01, p.eta^2 = 0.226)
- G.1 have decreasing RLT from ID.N to ID.H
- G.2 have have increasing RLT from ID.N to ID.H
- G.3 have decreased RLT from ID.N to ID.L, then an increase from ID.L to ID.H
- G*T N.S.
- T*ID sig. (F(2,66) = 36.75, p < 0.01, p.eta^2 = 0.527)
- RLT is lower in SC than SqP
- RLT increases from ID.N to ID.H in SC
- RLT decreases from ID.N to ID.H in SqP
- G*ID*T sig. (F(4,66) = 2.77, p = 0.03, p.eta^2 = 0.144)
- RLT increases across all age groups in SC from ID.N to ID.H
- RLT decreases in G.1, G.2 in SqP from ID.N to ID.H, except G.3, who increase from ID.L to ID.H
Within Tasks Analysis
- all analysis are 3 (G: age, BS) x 3 (ID: interruption demand, WS) mixed design
- age levels: G.1 = young, G.2 = pre-old, G.3 = old
- interruption demand levels: ID.N = no interruption, ID.L = low-demand interruption, ID.H = high-demand interruption
- the global results are not sensitive enough to the interactions taking place at the local and RLT scale; therefore, global completion time and score results are not reported in the within-tasks section (only age effects are present in global analysis at within-tasks level of analysis)
Sentence Comprehension
Completion Time (Local)
Younger adults are significantly faster than older adults, and marginally faster than pre-old adults. Pre-old adults are marginally faster than older adults. The no-interruption and low-demand conditions were significantly faster than the high-demand condition.- G sig. (F(2,33) = 12.00, p < 0.01, p.eta^2 = 0.421)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M.sig p = 0.07); G.2 < G.3 (M. sig p = 0.05)
- ID sig. (F(2,66) = 12.47, p < 0.01, p.eta^2 = 0.274)
- pairwise: ID.N < ID.H (p < 0.01); ID.L < ID.H (p < 0.01);
- G*ID N.S.
Score (Local)
Younger adults score significantly higher than pre-old and older adults. There is a marginal effect of interruption demand.- G sig (F(2,33) = 10.46, p < 0.01, p.eta^2 = 0.388)
- pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (p < 0.01)
- ID M.sig (F(2,66) = 2.52, p = 0.09, p.eta^2 = 0.071)
- pairwise: N.S.
- G*ID N.S
Task Resumption Lag Time (Local)
Times have been log-transformed to correct positive skew.RLT-L vs. RLT-H
Younger adults had significantly shorter task resumption lag times than pre-old and older adults. The low-demand condition's task resumption time was significantly shorter than the task resumption lag following a high-demand interruption. There was an significant interaction between age and interruption demand: there was no significant differences between resumption lag times for younger adults. The task resumption lag time following a low-demand interruption was significantly shorter than in the high-demand condition for pre-old and older adults. In the low-demand condition, younger adults were significantly quicker to resume the task than older adults. In the high-demand condition, they were significantly quicker to resume than both pre-old and older adults.- G sig.(F(2,33) = 11.89, p < 0.01, p.eta^2 = 0.419)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
- ID sig. (F(1,33) = 35.99, p < 0.01, p.eta^2 = 0.522)
- pairwise: ID.L < ID.N (p < 0.01)
- G*ID M.sig. (F(2,33) = 3.22, p = 0.05, p.eta^2 = 0.163)
- all age groups increase in RLT from ID.L to ID.H, however the slopes of G.2 and G.3 are steeper
- RM-ANOVA at G.1: ID N.S.
- RM-ANOVA at G.2: ID sig. (F(1,11) = 34.49, p < 0.01, p.eta^2 = 0.758)
- pairwise: ID.L < ID.N (p < 0.01)
- RM-ANOVA at G.3: ID sig. (F(1,11) = 12.06, p < 0.01, p.eta^2 = 0.523)
- pairwise: ID.L < ID.N (p < 0.01)
- one-way ANOVA at ID.L: G. sig (F(2,33) = 5.67, p < 0.01, p.eta^2 = 0.256)
- pairwise: G.1 < G.3 (p < 0.01)
- one-way ANOVA at ID.H: G. sig (F(2,33) = 13.45, p < 0.01, p.eta^2 = 0.449)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
IIA-N vs. RLT-L vs. RLT-H
Times have been log-transformed to correct positive skew. Younger adults had significantly shorter task resumption lag times than pre-old and older adults. The no-interruption condition's inter-action interval was significantly shorter than the task resumption lag following a high-demand interruption, and marginally shorter than the task resumption lag following a low-demand interruption.- G sig. (F(2,33) = 13.50, p < 0.01, p.eta^2 = 0.450)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
- ID sig. (F(2,66) = 34.00, p < 0.01, p.eta^2 = 0.507)
- pairwise: ID.N < ID.H (p < 0.01); ID.N < ID.L (M. sig. p = 0.08)
- G*ID N.S. (F(4,66) = 1.99, p = 0.11, p.eta^2 = 0.107)
- lines parallel from ID.N to ID.L; interaction from RLT-L vs. RLT-H still there
Questionnaire Results (Global)
- notes: scores are out of 10 before ART transformation (non-parametric factorial analysis)
- G N.S.
- ID N.S.
- G*ID N.S.
- G sig. (F(2,33) = 4.63, p = 0.02, p.eta^2 = 0.219)
- pairwise: G.1 < G.3 (M.sig. p = 0.05); G.2 < G.3 (p = 0.03)
- ID N.S.
- G*ID N.S.
- G sig. (F(2,33) = 3.31, p = 0.05, p.eta^2 = 0.167)
- pairwise: G.1 < G.2 (p = 0.05)
- ID sig. (F(2,66) = 13.17, p < 0.01, p.eta^2 = 0.285)
- pairwise: ID.N < ID.L (p < 0.01); ID.N < ID.H (p <0.01)
- G*ID N.S.
- G N.S.
- ID sig. (F(2,30) = 4.24, p = 0.05, p.eta^2 = 0.124)
- pairwise: ID.L < ID.H (p = 0.05)
- G*ID N.S.
- G M.sig. (F(2,33) = 3.05, p = 0.06, p.eta^2 = 0.156)
- pairwise: G.1 > G.3 (M sig. p = 0.06)
- ID N.S.
- G*ID N.S.
- G N.S.
- ID N.S.
- G*ID N.S.
Square Puzzles
Completion Time (Local)
Younger adults were significantly faster than older adults. There was a significant interaction between age and interruption demand: in the no-interruption and high-demand conditions, younger adults are significantly faster than older adults. Pre-old adults are marginally faster than older adults in the high-demand condition. There are no significant differences between the groups in the low-demand condition. Younger adults are marginally faster in the high-demand condition than in the low-demand condition. Pre-old adults do not differ significantly between conditions. Older adults are marginally faster in the low-demand condition than in the no-interruption condition.- G sig. (F(2,33) = 4.09, p = 0.03, p.eta^2 = 0.199)
- pairwise: G.1 < G.3 (p = 0.02)
- ID N.S
- G*ID sig. (F(4,66) = 3.28, p = 0.02, p.eta^2 = 0.166)
- G.1, G.2: log completion time increases from ID.N to ID.L, then decreases from ID.L to ID.H
- G.3: log completion time decreases from ID.N to ID.L, then increases from ID.L to ID.H
- one-way ANOVA at ID.N:
- G sig. (F(2,33) = 4.09, p = 0.03, p.eta^2 = 0.199)
- pairwise: G.1 < G.3 (p = 0.01)
- G sig. (F(2,33) = 4.09, p = 0.03, p.eta^2 = 0.199)
- one-way ANOVA at ID.L:
- G N.S.
- one-way ANOVA at ID.H:
- G sig. (F(2,33) = 4.89, p = 0.01, p.eta^2 = 0.229)
- pairwise: G.1 < G.3 (p = 0.01); G.2 < G.3 (M sig. p = 0.07)
- G sig. (F(2,33) = 4.89, p = 0.01, p.eta^2 = 0.229)
- RM-ANOVA at G.1:
- ID sig. (F(2,22) = 3.83, p = 0.04, p.eta^2 = 0.258)
- pairwise: ID.H < ID.L (M sig. p = 0.07)
- ID sig. (F(2,22) = 3.83, p = 0.04, p.eta^2 = 0.258)
- RM-ANOVA at G:2:
- ID N.S.
- RM-ANOVA at G.3:
- ID sig. (F(2,22) = 3.63 p = 0.04, p.eta^2 = 0.248)
- pairwise: ID.L < ID.N (M sig. p = 0.1)
- ID sig. (F(2,22) = 3.63 p = 0.04, p.eta^2 = 0.248)
Score (Local)
Younger adults scored marginally higher than older adults.- G M.sig (F(2,33) = 2.77, p = 0.08, p.eta^2 = 0.144)
- pairwise: G.1 > G.3 (M.sig p = 0.08)
- ID N.S.
- G*ID N.S.
Task Resumption Lag Time (Local)
RLT-L vs. RLT-H
Younger adults resumed the task significantly faster than older adults. There was a significant interaction between age and interruption demand: young adults resume the task significantly faster following a high-demand interruption than following a low-demand interruption. The pre-old adults did not differ significantly between conditions. The older adults resumed a task marginally faster in the low-demand condition than in the high-demand condition. There were no significant differences in resumption lag time between the age groups in the low-demand conditions. In the high-demand condition, younger adults resume the task significantly faster than older adults, and marginally faster than pre-old adults.- G sig. F(2,33) = 3.40, p = 0.05, p.eta^2 = 0.171)
- pairwise: G.1 < G.3 (M.sig p = 0.08)
- ID N.S.
- G*ID sig. F(2,33) = 5.60, p < 0.01, p.eta^2 = 0.253)
- RLT decreases for G.1, G.2 from ID.L to ID.H; RLT increases for G.3 from ID.L to ID.H
- RM-ANOVA at G.1: ID sig (F(1,11) = 7.48, p = 0.02, p.eta^2 = 0.405)
- pairwise: ID.H < ID.L (p = 0.02)
- RM-ANOVA at G.2: ID N.S.
- RM-ANOVA at G.3: ID M.sig (F(1,11) = 4.48, p = 0.06, p.eta^2 = 0.289)
- pairwise: ID.L < ID.H (p = 0.06)
- one-way ANOVA at ID.L: G N.S.
- one-way ANOVA at ID.H: G sig. (F(2,33) = 7.83, p < 0.01, p.eta^2 = 0.322)
- pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M sig. p = 0.06)
IIA-N vs. RLT-L vs. RLT-H
Younger adults resumed the task significantly faster than older adults. The time to resume the task following a low-demand or high-demand interruption was faster than the interaction interval in the no-interruption condition. There was a significant interaction between age and interruption demand: young adults resume the task significantly faster following a high-demand interruption than following a low-demand interruption, and this resumption lag time is faster than their inter-action interval in the no-interruption condition. The older adults' resumption lag time following a low-demand interruption is significantly faster than their interaction interval in the no-interruption condition. In the no-interruption and high-demand conditions, younger adults have a significantly shorter inter-action interval than older adults. There are no significant differences in between age groups in the low-demand condition. The pre-old adults do not differ significantly between conditions.- G sig. (F(2,33) = 4.83, p = 0.02, p.eta^2 = 0.226)
- pairwise: G.1 < G.3 (sig p = 0.01)
- ID sig. (F(2,66) = 11.60, p < 0.01, p.eta^2 = 0.260)
- pairwise: ID.L < ID.N (p < 0.01); ID.H < ID.N (p < 0.01);
- G*ID sig. (F(4,66) = 4.62, p < 0.01, p.eta^2 = 0.219)
- G.1, G.2 decreasing RLT trend from ID.N to ID.H; G.3 decreases sharply from ID.N to ID.L, then increases from ID.L to ID.H
- RM-ANOVA at G.1: ID sig (F(2,22) = 11.28, p < 0.01, p.eta^2 = 0.506)
- pairwise: ID.H < ID.N (p < 0.01); ID.H < ID.L (M.sig. p = 0.06)
- RM-ANOVA at G.2: ID N.S.
- RM-ANOVA at G.3: ID sig (F(2,22) = 9.63, p < 0.01, p.eta^2 = 0.467)
- pairwise: ID.L < ID.N (p < 0.01)
- one-way ANOVA at ID.N: G sig. (F(2,33) = 6.42, p < 0.01, p.eta^2 = 0.280)
- pairwise: G.1 < G.3 (p < 0.01)
- one-way ANOVA at ID.L: G N.S.
- one-way ANOVA at ID.H: G sig. (F(2,33) = 7.83, p < 0.01, p.eta^2 = 0.322)
- pairwise: G.1 < G.3 (p < 0.01)
Questionnaire Results (Global)
Fatigue There were no significant differences in self-reported fatigue levels across age groups or interruption demand conditions, nor were there interactions.- G N.S.
- ID N.S.
- G*ID N.S.
- G N.S.
- ID N.S.
- G*ID N.S.
- G N.S.
- ID sig. (F(2,66) = 3.25, p = 0.05, p.eta^2 = 0.090)
- pairwise: ID.N < ID.H (M.sig. p = 0.07); ID.L < ID. H (M.sig. p = 0.07)
- G*ID N.S.
- G N.S.
- ID N.S.
- G*ID N.S.
- G sig. (F(2,33) = 5.01, p = 0.01, p.eta^2 = 0.235)
- pairwise: G.1 > G.3 (p = 0.01)
- ID N.S.
- G*ID M.sig. (F(4,66) = 2.37, p = 0.06, p.eta^2 = 0.126)
- G.1 perceived performance decreases from ID.N to ID.L, then increases again to ID.H
- G.2 perceived performance remains relatively constant
- G.3 perceived performance increases from ID.N to ID.L, then decreases again to ID.H
- RM-ANOVA at G.1: ID M.sig. (F(2,22) = 2.86, p = 0.08, p.eta^2 = 0.206)
- pairwise: N.S. ID.N > ID.L, ID.N > ID.H, ID.H > ID.L
- RM-ANOVA at G.2: ID N.S.
- RM-ANOVA at G.3: ID sig. (F(2,22) = 4.10, p = 0.03, p.eta^2 = 0.271)
- pairwise: ID.L > ID.H (p < 0.01)
- one-way ANOVA at ID.N: G sig. F(2,33) = 4.43, p = 0.02, p.eta^2 = 0.211)
- pairwise: G.1 > G.3 (p = 0.02)
- one-way ANOVA at ID.L: G N.S.
- one-way ANOVA at ID.H: G sig. F(2,33) = 6.31, p < 0.01, p.eta^2 = 0.277)
- pairwise: G.1 > G.3 (p < 0.01); G.2 > G.3 (p = 0.03);
- G N.S.
- ID N.S.
- G*ID N.S.
Interviews
Task Difficulty
Which task was more difficult, regardless of whether or not they were interrupted?- G.1: SC (4), SqP (7), No answer (1)
- G.2: SC (4), SqP (8)
- G.3: SC (3), SqP (8), equally difficult (1)
- "[SC]"
- "equally difficult, but difficult in different ways: verbal vs. spatial"
- "SC: I couldn't always remember what I was supposed to be doing. SqP: I mainly could see what I was supposed to do, but I still blew it on occasion. I don't think I thought it out completely at times. I went ahead and I should have maybe considered my moves."
- [SC]
- "!SqP: I don't see how to make the boxes"
- "!SqP: because I didn't do as well on it."
- "!SqP: I couldn't get it! I found it difficult to figure it out. SC easier (colours made it easier)"
- "!SqP: I don't know. When you had to move 3 lines, you had to look at it more [strategically]."
- "!SqP: I couldn't see the image in my brain ahead of time"
- "!SqP: not entirely sure why - only one simple instruction. SC easier - directions were clear, very specific as to what to do. Distinction between general and specific instructions. Fatigue factored into performance."
- "!SqP, it took time after realizing I couldn't rotate the lines"
- "!SqP: I felt it required more mental activity to decide which lines to move than to remember the instructions. The most difficult part about [SC] was remembering the instructions, they were easy to follow if you could remember them. In [!SqP] the more difficult part was determining which lines to move, but the instructions were extremely simple."
Interruption Disruption
Which task was more disrupted by interruptions?- G.1: SC (9), SqP (1), equally affected (1), no answer (1)
- G.2: SC (6), SqP (2), equally affected (4)
- G.3: SC (12)
- 1. "[SC]"
- 2. "SC: I had an easier time remembering shapes than words. SqP: If I could figure it out, I could remember it."
- 3. "SC: it destroyed my concentration. No effect of interruptions on SqP"
- 4. "SC: more things to remember"
- 5. "SC: there were more things to make mental notes of."
- 6. "SC: I couldn't remember what I was supposed to do to finish it off. In [!SqP] I was able to pick it up and try to finish it."
- 7. "SC: I don't know. I couldn't remember, after an interruption I couldn't bring it up again. In SqP I think I could remember better, the squares."
- 8. "SC, because you had more to remember. [!SqP]: you didn't have as much to remember."
- 9. "SC: I had to remember shape and colour, whereas in [!SqP] there was just movement."
- 10. "SC, more of an interruption, even though I had more difficulty with [!SqP] - the memory as opposed to strategy."
- 11. "SC, instructions more complicated. In [!SqP] everything was pretty much the same and you knew what you were doing."
- 12. "SC, because the instructions had to be memorized, I felt the interruptions impeded my ability to remember what I was doing. With [!SqP], the instructions, which were simple to begin with, reappeared on the screen, there was in effect to memory to be done. Interruptions didn't affect [!SqP] performance."
Interruption Lag / Delay
Was the interruption lag delay helpful?- G.1: No (5), Yes/somewhat (6), no answer (1)
- G.2: No (6), Yes/somewhat (2), no/mixed opinion (3), no answer (1)
- G.3: No (7), Yes/somewhat (4), mixed opinion (1)
- 1. "no, not really"
- 2. "delay helpful. It allowed me to visualize what I was going to do: strategy was the same [between tasks]"
- 3. "delay not really helpful. It was frustrating because I was about to move something. I knew what I was going to do next."
- 4. "delay helpful. you could concentrate on what you should be remembering. SC: I tried to remember what I did last. SqP: I remembered the number of moves [That I already made], and how many I had left"
- 5. "yes and no. In one way it was helpful and in another way my mind says: "Okay, get it over with". I tried to remember in [!SqP] to move that line to that place or what I was going to next in [SC]."
- 6. "I just waited. In some of them I just looked where I was and tried to remember to just pick up where I left off. In [!SqP] I don't really think that I tried to remember anything because it was a case of just moving the lines, whereas in [SC] there was a result that was wanted."
- 7. "I said "darn it". frustrating."
- 8. "no, actually, it was a nuisance. You couldn't move anything so it was too late anyways."
- 9. "I would try to say to myself what the task was [in SC only]"
- 10. "not particularly, early on I was clicking on it because I wanted to get on with it. [re: strategies] I didn't think so, not consciously."
- 11. "annoying, very annoying. While it was flashing you knew you had to remember what you were doing and where things had to go when you finished. In SC you had to remember the specifics.In SqP you kinda knew what you had to do, it wasn't so important to remember"
- 12. "I did not, because I didn't make good use of it. In [!SqP] I did [remember current state] because it was readily apparent by looking at the screen what I had planned to do next, if I had planned to do anything next, whereas with [SC] looking at the screen didn't do any good. Because the interruption came so early in the primary task there was no evident pattern on the screen to help me to remember."
Task Resumption Strategies
What task resumption strategies were used?- G.1
- 1. "nothing in particular, I usually knew where I left off. [SC long sequences repeated internally]. [!SqP] I knew where I'd placed the last line"
- 2.
- 3. "recalling instructions, [SC] visualized clockwise in a starting corner. [!SqP] to remember what I was doing/thinking before interruption"
- 4. "[SC] recalling a sequence of colours, in [!SqP], I just looked at instructions again."
- 5. "I tried to retrieve what I had stored"
- 6. "in [SC] I tried to picture where the shapes are supposed to be and I follow that. in [!SqP] I don't really have to remember"
- 7. "SC: Other than remembering the instructions themselves, I'd place my eyes over the screen in order to remember what goes where"
- 8. "I tried to refocus on the screen I had left before the interruption. In [SC] I looked for some clues to help me remember if it was a shape or a size or colour, whereas in [!SqP] I just had to completely refocus on the puzzle to solve it."
- 9. "I usually had time to start the puzzle so then I would just go back to what I had already started to see what I was going to do next [same for SC and SqP]"
- 10. "Not really, usually I just tried to remember to the best of my ability - [in SqP what was done last, in SC, just key words]"
- 11. "I tried to memorize 3 colours [SC], in my head I went over what I had to go back to, [reciting just the key words], in [!SqP] I just remembered where the next line would be and complete the square. [during interruption lag], I would memorize where that line had to go so when the screen came back I would focus back there and drag the line"
- 12. "No because in my head I would still be thinking of the puzzle, so I would just go back into it, more for [SC]"
- G.3
- 1. "SC: I tried to remember key words; SqP: that just seemed automatic, not affected by interruption"
- 2. "SC: I tried to visualize the shapes. SqP: try and remember where I was going to put things."
- 3. "SC: that was the most difficult. SqP: I was frustrated when there were some lines left over."
- 4. "SC: I tried to remember the initial instruction."
- 5. "It was kind of a mental note I was making about what I was going to do next. In [SC] I had to remember what I was going to do period. [!SqP] was easier in that sense because there was only one thing to do. In [SC] when and interruption occurred, you had to remember more stuff. I remembered what I was going to do next."
- 6. "[remembered patterns, difficult to remember final steps in SC instructions]. Never thought about [how many moves already made in [!SqP]]"
- 7. "no, not really, maybe I should have. [Same for both tasks.]"
- 8. "[!SqP] was easy to get back to because you knew what you had to do whereas in [SC] you had to remember what you were supposed to do in that. "
- 9. "I didn't try to remember where I had left off, I tried to remember the signals I had given myself during the [interruption pending notification]. I tried to repeat that to myself. In [!SqP] it was just continuing where I had left off."
- 10. "not consciously."
- 11. "in [!SqP] I just tried to remember what lines I had to move. In [SC] it was kinda "which colour to where",different strategies. the flashy delay allowed you to do that. If it hadn't been for the flashy delay it would have been harder because you were shifting immediately to the interruption tasks."
- 12. "I don't think so. In [!SqP] it was evident what I had intended to do next. In [SC] it was a matter of returning my mind to the original instructions which quite frankly either were there or were not."
Interruption Types
Did strategy differ between passive and active interruptions?- G.1 - no difference / zoned out / harder to maintain concentration in passive interruption / played n-back practice with passive;
- 1. "strategy stayed the same" [pretended [passive] interruption was [active] interruption, played n-back game in her head]
- 2.
- 3. "no difference in strategy"
- 4. "no. just watched the images in the passive interruption to see if they change"
- 5. "I zoned out [during the passive interruption], [not thinking about puzzle task instructions] and that was the problem. I thought not having to do anything was harder. With [the active interruption] I was focused, I was still, like, doing stuff. The information I had stored was still there, whereas when you zone out, it's just like, things get lost."
- 6. "No difference. [during the passive interruption], I was trying to remember the main task."
- 7. "It would take a different type of focus. It was easier to take a break on [!SqP] than on [SC], in [SC] I had to play the instructions in my mind during the interruption. [During active interruption] I tried to retain [the instructions] somehow in my mind while allowing my eyes to do the thinking, if you want to put it that way."
- 8. "For the passive-type interruption I found it harder to maintain concentration on the interruption so I'd frequently go back in my mind to the puzzle I was doing. [Didn't have the opportunity to do this during the active interruption]"
- 9. "[during passive interruption] I had a chance to review the instructions in my head. I wasn't as focused [on the images] as when I had to click on them."
- 10. "I actually like the [active interruption] better than the passive one, in terms of stimulus, I guess. it was more interesting in that sense." [on passive interruption mind wandering]: "a little bit. It was easier to remember [the main task] during the passive one what the instructions were. During the [active interruption] you would have to remember partially"
- 11. "It did get more difficult when I had to click on images [in the active interruption]. In the [passive interruption], I could remember [the instructions]. In the [active interruption] when I came back I wasn't quite sure what to complete. [In passive interruption, may have played n-back game in her head (likely some misunderstanding of the question)]"
- 12. "For the active interruption, I may have doubted my memory, since the interruption required memory as well. In the [passive interruption] I just looked at [the images] but didn't try to think about them."
- G.2 - passive: relaxed, not think, practice for active interruptions
- 1. "yes. there was difference. [active]: I was dedicated to the interruption, I had made the map, I was kinda going "I'm doing my best on this one and when I get back the [main task] we'll see where I am and we'll deal with it then." [passive]: "I didn't really concentrate on the interruption at all when I realized that there was going to be no need count backwards or keep track of [the images]. There could have been a surpirse so I was consciously watching the images but I wasn't trying to remember anything. I glossed over that and was keeping [the main task] more so in mind so that I didn't lose track of that."
- 2. "It was easier to endure an interruption in which you didn't have to do anything. Your mind is still on the previous task." [Thinking more of [the puzzle task] during the [passive interruption]]. [active]: "I tried to do the interruption to the best of my ability."
- 3. "[passive interruption: wasn't able to rehearse SC task]. I just sat there, and while I was sitting there I was thinking about my plan."
- 4. "You could just go to sleep with the passive one if you want, you don't actually have to pay attention. The other one you do, so you need to be more alert for that one. For the passive one, I just kind of chilled out for a minute [wasn't thinking of main task]."
- 5. "In the passive type, I guess you could just basically relax while it was happening, not really [thinking of puzzle tasks]"
- 6. "No difference in strategy, I just looked out the window, taking a break [during passive interruption, wasn't trying to rehearse main task]"
- 7. "If the interruption was [active], you just very patiently try to stay where you were in the [main task], but in the [active interruption] you had to shelve your other memory and bring forward the new memory, and it was a lot of work. In the [passive interruption], you just put yourself in neutral, I guess is the best way to explain it, just wait for this to be over and we'll get back to it. It wasn't a break time, it was "wait, and you'll get back to [the main task]". Only in some cases [were instructions rehearsed]"
- 8. "I tuned the [passive interruption] right out, my mind was ready to get back to [the main task]. The active one needed some thought. [SC] was harder to return to."
- 9. "[No difference], during passive interruption, I was taking a break [during SqP, but tried to remember instruction in SC]
- 10. "If I didn't have to monitor, then I just let it pass, but I started reciting each one, saying it in my mind as a practice for the [active interruption]" [did not rehearse main task]
- 11. "I don't think so [no difference in strategy] performance didn't differ"
- 12. "the interruptions didn't really bother me. [in passive interruption, didn't think about main task]. I didn't really watch, I refocused."
- G.3 - relaxed during passive interruption, rehearsed instructions, mind went blank, ignored passive interruption, thought about main task,
- 1. "ignored passive interruption, I didn't have to put forth any energy, I relaxed. [active interruption] I had to think"
- 2. "If I didn't have to click I was able to rehearse what I was going to do. If I had to click then my was obviously involved in something else" [unable to rehearse]
- 3. "During passive interruption: I tried to think about [the SC task]. [active]: I could only do one thing at a time."
- 4. "the clicking one was just more destructive, make you tend to forget what it is you're supposed to be doing when you went back." [passive interruption]: "I watched the images but I was also focusing on what I should be doing when I get back. Every couple seconds reminding myself of what I should be doing when I go back."
- 5. [active]: "I paid attention to the interruption, but I had filed in my mind where I had left off, since I was warned that the interruption was coming. So I was just: "Okay, mental note." "The [passive] type was much easier because you could just ignore it. A lot of the time I didn't pay attention to it, I just tried to remember what I was going to get back to in the main task, like "just get this over with and get back to it." [Was thinking of main task during passive interruption]
- 6. "When I went back, regardless of what the interruption was, there was no difference in my strategy." [passive]: "half way through I realized that I didn't have to have my finger on the cursor and I could just look at [the images]. I didn't think about main task, I just took a break."
- 7. "the active was more difficult. I was concentrating more on the interruption than on the task." [passive]: "it was easier when you didn't have to click - all you did was watch, better able to concentrate on [main tasks]"
- 8. [passive]: "it was no big deal because you didn't have to do anything. just waiting for it to finish so you wouldn't forget too much, trying to remember what it was supposed to be." [active]: "you were far to busy"
- 9. [active]: "it was a little harder, harder to remember the more difficult ones" [unable to think of main tasks]. [passive]: "I let my mind go blank."
- 10. [passive]: "If I did it was not consciously."
- 11. "I found the passive one annoying, you were just sitting there, but it was easier. Didn't rehearse, I had done that already during the [interruption lag]. I was annoyed. I am an action person so I didn't mind shifting to a new activity whereas just sitting there was annoying."
- 12. "I found it extremely easy to ignore the [passive interruption], I turned my mind off. in the [active] interruption I had less opportunity to retain the instructions."
Other Factors (Local)
Other factors were examined to ensure the validity of results (at local level only).N-Back
N-back scores were at ceiling levels throughout the experiment, indicating that participants generally attended to the n-back task. Younger adults were score significantly higher than pre-old and older adults. There was no effect of task on n-back score, nor did age or task interact.- n-back scores negatively skewed, ceiling effect; non-parametric analysis using ART technique
- SC n-back means: G.1: 9.6 / 10; G.2: 8.8 / 10; G.3: 8.4 / 10; overall: 9.0 / 10
- SqP n-back means: G.1: 9.5 / 10; G.2: 8.7 / 10; G.3: 8.1 / 10; overall: 8.8 / 10
- G sig. (F(2,33) = 12.80, p < 0.01, p.eta^2 = 0.437)
- pairwise: G.1 > G.2 (p = 0.01); G.1 > G.3 (p < 0.01)
- T N.S. (no effect of task SC / SqP on n-back score)
- G*T N.S. (no interaction between age and task on n-back score
Interruption Condition Ordering (N-L-H)
The main task order had no main effect on score or completion time for Sentence Comprehension, but it interacted with age and interruption demand on log completion time. It had a marginal main effect on score in Square Puzzles.- x6 orderings: NLH, NHL, LNH, LHN, HNL, HLN; fully-counterbalanced within each group
- no main effect on SC log completion time; no interactions with G or ID; 3-way interaction G*ID*Ord (p = 0.03)
- no main effect or interactions on SC score
- no main effect or interactions on SqP log completion time
- marginal main effect (p = 0.06) on SqP score such that those with ordering NLH had lower scores; no interactions
Main Task Ordering (SC-!SqP)
The main task order had no main effect on score or completion time for both tasks, but it interacted with age on Square Puzzle log completion time.- x2 orderings: SC-!SqP,!SqP-SC, fully-counterbalanced within each group
- no main effect or interactions on SC log completion time
- no main effect or interactions on SC score;
- no main effect on SqP log completion time; interaction G*TOrd sig. (p = 0.04) such G.1 Ps with task order SC-!SqP had lower SqP logT than those with SqP-SC ordering; G.2 and G.3 had opposite trend;
- no main effect or interactions on SqP score;
SC Bank order (A-B-C)
The Sentence Comprehension bank order had no effect on score, but interacted with age and interruption demand on log completion time.- x6 orderings: ABC, ACB, BAC, BCA, CAB, CBA: isomorphic SC task banks, selected at random for each participant
- order number and distribution (G.1,G.2,G.3):
- ABC: 5 (2,2,1)
- ACB: 8 (2,4,2)
- BAC: 5 (2,1,2)
- BCA: 6 (2,2,2)
- CAB: 6 (2,1,3)
- CBA: 6 (2,2,2)
- interaction ID*BankOrd on log completion time sig. (p = 0.04)
- interaction G*ID*BankOrd on log completion time M.sig. (p = 0.06)
SqP Bank order (A-B-C)
The Square Puzzles bank order had no effect on log completion time, but interacted with age and interruption demand on score.- x6 orderings: ABC, ACB, BAC, BCA, CAB, CBA: isomorphic SqP task banks, selected at random for each participant
- order number and distribution (G.1,G.2,G.3):
- ABC: 8 (2,3,3)
- ACB: 2 (1,1,0)
- BAC: 5 (4,0,1)
- BCA: 5 (0,3,2)
- CAB: 7 (2,1,4)
- CBA: 9 (3,4,2)
- interaction G*ID*BankOrd on score
Experimenter & Apparatus
The experimenter and computer used had no effect on log completion time or score for either task.- MB (Lenovo thinkpad) (6 in group 1, 6 in group 2, 9 in group 3)
- CL (IBM thinkpad) (6 in group 1, 6 in group 2, 3 in group 3)
- no effects or interactions
Location
The location of the experiment had no effect on log completion time or score for either task.- A.D. Clinic (12 in group 1, 11 in group 2, 7 in group 3)
- Kerrisdale (4 in group 3)
- JM's condo (1 in group 3)
- UBC hosptial (1 in group 2)
- no effects or interactions
2-group analysis
- 2 age groups at median age of 64 (18 participants in either group)
Local completion time (Across Tasks)
Younger adults are significantly faster than older adults. Sentence Comprehension is faster than Square Puzzles. There are significant interactions between interruption demand and age, between task and interruption demand, and between age, interruption demand, and task.- G sig. (p < 0.01) young < old
- ID N.S.
- T sig. (p < 0.01) SC < SqP
- G*ID sig. (p = 0.03)
- G*T N.S.
- T*ID sig. (p = 0.02)
- G*ID*T M.sig (p = 0.1)
Local completion time (SC)
Younger adults are significantly faster than older adults. The no-interruption and low-demand interruption conditions are significantly faster than the high-demand condition.- G sig. (p < 0.01) young < old
- ID sig. (p < 0.01) ID.N < ID.H, ID.L < ID.H
- G*ID N.S.
Local completion time (SqP)
Younger adults are significantly faster than older adults. There is a significant interaction between interruption demand and age: younger adults are slower in low-demand interruption condition than in other conditions, while older adults are faster in low-demand interruption condition than in other conditions.- G sig. (p = 0.04) young < old
- ID N.S.
- G*ID sig. (p = 0.03)
- young SqP log completion times increase from ID.N to ID.L, then decrease from ID.L to ID.H
- old SqP log completion times decrease from ID.N to ID.L, then increase from ID.L to ID.H
Local score (Across Tasks)
Younger adults score significantly higher than older adults. There is a significant interaction between interruption demand and age: younger adults have lower scores in low-demand interruption condition than in other conditions, while older adults have highest score in no-interruption condition, and lowest score in high-demand interruption condition.- G sig (p = 0.02) young > old
- ID N.S.
- T N.S.
- G*ID M.sig. (p = 0.1)
- young scores decrease from ID.N to ID.L, and increase from ID.L to ID.H
- old scores decrease from ID.N to ID.L and again from ID.L to ID.H
- G*T N.S.
- T*ID N.S.
- G*ID*T N.S.
Local score (SC)
Younger adults score significantly higher than older adults. Scores decrease marginally as interruption workload increases.- G sig. (p < 0.01) young > old
- ID M.sig. (p = 0.08) ID.N > ID.L > ID.H (pairwise N.S.)
- G*ID N.S.
Local score (SqP)
Younger adults score significantly higher than older adults. There is a marginal interaction between interruption demand and age: younger adults have lower scores in low-demand interruption condition than in other conditions, while older adults have highest score in no-interruption condition, and lowest score in high-demand interruption condition.- G sig. (p = 0.05) young > old
- ID N.S.
- G*ID M.sig. (p = 0.09)
- young scores decrease from ID.N to ID.L, and increase from ID.L to ID.H
- old scores decrease from ID.N to ID.L and again from ID.L to ID.H
Global completion time(Across Tasks)
Younger adults are significantly faster than older adults. The no-interruption condition significantly faster than high-demand interruption condition. The no-interruption condition is marginally faster than the low-demand interruption condition. Sentence Comprehension is faster than Square Puzzles.- G sig (p < 0.01) young < old
- ID sig. (p = 0.03) ID.N < ID.H, ID.N < ID.L (M.sig)
- T sig. (p < 0.01) SC < SqP
- G*ID N.S.
- G*T N.S.
- T*ID N.S.
- G*ID*T N.S.
Global completion time(SC)
Younger adults are significantly faster than older adults. The no-interruption condition is significantly faster than the high-demand interruption condition.- G sig. (p < 0.01) young < old
- ID sig. (p < 0.01) ID.N < ID.H
- G*ID N.S.
Global completion time(SqP)
Younger adults are marginally faster than older adults. The no-interruption condition is marginally faster than the low-demand interruption condition.- G M.sig (p = 0.08) young < old
- ID M.sig (p = 0.08) ID.N < ID.L
- G*ID N.S.
Notes
One's performance on C-TOC and other similar applications will likely be affected by interruptions and distractions. This study examines the nature of disruptions caused by interruptions in the C-TOC test-taker's environment, and how these disruptions interact with the age of the test-taker. Previous research relating to interruptions in the field of human-computer interaction (HCI) focuses predominantly on younger adults in workplace settings. A body of HCI research on the design of information and communication technology for older adults is similarly well established, yet is disjoint from research pertaining to interruptions and distractions. This research project attempts to unify these research areas. A decline in higher cognitive functioning and prospective memory in old age has been well documented in the cognitive psychology literature. As such, it is hypothesized that the effect of interruptions may interact with the age of the test-taker. Our mixed experimental design involves 3 factors: a between-subjects factor of age and a within-subjects factor of interruption demand. Our dependent measures include performance scores (i.e. completion time and accuracy) on a small subset of tests in the C-TOC battery. We will be recruiting an equal number of cognitively-healthy participants from 3 age groups: 19-54, 55-69, and more than 70 years old. Throughout each of the C-TOC tests used in this study, participants will experience three levels of interruption demand in the form of 20-second distraction tasks. Our aim is to learn how interruption demand interacts with age. We will additionally rely on self-reports to acquire an understanding of task resumption strategies employed by participants in the three age groups. This knowledge will hopefully guide the design of features in C-TOC pertaining to interruption mitigation and task resumption. Such knowledge would also be an important contribution to the broader HCI community, especially valuable for those designing tools for older adults.Related work
See ongoing notes added to the literature review page.Experiment
- 3 levels of age (young vs. young-old vs. old-old, BS)
- 3 levels of interruption complexity (no interruption vs. low-demand vs. high-demand, ordering counterbalanced WS)
- 2 main tasks: (verbal: Sentence Comprehension Task (SC) vs. non-verbal: Square Puzzles (SqP), ordering counterbalanced WS)
- 8 trials in SqP, 10 trials in SC, 3 banks of isomorphic trials, which increase in difficulty;
- difficulty is gauged by number of actions required and complexity of instruction for SC puzzles, gauged by number of lines moved for SqP puzzles;
- ordering of 3 task banks randomly assigned; subsets of trials in which interruptions occur are semi-randomly generated, with the constraint that interruptions are spread across medium and hard trials; no interruptions occur on easy trials (1st trial in each bank);
- our dependent measures are speed and accuracy scores from the subset of interruption trials for each participant (4 in SC, 3 in SqP); dependent measures from other trials are not considered in this analysis
- for both tasks, interruptions occur on the same subset of trials between interruption conditions;
- in SC trial banks, interruptions occur after a fixed onset of time as a function of task difficulty;
- in SqP trial banks, interruptions occur after a short delay (500ms) following the first move in 2-move trials; for 3-move puzzles, there is a 50% probability of the interruption being triggered by the first move, and a 50% probability of the interruption being triggered by the second move;
Planned Analyses
Analysis 1: To determine the local effects of interruption disruption (specific to interrupted trials), we are to conduct a 3 (age, BS) x 3 (interruption demand, WS) x [3,4] (trial, WS) mixed-factor ANOVA for both SC and SqP. Main effect of trial is not interesting, neither are interactions of trial with age or interruption demand.- local performance measures - only on subset of trials that are interrupted compared with control condition; remainder of uninterrupted trial scores discarded;
- 3 x 4 = 12 data points per participant in SC, 3 x 3 = 9 data points per participant in SqP
- DV1: trial completion time;
- DV2: trial accuracy score, based on predefined scoring scheme;
- ?DV3: task resumption time vs. avg. interaction interval;?
- global performance measures, collapsed across all trials in a bank of trials
- 6 data points per participant (3 from each task)
- DV1: total completion time for each bank of trials - normalized to compare total completion time between SC and SqP trial banks; (if normally distributed, normalize using z-scores; else, subtract mean from each total completion time)
- DV2: total accuracy score for each bank of trials, score out of 100%;
- ?DV3: avg task resumption time vs. avg. interaction interval;?
- other recorded measures
- # moves, # adjustments, # clicks, # aborted moves, # invalid moves, SC instruction read time (SC only), first-action time, interruption dismissal time, resumption lag time (time to first completed action after interruption),interaction interval before interruption, interaction interval after interruption, total interaction time;
- for high-demand n-back interruptions: # correct, # incorrect, # false positives, # false negatives, # true positives, # true negatives;
- NASA-TLX workload assessment survey scores, administered after each bank of trials (fatigue, cognitive demand, physical demand, annoyance)
- MoCA test score - participant is excluded if scores less than 26
- English language test score - using the NAART (North American Adult Reading Test, [Uttl 02]); participant is excluded if less than 50% of words on page 1 are pronounced incorrectly;
- CJ: timing: advise against using instructions in time to complete - tests reading comprehension rather than task at hand (i.e. pattern construction) opportunity to separate criteria: item completion and task completion (including items, transitions, breaks in between, etc.)
Other Factors (unsupported or for future work)
- interruption type/modality (considered in addition to similarity/complexity): computerised (to simulate email / IM alert, antivirus notification, software update, browser crash (total screen change)) vs. uncomputerised (to simulate a telephone call, a caller at the door, a conversation with someone in the room, an alarm ringing, an errand or chore) - the latter forcing one away from the screen
- pop-up interruptions requiring simple vs. complex response (to simply dismiss interrupting pop-ups (easy) vs. pop-ups requiring responses to urgent forced-choise questions (difficult)).
- dimension: verbal (spoken) vs. written vs. interactive interruption
- unsupported: 2-3 levels of interruption frequency (no interruption/control, infrequent interruption, frequent interruption) (not sure if frequent interruption will be an issue for 1/2 hour cognitive test)
- partially supported: 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])
- task instruction (emphasis on speed vs. emphasis on accuracy)
- length of interruption lag (currently fixed at 2s)
Participants
- 3 age groups (19-55),(56-69),(70+) - rationale for 2-3 groups from [Moffatt 10], CJ
- CJ: young-old vs. old-old : age dif @ 70 in literature / 65 for AD/MCI research
Tasks
2 adapted and programmed C-TOC cognitive tests (one verbal, one nonverbal); Computerized Interruption tasks: one low-demand (passive n-back); the other high-demand (active n-back); interruptions have a fixed onset for SC tasks, depending on the difficulty of the SC task (number of moves and complexity of instruction taken into consideration), identical between 3 isomorphic SC task sets); interruptions occur in SqP after short random interval after moving 1st line (or with some probability after 2nd line in 3-line puzzles); 3 SqP task banks also isomorphic;- interrupting listing task used in [Farrimond 06] - one which would conflict with verbal memory), (puzzle task - sorting or arranging for nonverbal interruption)
- [Storch 92] shows on-screen interruptions more disruptive than phone/visitor interruptions with data entry tasks on GUI and CUI interfaces (GUI facilitating simultaneous execution of main task during interruption);
- phone interruption shows no difference from control group; screen interruption (similar modality) shows worst disruption, then walk-in interruption
- a simulated office - door, phone, desk, computer, etc.
- GUI allows users to work simultaneously on main task during an interruption
- [Speier 03] - did not vary simplicity/complexity of interrupting tasks, but main tasks, varied interruption condition BS; all interruptions were data acquisition tasks that could be carried out regardless of the interface/task complexity of the main task;
- models of interruptions identified; did not control social characteristics of interruption - will this factor be worse for older adults?
Materials
- 2 adapted C-TOC tests; verbal: sentence comprehension (SC); non-verbal: square puzzles (SqP)
Procedure
- Review and Sign Consent form
- Administer MoCA and NAART tests
- Demonstrate interruption practice tasks
- experimental tasks A & B, counterbalanced between task types, 3 banks of each counterbalanced on interruption demand
- NASA-TLX questionnaire administered between each bank
- Brief interview re: strategies and perceptions
- Compensation
- Regarding the state of the software, I have automated the main task ordering, the interruption sequence for 3 task sets (for each of the 2 main tasks), and the subset of tasks within each task set on which interruptions occur, which is completed using an initialization screen by the experimenter. This is followed by the main portal UI to the 12 possible interactive states (SC demo, SC training, SC_{A,B,C}, SqP demo, SqP training set, SqP_{A,B,C}, low-demand interrupt demo and high-demand interrupt demo). If the main task-order is set SC-SqP on it initialization screen, SqP options on the main portal UI are disabled until all SC states are completed, and vice versa; also after a task set is completed its portal link becomes disabled. The main 12-button portal is needed to allow users to experience the demo states more than once, and to permit users a short break between task sets, and to serve as visual indication to the user of what remains to be done. If you want I can show you this before our next meeting at your convenience.
Hypotheses
Age-interaction effects in H1-4 are explained by theories of cognitive ageing: a decrease in working memory [Craik 82], [Hasher 88] - inhibition theory, a loss of sensory acuity [Lindenberger 94], a drop in processing speed [Salthouse 96]; increased distractibility and interruptibility in old age, resulting from decreased ability to suppress some stimuli and enhance others;- H1: Age interaction effects for speed and accuracy; Older adults performing proportionally worse on interruption conditions
- H2: High-demand interruptions will incur worse performance than low-demand interruptions; Possible age interaction or age-related compensation (attention enhancement and suppression) low-demand interruptions may cause mind to wander, become distracted with internal thoughts; high-demand interrupting tasks to main task will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than low-demand interrupting tasks.
- following from [Gillie 89], interruptions that are similar in nature to the main task being performed will have a greater negative effect (again in terms of TWT, TOT, TOI, and error rate) than dissimilar interruptions. Once again a greater effect can be expected for older adults.
- From [Storch 92], interruptions using a similar modality (on screen vs. on the phone or in person) will have a greater detrimental effect on users.
- From [Speier 03], interruptions differ from distractions in that they exist in the same modality as the main task, leading to an overloading of cues. Distractions during complex tasks can still be detrimental to performance do to an overall cognitive overload.
- H3: Performance detriments greater for verbal task (SC) than non-verbal task (SqP); Age interaction possible; following from [Speier 03], detriments to performance following interruptions will be more severe for verbal tasks (SC) than non-verbal tasks (SqP). An interaction would exist in that older adults performance on verbal tasks following interruptions are proportionally worse than younger adults performance, compared to the performance of both groups on non-verbal tasks.
- H4: Self-reports regarding disruptive effects of interruptions to reinforce H1-H3
Former Hypotheses
- fH1: performance measures (speed): disruptions caused by interruptions will lead to greater total working time (TWT) for older adults than younger adults on time-intensive tasks. A similar difference should be found in increases in time on task (TOT). Time on interruption (TOI) differences should also be expected, as time to re-orient to and from the main task (change-over and resumption) are expected to be greater for older adults. Older adults will commit more errors and omissions than younger adults on the main task, both groups are expected commit more errors/omissions upon task resumption compared to prior to the interruption, however a greater effect is expected for older adults.
- fH2: performance measures (accuracy): disruptions caused by interruptions will lead to a decrease in TWT and TOT for younger adults (following from [Zijlstra 99]), and an increase in TWT and TOT for older adults. Similar differences in TOI and change-over/resumption times are to be expected as in H1. (Zeigarnik effect)
Implications of hypotheses
- types of prompts (how much info is too much info for C-TOC - worry about too much aid)
- [Parnin CHI 10] - relevant prompts (developer user)
- snapshots / instant replay of past actions
- task sketches - annotations of a task breakdown: steps, objectives, plans + goals (short animated clip or series of images)
- prospective cues - contextual reminders that are displayed when a condition is true;
- change summary - natural language summary of past actions in log format
- simple contextual cue of recent action: visual highlighting of last object moved/clicked (glow), display origin distinguished from current location of recently moved object
- repeat of selected instructions and speed/accuracy tradeoff - to avoid overcompensation on speed at expense of accuracy
- do older adults compensate to overcome deficiencies in attention suppression and enhancement; does interruption recovery lead to increased fatigue for older adults?
- use of interruption lag [Altmann 04], [Hodgetts 06] - do older adults make use of contextual cues from the interruption lag and resumption lag to resume a task, or does automatic encoding of task goals occur less in older age? what are the implications for prompting systems?
- from [Speier 03] spatial presentation formats able to mitigate effect of interruptions while symbolic formats were not for complex tasks;
- use of icons and graphics rather than text-based resumption prompts
- crossover point exists between perceptual and analytical processes - where is the crossover for older adults?
- graphical resumption prompts will be more effective than verbal resumption prompts, regardless of the type of main task. A proportionally greater benefit will be seen for older adults than younger adults.
- [Storch 92] - GUI better than CUI for concurrent execution of main task and interrupting task - likely not a factor with C-TOC
- [Parnin CHI 10] - relevant prompts (developer user)
| I | Attachment | History | Action | Size | Date | Who | Comment |
|---|---|---|---|---|---|---|---|
 docx |
BrainFreeze_expt_script.docx | r1 | manage | 18.9 K | 2011-01-11 - 22:33 | MatthewBrehmer | experiment script |
 pptx |
IDRG-MUX-Jan12.pptx | r1 | manage | 1383.0 K | 2011-01-11 - 19:48 | MatthewBrehmer | MUX presentation Jan 12/11 |
 xlsx |
mb_experiment_ordering.xlsx | r1 | manage | 13.4 K | 2011-01-07 - 02:24 | MatthewBrehmer | sequences and participant ordering |
Topic revision: r28 - 2011-05-31 - MatthewBrehmer
Site map
BETA web
Users
Groups
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback