• G N.S.

• G*ID sig. (F(4,66) = 3.25, p = 0.02, p.eta^2 = 0.165)

• • 1. "strategy stayed the same" [pretended [passive] interruption was [active] interruption, played n-back game in her head]

• • 5. "I zoned out [during the passive interruption], [not thinking about puzzle task instructions] and that was the problem. I thought not having to do anything was harder. With [the active interruption] I was focused, I was still, like, doing stuff. The information I had stored was still there, whereas when you zone out, it's just like, things get lost."
  • 6. "No difference. [during the passive interruption], I was trying to remember the main task."

• • 10. "I actually like the [active interruption] better than the passive one, in terms of stimulus, I guess. it was more interesting in that sense." [on passive interruption mind wandering]: "a little bit. It was easier to remember [the main task] during the passive one what the instructions were. During the [active interruption] you would have to remember partially"
  • 11. "It did get more difficult when I had to click on images [in the active interruption]. In the [passive interruption], I could remember [the instructions]. In the [active interruption] when I came back I wasn't quite sure what to complete. [In passive interruption, may have played n-back game in her head (likely some misunderstanding of the question)]"
  • 12. "For the active interruption, I may have doubted my memory, since the interruption required memory as well. In the [passive interruption] I just looked at [the images] but didn't try to think about them."

• • 1. "yes. there was difference. [active]: I was dedicated to the interruption, I had made the map, I was kinda going "I'm doing my best on this one and when I get back the [main task] we'll see where I am and we'll deal with it then." [passive]: "I didn't really concentrate on the interruption at all when I realized that there was going to be no need count backwards or keep track of [the images]. There could have been a surpirse so I was consciously watching the images but I wasn't trying to remember anything. I glossed over that and was keeping [the main task] more so in mind so that I didn't lose track of that."
  • 2. "It was easier to endure an interruption in which you didn't have to do anything. Your mind is still on the previous task." [Thinking more of [the puzzle task] during the [passive interruption]]. [active]: "I tried to do the interruption to the best of my ability."

• • 6. "No difference in strategy, I just looked out the window, taking a break [during passive interruption, wasn't trying to rehearse main task]"
  • 7. "If the interruption was [active], you just very patiently try to stay where you were in the [main task], but in the [active interruption] you had to shelve your other memory and bring forward the new memory, and it was a lot of work. In the [passive interruption], you just put yourself in neutral, I guess is the best way to explain it, just wait for this to be over and we'll get back to it. It wasn't a break time, it was "wait, and you'll get back to [the main task]". Only in some cases [were instructions rehearsed]"
  • 8. "I tuned the [passive interruption] right out, my mind was ready to get back to [the main task]. The active one needed some thought. [SC] was harder to return to."

• • 10. "If I didn't have to monitor, then I just let it pass, but I started reciting each one, saying it in my mind as a practice for the [active interruption]" [did not rehearse main task]

• • 2. "If I didn't have to click I was able to rehearse what I was going to do. If I had to click then my was obviously involved in something else" [unable to rehearse]

• • 4. "the clicking one was just more destructive, make you tend to forget what it is you're supposed to be doing when you went back." [passive interruption]: "I watched the images but I was also focusing on what I should be doing when I get back. Every couple seconds reminding myself of what I should be doing when I go back."
  • 5. [active]: "I paid attention to the interruption, but I had filed in my mind where I had left off, since I was warned that the interruption was coming. So I was just: "Okay, mental note." "The [passive] type was much easier because you could just ignore it. A lot of the time I didn't pay attention to it, I just tried to remember what I was going to get back to in the main task, like "just get this over with and get back to it." [Was thinking of main task during passive interruption]
  • 6. "When I went back, regardless of what the interruption was, there was no difference in my strategy." [passive]: "half way through I realized that I didn't have to have my finger on the cursor and I could just look at [the images]. I didn't think about main task, I just took a break."
  • 7. "the active was more difficult. I was concentrating more on the interruption than on the task." [passive]: "it was easier when you didn't have to click - all you did was watch, better able to concentrate on [main tasks]"
  • 8. [passive]: "it was no big deal because you didn't have to do anything. just waiting for it to finish so you wouldn't forget too much, trying to remember what it was supposed to be." [active]: "you were far to busy"
  • 9. [active]: "it was a little harder, harder to remember the more difficult ones" [unable to think of main tasks]. [passive]: "I let my mind go blank."
  • 10. [passive]: "If I did it was not consciously."
  • 11. "I found the passive one annoying, you were just sitting there, but it was easier. Didn't rehearse, I had done that already during the [interruption lag]. I was annoyed. I am an action person so I didn't mind shifting to a new activity whereas just sitting there was annoying."
  • 12. "I found it extremely easy to ignore the [passive interruption], I turned my mind off. in the [active] interruption I had less opportunity to retain the instructions."

• • 1. "ignored passive interruption, I didn't have to put forth any energy, I relaxed. [active interruption] I had to think"

• • 3. "During passive interruption: I tried to think about [the SC task]. [active]: I could only do one thing at a time."
  • 4. "the clicking one was just more destructive, make you tend to forget what it is you're supposed to be doing when you went back. [passive interruption]: I watched the images but I was also focusing on what I should be doing when I get back. Every couple seconds reminding myself of what I should be doing when I go back."
  • 5. "[active]: I paid attention to the interruption, but I had filed in my mind where I had left off, since I was warned that the interruption was coming. So I was just: "Okay, mental note:" [passive] type was much easier because you could just ignore it. A lot of the time I didn't pay attention to it, I just tried to remember what I was going to get back to in the main task, like "just get this over with and get back to it." [Was thinking of main task during passive interruption]."

• • 11. "I found the passive one annoying, you were just sitting there, but it was easier. Didn't rehearse, I had done that already during the flashy light. I was annoyed. I am an action person so I didn't mind shifting to a new activity whereas just sitting there was annoying"
  • 12. "I found it extremely easy to ignore the [passive interruption], I turned my mind off. in the [active] interruption I had less opportunity to retain the instructions"

Interviews

Task Difficulty

Which task was more difficult, regardless of whether or not they were interrupted?

• G.1: SC (4), SqP (7), No answer (1)

• G.2: SC (4), SqP (8)

• G.3: SC (3), SqP (8), equally difficult (1)
  • "[SC]"
  • "equally difficult, but difficult in different ways: verbal vs. spatial"
  • "SC: I couldn't always remember what I was supposed to be doing. SqP: I mainly could see what I was supposed to do, but I still blew it on occasion. I don't think I thought it out completely at times. I went ahead and I should have maybe considered my moves."
  • [SC]
  • "!SqP: I don't see how to make the boxes"
  • "!SqP: because I didn't do as well on it."
  • "!SqP: I couldn't get it! I found it difficult to figure it out. SC easier (colours made it easier)"
  • "!SqP: I don't know. When you had to move 3 lines, you had to look at it more [strategically]."
  • "!SqP: I couldn't see the image in my brain ahead of time"
  • "!SqP: not entirely sure why - only one simple instruction. SC easier - directions were clear, very specific as to what to do. Distinction between general and specific instructions. Fatigue factored into performance."
  • "!SqP, it took time after realizing I couldn't rotate the lines"
  • "!SqP: I felt it required more mental activity to decide which lines to move than to remember the instructions. The most difficult part about [SC] was remembering the instructions, they were easy to follow if you could remember them. In [!SqP] the more difficult part was determining which lines to move, but the instructions were extremely simple."

Interruption Disruption

Which task was more disrupted by interruptions?

• G.1: SC (9), SqP (1), equally affected (1), no answer (1)

• G.2: SC (6), SqP (2), equally affected (4)

• G.3: SC (12)
  • 1. "[SC]"
  • 2. "SC: I had an easier time remembering shapes than words. SqP: If I could figure it out, I could remember it."
  • 3. "SC: it destroyed my concentration. No effect of interruptions on SqP"
  • 4. "SC: more things to remember"
  • 5. "SC: there were more things to make mental notes of."
  • 6. "SC: I couldn't remember what I was supposed to do to finish it off. In [!SqP] I was able to pick it up and try to finish it."
  • 7. "SC: I don't know. I couldn't remember, after an interruption I couldn't bring it up again. In SqP I think I could remember better, the squares."
  • 8. "SC, because you had more to remember. [!SqP]: you didn't have as much to remember."
  • 9. "SC: I had to remember shape and colour, whereas in [!SqP] there was just movement."
  • 10. "SC, more of an interruption, even though I had more difficulty with [!SqP] - the memory as opposed to strategy."
  • 11. "SC, instructions more complicated. In [!SqP] everything was pretty much the same and you knew what you were doing."
  • 12. "SC, because the instructions had to be memorized, I felt the interruptions impeded my ability to remember what I was doing. With [!SqP], the instructions, which were simple to begin with, reappeared on the screen, there was in effect to memory to be done. Interruptions didn't affect [!SqP] performance."

Interruption Lag / Delay

Was the interruption lag delay helpful?

• G.1: No (5), Yes/somewhat (6), no answer (1)

• G.2: No (6), Yes/somewhat (2), no/mixed opinion (3), no answer (1)

• G.3: No (7), Yes/somewhat (4), mixed opinion (1)
  • 1. "no, not really"
  • 2. "delay helpful. It allowed me to visualize what I was going to do: strategy was the same [between tasks]"
  • 3. "delay not really helpful. It was frustrating because I was about to move something. I knew what I was going to do next."
  • 4. "delay helpful. you could concentrate on what you should be remembering. SC: I tried to remember what I did last. SqP: I remembered the number of moves [That I already made], and how many I had left"
  • 5. "yes and no. In one way it was helpful and in another way my mind says: "Okay, get it over with". I tried to remember in [!SqP] to move that line to that place or what I was going to next in [SC]."
  • 6. "I just waited. In some of them I just looked where I was and tried to remember to just pick up where I left off. In [!SqP] I don't really think that I tried to remember anything because it was a case of just moving the lines, whereas in [SC] there was a result that was wanted."
  • 7. "I said "darn it". frustrating."
  • 8. "no, actually, it was a nuisance. You couldn't move anything so it was too late anyways."
  • 9. "I would try to say to myself what the task was [in SC only]"
  • 10. "not particularly, early on I was clicking on it because I wanted to get on with it. [re: strategies] I didn't think so, not consciously."
  • 11. "annoying, very annoying. While it was flashing you knew you had to remember what you were doing and where things had to go when you finished. In SC you had to remember the specifics.In SqP you kinda knew what you had to do, it wasn't so important to remember"
  • 12. "I did not, because I didn't make good use of it. In [!SqP] I did [remember current state] because it was readily apparent by looking at the screen what I had planned to do next, if I had planned to do anything next, whereas with [SC] looking at the screen didn't do any good. Because the interruption came so early in the primary task there was no evident pattern on the screen to help me to remember."

Task Resumption Strategies

What task resumption strategies were used?

• G.1
  • 1. "nothing in particular, I usually knew where I left off. [SC long sequences repeated internally]. [!SqP] I knew where I'd placed the last line"
  • 2.
  • 3. "recalling instructions, [SC] visualized clockwise in a starting corner. [!SqP] to remember what I was doing/thinking before interruption"
  • 4. "[SC] recalling a sequence of colours, in [!SqP], I just looked at instructions again."
  • 5. "I tried to retrieve what I had stored"
  • 6. "in [SC] I tried to picture where the shapes are supposed to be and I follow that. in [!SqP] I don't really have to remember"
  • 7. "SC: Other than remembering the instructions themselves, I'd place my eyes over the screen in order to remember what goes where"
  • 8. "I tried to refocus on the screen I had left before the interruption. In [SC] I looked for some clues to help me remember if it was a shape or a size or colour, whereas in [!SqP] I just had to completely refocus on the puzzle to solve it."
  • 9. "I usually had time to start the puzzle so then I would just go back to what I had already started to see what I was going to do next [same for SC and SqP]"
  • 10. "Not really, usually I just tried to remember to the best of my ability - [in SqP what was done last, in SC, just key words]"
  • 11. "I tried to memorize 3 colours [SC], in my head I went over what I had to go back to, [reciting just the key words], in [!SqP] I just remembered where the next line would be and complete the square. [during interruption lag], I would memorize where that line had to go so when the screen came back I would focus back there and drag the line"
  • 12. "No because in my head I would still be thinking of the puzzle, so I would just go back into it, more for [SC]"

• G.3
  • 1. "SC: I tried to remember key words; SqP: that just seemed automatic, not affected by interruption"
  • 2. "SC: I tried to visualize the shapes. SqP: try and remember where I was going to put things."
  • 3. "SC: that was the most difficult. SqP: I was frustrated when there were some lines left over."
  • 4. "SC: I tried to remember the initial instruction."
  • 5. "It was kind of a mental note I was making about what I was going to do next. In [SC] I had to remember what I was going to do period. [!SqP] was easier in that sense because there was only one thing to do. In [SC] when and interruption occurred, you had to remember more stuff. I remembered what I was going to do next."
  • 6. "[remembered patterns, difficult to remember final steps in SC instructions]. Never thought about [how many moves already made in [!SqP]]"
  • 7. "no, not really, maybe I should have. [Same for both tasks.]"
  • 8. "[!SqP] was easy to get back to because you knew what you had to do whereas in [SC] you had to remember what you were supposed to do in that. "
  • 9. "I didn't try to remember where I had left off, I tried to remember the signals I had given myself during the [interruption pending notification]. I tried to repeat that to myself. In [!SqP] it was just continuing where I had left off."
  • 10. "not consciously."
  • 11. "in [!SqP] I just tried to remember what lines I had to move. In [SC] it was kinda "which colour to where",different strategies. the flashy delay allowed you to do that. If it hadn't been for the flashy delay it would have been harder because you were shifting immediately to the interruption tasks."
  • 12. "I don't think so. In [!SqP] it was evident what I had intended to do next. In [SC] it was a matter of returning my mind to the original instructions which quite frankly either were there or were not."

Interruption Types

Did strategy differ between passive and active interruptions?

• G.1 - no difference / zoned out / harder to maintain concentration in passive interruption / played n-back practice with passive;
  • 1. "strategy stayed the same, pretended [passive] interruption was [active] interruption, played n-back game in her head"
  • 2.
  • 3. "no difference in strategy"
  • 4. "no. just watched the images in the passive interruption to see if they change"
  • 5. "I zoned out [during the passive interruption], [not thinking about puzzle task instructions] and that was the problem. I thought not having to do anything was harder. With [the active interruption] I was focused, I was still, like, doing stuff. The information I had stored was still there, whereas when you zone out, it's just like, things get lost."
  • 6. "No difference. [During the passive interruption], I was trying to remember the main task."
  • 7. "It would take a different type of focus. It was easier to take a break on [!SqP] than on [SC], in [SC] I had to play the instructions in my mind during the interruption. [During active interruption] I tried to retain [the instructions] somehow in my mind while allowing my eyes to do the thinking, if you want to put it that way."
  • 8. "For the passive-type interruption I found it harder to maintain concentration on the interruption so I'd frequently go back in my mind to the puzzle I was doing. [Didn't have the opportunity to do this during the active interruption]"
  • 9. "[during passive interruption] I had a chance to review the instructions in my head. I wasn't as focused [on the images] as when I had to click on them."
  • 10. "I actually like the [active interruption] better than the passive one, in terms of stimulus, I guess. it was more interesting in that sense. [passive interruption mind wandering] a little bit. It was easier to remember [the main task] during the passive one what the instructions were. During the [active interruption] you would have to remember partially"
  • 11. "It did get more difficult when I had to click on images [in the active interruption]. [In passive interruption], I could remember [the instructions]. In the [active interruption] when I came back I wasn't quite sure what to complete. [In passive interruption, may have played n-back game in her head (likely some misunderstanding of the question)]"
  • 12. "For the active interruption, I may have doubted my memory, since the interruption required memory as well. In [passive interruption] I just looked at [the images] but didn't try to think about them."

• G.2 - passive: relaxed, not think, practice for active interruptions
  • 1. "yes. there was difference. [active interruption]: I was dedicated to the interruption, I had made the map, I was kinda going "I'm doing my best on this one and when I get back the [main task] we'll see where I am and we'll deal with it then. [passive interruption] I didn't really concentrate on the interruption at all when I realized that there was going to be no need count backwards or keep track of [the images]. There could have been a surpirse so I was consciously watching the images but I wasn't trying to remember anything. I glossed over that and was keeping [the main task] more so in mind so that I didn't lose track of that."
  • 2. "It was easier to endure an interruption in which you didn't have to do anything. Your mind is still on the previous task. Thinking more of [the puzzle task] during the [passive interruption]. [re: active interruption] I tried to do the interruption to the best of my ability."
  • 3. "[passive interruption: wasn't able to rehearse SC task]. I just sat there, and while I was sitting there I was thinking about my plan."
  • 4. "You could just go to sleep with the passive one if you want, you don't actually have to pay attention. The other one you do, so you need to be more alert for that one. For the passive one, I just kind of chilled out for a minute [wasn't thinking of main task]."
  • 5. "In the passive type, I guess you could just basically relax while it was happening, not really [thinking of puzzle tasks]"
  • 6. "No difference in strategy, I just looked out the window, taking a break [during passive interruption, wasn't trying to rehearse main task]"
  • 7. "If the interruption was [active], you just very patiently try to stay where you were in the [main task], but in the [active interruption] you had to shelve your other memory and bring forward the new memory, and it was a lot of work. In the [passive interruption], you just put yourself in neutral, I guess is the best way to explain it, just wait for this to be over and we'll get back to it. It wasn't a break time, it was "wait! and you'll get back to [the main task]". Only in some cases [were instructions rehearsed]"
  • 8. "I tuned the [passive interruption] right out, my mind was ready to get back to [the main task]. The active one needed some thought. [SC] was harder to return to."
  • 9. "[No difference], during passive interruption, I was taking a break [during SqP, but tried to remember instruction in SC]
  • 10. "If I didn't have to monitor, then I just let it pass, but I started reciting each one, saying it in my mind as a practice for the [active interruption] [did not rehearse main task]"
  • 11. "I don't think so [no difference in strategy] performance didn't differ"
  • 12. "the interruptions didn't really bother me. [in passive interruption, didn't think about main task]. I didn't really watch, I refocused."

• G.3 - relaxed during passive interruption, rehearsed instructions, mind went blank, ignored passive interruption, thought about main task,
  • 1. "ignored passive interruption, I didn't have to put forth any energy, I relaxed. [active interruption] I had to think"
  • 2. "If I didn't have to click I was able to rehearse what I was going to do. If I had to click then my was obviously involved in something else [unable to rehearse]"
  • 3. "During passive interruption: I tried to [think about SC task]. active: I could only do one thing at a time."
  • 4. "the clicking one was just more destructive, make you tend to forget what it is you're supposed to be doing when you went back. passive interruption: I watched the images but I was also focusing on what I should be doing when I get back. Every couple seconds reminding myself of what I should be doing when I go back."
  • 5. "[active]: I paid attention to the interruption, but I had filed in my mind where I had left off, since I was warned that the interruption was coming. So I was just: "Okay, mental note:" [passive] type was much easier because you could just ignore it. A lot of the time I didn't pay attention to it, I just tried to remember what I was going to get back to in the main task, like "just get this over with and get back to it." [Was thinking of main task during passive interruption."
  • 6. "When I went back, regardless of what the interruption was, there was no difference in my strategy. [passive] - half way through I realized that I didn't have to have my finger on the cursor and I could just look at [the images]. Didn't think about main task, just took a break."
  • 7. "the active was more difficult. I was concentrating more on the interruption than on the task. [passive] it was easier when you didn't have to click - all you did was watch, better able to concentrate on [main tasks]"
  • 8. "[passive] it was no big deal because you didn't have to do anything. just waiting for it to finish so you wouldn't forget too much, trying to remember what it was supposed to be. [active] you were far to busy"
  • 9. "[active]: it was a little harder, harder to remember the more difficult ones, [unable to think of main tasks]. [passive]: I let my mind go blank."
  • 10. "[passive] If I did it was not consciously."
  • 11. "I found the passive one annoying, you were just sitting there, but it was easier. Didn't rehearse, I had done that already during the flashy light. I was annoyed. I am an action person so I didn't mind shifting to a new activity whereas just sitting there was annoying"
  • 12. "I found it extremely easy to ignore the [passive interruption], I turned my mind off. in the [active] interruption I had less opportunity to retain the instructions"

An analysis with 2 age groups (split at the median age of 64) was also performed; however this did not reveal any new aspects in the data. Main effects and interactions as seen with 3 groups were largely still present with 2 groups. Outlier analysis using box-plots and cost-of-interruption scatter plots was also performed, however no significant outliers were found.

Across Tasks Analysis (Sentence Comprehension & Square Puzzles)

Within Tasks Analysis

• the global results are not sensitive enough to the interactions taking place at the local and RLT scale; therefore, global completion time and score results are not reported in the within-tasks section (only age effects are present in global analysis at within-tasks level of analysis)

Younger adults had significantly shorter task resumption lag times than pre-old and older adults. The no-interruption condition's inter-action interval was significantly shorter than the task resumption lag following a high-demand interruption, and marginally shorter than the task resumption lag following a low-demand interruption.

• notes: scores are out of 10 before ART transformation (non-parametric factorial analysis)

Fatigue

Younger adults reported marginally lower levels of mental demand than older adults. Pre-old adults reported significantly lower levels of mental demand than older adults.

Younger adults reported the task to be significantly less annoying pre-old adults. The no-interruption condition was reported to be significantly less annoying than the low-demand and high-demand conditions.

The low-demand interruptions were reported to be significantly less annoying than high-demand interruptions.

Younger adults reported to have marginally higher performance than older adults.

There were no significant differences in self-reported physical demand across age groups or interruption demand conditions, nor were there interactions.

Fatigue

Younger adults reported the task to be marginally less demanding than older adults. Pre-old adults reported the task to be significantly less demanding than older adults.

The no-interruption and low-demand condition were reported to be marginally less annoying than the high-demand condition.

There were no significant differences in self-reported interruption annoyance across age groups or interruption demand conditions, nor were there interactions.

Younger adults reported performing significantly better than older adults. They reported a drop in performance in the low-demand condition, then an increase in performance in the high-demand condition (marginally significant). Older adults reported an increase in performance in the low-demand condition (not significant), then a decrease in performance in the high-demand condition (significant). In the no-interruption condition, younger adults reported significantly higher performance than older adults. In the high-demand condition, both younger and pre-old adults reported significantly higher performance than older adults.

There were no significant differences in self-reported physical demand across age groups or interruption demand conditions, nor were there interactions.

Other Factors (Local)

• no main effect on SC log completion time; no interactions with G or ID; 3-way interaction G*ID*Ord (p = 0.03)
• no main effect or interactions on SC score
• no main effect or interactions on SqP log completion time
• marginal main effect (p = 0.06) on SqP score such that those with ordering NLH had lower scores; no interactions

• no main effect or interactions on SC log completion time
• no main effect or interactions on SC score;
• no main effect on SqP log completion time; interaction G*TOrd sig. (p = 0.04) such G.1 Ps with task order SC-!SqP had lower SqP logT than those with SqP-SC ordering; G.2 and G.3 had opposite trend;
• no main effect or interactions on SqP score;

• order number and distribution (G.1,G.2,G.3):
  • ABC: 5 (2,2,1)
  • ACB: 8 (2,4,2)
  • BAC: 5 (2,1,2)
  • BCA: 6 (2,2,2)
  • CAB: 6 (2,1,3)
  • CBA: 6 (2,2,2)
• interaction ID*BankOrd on log completion time sig. (p = 0.04)
• interaction G*ID*BankOrd on log completion time M.sig. (p = 0.06)

• order number and distribution (G.1,G.2,G.3):
  • ABC: 8 (2,3,3)
  • ACB: 2 (1,1,0)
  • BAC: 5 (4,0,1)
  • BCA: 5 (0,3,2)
  • CAB: 7 (2,1,4)
  • CBA: 9 (3,4,2)
• interaction G*ID*BankOrd on score

• MB (Lenovo thinkpad) (6 in group 1, 6 in group 2, 9 in group 3)
• CL (IBM thinkpad) (6 in group 1, 6 in group 2, 3 in group 3)
• no effects or interactions

• no effects or interactions

Local completion time (Across Tasks)

Younger adults are significantly faster than older adults. Sentence Comprehension is faster than Square Puzzles. There are significant interactions between interruption demand and age, between task and interruption demand, and between age, interruption demand, and task.

• G sig. (p < 0.01) young < old

• ID N.S.

• T sig. (p < 0.01) SC < SqP

• G*ID sig. (p = 0.03)

• G*T N.S.

• T*ID sig. (p = 0.02)

• G*ID*T M.sig (p = 0.1)

Local completion time (SC)

Younger adults are significantly faster than older adults. The no-interruption and low-demand interruption conditions are significantly faster than the high-demand condition.

• G sig. (p < 0.01) young < old

• ID sig. (p < 0.01) ID.N < ID.H, ID.L < ID.H

• G*ID N.S.

Local completion time (SqP)

Younger adults are significantly faster than older adults. There is a significant interaction between interruption demand and age: younger adults are slower in low-demand interruption condition than in other conditions, while older adults are faster in low-demand interruption condition than in other conditions.

• G sig. (p = 0.04) young < old

• ID N.S.

• G*ID sig. (p = 0.03)
  • young SqP log completion times increase from ID.N to ID.L, then decrease from ID.L to ID.H
  • old SqP log completion times decrease from ID.N to ID.L, then increase from ID.L to ID.H

Local score (Across Tasks)

Younger adults score significantly higher than older adults. There is a significant interaction between interruption demand and age: younger adults have lower scores in low-demand interruption condition than in other conditions, while older adults have highest score in no-interruption condition, and lowest score in high-demand interruption condition.

• G sig (p = 0.02) young > old

• ID N.S.

• T N.S.

• G*ID M.sig. (p = 0.1)
  • young scores decrease from ID.N to ID.L, and increase from ID.L to ID.H
  • old scores decrease from ID.N to ID.L and again from ID.L to ID.H

• G*T N.S.

• T*ID N.S.

• G*ID*T N.S.

Local score (SC)

Younger adults score significantly higher than older adults. Scores decrease marginally as interruption workload increases.

• G sig. (p < 0.01) young > old

• ID M.sig. (p = 0.08) ID.N > ID.L > ID.H (pairwise N.S.)

• G*ID N.S.

Local score (SqP)

Younger adults score significantly higher than older adults. There is a marginal interaction between interruption demand and age: younger adults have lower scores in low-demand interruption condition than in other conditions, while older adults have highest score in no-interruption condition, and lowest score in high-demand interruption condition.

• G sig. (p = 0.05) young > old

• ID N.S.

• G*ID M.sig. (p = 0.09)
  • young scores decrease from ID.N to ID.L, and increase from ID.L to ID.H
  • old scores decrease from ID.N to ID.L and again from ID.L to ID.H

Global completion time(Across Tasks)

Younger adults are significantly faster than older adults. The no-interruption condition significantly faster than high-demand interruption condition. The no-interruption condition is marginally faster than the low-demand interruption condition. Sentence Comprehension is faster than Square Puzzles.

• G sig (p < 0.01) young < old

• ID sig. (p = 0.03) ID.N < ID.H, ID.N < ID.L (M.sig)

• T sig. (p < 0.01) SC < SqP

• G*ID N.S.

• G*T N.S.

• T*ID N.S.

• G*ID*T N.S.

Global completion time(SC)

Younger adults are significantly faster than older adults. The no-interruption condition is significantly faster than the high-demand interruption condition.

• G sig. (p < 0.01) young < old

• ID sig. (p < 0.01) ID.N < ID.H

• G*ID N.S.

Global completion time(SqP)

Younger adults are marginally faster than older adults. The no-interruption condition is marginally faster than the low-demand interruption condition.

• G M.sig (p = 0.08) young < old

• ID M.sig (p = 0.08) ID.N < ID.L

• G*ID N.S.

Results

• Scope of analysis:
  • Local: includes the mean DV for subset of interrupted trials only; corresponding uninterrupted trials from the ID.N condition are also included; all other trials are discarded from analysis; the interrupted subset of trials is selected semi-randomly for each participant, with the condition that interruptions span a range of trial difficulty levels; since trials in each bank increase in difficulty, this makes 3 or 4 consecutive interrupted trials unlikely;
  • Global: included the mean DV for all trials in a bank (interrupted and uninterrupted) (except trial 1, which is treated as a practice trial and is excluded from all analysis);
    • the global results are not sensitive enough to the interactions taking place at the local and RLT scale; therefore, global completion time and score results are not reported in the within-tasks section

• Data transformations:
  • all completion time, inter-action interval (IIA: average time between valid moves) time, and resumption lag time (RLT: time between interruption dismissal and subsequent valid move) results are log-transformed to correct positive skew, to lower the effects of outliers
  • trial completion times do not include in the interruption
  • all score results are normalized per trial (out of 1.00); results are averaged in the local and global analysis; the ART aligned rank transformation is then applied to the data, allowing for non-parametric factorial analysis using ANOVA procedures; this corrects for the score ceiling effect and negative skew; for more information, see [Wobbrock, Findlater, Gergle, and Higgins. "The Aligned Rank Transformation for Nonparametric Factorial Analyses Using only ANOVA Procedures. Proceedings of CHI 2011, 143-146, Vancouver BC].
  • all questionnaire data is non-parametric and is therefore also analyzed using the ART technique;

• Notes:
  • N.S. = not significant
  • sig = significant (p < 0.05)
  • M.sig = marginally significant (p <= 0.1)

Across Tasks (Sentence Comprehension & Square Puzzles)

• all analysis are 3 (G: age, BS) x 3 (ID: interruption demand, WS) x 2 (T: task, WS) mixed design
• age levels: G.1 = young, G.2 = pre-old, G.3 = old
• interruption demand levels: ID.N = no interruption, ID.L = low-demand interruption, ID.H = high-demand interruption
• task levels: T.SC = sentence comprehension, T.!SqP = square puzzle

Completion Time

• note: A < B = A is faster than B (A has lower log completion time)

Local

• G sig. (F(2,33) = 9.90, p < 0.01, p.eta^2 = 0.375)
  • pairwise: G.1 < G.3 (p < 0.01); G.2 < G.3 (M.sig p = 0.07)

• ID N.S.

• T sig. (F(1,33) = 49.65, p < 0.01, p.eta^2 = 0.601)
  • pairwise: SC < SqP (p < 0.01)

• G*ID sig. (F(4,66) = 49.65, p = 0.02, p.eta^2 = 0.165)
  • G.3 are best in ID.L condition; G.1, G.2 are worse in ID.L condition
  • interaction explored in Within-Tasks Analysis (see below)

• G*T N.S.

• T*ID sig. (F(2,66) = 4.69, p = 0.01, p.eta^2 = 0.124)
  • SC logT increases linearly with ID, SqP logT highest in ID.L condition
  • interaction explored in Within-Tasks Analysis (see below)

• G*ID*T sig. (F(4,66) = 2.85, p = 0.03, p.eta^2 = 0.147)
  • G.3 is worse than G.1, G.2 on both tasks, but perform proportionally better (relative to themselves) in SqP ID.L condition
  • interaction explored in Within-Tasks Analysis (see below)

Global

• G sig. (F(2,33) = 12.96, p < 0.01, p.eta^2 = 0.440)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M. sig p = 0.05); G.2 < G.3 (p = 0.04)

• ID sig. (F(2,66) = 3.90, p = 0.03, p.eta^2 = 0.106)
  • pairwise: ID.N < ID.L (M.sig p = 0.08); ID.N < ID.H (M.sig p = 0.05)

• T sig. (F(1,33) = 125.93, p < 0.01, p.eta^2 = 0.792)
  • pairwise: SC < SqP (p < 0.01)

• G*ID N.S.
• G*T N.S.
• T*ID N.S.

• G*ID*T M.sig. (F(4,66) = 2.24, p = 0.07, p.eta^2 = 0.120)
  • G.3 are worse than G.1, G.2 on both tasks, but perform proportionally better (relative to themselves) in SqP ID.L condition

Score

• note: A > B = A has higher normalized score rank than B (after ART)

Local

• G sig. (F(2,33) = 7.39, p = 0.02, p.eta^2 = 0.309)
  • pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (p = 0.05)

• ID N.S.
• T N.S.
• G*ID N.S.
• G*T N.S.
• T*ID N.S.
• G*ID*T N.S.

Global

• G sig. (F(2,33) = 8.22, p < 0.01, p.eta^2 = 0.333)
  • pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (M. sig p = 0.05)

• ID N.S.

• T sig. F(1,33) = 14.97, p < 0.01, p.eta^2 = 0.312)
  • pairwise: SC > SqP (p < 0.01)

• G*ID N.S.
• G*T N.S.
• T*ID N.S.
• G*ID*T N.S.

Task Resumption Lag Time (Local)

• Analysis 1: RLT-L vs. RLT-H = log resumption lag times compared between ID.L and ID.H conditions
• Analysis 2: IIA-N vs. RLT-L vs. RLT-H = log inter-action interval (avg. time between valid moves) in ID.N condition compared against log resumption lag times in ID.L and ID.H conditions

• note: A < B = A has shorter log resumption lag time than B (A resumes task faster than B)

RLT-L vs. RLT-H

• G sig. F(2,33) = 8.93, p < 0.01, p.eta^2 = 0.351)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)

• ID sig. F(1,33) = 4.46, p = 0.04, p.eta^2 = 0.119)
  • pairwise: ID.L < ID.H (p = 0.04)

• T sig. F(1,33) = 23.03, p < 0.01, p.eta^2 = 0.411)
  • pairwise: SC < SqP (p < 0.01)

• G*ID sig. F(2,33) = 6.33, p < 0.01, p.eta^2 = 0.277)
  • G.1 resumes faster in ID.H than in ID.L; opposite holds for G.2, G.3 - steeper ascending RLT slope for older adults;
  • ID RM-ANOVA at G.1: ID N.S.
  • ID RM-ANOVA at G.2: ID N.S.
  • ID RM-ANOVA at G.3: ID sig. (F(1,23) = 13.41, p < 0.01, p.eta^2 = 0.368)
    • pairwise: ID.L < ID.H (p < 0.01)
  • G one-way ANOVA at ID.L: G N.S.
  • G one-way ANOVA at ID.H: G sig. (F(2,69) = 19.20, p < 0.01, p.eta^2 = 0.358)
    • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)
  • interaction explored in Within-Tasks Analysis (see below)

• G*T N.S.

• T*ID sig. F(1,33) = 13.21, p < 0.01, p.eta^2 = 0.286)
  • in SC, steep ascending slope from ID.L to ID.H; in SqP, downward slope from ID.L to ID.H
  • T RM-ANOVA at ID.L T sig. (F(1,35) = 30.90, p < 0.01, p.eta^2 = 0.469)
    • pairwise: SC < SqP (p < 0.01)
  • T RM-ANOVA at ID.H: T N.S.

• G*ID*T sig. F(2,33) = 3.59, p = 0.04, p.eta^2 = 0.179)
  • age, ID effects in SC (no interactions); age but not ID effect in SqP (G*I interaction) exists
  • interaction explored in Within-Tasks Analysis (see below)

IIA-N vs. RLT-L vs. RLT-H

• G sig. (F(2,33) = 7.66, p < 0.01, p.eta^2 = 0.410)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p = 0.01)

• ID M.sig. (F(2,33) = 2.86, p = 0.07, p.eta^2 = 0.732)
  • pairwise: N.S.

• T sig. (F(1,33) = 90.34, p < 0.01, p.eta^2 = 0.080)
  • pairwise: SC < SqP (p < 0.01)

• G*ID sig. (F(4,66) = 4.81, p < 0.01, p.eta^2 = 0.226)
  • G.1 have decreasing RLT from ID.N to ID.H
  • G.2 have have increasing RLT from ID.N to ID.H
  • G.3 have decreased RLT from ID.N to ID.L, then an increase from ID.L to ID.H
  • interaction explored in Within-Tasks Analysis (see below)

• G*T N.S.

• T*ID sig. (F(2,66) = 36.75, p < 0.01, p.eta^2 = 0.527)
  • RLT is lower in SC than SqP
  • RLT increases from ID.N to ID.H in SC
  • RLT decreases from ID.N to ID.H in SqP
  • interaction explored in Within-Tasks Analysis (see below)

• G*ID*T sig. (F(4,66) = 2.77, p = 0.03, p.eta^2 = 0.144)
  • RLT increases across all age groups in SC from ID.N to ID.H
  • RLT decreases in G.1, G.2 in SqP from ID.N to ID.H, except G.3, who increase from ID.L to ID.H
  • interaction explored in Within-Tasks Analysis (see below)

Within Tasks

• all analysis are 3 (G: age, BS) x 3 (ID: interruption demand, WS) mixed design
• age levels: G.1 = young, G.2 = pre-old, G.3 = old
• interruption demand levels: ID.N = no interruption, ID.L = low-demand interruption, ID.H = high-demand interruption
• the global results are not sensitive enough to the interactions taking place at the local and RLT scale; therefore, global completion time and score results are not reported in the within-tasks section

Sentence Comprehension

Completion Time (Local)

• G sig. (F(2,33) = 12.00, p < 0.01, p.eta^2 = 0.421)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M.sig p = 0.07); G.2 < G.3 (M. sig p = 0.05)

• ID sig. (F(2,66) = 12.47, p < 0.01, p.eta^2 = 0.274)
  • pairwise: ID.N < ID.H (p < 0.01); ID.L < ID.H (p < 0.01);

• G*ID N.S.

Score (Local)

• G sig (F(2,33) = 10.46, p < 0.01, p.eta^2 = 0.388)
  • pairwise: G.1 > G.3 (p < 0.01); G.1 > G.2 (p < 0.01)

• ID M.sig (F(2,66) = 2.52, p = 0.09, p.eta^2 = 0.071)
  • pairwise: N.S.

• G*ID N.S

Task Resumption Lag Time (Local)

RLT-L vs. RLT-H

• G sig.(F(2,33) = 11.89, p < 0.01, p.eta^2 = 0.419)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)

• ID sig. (F(1,33) = 35.99, p < 0.01, p.eta^2 = 0.522)
  • pairwise: ID.L < ID.N (p < 0.01)

• G*ID M.sig. (F(2,33) = 3.22, p = 0.05, p.eta^2 = 0.163)
  • all age groups increase in RLT from ID.L to ID.H, however the slopes of G.2 and G.3 are steeper
  • RM-ANOVA at G.1: ID N.S.
  • RM-ANOVA at G.2: ID sig. (F(1,11) = 34.49, p < 0.01, p.eta^2 = 0.758)
    • pairwise: ID.L < ID.N (p < 0.01)
  • RM-ANOVA at G.3: ID sig. (F(1,11) = 12.06, p < 0.01, p.eta^2 = 0.523)
    • pairwise: ID.L < ID.N (p < 0.01)
  • one-way ANOVA at ID.L: G. sig (F(2,33) = 5.67, p < 0.01, p.eta^2 = 0.256)
    • pairwise: G.1 < G.3 (p < 0.01)
  • one-way ANOVA at ID.H: G. sig (F(2,33) = 13.45, p < 0.01, p.eta^2 = 0.449)
    • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)

IIA-N vs. RLT-L vs. RLT-H

• G sig. (F(2,33) = 13.50, p < 0.01, p.eta^2 = 0.450)
  • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (p < 0.01)

• ID sig. (F(2,66) = 34.00, p < 0.01, p.eta^2 = 0.507)
  • pairwise: ID.N < ID.H (p < 0.01); ID.N < ID.L (M. sig. p = 0.08)

• G*ID N.S. (F(4,66) = 1.99, p = 0.11, p.eta^2 = 0.107)
  • lines parallel from ID.N to ID.L; interaction from RLT-L vs. RLT-H still there

Questionnaire Results (Global)

• notes: scores are out of 10 before ART transformation

Fatigue

• G N.S.
• ID N.S.
• G*ID N.S.

Mental Demand

• G sig. (F(2,33) = 4.63, p = 0.02, p.eta^2 = 0.219)
  • pairwise: G.1 < G.3 (M.sig. p = 0.05); G.2 < G.3 (p = 0.03)

• ID N.S.
• G*ID N.S.

General Annoyance

• G sig. (F(2,33) = 3.31, p = 0.05, p.eta^2 = 0.167)
  • pairwise: G.1 < G.2 (p = 0.05)

• ID sig. (F(2,66) = 13.17, p < 0.01, p.eta^2 = 0.285)
  • pairwise: ID.N < ID.L (p < 0.01); ID.N < ID.H (p <0.01)

• G*ID N.S.

Interruption Annoyance

• G N.S.

• ID sig. (F(2,30) = 4.24, p = 0.05, p.eta^2 = 0.124)
  • pairwise: ID.L < ID.H (p = 0.05)

• G*ID N.S.

Performance

• G M.sig. (F(2,33) = 3.05, p = 0.06, p.eta^2 = 0.156)
  • pairwise: G.1 > G.3 (M sig. p = 0.06)

• ID N.S.
• G*ID N.S.

Physical Demand

• G N.S.
• ID N.S.
• G*ID N.S.

Square Puzzles

Completion Time (Local)

• G sig. (F(2,33) = 4.09, p = 0.03, p.eta^2 = 0.199)
  • pairwise: G.1 < G.3 (p = 0.02)

• ID N.S

• G*ID sig. (F(4,66) = 3.28, p = 0.02, p.eta^2 = 0.166)
  • G.1, G.2: log completion time increases from ID.N to ID.L, then decreases from ID.L to ID.H
  • G.3: log completion time decreases from ID.N to ID.L, then increases from ID.L to ID.H
  • one-way ANOVA at ID.N:
    • G sig. (F(2,33) = 4.09, p = 0.03, p.eta^2 = 0.199)
      • pairwise: G.1 < G.3 (p = 0.01)
  • one-way ANOVA at ID.L:
    • G N.S.
  • one-way ANOVA at ID.H:
    • G sig. (F(2,33) = 4.89, p = 0.01, p.eta^2 = 0.229)
      • pairwise: G.1 < G.3 (p = 0.01); G.2 < G.3 (M sig. p = 0.07)
  • RM-ANOVA at G.1:
    • ID sig. (F(2,22) = 3.83, p = 0.04, p.eta^2 = 0.258)
      • pairwise: ID.H < ID.L (M sig. p = 0.07)
  • RM-ANOVA at G:2:
    • ID N.S.
  • RM-ANOVA at G.3:
    • ID sig. (F(2,22) = 3.63 p = 0.04, p.eta^2 = 0.248)
      • pairwise: ID.L < ID.N (M sig. p = 0.1)

Score (Local)

• G M.sig (F(2,33) = 2.77, p = 0.08, p.eta^2 = 0.144)
  • pairwise: G.1 > G.3 (M.sig p = 0.08)

• ID N.S.

• G*ID N.S.

Task Resumption Lag Time (Local)

RLT-L vs. RLT-H

• G sig. F(2,33) = 3.40, p = 0.05, p.eta^2 = 0.171)
  • pairwise: G.1 < G.3 (M.sig p = 0.08)

• ID N.S.

• G*ID sig. F(2,33) = 5.60, p < 0.01, p.eta^2 = 0.253)
  • RLT decreases for G.1, G.2 from ID.L to ID.H; RLT increases for G.3 from ID.L to ID.H
  • RM-ANOVA at G.1: ID sig (F(1,11) = 7.48, p = 0.02, p.eta^2 = 0.405)
    • pairwise: ID.H < ID.L (p = 0.02)
  • RM-ANOVA at G.2: ID N.S.
  • RM-ANOVA at G.3: ID M.sig (F(1,11) = 4.48, p = 0.06, p.eta^2 = 0.289)
    • pairwise: ID.L < ID.H (p = 0.06)
  • one-way ANOVA at ID.L: G N.S.
  • one-way ANOVA at ID.H: G sig. (F(2,33) = 7.83, p < 0.01, p.eta^2 = 0.322)
    • pairwise: G.1 < G.3 (p < 0.01); G.1 < G.2 (M sig. p = 0.06)

IIA-N vs. RLT-L vs. RLT-H

• G sig. (F(2,33) = 4.83, p = 0.02, p.eta^2 = 0.226)
  • pairwise: G.1 < G.3 (sig p = 0.01)

• ID sig. (F(2,66) = 11.60, p < 0.01, p.eta^2 = 0.260)
  • pairwise: ID.L < ID.N (p < 0.01); ID.H < ID.N (p < 0.01);

• G*ID sig. (F(4,66) = 4.62, p < 0.01, p.eta^2 = 0.219)
  • G.1, G.2 decreasing RLT trend from ID.N to ID.H; G.3 decreases sharply from ID.N to ID.L, then increases from ID.L to ID.H
  • RM-ANOVA at G.1: ID sig (F(2,22) = 11.28, p < 0.01, p.eta^2 = 0.506)
    • pairwise: ID.H < ID.N (p < 0.01); ID.H < ID.L (M.sig. p = 0.06)
  • RM-ANOVA at G.2: ID N.S.
  • RM-ANOVA at G.3: ID sig (F(2,22) = 9.63, p < 0.01, p.eta^2 = 0.467)
    • pairwise: ID.L < ID.N (p < 0.01)
  • one-way ANOVA at ID.N: G sig. (F(2,33) = 6.42, p < 0.01, p.eta^2 = 0.280)
    • pairwise: G.1 < G.3 (p < 0.01)
  • one-way ANOVA at ID.L: G N.S.
  • one-way ANOVA at ID.H: G sig. (F(2,33) = 7.83, p < 0.01, p.eta^2 = 0.322)
    • pairwise: G.1 < G.3 (p < 0.01)

Questionnaire Results (Global)

Fatigue

• G N.S.
• ID N.S.
• G*ID N.S.

Mental Demand

• G sig. (F(2,33) = 4.63, p = 0.02, p.eta^2 = 0.219)
  • pairwise: G.1 < G.3 (M.sig. p = 0.05); G.2 < G.3 (p = 0.03)

• ID N.S.
• G*ID N.S.

General Annoyance

• G N.S.

• ID sig. (F(2,66) = 3.25, p = 0.05, p.eta^2 = 0.090)
  • pairwise: ID.N < ID.H (M.sig. p = 0.07); ID.L < ID. H (M.sig. p = 0.07)

• G*ID N.S.

Interruption Annoyance

• G N.S.
• ID N.S.
• G*ID N.S.

Performance

• G sig. (F(2,33) = 5.01, p = 0.01, p.eta^2 = 0.235)
  • pairwise: G.1 > G.3 (p = 0.01)

• ID N.S.

• G*ID M.sig. (F(4,66) = 2.37, p = 0.06, p.eta^2 = 0.126)
  • G.1 perceived performance decreases from ID.N to ID.L, then increases again to ID.H
  • G.2 perceived performance remains relatively constant
  • G.3 perceived performance increases from ID.N to ID.L, then decreases again to ID.H
  • RM-ANOVA at G.1: ID M.sig. (F(2,22) = 2.86, p = 0.08, p.eta^2 = 0.206)
    • pairwise: N.S. ID.N > ID.L, ID.N > ID.H, ID.H > ID.L
  • RM-ANOVA at G.2: ID N.S.
  • RM-ANOVA at G.3: ID sig. (F(2,22) = 4.10, p = 0.03, p.eta^2 = 0.271)
    • pairwise: ID.L > ID.H (p < 0.01)
  • one-way ANOVA at ID.N: G sig. F(2,33) = 4.43, p = 0.02, p.eta^2 = 0.211)
    • pairwise: G.1 > G.3 (p = 0.02)
  • one-way ANOVA at ID.L: G N.S.
  • one-way ANOVA at ID.H: G sig. F(2,33) = 6.31, p < 0.01, p.eta^2 = 0.277)
    • pairwise: G.1 > G.3 (p < 0.01); G.2 > G.3 (p = 0.03);

Physical Demand

• G N.S.
• ID N.S.
• G*ID N.S.

Other Factors

• Other factors examined to ensure validity of results

N-Back

• n-back scores negatively skewed, ceiling effect; non-parametric analysis using ART technique
• SC n-back means: G.1: 9.6 / 10; G.2: 8.8 / 10; G.3: 8.4 / 10; overall: 9.0 / 10
• SqP n-back means: G.1: 9.5 / 10; G.2: 8.7 / 10; G.3: 8.1 / 10; overall: 8.8 / 10

• G sig. (F(2,33) = 12.80, p < 0.01, p.eta^2 = 0.437)
  • pairwise: G.1 > G.2 (p = 0.01); G.1 > G.3 (p < 0.01)

• T N.S. (no effect of task SC / SqP on n-back score)
• G*T N.S. (no interaction between age and task on n-back score

Interruption Condition Ordering (N-L-H)

• x6 orderings: NLH, NHL, LNH, LHN, HNL, HLN; fully-counterbalanced within each group

Main Task Ordering (SC-!SqP)

• x2 orderings: SC-!SqP,!SqP-SC, fully-counterbalanced within each group

SC Bank order (A-B-C)

• x6 orderings: ABC, ACB, BAC, BCA, CAB, CBA: isomorphic SC task banks, selected at random for each participant

SqP Bank order (A-B-C)

• x6 orderings: ABC, ACB, BAC, BCA, CAB, CBA: isomorphic SqP task banks, selected at random for each participant

Experimenter & Apparatus

• MB (Lenovo thinkpad)
• CL (IBM thinkpad)

Location

• A.D. Clinic (12 in group 1, 11 in group 2, 7 in group 3)
• Kerrisdale (4 in group 3)
• JM's condo (1 in group 3)
• UBC hosptial (1 in group 2)

2-group analysis

• 2 age groups at median age of 64 (18 participants in either group)

Methodology

It is hypothesized that the effect of interruptions may interact with the age of the test-taker and the type of task being interruped. This mixed experimental design involves three factors: age, level of interruption demand, and type of main task. We will additionally rely on self-reports to acquire an understanding of task resumption strategies employed by participants in three age groups.

Apparatus

Half of the experimental sessions were conducted using a Lenovo ThinkPad T400 with a 2.26Ghz Core 2 Duo processor and 1.92 GB RAM, while the remaining sessions were conducted using and an IBM ThinkPad T43 with a 2.0GHz Pentium M processor and 1GB + 512MB RAM. Both laptop computers were running the Windows XP operating system. Both computers were connected to a 17 inch diagonal display, with a resolution of 1024 by 768 pixels; the laptop displays were not used. A logitech M110 optical mouse was used with both laptops; identical mouse gains and tracking speeds were used. For the experimental tasks, the screen was positioned the screen at a comfortable viewing angle. The experimental software was an Adobe AIR application written using the Adobe Flex 4.0 SDK.

Participants

We are recruiting 12 participants from three age groups, for a total of 36 participants: Young (19-54), Pre-Old (55-69), and Old (70+). The naming of these age groups is based on accepted terminology in the ageing literature. [Rimkus, A., Melinchok, M. D., McEvoy, K., & Yeager, A. K. (Eds.) (2005). Thesaurus of Aging Terminology. Eighth ed. AARP.]The justification for these groupings rests on the age-related changes that occur in cognition, notably that higher cognitive function remains relatively stable up to about age 55, after which there is a small decline, followed by a much steeper one after 70. [Craik, F. I. M., & Salthouse, T. A. (Eds.) (1992). The Handbook of Aging and Cognition. 2nd ed. Hillsdale, NJ: Erlbaum.]

Participants receive $5 for each half hour of participation. We are recruiting younger participants through advertisements on campus and through word-of-mouth advertisement. Older participants were recruited through word-of-mouth advertisement and a postings in the community. To participate, all participants must be free of any diagnosed cognitive impairments or motor impairments to their right hand, and have normal or corrected-to-normal eyesight.

We administered the Montreal Cognitive Assessment (MoCA) test [CITATION] to help ensure that participants are not cognitively impaired. A score of 26 / 30 or higher is considered normal. Additionally, we administered the North American Adult Reading Test (NAART) [B. Uttl. North american adult reading test: Age norms, reliability, and validity. Journal of Clinical and Experimental Neuropsychology, 24(8):1123–1137, 2002.] to help ensure participants had sufficient English fluency to follow our instructions. The NAART is a quick to administer test measuring verbal intelligence, which requires participants to read a list of 59 words increasing in difficulty. Using a somewhat arbitrary threshold, we accept only participants who get at least 25% of the words correct. Participants who do not have a normal MoCA score or those who do not meet our NAART threshold completed a shorter version of the study, but their data is not included in our analysis.

Tasks

To gain a better understanding of how interruption demand and age interact with task type, two main tasks were used in this study. Both tasks were adapted from C-TOC tests: Sentence Comprehension and Square Puzzles.

Sentence Comprehension (Figure X) tests verbal memory, wherein each trial is comprised of an instruction step and an execution step. The instruction screen displays 1-2 sentences instructing the user to arrange coloured geometric figures. The user clicks a 'Continue' button to advance to the execution screen, where geometric figures are to be arranged as per the instruction.

Square Puzzles (Figure Y) tests non-verbal spatial reasoning, wherein each trial is comprised of a single screen. The user is instructed to move lines to create a certain number of complete squares in a specified number of moves, without leaving any incomplete squares.

A participant can proceed to the next trial at any time by clicking on the 'Next' button at the bottom-right corner of the screen.

Two interruption tasks were used in this study, corresponding to two levels of interruption demand. The interrupting tasks occupy the entire screen, occluding the main task. Interrupting tasks are preceded by an interruption notification, in which a red banner bearing the message 'Interruption Pending' flashes for 2 seconds at the top of the screen; during this time all interactivity is disabled. Once begun, both interruption tasks display an automated sequence of a dozen cartoon images inside a box outline in the middle of the display. There is no fixed order to the presentation of images; each image is selected semi-randomly from a bank of ten images. Each image is displayed for 1500ms; no image is displayed in the box for 100ms between successive images. The total length of the interrupting tasks is approximately 19 seconds. At the end of an interrupting task, the user is prompted to click in order to dismiss the interruption, returning to the interrupted main task.

Interruption types are visually differentiated by a coloured instruction at the top of the screen. In the low-demand interrupting task, the user is prompted, in red font, to watch the sequence of images passively. The high-demand interrupting task (Figure Z) is a visual n-back memory game [CITATION], which is known to place high demand on working memory. In this task, the user is prompted, in green font, to click inside the box whenever the current image repeats what was shown 2 images prior to the current image. Feedback is displayed adjacent to the box outline in the form of a green check icon for correct responses (true positives), while a red 'x' icon is displayed for incorrect answers (false positives and false negatives). No feedback is shown for true negatives. The random sequence of images is weighted such that there is a 40% probability that any given image will repeat what was shown 2 images prior, otherwise the image is selected randomly.

Design

The experiment used a mixed design with two counterbalanced main tasks and three counterbalanced levels of interruption demand: 3 (age groups: young, pre-old, and old) x 2 (main tasks: Sentence Comprehension and Square Puzzles) x 3 (interruption demand: none, low, and high). Age was the only between-subjects factor. Each participant completed 30 trials in the Sentence Comprehension task, and 24 trials in the Square Puzzles task.

In both main tasks, three isomorphic trial blocks were used for the three conditions of interruption demand, comprised of trials increasing in difficulty. An equivalent level of difficulty between trials was not attempted, as the original C-TOC tests from which our tasks were adapted also increase in difficulty. For both main tasks, each participant was assigned one of the six possible trial block permutations at random.

Each Sentence Comprehension trial block contained 10 trials, wherein a subset of 4 trials were interrupted. This subset of trials was determined randomly with the restriction that interruptions could not occur on 4 successive trials and that no interruption may occur on the first trial in a block of trials. The same subset of trials received interruptions in conditions of low and high interuption demand. Interruption onsets were fixed in that they would occur shortly after beginning the execution phase of a trial, with a short lag corresponding to increasing trial difficulty.

Each Square Puzzles trial block contained 8 trials, wherein a subset of 3 trials were interrupted. This subset of trials was determined in the same fashion as in the Sentence Comprehension task. The same subset of trials received interruptions in conditions of low and high interuption demand. Interruption onsets were fixed in that they would occur after a 500ms lag upon the completion of the first move operation in a 2-move puzzle; in a 3-move puzzle, an interruption would occur after the completion of the first or second move operation, with equal likelihood.

Procedure

The experiment was designed to fit into a single 120 minute session. All participants finished in between 60 minutes and 100 minutes.

We began with the Montreal Cognitive Assessment and the North American Adult Reading Test.

Participants then received practice on both types of interrupting tasks: 1 practice trial was performed of the low-demand interrupting task and two practice trials was performed of the high-demand interrupting task. A 3rd practice trial was offered if performance on the high-demand interrupting task was still poor after two practice trials.

We then presented the first task (either Sentence Comprehension or Square Puzzles). An example trial was provided to familiarize the participants with the mechanics of the task. Participants were instructed to perform each task both as quickly and as accurately as possible. For the Square Puzzles task, participants were additionally instructed to not perform more than the specified number of moves, and to not leave remaining incomplete squares. Following this, a practice block of three trials was completed; during this set, the second trial was interrupted with a low-demand interrupting task, while the third trial was interrupted with a high-demand interrupting task. Participants then completed 3 blocks of trials of the first task. After each block, participants completed a short workload and fatigue survey. They then completed their second task.

Upon completion of the experimental trials, participants were subject to a brief interview regarding their perceptions of task difficulty and their strategies for task resumption following an interruption.

Measures

We included measures of speed and accuracy.

In Sentence Comprehension, we measured trial time as the uninterrupted time elapsed during the execution step of the trial until the participant clicks to proceed to the next trial; time spent in the instruction step was recorded separately. In Square Puzzles, we measured trial time as the total uninterruped time elapsed between the start of the trials and when the participant clicks to proceed to the next trial. Additionally, the average time interval between valid move operations was determined for each trial. For interrupted trials, we also recorded the interruption dismissal lag time, the task resumption time following an interruption, and the average interval between valid move operations prior to and following an interruption.

We included several measures of accuracy in both main tasks. The scoring scheme for these tasks was adapted from clinical scoring schemes and from the scoring scheme used in the C-TOC validity testing project. Accuracy scores were determined by the experimenter. A screen capture video of the entire experimental sesssion was recorded, which could be used to confirm scores. Each Sentence Comprehension trial has a possible total score based on the number of geometric figures the participant is instructed to arrange and the relative positioning of these figures upon completion. Each Square Puzzles trial has a possible total score based on the number of complete squares; points are deducted for each additional move operation and each incomplete square. For both main tasks, we additionally recorded the number of completed move operations, invalid moves, aborted moves, and clicks in each trial.

Performance accuracy on the high-demand interrupting task was also recorded, including the number of true positives, true negatives, false positives, and false negatives for each trial.

Hypotheses

We had the following hypotheses for this study:

H1. Age will interact with the presence of interruptions; older adults will perform proportionally worse on interruped trials than on corresponding uninterrupted trials.

H2. High-demand interruptions will incur worse performance than low-demand interruptions; interruption demand may interact with age such that older adults perform proportionally worse on trials interrupted with a high-demand interrupting task than on corresponding trials interrupted with a low-demand interrupting task.

H3. Interruptions will incur worse performance on the Sentence Comprehension task than on the Square Puzzles task; a three-way interaction between level of interruption demand, main task type, and age is also expected.

H4. Self-reports regarding the disruptive effects of interruptions are expected to reinforce H1-H3.

Planned Analysis

Analysis 1: To determine the local effects of interruption disruption in terms of speed and accuracy, we plan to conduct a 3 (age) x 3 (interruption demand) x [3,4] (trial) mixed-factor ANOVA for both Sentence Comprehension and Square Puzzles. We will only consider performance data from the subset of trials that are interrupted in the two interruption conditions, along with the corresponding subset of trials in the uninterrupted condition; results from the remainder of uninterrupted trials in each condition are discarded. Therefore, each subject will generate 12 Sentence Comprehension data points (4 in each interruption condition), and 9 Square Puzzle data points (3 in each interruption condition). The main effect of trial is not of interest, as the difficulty differences between trials are known to us. Similarly, the interactions of trial with age or interruption demand are also of little interest. This analysis will address H1 and H2, to determine if there are interactions between age and level of interruption demand.

Analysis 2: To determine the global effects of interruption disruption, we are to conduct a 3 (age) x 3 (interruption demand) x 2 (main task) mixed-factor ANOVA. A main effect of task is expected and is not of interest. We are, however, interested in the presence of any interaction between of task, age, and interruption demand, which will address H3. For this analysis, a normalized performance score and completion time is calculated for each block of trials, such that each participant will generate 6 data points (3 from each main task).

Preliminary Results

Preliminary findings indicate that there may be a three-way interaction between user age, level of interruption demand, and the requirements of the C-TOC test undertaken by the user. Research participants have reported that interruptions are more disruptive to their performance during a verbal Sentence Comprehension task than during the non-verbal Square Puzzles task. Our analyses will examine whether test performance is proportionally worse when high-demand interruptions are present, versus low-demand interruption and control conditions. The initial findings appear to suggest that technological interventions for mitigating the impact of interruptions and for ensuring the validity of the test scores could be developed to be specific to each type of test in the C-TOC battery.

Notes

• • our dependent measures are speed and accuracy scores from the subset of interruption trials for each participant (4 in SC, 3 in SqP); dependent measures from other trials are not considered in this analysis

• • 3 x 4 = 12 data points per participant in SC, 3 x 3 = 9 data points per participant in SqP

• • ?DV3: task resumption time vs. avg. interaction interval;?

• • DV1: total completion time for each bank of trials - normalized to compare total completion time between SC and SqP trial banks; (if normally distributed, normalize using z-scores; else, subtract mean from each total completion time)

• • ?DV3: avg task resumption time vs. avg. interaction interval;?

Other Factors (unsupported or for future work)

Planned Analyses

• • 12, 9 data points per participant in SC, SqP
  • DV1: trial completion time;
  • DV2: trial accuracy score, based on predefined scoring scheme;

• • 6 data points per participant (3 from each task)
  • DV1: total completion time for each bank of trials - normalized to compare total completion time between SC and SqP trial banks;
  • DV2: total accuracy score for each bank of trials, score out of 100%;

• • ordering of 3 task banks randomly assigned; subsets of trials in which interruptions occur are semi-randomly generated, with the constraint that interruptions are spread across medium and hard trials; no interruptions occur on easy trials (1st trial in each bank);
  • our dependent measures are speed and accuracy scores from the subset of interruption trials for each participant; dependent measures from other trials are not considered in this analysis

• • trial completion time;
  • trial accuracy score, based on predefined scoring scheme;

Analysis 2: To determine the global effects of interruption disruption, we are to conduct a 3 (age, BS) x 3 (interruption demand, WS) x 2 (main task, WS) mixed-factor ANOVA.

• • total completion time for each bank of trials - normalized to compare total completion time between SC and SqP trial banks;
  • total accuracy score for each bank of trials, score out of 100%;

Potential covariates, other recorded DVs:

• MoCA test score - participant is excluded if scores less than 26
• English language test score - using the NAART (North American Adult Reading Test, [Uttl 02]); participant is excluded if less than 50% of words on page 1 are pronounced incorrectly;

Other potential factors (not addressed)

• interruption type/modality (considered in addition to similarity/complexity): computerised (to simulate email / IM alert, antivirus notification, software update, browser crash (total screen change)) vs. uncomputerised (to simulate a telephone call, a caller at the door, a conversation with someone in the room, an alarm ringing, an errand or chore) - the latter forcing one away from the screen
  • pop-up interruptions requiring simple vs. complex response (to simply dismiss interrupting pop-ups (easy) vs. pop-ups requiring responses to urgent forced-choise questions (difficult)).
  • dimension: verbal (spoken) vs. written vs. interactive interruption
• unsupported: 2-3 levels of interruption frequency (no interruption/control, infrequent interruption, frequent interruption) (not sure if frequent interruption will be an issue for 1/2 hour cognitive test)
• partially supported: 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])
• task instruction (emphasis on speed vs. emphasis on accuracy)
• length of interruption lag (currently fixed at 2s)

Participants

• 3 age groups (19-55),(56-69),(70+) - rationale for 2-3 groups from [Moffatt 10], CJ
  • CJ: young-old vs. old-old : age dif @ 70 in literature / 65 for AD/MCI research

• Administer MoCA and NAART tests

• 2 main tasks: (verbal: Sentence Comprehension Task (SC) vs. non-verbal: Square Puzzles (SqP), ordering counterbalanced WS)
• 8 trials in SqP, 10 trials in SC, 3 banks of isomorphic trials, which increase in difficulty;
  • difficulty is gauged by number of actions required and complexity of instruction for SC puzzles, gauged by number of lines moved for SqP puzzles;
  • ordering of 3 task banks randomly assigned; subsets of trials in which interruptions occur are semi-randomly generated, with the constraint that interruptions are spread across medium and hard trials;
  • no interruptions occur on easy trials (1st trial in each bank);
• for both tasks, interruptions occur on the same subset of trials between interruption conditions;
  • in SC trial banks, interruptions occur after a fixed onset of time as a function of task difficulty;
  • in SqP trial banks, interruptions occur after a short delay (500ms) following the first move in 2-move trials; for 3-move puzzles, there is a 50% probability of the interruption being triggered by the first move, and a 50% probability of the interruption being triggered by the second move;

Other potential factors (not addressed)

• interruption type/modality (considered in addition to similarity/complexity): computerised (to simulate email / IM alert, antivirus notification, software update, browser crash (total screen change)) vs. uncomputerised (to simulate a telephone call, a caller at the door, a conversation with someone in the room, an alarm ringing, an errand or chore) - the latter forcing one away from the screen

• length of interruption lag (currently fixed at 2s)

Dependent measures and potential covariates

• For each of the 2 main tasks, and for both normalized time and accuracy, we will run a 3 (age, BS) x 3 (interruption demand, WS) mixed factor ANOVA

• local performance measures - only on subset of trials that are interrupted compared with control condition; remainder of uninterrupted trial scores discarded;
  • normalized trial completion time;
  • normalized trial accuracy score, based on predefined scoring scheme;
• global performance measures, collapsed across all trials in a bank of trials
  • total completion time for each bank of trials;
  • total accuracy score for each bank of trials;
• other recorded measures

• NASA-TLX workload assessment survey scores, administered after each bank of trials (fatigue, cognitive demand, physical demand, annoyance)
• MoCA test score
• language test score

Notes:

• Review and Sign Consent form
• Administer MoCA and language test
• Demonstrate interruption practice tasks
• experimental tasks A & B, counterbalanced between task types, 3 banks of each counterbalanced on interruption demand
  • NASA-TLX questionnaire administered between each bank
• Brief interview re: strategies and perceptions
• Compensation

• Regarding the state of the software, I have automated the main task ordering, the interruption sequence for 3 task sets (for each of the 2 main tasks), and the subset of tasks within each task set on which interruptions occur, which is completed using an initialization screen by the experimenter. This is followed by the main portal UI to the 12 possible interactive states (SC demo, SC training, SC_{A,B,C}, SqP demo, SqP training set, SqP_{A,B,C}, low-demand interrupt demo and high-demand interrupt demo). If the main task-order is set SC-SqP on it initialization screen, SqP options on the main portal UI are disabled until all SC states are completed, and vice versa; also after a task set is completed its portal link becomes disabled. The main 12-button portal is needed to allow users to experience the demo states more than once, and to permit users a short break between task sets, and to serve as visual indication to the user of what remains to be done. If you want I can show you this before our next meeting at your convenience.

• task instruction (emphasis on speed vs. emphasis on accuracy)
• length of interruption lag

• psycho-physiological state: mental effort/expenditure, affect, well-being, activity, mood, anxiety, fatigue (self-reports, experimenter observations)

• H1: Age interaction effects for speed and accuracy; Older adults performing proportionally worse on interruption conditions

• H2: High-demand interruptions will incur worse performance than low-demand interruptions; Possible age interaction or age-related compensation (attention enhancement and suppression) low-demand interruptions may cause mind to wander, become distracted with internal thoughts; high-demand interrupting tasks to main task will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than low-demand interrupting tasks.

• H3: Performance detriments greater for verbal task (SC) than non-verbal task (SqP); Age interaction possible; following from [Speier 03], detriments to performance following interruptions will be more severe for verbal tasks (SC) than non-verbal tasks (SqP). An interaction would exist in that older adults performance on verbal tasks following interruptions are proportionally worse than younger adults performance, compared to the performance of both groups on non-verbal tasks.

• H4: Self-reports regarding disruptive effects of interruptions to reinforce H1-H3

Former Hypotheses

• fH1: performance measures (speed): disruptions caused by interruptions will lead to greater total working time (TWT) for older adults than younger adults on time-intensive tasks. A similar difference should be found in increases in time on task (TOT). Time on interruption (TOI) differences should also be expected, as time to re-orient to and from the main task (change-over and resumption) are expected to be greater for older adults. Older adults will commit more errors and omissions than younger adults on the main task, both groups are expected commit more errors/omissions upon task resumption compared to prior to the interruption, however a greater effect is expected for older adults.

• fH2: performance measures (accuracy): disruptions caused by interruptions will lead to a decrease in TWT and TOT for younger adults (following from [Zijlstra 99]), and an increase in TWT and TOT for older adults. Similar differences in TOI and change-over/resumption times are to be expected as in H1. (Zeigarnik effect)

Implications of hypotheses

• • do older adults compensate to overcome deficiencies in attention suppression and enhancement; does interruption recovery lead to increased fatigue for older adults?

• 3 levels of interruption complexity (no interruption vs. low-demand vs. high-demand, ordering counterbalanced WS)
• 2 types of task (verbal (SC) vs. non-verbal (SqP), ordering counterbalanced WS)

• 3 isomorphic task banks for each of the 2 main tasks, with increasing difficulty; difficulty gauged by number of actions required and complexity of instruction for SC puzzles, gauged by number of lines moved for SqP puzzles;

Other potential factors (not addressed)

• performance measures per trial:
  • # moves, # adjustments, # clicks, # aborted moves, # invalid moves, SC instruction read time (SC only), first-action time, interruption dismissal time, resumption lag time (time to first completed action after interruption),interaction interval before interruption, interaction interval after interruption, total interaction time;
  • for high-demand n-back interruptions: # correct, # incorrect, # false positives, # false negatives, # true positives, # true negatives;

• • incentive to block at random point during task (not just during item completion); post-hoc blocking on when interruption occurred

• interrupting listing task used in [Farrimond 06] - one which would conflict with verbal memory), (puzzle task - sorting or arranging for nonverbal interruption)

• 2 adapted C-TOC tests; verbal: sentence comprehension (SC); non-verbal: square puzzles (SqP)

• screening NSP pencil+paper tests for NSP validity and post-experiment screening

Regarding the state of the software, I have automated the main task ordering, the interruption sequence for 3 task sets (for each of the 2 main tasks), and the subset of tasks within each task set on which interruptions occur, which is completed using an initialization screen by the experimenter. This is followed by the main portal UI to the 12 possible interactive states (SC demo, SC training, SC_{A,B,C}, SqP demo, SqP training set, SqP_{A,B,C}, low-demand interrupt demo and high-demand interrupt demo). If the main task-order is set SC-SqP on it initialization screen, SqP options on the main portal UI are disabled until all SC states are completed, and vice versa; also after a task set is completed its portal link becomes disabled. The main 12-button portal is needed to allow users to experience the demo states more than once, and to permit users a short break between task sets, and to serve as visual indication to the user of what remains to be done. If you want I can show you this before our next meeting at your convenience.

As per subject-task-interrupt sequence/ordering in my experiment, the following factors are involved:

- 2 main tasks (SC and SqP) - important to counterbalance within each age group - 3 task sets (ABC) with 6 possible isomorphic permutations, so not necessary to counterbalance - 3 interruption conditions (none, low, high) with 6 possible permutations, also important to counterbalance within each age group - 6 subsets of trials within each task set on which interruptions occur (i.e. interruptions on trials #3,5,7,9 or interruptions on trials #2,4,6,8), which I created myself and were derived based on individual puzzle difficulty

This gives us a total of 436 possible orderings, x 3 age groups, with a maximum of 24 participants per age group. Since task sets are deliberately isomorphic in difficulty, we can limit our choices to two task set orderings (i.e. 12 participants in each group using ABC, and the other 12 using CAB), such that no two participants in a single age group receive the same combination of interruption trial subset, interruption set ordering, and task set ordering. Since the 2 main tasks are independent of one another, the same ordering (ABC or CAB) can be used for both SC and SqP.

There are 24 unique orderings for each subject group, see the attached .xls. Interruption trial subsets and interruption conditions could be specified advance. It will be necessary for the experimenter to refer to this lookup table while completing the initialization screen before the participant begins the experiment.

• H3: high-demand interrupting tasks to main task will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than low-demand interrupting tasks. A greater effect can be expected for older adults.

• H4: following from [Speier 03], detriments to performance following interruptions will be more severe for verbal tasks (SC) than non-verbal tasks (SqP). An interaction would exist in that older adults performance on verbal tasks following interruptions are proportionally worse than younger adults performance, compared to the performance of both groups on non-verbal tasks.

• • • graphical resumption prompts will be more effective than verbal resumption prompts, regardless of the type of main task. A proportionally greater benefit will be seen for older adults than younger adults.

• • following from [Gillie 89], interruptions that are similar in nature to the main task being performed will have a greater negative effect (again in terms of TWT, TOT, TOI, and error rate) than dissimilar interruptions. Once again a greater effect can be expected for older adults.
  • From [Storch 92], interruptions using a similar modality (on screen vs. on the phone or in person) will have a greater detrimental effect on users.
  • From [Speier 03], interruptions differ from distractions in that they exist in the same modality as the main task, leading to an overloading of cues. Distractions during complex tasks can still be detrimental to performance do to an overall cognitive overload.
  • [Farrimond 06] found no effect of noise and movement distractions on virtual shopping PM task; one could argue that the task was not complex enough.
  • Older adults are more distractible [CITATION NEEDED], thus an interaction may exist in that interruptions in different modalities, or simply distractions may be detrimental to performance on complex main tasks.

• • following from [Speier 03], graphical resumption prompts will be more effective than verbal resumption prompts, regardless of the type of main task. A proportionally greater benefit will be seen for older adults than younger adults.

• 2 adapted C-TOC tests; verbal: sentence comprehension; non-verbal: square puzzles

• • nonverbal: pattern (re)construction and square puzzles may be difficult to score - square puzzles easier than pattern (reconstruction) - include reset functionality, lock shapes after x moves, give-up button. How to code?

Hypotheses

• H1: performance measures (speed instruction condition): disruptions caused by interruptions will lead to greater total working time (TWT) for older adults than younger adults on time-intensive tasks. A similar difference should be found in increases in time on task (TOT). Time on interruption (TOI) differences should also be expected, as time to re-orient to and from the main task (change-over and resumption) are expected to be greater for older adults. Older adults will commit more errors and omissions than younger adults on the main task, both groups are expected commit more errors/omissions upon task resumption compared to prior to the interruption, however a greater effect is expected for older adults.

• H2: performance measures (accuracy instruction condition): disruptions caused by interruptions will lead to a decrease in TWT and TOT for younger adults (following from [Zijlstra 99]), and an increase in TWT and TOT for older adults. Similar differences in TOI and change-over/resumption times are to be expected as in H1.

• H3: complex/similar interrupting tasks to main task will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than simple/dissimilar interrupting tasks. A greater effect can be expected for older adults.
  • following from [Gillie 89], interruptions that are similar in nature to the main task being performed will have a greater negative effect (again in terms of TWT, TOT, TOI, and error rate) than dissimilar interruptions. Once again a greater effect can be expected for older adults. From [Storch 92], interruptions using a similar modality (on screen vs. on the phone or in person) will have a greater detrimental effect on users. From [Speier 03], interruptions differ from distractions in that they exist in the same modality as the main task, leading to an overloading of cues. Distractions during complex tasks can still be detrimental to performance do to an overall cognitive overload. [Farrimond 06] found no effect of noise and movement distractions on virtual shopping PM task; one could argue that the task was not complex enough. Older adults are more distractible [CITATION NEEDED], thus an interaction may exist in that interruptions in different modalities, or simply distractions may be detrimental to performance on complex main tasks.

• H4: following from [Speier 03], detriments to performance following interruptions will be more severe for verbal tasks than non-verbal tasks. An interaction would exist in that older adults performance on verbal tasks following interruptions are proportionally worse than younger adults performance, compared to the performance of both groups on non-verbal tasks.

• (future work) H5: also following from [Speier 03], graphical resumption prompts will be more effective than verbal resumption prompts, regardless of the type of main task. A proportionally greater benefit will be seen for older adults than younger adults.

• 3 levels of age (young vs. young-old vs. old-old, BS)
• 2 types of interruption complexity/similarity (simple/unrelated vs. complex/conflicting, WS) - both on computer? on complex task on computer?

• • dimension: verbal (spoken) vs. written vs. interactive interruption

• partially supported: 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])

• • block at random point during task (not just during item completion); post-hoc blocking on when interruption occurred

• • phone interruption shows no difference from control group; screen interruption (similar modality) shows worst disruption, then walk-in interruption
  • a simulated office - door, phone, desk, computer, etc.
  • GUI allows users to work simultaneously on main task during an interruption
• [Speier 03] - did not vary simplicity/complexity of interrupting tasks, but main tasks, varied interruption condition BS; all interruptions were data acquisition tasks that could be carried out regardless of the interface/task complexity of the main task;
  • models of interruptions identified; did not control social characteristics of interruption - will this factor be worse for older adults?

• 2 adapted C-TOC tests; verbal: sentence comprehension; non-verbal: ???
  • must consider needs for coding / ways to score results!
  • symbol digit matching (time to complete too short, length an issue)
  • nonverbal: pattern (re)construction and square puzzles may be difficult to score
  • misplaced object recall (clear right/wrong answer + time to execute) (difficult to generate enough stimuli?

• 3 age groups (19-55),(56-69),(70+) - rationale for 2-3 groups from [Moffatt 10], CJ

Hypotheses - refine!

• • from [Speier 03] spatial presentation formats able to mitigate effect of interruptions while symbolic formats were not for complex tasks;
    • use of icons and graphics rather than text-based resumption prompts
    • crossover point exists between perceptual and analytical processes - where is the crossover for older adults?
  • [Storch 92] - GUI better than CUI for concurrent execution of main task and interrupting task - likely not a factor with C-TOC

• [Storch 92] shows on-screen interruptions more disruptive than phone/visitor interruptions with data entry tasks on GUI and CUI interfaces (GUI facilitating simultaneous execution of main task during interruption);

• • from [Speier 03] patial presentation formats able to mitigate effect of interruptions while symbolic formats were not;

• • CJ: timing: advise against using instructions in time to complete - tests reading comprehension rather than task at hand (i.e. pattern construction) opportunity to separate criteria: item completion and task completion (including items, transitions, breaks in between, etc.)

• • CJ: young-old vs. old-old : age dif @ 70 in literature / 65 for AD/MCI research

• • use of interruption lag [Altmann 04], [Hodgetts 06] - do older adults make use of contextual cues from the interruption lag and resumption lag to resume a task, or does automatic encoding of task goals occur less in older age? what are the implications for prompting systems?

• 2 levels of age (young-old vs. old-old, BS)
• 2 types of interruption complexity/similarity (simple and unrelated vs. complex and conflicting, WS)
• 2 types of task instruction (emphasis on accuracy vs. emphasis on speed, counterbalanced WS)
• 2 types of task (verbal vs. non-verbal, counterbalanced WS or BS)

Comments:

• JM: tasks WS, instruction type WS results don't hinge on particular task type (more generalizable); if 4 tasks, could make tasks BS;
  • CJ: counterbalancing important, but doable;
• CJ: verbal or non-verbal main task; similarity could mean verbal vs. non-verbal; complexity manipulated once an interrupting task is chosen
• CJ: e.g. letter fluency: low level complexity => one letter; high-level complexity => three letters

• interruption type (considered in addition to similarity/complexity): computerised (to simulate email / IM alert, antivirus notification, software update, browser crash (total screen change)) vs. uncomputerised (to simulate a telephone call, a caller at the door, a conversation with someone in the room, an alarm ringing, an errand or chore) - the latter forcing one away from the screen
  • pop-up interruptions requiring simple vs. complex response (to simply dismiss interrupting pop-ups (easy) vs. pop-ups requiring responses to urgent forced-choise questions (difficult)).
  • dimension: verbal vs. written vs. interactive interruption
• unsupported: 2-3 levels of interruption frequency (no interruption/control, infrequent interruption, frequent interruption) (not sure if frequent interruption will be an issue for 1/2 hour cognitive test)
• unsupported: 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])

Tasks

Materials

• 2 adapted C-TOC tests (i.e. sentence comprehension or sentence production for verbal, symbol digit matching (length an issue), square puzzles, or pattern reconstruction for non-verbal) with inserted interruptions

Procedure

• questionnaire about background: health, vision, fatigue, computer use
• practice tasks
• experimental tasks, counterbalanced for interruption style and task type
• questionnaire about psycho-physiological state

• 2 age groups (19-69), (70+) - rationale for 2-3 groups from [Moffatt 10]
  • CJ: young-old vs. old-old

Hypotheses

• H1: performance measures (emphasis on speed tasks): disruptions caused by interruptions will lead to greater total working time (TWT) for older adults than younger adults on time-intensive tasks. A similar difference should be found in increases in time on task (TOT). Time on interruption (TOI) differences should also be expected, as time to re-orient to and from the main task (change-over and resumption) are expected to be greater for older adults. Older adults will commit more errors and omissions than younger adults on the main task, both groups are expected commit more errors/omissions upon task resumption compared to prior to the interruption, however a greater effect is expected for older adults.

• H2: performance measures (emphasis on accuracy tasks): disruptions caused by interruptions will lead to a decrease in TWT and TOT for younger adults (following from [Zijlstra 99]), and an increase in TWT and TOT for older adults. Similar differences in TOI and change-over/resumption times are to be expected as in H1.

• H3: complex interruptions similar to main task will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than simple interruptions dissimilar to main task. A greater effect can be expected for older adults.
  • following from [Gillie 89], interruptions that are similar in nature to the main task being performed will have a greater negative effect (again in terms of TWT, TOT, TOI, and error rate) than dissimilar interruptions. Once again a greater effect can be expected for older adults.

• H4: older adults' performance on a prospective memory task will be more severely affected by interruptions than by younger adults.

Implications of hypotheses

• types of prompts (how much info is too much info for C-TOC - worry about too much aid)
  • [Parnin CHI 10] - relevant prompts (developer user)
    • snapshots / instant replay of past actions
    • task sketches - annotations of a task breakdown: steps, objectives, plans + goals (short animated clip or series of images)
    • prospective cues - contextual reminders that are displayed when a condition is true;
    • change summary - natural language summary of past actions in log format
  • simple contextual cue of recent action: visual highlighting of last object moved/clicked (glow), display origin distinguished from current location of recently moved object
  • repeat of selected instructions and speed/accuracy tradeoff - to avoid overcompensation on speed at expense of accuracy

Related work

Experiment

tentative 2 (age, BS) x 2 (complexity, WS) x 2 (similarity, WS) design, one for task A (time-sensitive), one for task B (time insensitive)

• 2 types of task (timed, un-timed) - the latter favouring accuracy and precision, the former favouring reaction time. (Experiment 1, 2) - 1/2 hour - 1 hour each?

Factors

• 2-3 age groups (19-45), (46-69), (70+) - rationale for 2-3 groups from [Moffatt 10]
• 2 levels of interruption complexity (easy, difficult)
• 2 levels of task similarity (similar, dissimilar) (required for both types of task?) (not sure if this is necessary for cognitive testing)

Other potential factors

• 2-3 levels of interruption frequency (no interruption/control, infrequent interruption, frequent interruption) (not sure if frequent interruption will be an issue for 1/2 hour cognitive test)
• 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])

Dependent measures

• performance measures: time (TWT, TOT, TOI), error rate + omissions (prior to and after interruptions)
• psycho-physiological state: mental effort/expenditure, affect, well-being, activity, mood, anxiety, fatigue (self-reports, observations)

Hypotheses

• H1: performance measures (timed tasks): disruptions caused by interruptions will lead to greater total working time (TWT) for older adults than younger adults on time-intensive tasks. A similar difference should be found in increases in time on task (TOT). Time on interruption (TOI) differences should also be expected, as time to re-orient to and from the main task (change-over and resumption) are expected to be greater for older adults. Older adults will commit more errors and omissions than younger adults on the main task, both groups are expected commit more errors/omissions upon task resumption compared to prior to the interruption, however a greater effect is expected for older adults.

• H2: performance measures (non-timed tasks): disruptions caused by interruptions will lead to a decrease in TWT and TOT for younger adults (following from [Zijlstra 99]), and an increase in TWT and TOT for older adults. Similar differences in TOI and change-over/resumption times are to be expected as in H1.

• H3: complex interruptions will have a greater negative effect (in terms of TWT, TOT, TOI, and error rate) than simple interruptions. A greater effect can be expected for older adults.

• H4: following from [Gillie 89], interruptions that are similar in nature to the main task being performed will have a greater negative effect (again in terms of TWT, TOT, TOI, and error rate) than dissimilar interruptions. Once again a greater effect can be expected for older adults.

• H5: again following from [Gillie 89], the length of interruptions should have no effect on younger adults on main task performance. (Does length of interruption have an effect on older adults? On both timed and un-timed tasks).

• H6: an increased frequency of interruptions will have an adverse effect on the psycho-physiological state of both older and younger adults. Frequency of interruptions will have no effect on performance for younger adults on un-timed tasks. (Older adults? timed tasks? is frequency an issue we would need to contend with? in a 1/2 hour test?)

• H7: older adults' performance on a prospective memory task will be more severely affected by interruptions than by younger adults.

An embedded prospective memory retrieval/cue task in the open-ended test?

Interruption tasks: one simple (and unrelated to main tasks, i.e. simulated phone call/registration task?); one complex (verbal fluency task, listing task [Farrimond 06] - one which would conflict with verbal memory), (puzzle task - sorting or arranging)

• questionnaire about background: health, vision, fatigue, computer use
• practice tasks for test A
• test A
• questionnaire about psycho-physiological state
• practice tasks for test B
• test B
• questionnaire about psycho-physiological state

• 2 adapted C-TOC tests (i.e. word recognition recall, sentence comprehension, sentence production) with inserted interruptions

• 2 types of task (timed, un-timed) - the latter favouring accuracy and precision, the former favouring reaction time. (Experiment 1, 2) - 1/2 hour - 1 hour each?

factors:

• 2-3 age groups (19-45), (46-69), (70+) - rationale for 2-3 groups from [Moffatt 10]
• 2 levels of interruption complexity (easy, difficult)
• 2 levels of task similarity (similar, dissimilar) (required for both types of task?) (not sure if this is necessary for cognitive testing)

Other potential factors:

• 2-3 levels of interruption frequency (no interruption/control, infrequent interruption, frequent interruption) (not sure if frequent interruption will be an issue for 1/2 hour cognitive test)
• 2-3 levels of interruption length (short, med, long), (no support for length from [Gillie 89])

• performance measures: time (TWT, TOT, TOI), error rate + omissions (prior to and after interruptions)
• psycho-physiological state: mental effort/expenditure, affect, well-being, activity, mood, anxiety, fatigue (self-reports, observations)

Interruption Lab Study

Abstract

Related Work

Hypotheses

Experimental Design

Participants

Tasks

Procedure

Materials

Design / Factors

Dependent Measures

-- MatthewBrehmer - 15 Sep 2010