Dear Karyn Moffatt

We are pleased to tell you that your paper submission, 230,
Steadied-Bubbles: Combining Techniques to Address Pen-Based Pointing
Errors for Younger and Older Adults, has been conditionally accepted
for inclusion in the CHI 2010 Papers and Notes Program. The final
reviews for your submission are included below and should now be
available to you on the PCS system at
https://precisionconference.com/~sigchi.  We hope these will be useful
to you in preparing your final revision.

Please read the remainder of this email carefully, as it lists several
deliverables and deadlines that you must adhere to, as well as a
schedule.

We received 1346 papers and notes. We were able to accept 22% of them.
The decisions as to which submissions to accept were made after
careful reviews by a number of highly qualified professionals around
the world. As you can see, we had to be very selective. You should be
very proud of your accomplishment.

Conditional acceptance means that you must fulfill particular requests
(if any were made) by the Associate Chair to revise your paper. What
this means is that Associate Chairs will be looking at your final
version again to see if you have followed their specific
recommendations. Please check the meta-reviews of your paper (written
by the Associate Chairs) to see if any such specific demands were
made. As well, the Associate Chairs and Reviewers will likely have
made other suggestions -- we suggest you read them very carefully and
use these to craft an even better final paper.

You must submit this final version to the PCS system (see
http://www.precisionconference.com/~sigchi) by *January 10* as
described in the schedule below. You will be notified only if there
are problems with your final submission.

Along with your submission, you will also be required to provide other
information by *January 10* via PCS, which will be used in the final
program to help people decide what papers/notes to go see. You will
receive an email message with details about how to upload the
following:

1. A 30-word contribution & benefits statement

2. A black and white thumbnail image that captures the essence of your
paper. This thumbnail should be 300 x 200 pixels, in TIFF or JPEG
format and at least 300 dpi.

You will also shortly receive an email from the publisher, Sheridan
Press, with instructions to prepare your paper for publication,
including insertion of the copyright notice and filling out the
copyright assignment form. Upload your camera-ready paper to the
publisher's site as per their instructions (do not send it to us
Chairs). It is due *January 10, 2010*.  Note that this means you will
upload copies of your paper to TWO DIFFERENT SITES: one to PCS and one
to the publisher's site.

As you start getting ready for your CHI presentations, we invite you
to prepare a 30-second preview of your work to present at CHI Madness.
CHI Madness is a single track event with no competing sessions that
attracts over a thousand attendees. It is one of the only events at
CHI that brings all attendees together. To participate in CHI madness,
you will create and submit your 30-second presentation slides by March
2nd via PCS. You can find detailed information at:
http://www.chi2010.org/Authors/Madness.html

We will follow up closer to the conference with additional information
about your presentation.

The CHI Conference requires that at least one author of an accepted
paper registers and attends the conference to present the paper.
Online registration will be open shortly. We urge you to register
early at http://www.chi2010.org <http://www.chi2010.org/> to secure
the pre-registration discount.  You can also find the Advance Program
there shortly.

Schedule:

*January 10*:
 - final submission posted to PCS, which will be checked by Associate Chairs.
 - 30 word contributions and benefit statement
 - thumbnail image

*January 10*: final submission posted to publisher's web site
(instructions to be sent by email)

*March 2*: CHI 2010 Madness Slide Submission Deadline

We are looking forward to meeting you at the conference.

Congratulations!

Geraldine Fitzpatrick and Scott Hudson
CHI 2010 Papers and Notes Chairs

===================== REVIEWS ========================

------------------------ Submission 230, Review 4 ------------------------

Title: Steadied-Bubbles: Combining Techniques to Address Pen-Based
Pointing Errors for Younger and Older Adults

Reviewer:           primary

Contribution Type Specific Rating (Meets Contribution Type Specific
Criteria Well)

  3.0 - Neutral

Overall Rating

  3.0 - Neutral: Overall I would not argue for accepting or rejecting
this paper.

Expertise

  3  (Knowledgeable)

Contribution to HCI

  This is a well written incremental study that takes further earlier work
  on the use of pen technologies with older adults.

  The paper reports a study with three techniques and reports findings.

The Meta-Review

  The reviewers of this paper all found the work to be interesting and all
  reported that the paper was well written and well described.  Two areas
  for concern however were the motivations for the work and the strength of
  the conclusions given the size of the sample.  It is suggested that the
  authors attend to these concerns by at least discussing the limitations
  should the paper be accepted.

  A key question was the extent to which the study made sense given the
  poor results that older adults have with pen based tasks.  This is an
  interesting question and one that the authors might like to consider.

  With regard to the size of the sample that were used, if it could be
  resonably argued that there was a point to this study, a larger sample
  would add validity to the work

Associate Chairs Additional Comments

------------------------ Submission 230, Review 1 ------------------------

Title: Steadied-Bubbles: Combining Techniques to Address Pen-Based
Pointing Errors for Younger and Older Adults

Reviewer:           external

Contribution Type Specific Rating (Meets Contribution Type Specific
Criteria Well)

  3.5     . . . Between agree and neutral

Overall Rating

  4.0 - Possibly Accept: I would argue for accepting this paper

Expertise

  4  (Expert )

Contribution to HCI

  This research builds on previous work and presents a user study that
  investigates the applicability of the previously known Bubble Cursor, the
  Steady Clicks techniques and a (novel) combination thereof to reduce
  target selection error rates for older users during pen-based
  interaction. The investigated interaction techniques are not novel but
  the studied usage context – older adults and pen-based interaction – is
  novel for these techniques, thus the main contribution of the paper is
  the presented empirical data that incrementally extends previous
  knowledge about these assisting pointing techniques. In short, the Bubble
  Cursor, the Steady Clicks and the combination, all can reduce selection
  errors (and selection times) when a pen is used as input device for both
  younger and older adults, but mainly for selection tasks involving small,
  non-adjacent targets for older adults.

The Review

  Top scores on formality and perfection: the paper is exceptionally
  well-written, the used language is easy to understand for a non-native
  English speaker, the structure of the paper follows a classic CHI paper
  on new interaction techniques and includes suitable references: the paper
  provides enjoyable reading, the figures and the text are nicely
  formatted, congratulations! The conducted study carefully designed and is
  described in a suitable way which allows for easy replication.

  The superb presentation has to be weighted against the following somewhat
  weaker issues.
  1) No new and innovative interaction technique is introduced and tested,
  instead already known techniques (and a combination thereof) are
  assessed. Noted is the novelty of investigating the usefulness of these
  techniques for pen-based interaction and comparing the results across
  different age groups.
  2) The overriding objective with the research/study, to make pen-based
  interaction less error prone for older adults, is motivated by previous
  research (Moffatt and McGrenere, Assets 2007) which report on older
  adults’ problems with pen-based interaction. However, to my knowledge,
  the error rate in the Moffatt and McGrenere study was rather low: in the
  Tapping Task, on average the older users made 11 ‘Slip’ errors out of 288
  trials, resulting in an error rate of about 3.8% (excluding ‘Near-misses’
  errors since a non-significant difference is reported between older and
  younger users for this error type). Arguably, it is questionable whether
  the previously reported error rate really motivates extensive hunts for
  assisting pointing enhancement technologies to further tweak down an
  already low error rate (which is below the typical 4% error rate in
  repetitive selection task studies). It is somewhat disappointing that the
  authors were not more sensitive to this fact and did not provide a more
  straight forward discussion whether or not this really is a problem
  worthwhile to try to tackle. I am not convinced that we are dealing with
  a significant problem here.
  3) The results are not very exciting, only limited improvements and only
  for rather small targets and in sparse layouts. Unfortunately, several
  important points are missed out or are unclear in the Result section.
  This makes it hard to untangle the possible benefit of using the
  non-standard cursors/techniques. For example, the authors do not report
  the total average error rate, only for the smallest target condition the
  average number of Miss-errors is provided (in the Slip-error analysis the
  reader is left with only Figure 4). We can only assume that the error
  rate was about as low as in the previous Moffatt and McGrenere study,
  perhaps a further indication pointing at a non-existing problem (as
  discussed above)? However, the 10.4% error rate for older adults in the
  WID12 condition is considerable. Unfortunately, it is impossible to
  deduce the exact number of Miss-Errors made in EWR=2 and EWR=3 (the
  target density conditions where the Bubble or Steadied-Bubble techniques
  were effective (Steady was not significantly different from Control for
  any EWR)) by old and younger from the Figures 1 alone. Similar for
  Slip-Errors. Here we do not know the exact number of errors (one can only
  use the bars in Figure 4 to add up). I am a bit unsure how to interpret
  Figure 4 (and Figure 3), but I guess, in Figure 4 in the Control and
  EWR=1 condition, on average Slip-Errors were made in five trials out of
  36 trials and in the Steady and EWR=1 condition, on average one
  Slip-Error was made, i.e., a reduction in error rate from 5/36 = 13.8% to
  1/36 = 2.7%. Somehow it is difficult for the reader to pin-down/interpret
  and judge the benefit of the proposed techniques. Furthermore, the
  possible benefits regarding error rate has to be balanced against the
  results from the Movement Time analysis. Also here, the exact movement
  times for the different conditions are missing. But according to Figure 7
  and 8, the differences do not seem drastic, also several nasty threefold
  interactions make it hard to interpret the results. (The sentence about a
  substantially reduction of errors in the conclusion section seems to be
  somewhat over exaggerated. In the Summary section, once again, it should
  be pointed out that the results only correspond to the hypotheses for the
  12 pix condition, for all other size-conditions all the hypotheses were
  falsified.) Additional, the potential advantage of the non-standard
  cursors has to be weighted against the cost of hitting a distracter
  target, as discussed later on by the authors.
  4) Furthermore, I was wondering about any possible learning effect. The
  test participants were all novice pen user, thus one might perhaps
  suspect some kind of improvement. Could it be that, after an extended
  period of usage (possibly longer than the 1296 selections made in this
  study), the already low error rate will further decrease?
  5) Since the Bubble and Steadied-Bubble seem to be the most effective
  techniques, one has to ask how feasible would it be to implement the
  Bubble-functionality and use it in real applications? This issue is
  touched upon in the Discussion Section but perhaps a bit too late. A
  reader who is familiar with the Bubble cursor “knows from the beginning”
  (and is thus somewhat “irritated” throughout reading until the Design
  Considerations section) that the Bubble-technique is somewhat limited and
  can only be effective in displays with few targets, a rare situation in
  desktop applications with toolbars, drop-down boxes and icons etc.
  Furthermore, a ‘target aware’ technique (appreciated the discussion by
  the authors) seems rather unrealistic in the end.

  In summary, given the rather low error rate in previous pen-based
  interaction studies e.g. Moffatt and McGrenere (somehow confirmed in this
  study) it is questionable whether this study really is motivated and thus
  the value of the incremental findings (and the somewhat limited
  improvements caused by the tested techniques) is significantly reduced.
  This, in combination with the subtle issues pointed out above, which are
  somewhat glossed over by the authors, make me hesitate and it is easy to
  get blinded by the exceptionally excellent presentation. Thus, for me
  (perhaps a bit too unforgiving) the substance/contribution vs.
  formality/perfection weighting has to result in a borderline rating.

  REBUTTAL RESPONSE
  Sorry, admittedly parts of my 3rd point are unclear. The core point I
  wanted to express was that I have a hard time “to untangle the possible
  benefit of using the non-standard cursors/techniques” and “to
  pin-down/interpret and judge the benefit of the proposed techniques”.
  Since data from the two largest size conditions are ‘excluded’ from most
  of the analysis on page 6, except from Figure 2 and the old vs. young
  miss-rate in the middle of the left column, it is hard to get the overall
  picture and estimate what the benefits from steady, bubble, and
  bubble/steady could be in a real world situation where not only 12pix
  small targets are used and the displays are mostly full of tightly
  located targets. At the moment, the reader is left to estimate and add up
  the data points in Figure 2 and do some dividing. So yes, a clear
  presentation of the error rates per error type, cursor, size, ewr, and
  age–possibly done by using a table–would be helpful.

  Unfortunately, I still cannot find any data in the paper similar to the
  overall error data the authors present in their rebuttal. Possibly they
  are referring at Figure 2 and data point estimations and calculations.
  However, reading the rebuttal and its presented overall error data, I am
  surprised the authors did not point out these high error rates (e.g., on
  average, of the total 324 trials made with the standard cursor, an older
  person made 72 errors and on average 34 errors were made with the
  steady-bubble) more explicitly and discussed the differences to the low
  error rates reported in [19] (the only discussion regards the age-factor
  late in the paper, p. 8). Obviously the very majority of the errors were
  made with the smallest targets, which were much smaller (38%?) than the
  smallest targets used in [19]. It is also obvious that it would be tricky
  to use the high error rate found in this study as a motivation for the
  very study itself. But it certainly seems as if some (the variance in the
  data seems to be large and as pointed out it was large in [19]) novice
  users (both here and in [19] most of the participants had not experienced
  pen-based interaction before) indeed have difficulties with very small
  targets. As I say in my review, it would have been interesting to know
  how the recorded errors were distributed throughout the test sessions? Do
  novice pen-users improve? Do they learn how to better use the pen and how
  to reduce mistakes? Or can we assume that the error rate for non-assisted
  pointing will be constant on the level reported after some practice
  trials and 81 test trials? Likewise, did participants improve during the
  assisting-conditions?

  And, yes I agree it is difficult to determine what is an acceptable error
  rate for computer use, thus when it is worth investigating
  error-preventing techniques. I am not sure, but if the consequences of
  errors made in controlled experiments are arguably under-estimated by the
  participants, would not that also imply that participants are more likely
  willing to sacrifice bad accuracy performance for good speed performance
  in such experiments (perhaps “to get over with the test quickly”)?
  Perhaps, they are less accurate in a controlled experiment than during
  real-world usage (we do not know, lacking data from real usage), thus the
  reported error rates in previous controlled studies are higher than what
  we would see during real-world computing. Consequently, I am not sure
  that the argumentation in the rebuttal on this issue strengthens the
  motivation for the paper/research. Having said that, I point out that the
  rebuttal has provided new “numbers” (the high error rates found in this
  study, discussed above) that were invisible to me when reading the
  submitted version, and which, in hindsight after the study was conducted,
  contribute to a strengthening of the very motivation. Furthermore, these
  new numbers make the results a bit more exiting, somewhat invalidating of
  my major criticism.

  Regarding target-awareness: yes I agree, most of the pointing techniques
  discussed in [iii] depend on target-awareness, but does this motivate
  following this path further? As argued in [iii], the target-aware
  techniques have been successful in controlled studies, few (none) has
  been proven useful/successful in real-world situations, most likely due
  to the impracticality of target-awareness in real applications.

  As my review says, I was perhaps a bit too unforgiving, and with the new
  “numbers” revealed in the rebuttal, which strengthens both the motivation
  and findings, I am happy to raise my score one step.

Areas for Improvement

  I only spotted a few very minor typos.
  1) top second column, page 4: “for a total of 324 trials per condition”
  -> the word ‘condition’ should be changed to ‘cursor condition’.
  2) last paragraph page 8: the punctuation mark after reference 5 should
  be replaced by a comma.
  3) including Fitts’ Law under Author Keywords seems somewhat misleading
  since not much in the paper relate to Fitts’ Law (only one direct
  reference in section ‘Bubble Cursor’.
  4) Caption to Figure 4: ‘At EWR=1, Control was not effective’, I suppose
  Bubble was not effective at EWR=1?
  5) There is an unfortunate line break on page 8, Subjective Findings, by
  the z-statistics for the Steadied-Bubble-Steady ranking.

------------------------ Submission 230, Review 2 ------------------------

Title: Steadied-Bubbles: Combining Techniques to Address Pen-Based
Pointing Errors for Younger and Older Adults

Reviewer:           external

Contribution Type Specific Rating (Meets Contribution Type Specific
Criteria Well)

  4.0 - Agree

Overall Rating

  4.0 - Possibly Accept: I would argue for accepting this paper

Expertise

  4  (Expert )

Contribution to HCI

  In this paper the authors present a technique to address pen-based
  pointing errors for younger and older adults. Based on the work of
  Moffatt and McGrenere they distinguish between two types of target
  acquisition difficulties using a pen: missing and slipping. Missing was
  common to both older and younger adults, whereas slipping was unique to
  the older users. To address these problems they propose the combination
  of two mouse-based techniques: Steady Clicks and Bubble cursor. The paper
  also reports on a laboratory experiment with younger and older adults
  showing that both techniques can be adapted for use in a pen interface,
  and that they can be successfully combined to provide greater support.

The Review

  In this paper the authors examined the combination of existing techniques
  to solve a relevant prolbem in the are of human-computer interaction for
  elderly people.
  The paper is well motivated, very well written, and has a clear
  structure. Moreover, the relevant literature review seems to be
  adequatate.
  The two mouse-based techniques as well as the pros and cons seem to be
  well explained. The presentation of the combined technique, though, lacks
  details. Here, the authors could have spend a little bit more effort in a
  detailed description. Have any problems occured when combining the
  techniques? What about the dynamic resizing of the bubble?
  Furthermore, the authors present a detailed laboratory experiment, that
  is convincing and seems valid.
  In summary, the paper is well written, the idea presented is interesting,
  and the study seems correct. Although the paper, in my opinion, could
  benefit from some minor improvements, I do believe that it will be
  interesting to the community and I would argue for its publication at
  CHI.

Areas for Improvement

  o Add some figures to explain the combined technique and some explaining
  text as well.
  o Use different colors or pattern for the figures showing the statistics.

------------------------ Submission 230, Review 3 ------------------------

Title: Steadied-Bubbles: Combining Techniques to Address Pen-Based
Pointing Errors for Younger and Older Adults

Reviewer:           external

Contribution Type Specific Rating (Meets Contribution Type Specific
Criteria Well)

  3.0 - Neutral

Overall Rating

  3.5 . . . Between neutral and possibly accept

Expertise

  3  (Knowledgeable)

Contribution to HCI

  The paper compares 3 different styles of pointer adjustment (Bubble,
  Steadied, Steadied bubble) in respect to older adults and their
  performance in pen-based interfaces.

  The authors present the work in context, however the discussion about
  age-related mobility impairments is limited, and does not show deep
  understanding of the issues involved.  In particular, the authors talk
  about cognitive and sensory impairments in the introduction and yet not
  much about mobility/dexterity issues in older adults.

  I also find it odd to look at pen based interfaces with older adults.
  Given the problems with arthritis and other stresses on the hands/arms,
  is a pen based interface one that we would expect older adults to use?
  Do they use them now?  Is there evidence that this will be a substantial
  problem in the future?

The Review

  The methodology proposes a proper empirical evaluation of the original
  Bubble and Steadied pointers in comparison to both mainstream pointing
  and the proposed Steadied-bubble.  This is done in the context of
  modifying the independent variable of age, having 6 young adults and 6
  older adults in the study.

  While the results are significant, I am concerned that the authors would
  have 6 older adults from a wide variety of ages (65-80+) and group them
  together.  The mobility and dexterity issues between a 65 year old and an
  85 year old are substantially different, with one aspect being tactile
  sensitivity.  To group such a small number of users that are so varied
  together raises concerns about the conclusions drawn at the end of the
  paper.  I do not believe that you can have such general conclusions from
  such a collection of users.

  Having seen the rebuttal - I have considered to raise my score

Areas for Improvement

------------------------ Submission 230, Review 5 ------------------------

Title: Steadied-Bubbles: Combining Techniques to Address Pen-Based
Pointing Errors for Younger and Older Adults

Reviewer:           secondary

Contribution Type Specific Rating (Meets Contribution Type Specific
Criteria Well)

  3.0 - Neutral

Overall Rating

  4.0 - Possibly Accept: I would argue for accepting this paper

Expertise

  3  (Knowledgeable)

Contribution to HCI

  This paper evaluates a few existing interaction techniques (and combines
  two of the techniques) for older adults' pen-based interaction. The
  contribution is a bit incremental, but does provide evidence that some
  interaction techniques are better for older adults' (for pen-based input)
  and do not hurt younger adults' performance.

  Yes, the paper provides context, clearly reviews what is already known,
  and discusses limitations that exist in knowledge about this technique.
  Yes, the paper motivates a real problem that is worth solving, however, I
  am concerned that the problem is relatively small, and may not be
  worthwhile if users' performance improve dramatically with
  training/experience. Yes, the paper describes the technique in sufficient
  detail for others to replicate it. Yes, the paper includes a rigorous and
  convincing validation of the technique, where it clearly shows that
  incremental gains not only exist. However, the practical significance of
  these gains are questionable.

2AC Review

  The main thing that struck me about this paper was that it was submitted
  to the wrong committee.  It is primarily a detailed evaluation of several
  interaction techniques that happens to be explored for a different
  population. It would have been better reviewed in the "Interaction
  Techniques and Devices" sub-committee, especially since some of the
  keywords for that committee are:
  * Input and Interaction Technology
  * Universal (or Disability) Access
  * Pen-based UIs
  * Pen and Tactile Input

  These keywords seem to fit this paper perfectly. My guess is that the
  authors were looking for a committee that would be more forgiving to the
  fact that the techniques explored are not new. However, I think the ACs
  on the other committee have better expertise to evaluate the study
  performed.

  That being said, I am somewhat knowledgeable in this area and the study
  seems to be well done and the evaluation looks correct (although I would
  like an expert in the area of interaction techniques to look at it
  because the methodology and statistics are quite complicated).

  Overall, I am inclined to suggest that this paper be accepted. It is an
  incremental contribution, but does provide information that is useful for
  interaction design for universal access.  The paper is well written and
  the study seems thorough and well done.

  Here are a few additional comments/questions:
  - Why was the table put on the stand? I don’t believe that this is the
  natural way that people use a tablet, and it makes the interaction more
  difficult.
  - Why were the different aged participants paid different honorarium
  amounts?  Was the study all done at the same time?
  - The fact that only novices were used could be problematic. It could be
  that with a small amount of practice older adults would be fine with the
  smaller targets, and as such the results of this work become unimportant.
  - The fact that the results are only significant for the smallest target
  is a bit problematic. How practically relevant is this? How common is
  this size?  Would it be easier to just make sure that UI elements are
  large enough? What about typical target density for this size?
  - Make sure you define CT before it is used.
  - How did the graphical feedback help? Why was it necessary?