Ducky's Homework / Research

I am working for Gail Murphy, looking at programmer productivity. I will be writing an Eclipse plug-in that is designed to help people navigate through code more easily, then running a user study to see if it really does help.

I am also the coach of the UBC programming team.

Current TBDs

• design experiment
  • buncha people doing same task in controlled environment or lotsa people doin' their own thing?
  • each person do all conditions (e.g. control AND "try BFS" AND "use BFN tools") or each person does one condition?
  • which tasks? use one previously done?
  • which participants? Gail thinks pros are more publishable.
• write tool to analyze the nav tracks of BSD's data
  • investigate Prefuse (Java, easier)
    • download
    • read manual
    • do a toy problem
  • if needed, investigate GraphViz (http://www.graphviz.org/)
  • Can GnuPlot do it for me?
  • port BSD's utilities
• Talk to Leah re finding BFN
• come up with numerical measures for BFN
  • back button presses
  • edit ratio
  • Rob's thesis mentions that 58-81% of web navigation is to previously seen pages [Dumais et al], but passes on saying what % is for source. That would be interesting to try to pull out of Mylar data.
• write fake intro
  • get solid on the citations
• write the actual plug-in, perhaps using quickmarks as a bit of a guide

• Bannon et al: "[D]igressions are frequent. When a record is made of these commands as they occur temporarily, as in the history list, information on the tasks and goals of the users is lost." (where is this from?)

Current hypotheses

• H0: Most of time "wasted" is due to false hypotheses. (is there a ko paper that says so?)
• H1: Breadth-first search of the hypothesis space reduces the amount of time spent on false hypotheses on average.
• H2: Breadth-first navigation helps BFS => BFN when seeking verification for a hypothesis is on average faster than a depth-first navigation
• H3: The following things will result in users using BFN more often.
  • H3.1: tabs staying in once place
  • H3.2: tabs being containers for a task rather than a file (more like Firefox)
  • H3.3: making the back/forward button history only contain what was looked at in that tab, vs. everything in the global history
  • H3.3: marking "special" places in the history drop-down with icons
    • classes/methods reached via a search
    • classes/methods reached via the package explorer
    • (Question: what about reached from Outline View? Class Hierarchy view? QuickOutline)
    • (Question: do I want to insert a right arrow if they did declaration and left if they did references? What about a refs/declr search?)
  • H3.4: making search results visually distinctive if they have gone there recently
  • H3.5: allowing people to see two places in one file more easily is useful
• H4: telling people that BFN is a better approach
• (H5: telling people to write down three hypotheses is useful)

Research

• DuckyThesisProposalNotes

Code base -- code obfuscators?

• Low Crossroads paper on obfuscation

More generic, old

• Potential future study
  • Read ethics materials, including Guidelines
  • Discuss ethics w/Gail
  • Look at previous ethics submissions
  • DuckyEthicsDraft

Need academic ref

• reference to Stanley Eisenstat yale cs 323 time vs. outcome studies

Interesting references for me to chase down

• Cross, E. The behavioral styles of computer programmers. in Proc 8th Annual SIGCPR Conference. 1970. Maryland, WA, USA.

• Mayer, D.B. and A.W. Stalnaker. Selection and Evaluation of Computer Personnel – the Research History of SIG/CPR. in Proc 1968 23rd ACM National Conference,. 1968. Las Vegas, NV, USA.

• Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben- David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, and Tadeusz Wilusz. A multinational, multi-institutional study of assessment of programming skills of first-year CS students. In Working group reports from ITiCSE on Innovation and technology in computer science education, Canterbury, UK, 2001. ACM Press.

• B Adelson and E Soloway. The role of domain experience in software design. IEEE Transactions on Software Engineering, 11(November):1351–1360, 1985.
• Jeffrey Bonar and Elliot Soloway. Uncovering principles of novice programming. In 10th ACM POPL, pages 10–13, 1983.

and other references from This Camel Has Two Humps and Testing Programming Aptitude

follow-on to the camel

Links to software tools

• http://www.sei.cmu.edu/publications/documents/97.reports/97tr012/97tr012title.htm
• http://valgrind.org/downloads/current.html
• http://valgrind.org/downloads/variants.html
• http://www.ndparking.com/stlabs.com
• http://lclint.cs.virginia.edu/
• http://www.st.cs.uni-sb.de/eclipse/
• http://www.cs.wisc.edu/cbi/

daikon

• http://c2.com/cgi/wiki?DoNotUseAssertions
• http://www.research.att.com/viewProject.cfm?prjID=67
• http://www.parasoft.com/jsp/home.jsp
• http://www.csse.monash.edu.au/~davida/guard/
• http://www.sos.cs.ru.nl/research/escjava/
• http://www.cs.iastate.edu/~leavens/JML/
• http://www.gnu.org/software/ddd/
• http://www.codesurfer.com/products/codesurfer/overview.html
• http://www.st.cs.uni-sb.de/dd/
• http://www.cs.nmsu.edu/~jeffery/aadebug.html
• http://www.checkpointing.org/
• http://www.gnu.org/software/nana/nana.html
• DDD - gui debugger front-end, probably not that interesting since Eclipse has a gui debugger
• http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-830.html AskIgor

Research aids

• ecleTeX -- has anyone used this?
• Zotero -- firefox-based note keeper

• Set ALLOWTOPICCHANGE = DuckySherwood, GailMurphy