Our Workshop Notes

DOCUMENTING COLLABORATIVE INTERACTIONS: ISSUES AND APPROACHES

From the CSCL web site: "The aim of the workshop is to look at the different approaches used to document collaborative learning and inform the design of the next generation of CSCL tools. We will be looking into different ways to document collaborative interactions, factors that affect collaboration as well as their effect on learning outcomes, and the development of a community of learners."


Tues 9:00-9:30 Sadhana Puntambekar, Rosemary Luckin
Introductions

- collaborative learning
- data approaches to documenting interactions
- in educational games, in courses

Tues 9:30-9:50 Cindy Hmelo, Rutgers University, USA
Analyzing collaborative interactions in Computer-based simulations

- importance of discourse on learning

- procedure
  - make a hypothesis
  - formulate coding categories
  - as look at data, get feedback on categories
  - get lots of stats
  - but lots of things we don't capture
  - want to use qualitative and quantitative analysis

example 1 CaMille
  - asynchronous discussion tool
  - where are students productive? how are they productive?
  - looked at what kids are saying..
  - students did 80% of posts
    - students talked about cog sci in context of their projects

example 2 Synchronous Collaboration
  - simulations of clinical trials for students
  - students worked in groups of 4-6 students (4th year medical students)
  - how do students with diff knowledge learn collaboratively from simulation?
  - quantitative results
  - BUT also qualitative results sought
    - looked at how the groups actually constructed the problem space...
      - high knowledge group used strategies

example 3 Studying a face-to-face Problem Based Learning Tutorial
- focus on what tutor did
- how is knowledge collaboratively constructed?

- discussion
  - how to representations of sequences, especially when they are long schemes
  - conversation analysis 
    - structure is based on explanation! codable.. but a lot of work

Tues 9:50-10:10 A Martinez, Y. Dimitriadis, et al., University of Valladolid, Spain
Combining qualitative evaluation and social network analysis for the study of classroom interactions

- naturalistic - curriculum based experiences
- how to automate process of qualitative analysis
- educational setting, university course Computer Architecture
- goal is to provide integrated and meaningful learning in course
  - student groups are to build some solutions for a client
- class structure, students divided into groups, each group assigned to a client
- in each session, pairs of students work together (intra-group interaction)
- some groups interact with other groups of other sessions
- tools used: debate organiser, bscw (group), e-mail (all asynchronous comm)

- methodology (the below utilize the data gathered from the above mentioned tools)
  - analysis procedures
  - comparative analysis of collaboration aspects (Nud*IST)
  - social network analysis (EL2AM)

- look at which things students will use to collaborate
- objects can be shared through bscw tool
- graphs analysis shows trends etc in use of each tool
- integration between qualitative mehods and social network analysis
- automatic support is needed for analyzing all that data

Tues 10:10-10:30 Reuma De. Groot, Hebrew University, Israel
Animating argumentative activities and CSCL

- how to use computer tools to foster argumentative activities in the classroom
- target people: elementary and junior high schools
- teachers brings problem to students, students must construct their hypothesis 
  for the dilemma
- students must work together and must construct their reasons together
- goal is to train students to argue, debate
- basic assumptions: 
    collaboration is a way of social interaction
    -> social interaction helps with individual cognitive processes
    -> cognitive processses help with knowledge construction
    -> in learning environments we look for facilitators that will help with 
       knowledge construction of which we assume resembles a better understanding
- methodology
  - social analysis, method of Resnick et al as modified by Dreyfus and Schwartz
    * build a map that shows the interactions and their relationships
    - kind of interactions!!! (MW: helpful for PrimeClimb later)
        declaration of new idea
	modification
	repeat
	explanation
	agreement
	done
	question - clarification
             objection
             direction
             encourage
	(arrows) --> relationships between the kinds of interactions (boxes)
  - examination of interaction dimentions as suggested by Baker
  - epistemic actions defined by Dreyfus, Hershkoviz and Schwartz as:
    recognizing, building with, and construction knowledge

Tues 10:30-11:00 Break



Tues 11:00-11:20 Richard Joiner & Kim Issroff, University of Bath, University college, London, UK
Spatial representation of collaborative interactions

- report a study of adults collaboratively completing a common multi-robot foraging
  task
- used an analysis method, trace diagrams

- multi-robot systems
  - example, foraging task done by multiple computer robots --> looked at 
    collective behaviour
  - used a trace diagram
    - can understand a bit more about the nature of robots collaborating
  - similarities to student tasks.. so this robot learning behaviour might provide
    some insight

- MooBees
  - aim to analyse their joint problem solving with trace diagrams
  - text-based environment
  - participants solved tasks in pairs and in separate rooms
  - as a bee, find as many flowers as possible
  - communication doesn't help robots!?
  - trace diagram shows where pair went
    - can see if students divided the search areas between them
      - joint information searching
    - problem: interactions like communication not represented
    - problem: does not represent time
    - problem: scalability
    - advantage: represents an aspect of collaboration which is not normally 
      looked at before

Tues 11:20-11:40 Jonathan Cohen, James Dai, Michael Wu, Sarah Yang, University of British Columbia, Canada
Colourful hints for collaborative climbing

- EGEMS ROCKS THE HOUSE!
- James gets beaned in the head

- discussion
  - seating can be extended to different situations that allow us to see things
    like turning head to look at partner
  - granularity is important in game events just as it is in social interaction 
    events

Tues 11:40-12:00 Rosemary Luckin, University of Sussex, UK
Toys, TV and Ubiquity: the challenges of new educational technologies for documenting collaborative interactions

- how adaptable is documentation technique in other forms of multi-person 
  collaboration

- desktop computer
  - challenges
    - can techniques that have proved succcessful in CSCL be used in personal 
      computers? YES!
    - example Ecolab, Galapagos - we ask if doc techniques are still applicable 
      to toys etc

- Ecolab
  - Adaptive Interactive Learning Environment
  - offered scaffolding
  - single user, collaboration btwn system and user
  - interaction summary records (multidimension timeline chart)
    - chronology of interaction events VS time (when does it matter!)
    - useful cuz we can add annotations to that
    - particularly interested how they asked for help
    - pinpoint places where collaboration btwn child and computer system
    - could look at number of times they asked for help, what level of help
      - through this, build profiles of users

- Galapagos
  - groups of users      
  - collaboration btwn students and user and students and students
  - video recording (similar to PrimeClimb)
    - clapper board (didn't need precise synchronization)
      -> result is mixed tape 
    - CORFU Charts (Chronologically Ordered representation of Features Used)
      - multidimentional timeline charts with events as boxes
      - know what happens btwn system and learner
      - BUT want to know what happens within group of learners
        each line (duration) represents group of user and system
      - also ADD social interaction events to chart (what type of task, whether 
        they were to form a sub-goal, whether or not they were reactions to 
        things) and synchronizing
        * found it was useful! adds value to what's happening to what happens to 
          user and system
        - example found that while they were looking at info, students weren't 
          talking... 

      * hard to define what event is and then set it up
        have to define this ourselves, units of length of sentences

- Toys
  - can new toys offer a useful way to present learning experiences for children
  - various interactions very different from interactions with computer
    - hugging, squeezing, cuddling
  - some toys can be connected to a computer
  - example DW dolls

- order and stages of school studies
- lots of techniques too

Tues 12:00-12:20 Jakob Tholander, Stockholm University, Sweden
Colaborations in children's game programming

- Playground
  - compuatational learning environment for kids to learn the mechanisms of 
    programming computer games
  - target children: aged 6 to 8 
  - two kinds of collaboration:
    - pairs of children building games
    - resesarchers guiding and helping children
    * these are different kinds of learning
  - children at such small age can't verbalize their wants if they are sitting 
    on the side..
  - pairing of children (experienced with less experienced)

- ToonTalk
  - build computers by using robots
  - child could be quiet but still learning, just not verbalizing
    - watch to see if more was done when partner made commands, watch to see if
      student does stuff that is discussed between researcher and other partner
      --- SILENT ACTIVITY

Tues 12:30-1:30 Lunch



Tues 1:30-2:30 Rosemary Luckin
Video analysis
- case studies
  - operational interaction (interact with technology) vs conceptual interaction 
    (interact with subject manner)

Tues 2:30-3:30 Discussion - How can you trace the development of collective knowledge?
Other issues from presentations.
- online course in which cases were given out and students would have to solve it 
  using two tools
- students were to post solutions together
- analyzed 1000's of boards, had quantitative analysis... but many interactions 
  also (example what's happening with the students' relationships with each other, 
  etc)
- want to look at how collective knowledge developed

- characteristics used to examine how collective discourse develops
  - evidence of initiation as course progressed
  - integration from other student's responses and does this improve over time
  - internalization
  * how do we track all that?

- discussion was based on the topic of the case
- webboard used for posting
  - difficult to post and reply from
  -> this affected response

- culture is very different as it is an online course
  - instructor took a back seat
  - feedback was that there wasn't any feedback on the board
  - culture affected way 

- some events triggered by outside influences, midterms from other courses can 
  affect coursework
  
- other issues from presentations
  - is it time or chronology? 
  - meta level knowledge vs domain level knowledge
  - co-occurring events, how to handle big chunks of time and small chunks of time
    - how to handle the different interactions, sequences
    - what is the appropriate unit of analysis? different units for diff 
      interactions.. how do we combine them?
  * very much depends on what we want to measure... we must question what we're 
    trying to measure, then we can answer the above questions specifically
    - but what about exploratory research?
    - we invent these tools or companies make them, but how many times do we have 
      to persuade people to use them?
  - operational interaction (interact with technology) vs conceptual interaction 
    (interact with subject manner)? how do we know we're not just documenting the 
    mechanics of it?
    - ? maybe we have to correlate multiple sources
  - PrimeClimb tying in with mathematical learning (measure)
    - learning is not limited to just how well students do on pre-post tests
    - but what if we're specific and say that students learn about the stuff they 
      talk about?

Tues 3:30-4:00 Closing remarks