A Research-Based Masters Program in the Workplace

The Software Engineering Research Network (SERN) is funded by industry and administered by the Industrial Software Engineering Chair at the University of Calgary to support the dissemination of good practice in software engineering. One component of SERN's activities is a thesis-based masters program with a specialization in software engineering targeted on students with industrial experience in full-time employment. This program has dual objectives: to develop highly-qualified personnel; and to encourage industry-based software engineering research with a focus on good practice. The program commenced in September 1996, and this article describes experience in its operation.

The primary focus of SERN is on software development as a manufacturing discipline, with objectives of establishing: a centre of excellence in research and development concerned with the knowledge flows and processes underlying the software manufacturing life cycle; research and graduate training activities that will develop highly qualified personnel in industrial software manufacturing; an organizational infrastructure at the University of Calgary supporting a long-term program of university-industry collaborative research on software manufacturing.

The organization of SERN is shown in Figure 1, and involves cross-disciplinary laboratories at the University undertaking fundamental research, joint projects with the Industrial Partners, and maintaining a database of knowledge, experience, problems and issues that can be widely accessed and will be used to manage the activities of the network. SERN operates as a networked organization providing associated university and industry personnel and students with a virtual institution which manages joint projects. There is a two-way flow of knowledge and personnel in which the Industry Partners introduce new problems, the University researchers introduce new technologies, and the relation between the problems and technologies is investigated through joint projects. Graduate students and university personnel work on the projects at the industry sites, and that industry personnel are seconded to the University for varying periods of time. The network is open to new partners and to the introduction of new technologies from elsewhere. In particular, active links with a range of other relevant institutions are being developed.

2 Software Engineering Thesis-Based Masters Specialization

An overlapping cohort model has been found to be highly supportive of the processes outlined above. There are two major intakes into the program, one in September and one in January, of approximately equal size. The core courses are taught only once a year so that on each course there is a roughly equal mix of new students and those who have already taken courses and are experienced in the learning environment. Students work in teams to develop presentations and those with prior experience of doing this essentially mentor those who are new to it. This has proved very effective in bringing new students rapidly into the learning culture of the program.

As in our undergraduate software engineering course (Shaw and Gaines, 1997c) student assignments are submitted on the World Wide Web making them accessible to others. A list server is used for continuing discussion of the course topics outside the class environment. These two features have made it possible for students whose companies allocate them to work in other locations, or who change jobs to companies in other locations world-wide, to continue to participate in the courses. Remote students who have already come to know their colleagues can continue to present and share material through the web and participate in discussion through the list server. We are currently beginning to address the challenge of making more material formally available to remote students and this early experience of students becoming remote through circumstances beyond their control, but maintaining effective ongoing participation, is proving very valuable in designing for the distance mode.

5 Student Experience

The students put all their work on the web, including the comments from others. For example, the pair who took the KPA of Software Quality Assurance included the section "Feedback from Others":

It is believed that a 'techniques map' as illustrated in Figure 1 presents a nice outline of the Requirements Engineering techniques that will be addressed in this report. It not only illustrates the techniques that will be discussed but also shows the possible combinations between them. This map will be used as a reference throughout this report. The reader is encouraged to point and click on any technique in order to learn about a short description and about its strengths and weaknesses. Also, pointing to any labeled relationship in the map enables the reader to learn about possible combinations between these techniques.

Another group used repertory grids to compare the techniques used in their report:

A repertory grid was used to compare various Requirements Engineering techniques. The techniques covered in this course were used as elements. Constructs consisted of quantitative vs. qualitative, understandability, scalability, usability, and time to execute.

A FOCUS hierarchical structure and PrinCom analysis of the grid are shown. The results show a similarity between Repertory Grids and Group Comparisons, which is expected since the latter technique is an extension of the first. The results also show a similarity between Interviewing and Use-Case Analysis. This may be due to a common involvement with users and a shared goal of acquiring information about how they work. Analytic Hierarchy Process is different from the other techniques. AHP is quantitative whereas most of the other approaches are primarily qualitative, and it is the least scaleable technique.

This comparison could be further enhanced by including more constructs to differentiate between Interviewing and Use-Case Analysis. The grid could also be exchanged with another group for comparison.

In 613, each group also had to produce a practical exercise for the other groups to do in class, to make sure they had a full understanding of the topic being presented. The group looking at Checkland's (1981) Soft System Methodology (SSM) set the following exercise:

Objective: This exercise is designed to help you to have a good understanding of what is SSM and how to use it in practice. Remember: There is no 'right' or 'wrong' answer for this exercise, what you need to present is your thinking about the problem situation and a relevant system model for your thinking. There are five requirements for this exercise, do as much as you can within 20 minutes. If you need hints for these questions, please refer the appendix.

Problem situation: A local Calgary choral society always has the difficulty in obtaining nominations for its officers and committee and attracting people to participation in choral related works. As it is a performing society a number of non-choral tasks must be managed. How could this difficulty be addressed and examined?

Requirement 1: Use a rich picture to address the "Problem Situation Expressed" stage in SSM for this problem situation.

Requirement 2: Define a "Root Definition" for this problem situation.

Requirement 3: Do a CATWOE analysis on your Root Definition.

Requirement 4: Produce a "Conceptual Model" based on your Root Definition. (activities such as identify needs of local community, attract membership, general funds, attract audiences could be used for this problem situation)

Requirement 5: Use the "Tabular Display" comparison method to perform the comparison between the real world and intellectual world.

The list server provided not only a method for communicating messages and discussing issues which had been unresolved in the class, but also as a continuation of the presentation and exercise. For example, after the group exercise on the comparison of approaches to requirements from the perspectives of Structured Analysis/Design and Object Oriented Analysis/Design, the group used the list server to promote more discussion during the following week, before the next topic was introduced.

Before we go on with the next approach (SSM), I would like to post one more issue regarding the problem analysis activity.

In his book "An integrated approach to software engineering", Jalote argues that there are a number of approaches to the problem analysis. They include (1) the traditional approach where the requirements elicited and the problem is analyzed through the use of interviews, questionnaires, forms, (2) a 'modeling' approach in which the information is 'transferred' into a model of the problem and (3) a 'hands on' approach in which prototypes are built and the problem is analyzed and the requirements are validated throughout the process by the use of a prototype system.

However, it seems to me that these approaches can be combined and used together into a powerful approach. The issue I want to discuss is the involvement of customers in such an approach. The analysis of information in the problem domain, as we saw last class, seems to be in the analysts' world. I agree that the structuring of information is not an easy task and it should be done properly by experienced people (like analysts) in order to check for completeness, inconsistency and other issues like these ones. However, the customers are the primary source of information for this analysis (isn't the problem analysis the activity where the needs of the customers and what is desired from the software is understood?). Issues like requirements completeness, inconsistencies or conflicts should be discussed with the customers. Then, the question is "to what extent and in what way do we involve the customers in the 'modeling' of the information?" what modeling approach to use so that (1) the customer understands what the analyst is doing (especially when the models become big and complicated) -- and we come here to the issue discussed in class, of the communication with the customers, and (2) the customer is able to validate the information for completeness, etc.?

6 Research Methods Course

Since the primary deliverable for a research degree is the thesis and the degree will be awarded principally through an examination committee assessing the thesis and the student's defence of it, a research methods course is essential to the program. The majority of students already have substantial experience of software development in industry, and some have come into contact with research issues, but virtually none of them have any experience in developing a research program, setting up experiments, collecting data, analyzing it appropriately, drawing appropriate conclusions and presenting all this effectively in a substantial, coherent document.

We have found in the past that the basics of research methods are best assimilated when students can perceive a need in their own experience or that of others, and hence a case-based approach has been adopted for this course with the cases being the ongoing research of students in the program. The course commences with the presentation and detailed analysis of the concept map for a research program shown in Figure 4.

The objectives defined on the left follow a logical sequence and also correspond to the typical chapter structure of a computer science masters thesis. The required activities to carry out these objectives provide the students with the framework for a research program. The students develop a concrete instance of this concept map for their own research program, and can do this in an active concept mapping tool (Gaines and Shaw, 1995b) that also links to material on the World Wide Web (Gaines and Shaw, 1995a; Gaines and Shaw, 1995c).

The assessment for the research methods course aims to develop the student's thesis:

I Plan 1: 10%: A statement of your current progress toward choosing a thesis topic including a list of possible topics. For each possible topic include:

1. possible supervisor at U of C
2. possible supervisor at work
3. a plan for how you propose to complete the research and thesis including milestones and schedule
4. a short list of the fundamental reading material for the research area.

II Plan 2: 20%: A 2000 word document on the web to include:

1. a short introduction stating the main topic area and importance of the topic, and background
2. aim and objectives of the work
3. current plan as in (3) above
4. revised reading list
5. concept map completed for your topic (see below)

III Presentation 1: 10%: Presentation of research ideas. To be done after Plan 2 to expose your ideas and get feedback. 20 mins + 10 mins discussion

IV Literature Survey. 20%: A 4000 word document on the web surveying the current state of the art relating to the aim, including extended reference section in standard format (check out one of our papers as a model).

V Critical Analysis. 20%: A word document on the web of a critical analysis of the current state of the art from IV above, leading to a research design based on this analysis. Include a short section on how you plan to implement your research design and a section on how you will evaluate your research implementation. Include also milestones and schedule for completion.

VI Presentation 1I: 10%: Presentation after V (critical analysis). 20 mins + 10 mins discussion

VII Participation 10%: Participation in discussion of other research presentations both in class and on the web.

7 Conclusions

The program now has over 30 students after two years of operation. This means that, since we run the required courses with overlapping cohorts, the average class size of these courses is 15 students. The students work in teams which are changed (usually drawn by lot) for each assignment. Every student is equally responsible for the work produced, and the same grade is given to each team member. All the notes and the student work is on the web so anyone can benefit from the course materials. The first students are expected to graduate during the summer of 1998. We are currently extending the program to include a course-based version where, instead of a research thesis, the student will carry out a project which will contribute to the database of experience and best practice accessible through the web.

The program builds on our objectives and experience in designing new learning environments based on facilitating collaborative learner interactions in what we have termed a learning web (Gaines and Shaw, 1997; Shaw and Gaines, 1997a; Shaw and Gaines, 1997b; Shaw and Gaines, 1997c). The learning web is a systemic approach to the modeling and support of knowledge processes in a learning society. It models various aspects of education as a means of disseminating expertise to create continuing and large-scale resources from short-lived and limited-capacity individual agents. The role of information technology is modeled as one of providing knowledge support systems that expedite the processes of knowledge formation and dissemination.

The learning web approach has been introduced into existing university courses and doing so provides the basis for its implementation on a larger scale transcending institutional boundaries. However, an essential prerequisite is the restructuring of existing pedagogical approaches, de-emphasizing received wisdom and authority of the instructor, and emphasizing collaborative learning and meta-reflection on all aspects of the learning process itself. The instructor becomes a facilitator of students learning to learn, and eventually that process of learning to learn must itself become an overt topic for discussion by students fully participating in its management. In the same way that in the current educational system the home prepares for the school and the school prepares for the university, the university must come to see itself as preparing students for a role in a learning society in which they have come to understand and manage their own processes of lifelong learning.

We are now involved in extending the approach reported in this paper to support professional development and lifelong learning in software engineering. This entails further modularization of course material, more extensive coverage of topics at senior undergraduate and graduate level, and more effective use of Internet technology to support workplace and distance learning. We see the need to partner with other institutions to do this cost-effectively and to sustain excellence in the program, and hope that this article will lead to new partnerships.

Acknowledgments

Internet Access

Further information about SERN can be found at its web site

The SE MSc course material is linked from: MSc Software Engineering Specialization

Other relevant articles are available.

Articles on the learning environment are also available.

References

Gaines, B.R. and Shaw, M.L.G. (1995a). Collaboration through concept maps. Schnase, J.L. and Cunnius, E.L., Ed. Proceedings of CSCL95: Computer Support for Collaborative Learning. pp.135-138. Mahwah, New Jersey, Lawrence Erlbaum.

Gaines, B.R. and Shaw, M.L.G. (1995b). Concept maps as hypermedia components. International Journal Human-Computer Studies 43(3) 323-361.

Gaines, B.R. and Shaw, M.L.G. (1995c). WebMap: concept mapping on the web. World Wide Web Journal 1(1) 171-183.

Gaines, B.R. and Shaw, M.L.G. (1997). Institutional transformation to a learning web. Proceedings of ED-TELECOM 97 : World Conference on Educational Telecommunications. pp.384-389. Charlottesville, VA, Association for the Advancement of Computing in Education.

Shaw, M.L.G. and Gaines, B.R. (1997a). Collaboration and negotiation through the learning web. Proceedings of ED-TELECOM 97 : World Conference on Educational Telecommunications. pp.1390-1391. Charlottesville, VA, Association for the Advancement of Computing in Education.

Shaw, M.L.G. and Gaines, B.R. (1997b). The learning web in the workplace. Proceedings of ED-TELECOM 97 : World Conference on Educational Telecommunications. pp.461-466. Charlottesville, VA, Association for the Advancement of Computing in Education.

Shaw, M.L.G. and Gaines, B.R. (1997c). Using the web to support cooperative learning in software engineering. Tsiknis, G.K., Ed. Proceeedings of WCCCE'97: 2nd Western Canadian Conference on Computing Education. pp.63-72. Nanaimo, BC, Malaspina College.