Teaching Software Engineering through Real Projects

Desanka Polajnar and Jernej Polajnar
Department of Computer Science
University of Northern British Columbia
Prince George, BC, V2N 4Z9
{desa, polajnar}@unbc.ca

 

 

ABSTRACT

Software engineering instruction often relies on semester-long student team projects in trying to recreate a realistic experience of professional software development in a classroom setting. A common weakness arises in the emulation of customer role within the requirements engineering process. This paper explores the merits and risks of teaching requirements elicitation skills through term projects with real external customers in the case of smaller class sizes. The analysis draws from concrete experiences of the authors. In the framework of a two-course software engineering sequence in the third year of an undergraduate computer science curriculum, the focus is on a requirements engineering project within the first course. The project organization is described, followed by a discussion of different approaches to the emulation of customer role. The approach involving real customers is illustrated with a comparative case study of two student teams working on parallel projects within the same course and with the same real customer. Some general conclusions are offered on how one might achieve a balance between exposing the students to a realistic requirements elicitation experience and keeping the project load, pace, and profile within the boundaries of the course syllabus.


Keywords: Software engineering education, requirements engineering, student term projects, requirements elicitation, customer role.

 

1. Introduction

  Software engineering courses often include team projects, in which instructors try to recreate, on a small scale and in classroom environment, a realistic sense of the tasks and phenomena that naturally arise in larger, real-life projects. At the same time, they try to preserve, as much as possible, the controlled classroom setting with carefully paced progress, predictable completion times, adherence to the course syllabus, and reasonable grading curves. These frequently conflicting requirements make software engineering courses hard to teach.

In trying to emulate professional software development in the classroom, one of the weakest spots is requirements elicitation. Whether the sources of domain knowledge and customer expectations happen to be the team members themselves, other students, members of the teaching staff, invited guests, or external interviewees, the `virtual' customer is usually a poor substitute for the real one. While virtual customers may or may not have genuine expertise in the application domain, their requirements typically bypass many areas that are relevant and difficult to address in real projects. Such areas include legacy system or personnel constraints, preferences for preserving existing operating procedures, itemized scrutiny of individual features based on cost, sequencing imposed by budget schedules and, in particular, semantic gaps between customers and developers that in reality often pose significant peril. The lack of a realistic requirements process is a serious problem, since the elicitation of project requirements is one of the more critical skills that software engineering curricula are supposed to teach.

In this paper we explore one possible approach to the problem, namely the adoption of class projects that are done for real external customers (cf. [1]). Those customers may or may not belong to the educational institution offering the course, but their own objectives must be real and unrelated to the pedagogical objectives of the course. They must stand to seriously benefit from the product and have a genuine interest in it.

When pondering how to fill the customer role in a class term project, one needs to find a balance between, on one side, the desire to expose the students to a realistic experience that helps develop their requirements elicitation skills -- and, on the other side, the need to keep the project's workload, pace, profile, and outcomes within the tight limits imposed by the one-semester course syllabus. In this paper we examine some of the factors that affect this balance, in the context of our teaching experiences based on smaller class sizes. We describe the organization of a requirements engineering term project used in a specific software engineering course, analyze different ways of filling the customer role in the project's requirements elicitation process, follow the progress of two teams working on parallel projects in the same course with the same real customer, and use those concrete experiences to suggest some general conclusions.

2. Term project organization

Our analysis is based on existing experiences with smaller software engineering classes consisting of computer science majors at the upper-division undergraduate level. They include our observations from teaching CPSC 300/301 Software Engineering I and II (first author), as well as research (CPSC 495/496) and project (CPSC 490/491) courses with software engineering topics (second author) at the University of Northern British Columbia. In this paper we shall mainly refer to the Software Engineering two-course sequence. The main subject of the present section is the term project organization in the first course.

Software Engineering I and II are electives that are strongly recommended to Computer Science majors. Typical enrollment is 15-20 for the first course, somewhat less for the second. The student population is a mixture of third and fourth year students. All have completed the four lower-division courses on programming, algorithms, and data structures, a course on system analysis and design, and (usually) a course on user interface design. Some have also had upper-division courses on operating systems, database systems, programming languages, and other software areas. They are advised to get a balanced background before entering the software engineering courses.

The first course is self-contained (but designed with the expectation that most students will continue into the second course). It has a substantial theoretical component (based on [2, 3]). Its emphasis is on software process models, system models, and requirements engineering. The project component focuses on the requirements engineering phase of software development. It includes requirements elicitation, generation of requirements documents that comply with the IEEE/ANSI 830-1993 standard [2], development of a limited prototype (not `throw-away'), definition of test cases for a key subset of the product's requirements specification, and validation of requirements. The second course includes detailed design of selected functionality (based on UML models), implementation, and testing. Design and implementation follow an incremental process that started with the prototype in the previous term.

The student project teams typically consist of three to five students each. They are formed in the second week of classes by students themselves, with occasional balancing intervention from the instructor. The project topics are determined in the third week of classes. An initial list of choices is proposed by the students and the instructor, and discussed in class. Each team then presents a prioritized list of three topics to the instructor. After the resolution of conflicting choices and some further discussion, each team adopts one topic. When approving the topic, the instructor assists the team in appropriately scaling the project size. By that time, the lectures have covered an overview of the requirements engineering process.

In its initial meeting, each team is required to do the following:

  1. select a team leader;
  2. distribute project duties among team members;
  3. schedule regular team meetings for the term;
  4. propose the time of an initial team meeting with the instructor and the customer;
  5. propose a schedule of regular team meetings with the customer;
  6. propose a schedule of regular team meetings with the instructor;
  7. generate a preliminary project description.


The team leader negotiates the frequency of meetings with the instructor and with the customer. In the first joint meeting with the course instructor and the customer, the instructor's role is to demonstrate to the team how to interview the customer. Within two weeks from the topic approval, students may request to switch teams in consultation with team leaders and the instructor.

Project topics proposed by students are sometimes planned well before they enroll in the course. They reflect individual or team interests and in some cases explore project ideas that are later applied in professional context. In this paper we are primarily interested in situations where there is a real external customer interacting with the student team during the requirements elicitation stage.

3. Requirements elicitation with virtual and real customers

In this section we analyze the ways in which the customer role can be filled for the purpose of requirements elicitation in the class project. In particular, we focus on the distinction between `real' and `virtual' customers. A more complete classification of customer roles, with analysis and characteristic examples, is presented in [4].

A virtual customer is someone who is willing to play the customer role as a part of the requirements process in the course, but is not a real stakeholder or user of the software product. This can be the course instructor, teaching assistant, a student not belonging to the team, a fellow faculty member, or other volunteer.

Requirements elicitation with virtual customers involves several potential difficulties. One is a possible lack of depth in the application domain knowledge. This need not be the case. For instance, the course instructor may have relevant professional experience in the application area, or the team may be lucky to recruit a domain specialist as volunteer. Another danger is that the virtual customer may know too much about software development to be able to model the semantic gap that typically arises in real life. Moreover, not having a stake in any particular features of the product, such a `customer' might in fact be guided, partly at least, by process rather than application needs. An external domain specialist may be sufficiently removed from software development to provide a semblance of a real semantic gap, but will likely act as a disinterested consultant friendly to the team rather than as a firm negotiator defending own perceived interest. In addition, the specialist consultant may not be available for sufficiently frequent regular meetings.

An inherent downside of virtual customers, even assuming independence and domain expertise, is that they primarily emphasize the general functional requirements. Realistic performance requirements are often difficult to emulate. Safety concerns, workplace regulations, training and practices of existing personnel, administrative structure of the client organization, legacy equipment and software, externally dictated timelines -- these are some of the constraints typically present in real-life requirements processes that cannot be realistically expected of an elicitation process involving virtual customers.

A real customer is a stakeholder who is expected to use the product, stands to benefit from it, has a direct, genuine interest in the product requirements, and wants a decisive control over them. The requirements of a real customer do not capture only the set of intended functional `features' of the product, but also insist on its practical usefulness within a realistic set of constraints. The real customer normally has the requisite domain knowledge, but may speak a rather different language than the project team, and may have insufficient knowledge of the technology to optimally set the level of specific requirements. Misunderstandings with a real customer can cause real harm. This provides the necessary subjective sense of risk, but may also introduce risks to the pedagogical outcome of the course. A further realistic feature which is seldom present in courses is the existence of a formal contract between the customer and the team. If the team is bound by contract to produce deliverables that meet the requirements specification, their experience is closer to reality but the risk is also enhanced.

Requirements engineering with virtual customers has inevitable shortcomings, but allows the course instructor to do the necessary planning and maintain control. Requirements engineering with real customers is clearly preferable if the risk of the course getting out of control can be kept to a minimum at a reasonable cost. In the next section we explore that possibility through specific project examples.

4. A case study involving real customers

In this section we discuss specific experiences with term projects in which the requirements elicitation process involved real customers. We study the case of two student teams in Software Engineering I, working on parallel projects with the same real customer during the same term. The projects followed the organization described in Section 2. The customer was a University research team in environmental science performing large simulation experiments in the area of climate studies. The customer was cooperative and prompt in providing information. The environmental research team did not include computer scientists, and the software engineering student teams had no prior knowledge of the application domain, resulting in an initial semantic gap.

Before the topic definition stage, the customer presented an overview of their research to the entire class and the course instructor. The talk also introduced the two simulation systems and the visualization system they use, and identified some desired improvements. In particular, the two simulation systems generated their output files in different formats, only one of which was accepted as input by the visualization system. This limited the possibilities of visual comparison of the simulation results produced by the two systems. Furthermore, the customer controlled the visualization system through a command-line interface, relying on memorized syntax of various weakly documented commands and their parameter lists. The overall impression was that the power of the simulation and visualization tools was exploited to a limited extent due to weaknesses in interfacing.

In the topic definition stage, the course instructor explained the likely advantages and disadvantages of working with real customers in a term project. The advantages included: a more realistic elicitation process, resulting in valuable requirements engineering experience; learning how to acquire domain knowledge from a specialist; and the domain knowledge itself, including the approaches, tools, and methods used in climate research. The disadvantages included: more time invested; more revision of initial requirements as the context becomes better understood; and risks of misunderstanding the customer and the problem, which could compromise the project objectives. In order to alleviate the risk, the instructor offered to attend several early meetings with the customer (taking the `back seat' and helping ensure that all key issues are adequately covered). The instructor did not make any suggestions to the teams as to the topics they should select.

In the topic selection stage, two three-member teams expressed interest in working on some of the presented problems with the real customer. Team 1 was to address the issue of format conversion and better connect the simulation and visualization stages. Team 2 was to determine the full capabilities of the visualization system and build a graphical user interface (GUI) for it. Each team was to have several meetings with the customer until the task and the context were fully understood, draft the user requirements and system requirements documents, have them validated by the customer, and then proceed to build a limited-functionality prototype that should be validated by the customer and used as a basis for detailed design and implementation in the follow-up Software Engineering II in the second term.

As they set to explore the background, the students encountered various difficulties typical of real projects. The systems were public-domain, open-source free software, developed over several decades in a decentralized manner by a number of researchers in the application field, and still evolving. The programming language choices (Fortran being gradually replaced by C), the lack of architectural uniformity, and the lack of systematic documentation were characteristic of the systems' life cycles. As a result, it was difficult for a system user to get a complete insight into the system's capabilities, and it seemed that a typical user would probably be familiar with only a subset of its full functionality.

At the end of the term, both teams successfully met their requirements engineering objectives. They both proceeded with successful detailed design and implementation stages in the second term and came up with useful deliverables that met (and in some aspects exceeded) the initial expectations. However, their paths in getting to the goal were rather different.

Team 1 took a cautious approach and invested significant time into early interactions with the customer. The course instructor attended the meetings and initially ensured that the interview process covered all the necessary angles. The instructor was subsequently consulted when the parties had doubts about mutual understanding of details, but the need for such intervention gradually disappeared. The initial requirements documents that the team generated were approved by the customer. The team collected information about the relevant file formats and inspected the legacy code to extract the data formatting detail, clarifying the system architecture in the process. They went on to design and implement a limited working prototype of the converter utility in C++, with its own graphical user interface in Java. The prototyping phase revealed further imperfections in the requirements documents, which were subsequently polished to provide a solid basis for the design and implementation phases.

Team 2 started with a more confident approach, possibly because their task seemed less intricate at the outset. Following the initial meeting with the customer and the course instructor, they started to work on the requirements documents. They had just one additional ad hoc meeting with the customer alone, and reported to the instructor that everything was clear. Their interactions were not thorough enough to let them appreciate the semantic gap, and they were shocked to find their draft requirements documents rejected by the customer. They believed in their proposed solution and only belatedly understood that that was not what the customer had been asking. The elicitation process had to be rolled back. Fortunately, the team managed to recover, revise the requirements documents, and get back on schedule. It should be pointed out that the rescue was accomplished without a need for instructor's intervention. The team inspected the legacy system code to extract the full command-line syntax and identify its full functionality. They went on to implement the prototype, and later a more complete version, of the GUI in Java, using the Java Native Interface to connect with a system routine in C that invoked Unix kernel calls to interact, through standard input/output, with the command-line driven visualization system.

One can conclude that in this case the student teams were exposed to a rather realistic experience in the requirements elicitation process, while at the same time the project work did not disrupt the balance within the course syllabus. As far as we know, the performance of team members in their other courses was not affected. The shortcut that one of the teams took may have coincided with time pressures in another course. However, on the whole the project activity seemed to fit well into the course and the students' study load. The students reacted with satisfaction, in particular at the end of the second course as they completed the implementations.

How should this positive result be interpreted and does it suggest some general conclusions? One should first note that the course instructor was able to closely monitor each team's progress and interact with teams and customers in a timely manner. With a total of four teams, this was a substantial task. In a larger class one would need competent teaching help, e.g., graduate teaching assistants specializing in software engineering, to maintain comparable attention to project dynamics. An important aspect of such attention is that each project was initially carefully scaled to an appropriate size and level of difficulty; this is what allowed a team to recover from early requirements errors. In order to help properly scale the projects, the instructor needed a good insight into the students' preparation and capabilities.

The customer in this case study was real in the sense of having a strong interest and commitment to the project, planning to directly use the product, being independent of the course, having the application domain knowledge, and being available for detailed interaction. However, several factors moderated the possible project pressures: the customer did not directly fund the work, understood the academic limitations and agreed that they should be respected, and did not have their own progress critically tied to the project outcomes. In our view, this was a reasonable balance that provided a good sense of reality while keeping the risks manageable.

5. Conclusions

In this paper we have explored the benefits and risks of relying on term projects with real customers in software engineering courses. The background for our analysis includes some of our own observations from teaching software engineering topics to undergraduate computer science majors. In particular, we discuss the ways of emulating the customer role in a requirements engineering term project, and report experiences from projects with real customers.

The analysis emphasizes the importance of finding a proper balance between exposing students to realistic experiences and ensuring that the project's workload, pace, and profile remain within the reasonable boundaries as outlined by the course syllabus. Some of the relevant factors are: the class size and the ability of instructors to closely monitor individual projects; careful initial scaling of team projects, based on sufficient insight into the students' educational backgrounds, practical experience, and potential; choice of real external customers who are able and willing to respect certain limitations of course projects, are not under extreme time pressures, have a strong interest in the product, and expect to be available for timely interactions.

With the proper stipulations, precautions, and resources in place, the value of having real customers in requirements engineering class projects can hardly be overestimated. A realistic software project experience is not only a necessary prerequisite for grasping the intent of software engineering methods, but also a prerequisite for proper understanding of why the discipline exists. One learns most from unexpected situations, and project organization should allow for them to happen. A study of software engineering methods in an overly controlled environment may create a false feeling of security -- a rather undesirable educational outcome. Working with real customers is definitely one way of maintaining a healthy balance between theory and experience.

References

  1. Bruegge, B. and Dutoit, A. H. Object-Oriented Software Engineering: Conquering Complex and Changing Systems, Prentice Hall, 2004.
  2. Sommerville, I., Software Engineering, 6-th. ed., Addison-Wesley, 2001.
  3. Sommerville, I., Sawyer, P., Requirements Engineering -- A Good Practice Guide, Wiley, 1997.
  4. Polajnar, D., Polajnar, J., ``Customer role in class projects on requirements engineering'', in preparation.


 
Name Desanka Polajnar Jernej Polajnar
Department Computer Science Computer Science
Institution University of Northern British Columbia            University of Northern British Columbia           
Postal address                3333 University Way 3333 University Way
Prince George, B. C. Prince George, B. C.
Canada, V2N 4Z9 Canada, V2N 4Z9
E-mail address desa@unbc.ca polajnar@unbc.ca