Designing Software Quality Assurance Course:An Effective Framework for Teaching

Designing and teaching a course like Software Quality Assurance (SQA) always poses enormous challenges [1]. Most of the students taking the SQA course lack practical exposures to the quality issues addressed by the software industries, and many of the SQA topics are of theoretical in nature. The challenge for the course designer is to select the appropriate subject areas from the enormous amount of available theoretical body of knowledge, and to provide students with a balanced framework that would enable them to apply these theoretical skills to software quality issues of information technology industries. The difficulties encountered in previous years in delivering the course, and experiences gained there-from, provided the impetus to re-evaluate the courseware and to re-examine the strategies for the delivery of the courseware. This paper discusses the strategies used in designing and teaching the Software Quality Assurance course for the fourth year students of the Bachelor of Technology in Information Technology (IT) degree program at Kwantlen University College. The paper provides an overview of the curriculum design, discusses the strategies used in integrating the theory with relevant practices commonly encountered in IT industries, outlines the challenges encountered in delivering the courses, and describes the successes that have been achieved.

1. INTRODUCTION

Kwantlen University College is offering the Software Quality Assurance course to the Bachelor of Technology in Information Technology degree program since summer 2000. The course was initially designed to meet the existing demand in the industry for technical personnel with strong background in Software Quality Assurance theory and techniques. This course is mandatory for all fourth year Bachelor of Technology in Information Technology degree students. The course will address the fundamental concepts of Software Quality Assurance. This course provides students with a solid foundation of knowledge on which to build a career in Software Quality assurance, and covers topics pertaining to the Software Quality Engineer certification of American Society of Quality (ASQ).The curriculum is designed to make academic theory relevant through exercises and projects normally encountered in a real world environment.

2. COURSE OVERVIEW

In the Software Quality Assurance course [1] students will learn the essential features involved in developing timely, cost-effective and high quality software products that meet the user's requirements. They will examine the effective deployment of quality assurance procedures throughout the entire software development process. The subject material is presented from a very broad base, with each lesson designed to target specific software quality assurance tasks. The course is designed in a modular fashion with each module focusing on specific Software Quality Assurance topic areas. With this modular framework approach, the students will achieve core competencies in the field of Software Quality Assurance. The framework comprises of the following two modules [see Appendix 1: Table of Modules], with each module being further subdivided to cover specific topic areas:

• Module 1 (Body of Knowledge): focuses on comprehensive understanding of software quality development and implementation methodologies. Specifically the following topic areas are included:
  • Cost of Quality,
  • Quality Tools and Statistical Techniques,
  • Software QA Planning (which incorporates ISO 9001 and Capability Maturity Model),
  • Software Development Life Cycles
  • Software Maintenance,
  • Software Configuration Management
  • Software Auditing
  • Software Metrics and Measurement

• Module 2(Software Testing): places emphasis on methods and techniques of software inspection and testing, verification, and validation and includes:
  • Software Verification and Validation
  • Software Test Overview
  • Test Types
  • Error Types
  • Test Planning and Documentation
  • Software Testing in Product Development
  • Testing Internet Applications
  • Automated Testing.

The supporting resources pertaining to Module 1: may be obtained from the Software Quality Engineer Certification (CSQE) Body of Knowledge [2]. This is a remarkable compendium of concepts and resources which is unrivalled.

Material pertaining to Module 2: Software Testing is referenced primarily from the best-selling Software Testing text Testing Computer Software by Kaner, Falk, Nguyen [3], from Wiley publication. The content was supplemented with the applicable IEEE 829-1998 standards for Software Test Documentation [5], and multiple articles available on software testing websites [4].

3. TEACHING STRATEGY

Classes for the course consist of a series of lectures, assignments, in class projects, term project and project presentation. The lecture will serve to introduce topics. However, there is a strong focus on reading, appraisal and assimilation of appropriate material in the textbook to provide further detail and context . Students are advised to read to relevant materials, as they will help in their assignment, and provide additional in-depth information. The course is presented and delivered with a strong emphasis on class interaction and participation. Students can be easily distracted in an evening class, particularly if they have access to a computer network with a browser. Given the strong positive correlation linking student participation to student performance, it is felt that the interests of the students would be best served with a direct incentive to participate, and rewards for such participation. As such (25%) of total course marks are assigned for student participation.

The course materials are segmented into independent modules [Appendix 1: Table of Modules]. The modular approach allows the courseware to be modified as required without disrupting the entire course. Students are provided with the course materials at the start of the class. Lectures are supplemented by a corresponding case study [Appendix 2: Assignments and Case Studies], which requires the students to apply the course material to a given scenario.

Students are encouraged to contribute to the collective wisdom of the class. There are areas where students have particular expertise. This insight and experience are drawn out to assist the other students understand the concepts. This also helps to validate the course material and reinforce the importance of the software quality assurance principles. After 60-70% of the course materials have been covered, the rest of the schedule times are devoted to application of the material completed using number of case studies [Appendix 2: Assignments and Case Studies]. Students are then required to apply the knowledge gained through course materials to typical situations, and elaborate on particular areas.It is observed that this segment of the schedule is proven to be very valuable in expanding everyone's knowledge and perspective. It also shifts the burden of enlightenment from instructor to many students, expanding the knowledge base exponentially.

There are two midterm examinations to verify that the students are competent and knowledgeable in the subject matter. The first midterm examination is devoted to the Software Quality Engineering topics, and second one to Software Testing.

There is no final examination for this course. Students are required to complete an assigned project. A project assignment [Appendix 3: Final Project Assignment] contains two parts: case studies and presentations. The case studies require significant works to be done on different aspects of Software Quality Assurance. The oral presentations permit students to provide their personal perspective on an area which particularly interest them, or on which they had experience or competency. This expanded the scope of the course considerably, and gave the students ownership of the course.

4. BENEFITS

The course adds value to the students in a variety of ways:

• Establishes the purpose and cost justification for quality assurance.
• Introduces the formal systems and methods for quality management.
• Demonstrates quality assurance planning process.
• Explains application of quality assurance methodologies to the various software development life cycles.
• Provides contextual overview of software maintenance, configuration management, auditing, and metrics as related to software quality assurance.
• Explains software verification and validation practices.
• Highlights detailed description and application of software test planning considerations.
• Reviews test types and error types for consideration in test case preparation.
• Reveals the industry standard for software test documentation to assist with Test Plans, Test Designs, Test Cases, Test Procedures, Test Logs, and Test Reports.
• Expands software testing knowledge to consider internet testing and automated testing.
• Provides opportunities for both individual and group work.
• Provides opportunities for presentations.
• Discloses references for additional insight and information for further subject matter study.


5. CHALLENGES

There were many challenges that need to be addressed throughout the course:

• Student attentiveness: This was addressed by increasing student participation and interaction, and by having explicit rewards for such participation.
• Significant number of students with English as Second Language: This was addressed by providing notes in advance, and getting confirmation of understanding from students after class.
• Course expectation: Software QA and Software Testing are not the only focus of study This issue is addressed by establishing the relevance to programming, design, network administration, and other areas of primary interest for students.
• Usefulness of the course : The catastrophic real life examples are presented to demonstrate the effect of lack of Software Quality Assurance on different projects.
• Lack of appropriate text book : Additional resource materials are provided to support the lecture topics.


6. RECOMMENDATIONS

The delivery of the courseware based on modularized framework has proven to be successful. However, there are number of issues that must be considered to improve the courseware:

• The pace of the class should be increased at the start so that the projects and presentations can begin earlier. By having these at the very end of the course, the instructor is unable to review the results with the class. This review would add considerable value to the class.
• The modules should have explicit learning objectives stated. The instructor and the students can assess the progress based on the fulfilment of these objectives.
• Content is repeated in the modules, particularly in the ones related to software testing. The course should be streamlined to reduce or remove redundant items.
• Kwantlen has an excellent, award-winning website. This creates a direct, relevant opportunity for testing. The course should collaborate with the Kwantlen Web Team to determine the testing methods and involve them in the lectures and presentations.
• In every class, there are outstanding students who distinguish themselves with an excellent grasp of the subject material. These students should be encouraged to mentor other students and support the instructor’s efforts. This would contribute to the overall acceptance of the material by the entire class, and help struggling students.
• The course should permit students to shine based on their particular strengths. Students, who do poorly at exams, may be more inclined to hands-on activities. This should be considered with case studies.
• There are multiple examples of cheap or free software testing tools available from the internet. These tools could be applied to expand the knowledge of automated testing and internet testing.
• The final class should have industry representatives address the class. This would be a refreshing change of pace, and would provide special insight into the actual industry. Students could expand their network and obtain insight into a potential career path.


7. CONCLUSION

The modularized approach used to teach the Software Quality Assurance course with extensive application of case studies has positive impact on the students learning outcome. The separation of theoretical components of Software Quality Assurance knowledge base from specific issues of Software Testing and Validation components enable the students to understand the concept of software quality and its importance in industry. With real life examples as the motivating factors, the courseware provides in-depth and meaningful content, and conveys to students the theoretical principles of Software Quality Assurance, and its potential practical usage. The SQA knowledge and training acquired through the project and case studies will provide the students skills and valuable experiences required by the IT industries. It is our belief that the introduction to modular approach to teach Software Quality Assurance principles and practices is successful.



8. ACKNOWLEDGEMENT

We are very appreciative to our students who generously shared their ideas and suggestions, for which we are grateful.



9. REFERENCES

[1] Abhijit Sen, Elsie Au, Software Quality Assurance: A Perspective in Teaching , Western Canadian Conference on Computing Education, Nelson, British Columbia, 2001, html version at, http://www.cs.ubc.ca/wccce/program01/papers01/abhijit.html
[2] Software Quality Engineer Certification (CSQE) Body of Knowledge, American Society of Quality, http://www.asq.org/cert/types/csqe/references.html
[3] Cem Kaner, Jack Falk, Hung Q. Nguyen, Testing Computer Software, 2nd Edition, John Wiley & Sons, 1999.
[4] Resources on testing techniques and expertise, http://www.qacity.com , http://stickyminds.com
[5] IEEE Standard for Software Test Documentation, ANSI/IEEE STD 829-1983 - Description, http://standards.ieee.org/reading/ieee/std_public/description/se/829-1983_desc.html

Appendix 1: Table of Modules

Appendix 2: Assignments and Case Studies

You are in the role of a Quality Advisor to a two-person start up that wishes to develop a brand new software product. The product is a software program which monitors the maintenance and calibration schedules of surgical equipment. It will be used in hospitals and clinics. The intended users are the doctors and clinical specialists.

Justify to the two entrepreneurs why there should be Quality Plan in place. Provide a method of how a Quality Plan might be constructed. Identify Quality Techniques which may be used to create a customized plan that is appropriate for the business.

Case Study 2 (Quality Tools and Techniques, Test Planning, Test Case Preparation):

You are the Software QA Manager for a company that makes commercial games. The company has had a history of the following problems:
- Late development schedules;
- Reduced times allowed for QA testing;
- High incidents of product rejects due to poor product performance and functionality;
- A high number of customer complaints, most focusing on 10 or 12 main issues;
As this is a game manufacturer, the corporate culture is very loose and playful. There is no sense of structure or formality.

What steps would you take to use your team (20 people) and resources to have the most impact in finding the most defects possible and creating accountability for quality assurance?

Case Study 3 (Software Metrics, Verification and Validation, Configuration Management):

You are a Software Quality Auditor, hired by a customer to follow up on a software company that has been delivering software with performance problems. How would you plan and execute the audit so that you could obtain sufficient objective evidence to determine:
- Is the company conducting a sufficient amount of inspections, tests, verifications, and validations?
- Is the product captured in a configuration management system?
- Does the company track its performance, and demonstrate a commitment to continual improvement?

Case Study 4 (Project Management, Software Maintenance, and Configuration Management):

You are the Project Manager, responsible for ensuring that your existing products, as well as those installed software-only products at your 140 customer sites, have the correct Configuration Management.
- Identify those steps needed to ensure that your products are tested, approved, and that customers have the “latest and greatest” version.
- Describe the steps taken to estimate the time and resources needed to plan this project.

Case Study 5 (Software Quality Planning, Software Development Life Cycles:

Consider the following Software Development project plan for accounting software:
Step 1: Code new software program for automating accounting tasks.
Step 2: Show program to marketing people.
Step 3: Modify program to make it more marketable.
Step 4: Research for innovative product features.
Step 5: Modify program to incorporate new features.
Step 6: Take program to trade shows and conferences.
Step 7: Sell program to customers.
Step 8: Collect complaints from customers about program quality.
Step 9: Modify program to address bugs and quality issues.
Step 10: Release program fixes to customers as needed, reflective of their particular environments and reported problems.
Step 11: Have focus groups with customers to determine acceptance requirements.
Step 12: Modify software to address requirements gaps.
Describe how you would modify this project plan to obtain better results.

Case Study 6 (Software Audits):

Consider the following Internal Audit project plan for reviewing software practices:
Step 1: Arrive at 4:00 AM to review documentation and files in secret.
Step 2: Obtain initial conclusions about department.
Step 3: Catch employees by surprise so they are unprepared for audit.
Step 4: Conduct interviews with employees.
Step 5: Point out contradictions and inconsistencies to interviewees.
Step 6: Have an audit closing meeting and introduce audit purpose and scope.
Step 7: Threaten participants that a bad report could have bad consequences.
Step 8: Offer to compromise on audit findings for improved cooperation on other issues.
Step 9: Tell client what they want to hear about the auditee.
Describe how you would modify this project plan to obtain better results.

Appendix 3: Final Project Assignment

The Project consists of two parts: Preparation and Presentation

1. Preparation:

You are required to provide detailed responses to each of the Case Studies. Please read the situation, and provide responses for the following:
- Summary of Quality Assurance approach;
- Project Plan;
- Test Plan;
- Summary of up to 5 proposed Test Cases (brief statement of each Test Case).

Please use your own words and respond as if in a real-life situation.

You may collaborate with other students, but you will be marked for your individual submission. Responses are due at the beginning of class.

2. Presentation:

Each student will have a five minute opportunity to elaborate on a Software Quality topic in a particular area of interest. Students have the option of choosing their area of interest, or one will be provided for them.

Examples could be:
- Modifying a Corrective Action system for Software Deficiencies;
- Training and mentoring new testers;
- Effective testing tools for software

As this is an interactive exercise, students should be prepared for questions and comments from other students.

Students will be called upon randomly to give their presentations.

Case Study 1:

You are in the role of a Quality Manager to a company which provides software to Automobile Dealerships. Your company has sold and maintained five generations of client-server software for managing Parts and Service activities. Version 6 has internet and e-commerce capabilities, to allow dealerships to manage inventories and customer relationships. You have four subordinates and one year before product release.

Case Study 2:

You are the Software QA Manager for a company that contracts its services to the Canadian Federal Department of Agriculture. You have to modernize the Farm Subsidy information technology program, to allow farmers to claim subsidies on their crops in relation to the global commodity value for those products. Your program must monitor the value of the crops; the subsidy levels assigned for each farmer, and must process this information so that farmers have their payments rapidly. Since this program will be implemented in every town with a grain elevator, the training and implementation needs are complex, and the service and support requirements are extensive.

Case Study 3:

Your company makes entertainment and strategy games involving Canadian Politicians. This game allows you to simulate Federal elections and form your own Parliaments. Your program can create new politicians with different characteristics (Wealth, Charisma, Intellect, etc.) as well as areas of expertise (Law, Health, Finance, Environment, etc.). The game must be both entertaining and politically accurate. The most recent version can be internet-enabled to allow multiple players to play remotely.

About the Authors

Mr. Daniel J. Zrymiak, ASQ - CQMgr, CSQE
Instructor, Department of Information Technology
Kwantlen University College
Daniel.Zrymiak@kwantlen.ca
http://www.kwantlen.bc.ca/

Dr. Abhijit Sen
Chair, Department of Information Technology and Computer Information Systems
Kwantlen University College
Abhijit.Sen@kwantlen.ca
http://www.kwantlen.bc.ca/
 

Created:  04/04/2002
Last Modified:  16/04/2002