SQA

Classification of Software Qualities

There are many desirable software qualities. Some of these apply both to the product and to the process used to produce the product. The user wants the software products to be reliable, efficient, and easy to use.

The producer of the software wants it to be verifiable, maintainable, portable, and extensible. The manager of the software project wants the process of software development to be productive and easy to control.

In this section, we consider two different classifications of software-related qualities: internal versus external and product versus process.

External versus Internal Qualities

We can divide software qualities into external and internal qualities. The external qualities are visible to the users of the system: the internal qualities are those thatconcern the developers of the system. In general, users of the software only care about the external qualities, but it is the internal qualities, which deal largely withthe structure of the software, that help developers achieve the external qualities.

For example, the internal quality of verifiability is necessary for achieving theexternal quality of reliability. In many cases, however, the qualities are relatedclosely and the distinction between internal and external is not sharp.

Product and Process Qualities

We use a process to produce the software product. We can also attribute some qualities to the process, although process qualities often are closely related to product qualities. For example, if the process requires careful planning of system test data before any design and development of the system starts, products reliability will increase. Some qualities, such as efficiency, apply both to the product and to the process.

It is interesting to examine the word product here. It usually refers to what is delivered to the customer. Even though this is an acceptable definition from the customer’s perspective, it is not adequate for the developer who requires a general definition of a software product that encompasses not only the object code and the user manual that are delivered to the customer but also the requirements, design, source code, test data, etc. In fact, it is possible to deliver different subsets of the same product to different customers.

SOFTWARE QUALITY

We examine the qualities that are pertinent to software products and software production processes. These qualities will become our goals in the practice of software engineering.

The basic goal of software engineering is to produce quality software. Software quality is a broad and important field of software engineering addressed by several standardization bodies, such as ISO, IEEE, ANSI, etc.

Software quality is the:

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.

The above definition emphasizes three important points:

1. Software requirements are the foundation from which quality is measured. Lack of conformance to requirements is lack of quality.

2. Specified standards define a set of development criteria that guide the manner in which software is engineered. If the criteria are not followed, lack of quality will almost surely result.

3. There is a set of implicit requirements that often goes unmentioned. If software conforms to its explicit requirements but fails to meet implicit requirements, software quality is suspect.

SQA Plan

An SQA plan defines the quality processes and procedures that should be used. This involves selecting and instantiating standards for products and processes and defining the required quality attributes of the system.

The SQA plan provides a roadmap for instituting software quality assurance. Developed by the SQA group (or the software team if a SQA group does not exist), the plan serves as a template for SQA activities that are instituted for each software project.

The quality plan should select those organizational standards that are appropriate to a particular product and development process. New standards may have to be defined if the project uses new methods and tools.

An outline structure for a quality plan includes:

1. Product introduction: A description of the product, its intended markets, and the quality expectations for the product.
2. Product plans: The critical release dates and responsibilities for the product along with plans for distribution and product servicing.
3. Process descriptions: The development and service processes that should be used for product development and management.
4. Quality goals: The quality goals and plans for the product including an identification and justification of critical product quality attributes.
5. Risks and risk management: The key risks that might affect product quality and the actions to address these risks.

Preparation of a Software Quality Assurance Plan for each software project is a primary responsibility of the software quality assurance group.

Topics in a Software Quality Assurance Plan include:

Purpose-scope of plan;
List of references to other documents;
Management, including organization, tasks, and responsibilities;
Documentation to be produced;
Standards, practices, and conventions;
Reviews and audits;
Testing;
Problem reporting and corrective action;
Tools, techniques, and methodologies;
Code, media, and supplier control;
Records collection, maintenance, and retention;
Training;
Risk management—the methods of risk management that are to be used.

SQA Objectives/Goals

The various objectives of SQA are as follows:

Quality management approach.
Measurement and reporting mechanisms.
Effective software-engineering technology.
A procedure to assure compliance with software-development standards where applicable.
A multi-testing strategy is drawn.
Formal technical reviews that are applied throughout the software process.

The major goals of SQA are as follows:

SQA activities are planned.
Non-compliance issues that cannot be resolved within the software project are addressed by senior management.
Adherence of software products and activities to the applicable standards, procedures, and requirements is verified objectively.
Affected groups and individuals are informed of SQA activities and results.

The process of the SQA

The aim of the Software Quality Assurance (SQA) process is to develop a highquality software product. Software Quality Assurance is a set of activities designed to evaluate the process by which software is developed and/or maintained.

Quality assurance is a planned and systematic pattern of all actions necessary to provide adequate confidence that the item or product conforms to established technicalrequirements (IEE83).

The purpose of a software quality assurance group is to provide assurance that the procedures, tools, and techniques used during product development and modification are adequate to provide the desired level of confidence in the work products.

The process of the SQA:

1. Defines the requirements for software controlled system fault/failure detection, isolation, and recovery;
2. Reviews the software-development processes and products for softwareerror prevention and/or controlled change to reduced functionality states; and
3. Defines the process for measuring and analyzing defects as well as reliability and maintainability factors.

What steps are required to perform statistical SQA?

STATISTICAL SOFTWARE QUALITY ASSURANCE
Statistical quality assurance reflects a growing trend throughout industry to become more quantitative about quality. For software, statistical quality assurance implies the following steps:

1.Information about software defects is collected and categorized.
2.An attempt is made to trace each defect to its underlying cause (e.g., non-conformance to specifications, design error, violation of standards, poor communication with the customer).
3. Using the Pareto principle (80 percent of the defects can be traced to 20 percent of all possible causes), isolate the 20 percent (the "vital few").
4. Once the vital few causes have been identified, move to correct the problems that have caused the defects.

This relatively simple concept represents an important step towards the creation o: an adaptive software engineering process in which changes are made to improve those elements of the process that introduce error.

To illustrate this, assume that a software engineering organization collects information on defects for a period of one year. Some of the defects are uncovered as software is being developed. Others are encountered after the software has been released to its end-users.

Although hundreds of different errors are uncovered, all can be tracked to one (or more) of the following causes:

•incomplete or erroneous specifications (IBS)
•misinterpretation of customer communication (MCC)
•intentional deviation from specifications (IDS)
•violation of programming standards (VPS)
•error in data representation (EDR)
•inconsistent component interface (ICI)
•error in design logic (EDL)
•incomplete or erroneous testing (IET)
•inaccurate or incomplete documentation (IID)
•error in programming language translation of design (PLT)
•ambiguous or inconsistent human/computer interface (HCI)
•miscellaneous (MIS)

To apply statistical SQA, Table 1.1 is built. The table indicates that IES, MCC, and EDR are the vital few causes that account for 53 percent of all errors. It should be noted, however, that IES, EDR, PLT, and EDL would be selected as the vital few causes if only serious errors are considered. Once the vital few causes are determined, the software engineering organization can begin corrective action.

For example, to correct MCC, the software developer might implement facilitated application specification techniques to improve the quality of customer communication and specifications.

To improve EDR, the developer might acquire CASE tools for data modeling and perform more stringent data design reviews. It is important to note that corrective action focuses primarily on the vital few. As the vital few causes are corrected, new candidates pop to the top of the stack.

Statistical quality assurance techniques for software have been shown to provide substantial quality improvement [ART97]. In some cases, software organizations have achieved a 50 percent reduction per year in defects after applying these techniques.

In conjunction with the collection of defect information, software developers can calculate an error index (El) for each major step in the software process [IEE94]. After analysis, design, coding, testing, and release, the following data are gathered:

E i = The total number of errors uncovered during the ith step in the software engineering process

Table 1.1: DATA COLLECTION FOR STATISTICAL SQA

S_i = the number of serious errors

M_i = the number of moderate errors

T_i = the number of minor errors

PS = size of the product (LOG, design statements, pages of documentation) at the ith step

W_s, W_m, W_t = weighting factors for serious, moderate, and trivial errors, where recommended values are W_s = 10, W_m = 3, W_t = 1. The weighting factors for each phase should become larger as development progresses. This rewards an organization that finds errors early.

At each step in the software process, a phase index, PI_i, is computed:

Pl_i = W_s (S_i/E_i) + W_m (M_i/E_i) + W_t (T_i/E_i)

The error index is computed by calculating the cumulative effect on each PI_i, weighting errors encountered later in the software engineering process more heavily than those encountered earlier:

EI= Σ(I x PI_i)/PS = (PI_i + 2PI₂ + 3PI₃ + .. . iPI_i)/PS

The error index can be used in conjunction with information collected.in Table 1.1.To develop an overall indication of improvement in software quality.

The application of the statistical SQA and the Pareto principle can be summarized in a single sentence: Spend your time focusing on things that really matter, but first be sure that you understand what really matters!

A comprehensive discussion of statistical SQA is beyond the scope of this book. Interested readers should see [SCH98], [KAP95], or [KAN95].

What is the role of an SQA group?

Prepares an SQA plan for a project

The plan is developed during project planning and is reviewed by all interested parties. Quality assurance activities performed by the software engineering team and the SQA group are governed by the plan. The plan identifies

•Evaluations to be performed
•Audits and reviews to be performed
•Standards that are applicable to the project
•Procedures for error reporting and tracking
•Documents to be produced by the SQA group
•Amount of feedback provided to the software project team

Participates in the development of the project's software process description.

The software team selects a process for the work to be performed. The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.

Reviews software engineering activities to verify compliance with the defined software process.

The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have been made.

Audits designated software work products to verify compliance with those defined as part of the software process.

The SQA group reviews selected work products; identifies, documents, and tracks deviations; verifies that corrections have been made; and periodically reports the results of its work to the project manager.

Ensures that deviations in software work and work products are documented and handled according to a documented procedure.

Deviations may be encountered in the project plan, process description, applicable standards, or technical work, products.

Records any noncompliance and reports to senior management.

Noncompliance items are tracked until they are resolved. In addition to these activities, the SQA group coordinates the control and management of change and helps to collect and analyze software metrics.

Shell Procedure Overview

SQA Robot lets you group several test procedures into a shell procedure that plays back the test procedures in sequence. For example..you may have one test procedure that starts your application a second that searches for and opens a particular file,a third that modifies the file and a fourth that closes the application and returns to the starting point.

Combined into a single shell procedure.Test Procedures can run in unwanted mode and perform comprehensive test coverage. Moreover,the results from all test procedures and test cases are stored in the same test log. Which simplifies results analysis in the SQA Test Log Viewer. For unattended testing,each test procedure should return to a common point in the application under test.This common point could be a main menu.A specific window or dialog box, or even Program Manager.This assures that test procedure playback remains synchronized with the application test.

Using Record and Insert Toolbars

By default, the SQA Robot insert toolbar is dynamic. It is displayed when you click the Insert button on the record toolbar,and hidden after the test case is created. To market test case buttons visible at all times while recording, you may add test case buttons to the record toolbar or manually display the insert toolbar.

To manually display the Insert toolbar so it is visible while recording, do one of the following:

1. Check SQA Robot insert in the toolbar popup menu.

2. Choose Window toolbar and check SQA Robot Insert in the toolbars dialog box.

To add test case buttons to record toolbar:

1. Display both record and insert toolbar.
2. Use the right mouse button to drag a test case button from the insert toolbar and drop it within the record toolbar.Alternatively,you may:
3.Display the Record toolbar.
4.Choose Add Button in the toolbar popup menu.
5. Use either mouse button to drag a test case button from the Add Toolbar Button dialog box and drop it whithin the Record toolbar..

Note:If you make any unwanted changes to the record or insert toolbar, you can choose Reset Toolbar in the toolbar popup menu to reset the toolbar to its default configuration. Recording actions in a Test Procedure while recording, you can insert any of the following into a test procedure.

1. Test Cases - Capture specific conditions,states,or values during a test procedure recording and store the data as the expected baseline for the application under test.During playback,test cases recapture the data from the current build of the application under test,and compare the captured data to the baseline data.

2. Wait States - Provide a way to synchronize test procedure playback with application under test. During playback, your application may require additional time to complete certain tasks. Setting wait states lets you define specific conditions for which the test procedure waits before continuing with subsequent test sections.

3. Timers - Let you insert start and stop timer commands to record the duration of events in a test procedure,such as database or network access. After playback, the elapsed time is displayed in the SQA Test Log Viewer.

4. Synchronization Points - Let you coordinate virtual users at test runtime. To use this feature, you must have Virtual User Recording installed.

The remaining Insert menu commands let you call existing test procedures from the script being recorded, insert comments into the test procedure script or test log,or invoke an application while recording without having to go through the desktop.

Pausing and Resuming a Recording Session

While recording,if you choose a valid SQA Robot menu command or toolbar button, recording is suspended. After SQA Robot completes your selected action,recording resumes and you can continue working with the application under test. To pause recording manually, choose RecordPause. SQA Robot indicates a paused state by:
1. Depressing the Pause toolbar button.
2. Displaying "Recording Suspended" in the status bar.
3. Displaying a check next to the RecordPause menu command.

To resume recording,choose RecordPause again. You should always resume recording with the application under test in the same state it was in when you paused.

Note: You cannot use the Pause feature while recording a virtual user procedure.

Hiding and Displaying SQA Robot

By default, the SQA Robot main window minimizes while recording to allow unobstructed access to the application-under-test. To display the SQA Robot main window at any time while recording, click the Open Robot Window button on the Record toolbar.

You can also use hot keys to display or hide SQA Robot while recording:

1. To bring the SQA Robot window to the foreground, press CTRL+SHIFT+F. This works whether SQA Robot is in the background or completely hidden.

2. To hide the SQA Robot main window completely, Press CTRL+SHIFT+H. The SQA Robot window disappears altogether but continue executing.

Hot keys work only while recording. If you are not recording, use the taskbar or ALT+TAB to bring SQA Robot to the foreground. You can also use the Recording Options dialog box to change the While recording options,which let you view part of the SQA Robot window while recording or send the window to the background.

Test Procedure Overview

A test procedure is an SQABasic or Visual Basic script that you record with SQA Robot.When you record a test Procedure,SQA Robot automatically captures user activities and stores them as a series of SQA Basic commands.Test Procedure script files have the same name as the test Procedure ID, but with a .REC extension.Test Procedure files are stored in the \Repository\PROCS sub directory.

If SQA Basic is your scripting language,the test procedure script is displayed in a child Window called the test procedure wi ndow.SQA Robot does not display the test procedure window if Visual Basic is your scripting language.

When you play back the test procedure,SQA Robot executes the activities You recorded.There are two types of test procedures:

•GUI procedure s:This contain the user`srecorded GUI actions,such as Keystrokes and mouse clicks.In regression tests.GUI procedures also contain one or more GUI test cases.

•Virtual user procedures:These contain the user`s recoded HTTP requests to aWeb server.

GUI Procedure Script Struture:

A typical GUI procedure has four major sections:
• Intialization commands
• Window restoration commands (optinal)
• User action and test case commands
• Closing commands

Intialization Commands:

All SQA Robot GUI procedure scripts must begin with the following commands:

SUB mAIN:

This defines the test procedure as a subroutine named Main.this is normally the first command in the script and should not be edited.

Dim result As Integer:

This defines the variable Result as an integer variable.SQA Robot returns the value from each test case function in the variable Result. The value for Result is local to the Main subroutine.
`Intially Recorded: 03/12/96 11:12:02
`Test procedure Name:Mortgage Prequalifier History:

SQA Robot writes two comment lines (which begin with a single quotation mark) in the initialization section of each script. The first tells when the the test procedure was recorded,and the second is the text entered as the name(not the ID)of the procedure.

InitPlay:

(visual Basic only) This is a subroutine defined by SQA Robot that initializes test procedure play back. It must always be the first command in a Visual Basic script.

Set ProcID “QBPRQ01”:

(Visual Basic only) This is a subroutine defined by SQA Robot that identifies the ID of the test Procedure. This always follows Init Play and always Precedes any user action commands in a Visual Basic script.

Window Restoring Commands:

The following commands record the sizes and positions of open windows so that SQA Robot can restore the testing environment during playback. They also indicate the windows within whin user actions occur.

StartSaveWindowPositions
EndSaveWindowPositions

These statements tell SQA Robot that, during plackback,the intervening Window SetContext, Window MoveTo, and Window SetPosition commands are for windows restoration only.All Playback timing defaults are set to zero to process these commands as quickly as possible.if any command fails b/t StartSaveWindowPositions, that failure is reported to the log as a Warning, not a script command failure.

Window SetContext
WindowSetPosition

The Window SetContext statement defines the context window for subsequent test cases and user actions,and Window SetPosition indicates the position,size, and status of that context window.These statements can vary,depending on the window within which user actions occur.During recording,SQA Robot saves the positions of Windows that are normal,minimized, or maximized if Auto Record Window Size is checked in the Recording Options dialog box. It does not save the positions of hidden windows. During playback, SQA Robot restores the Windows to their positions When the test procedure nwas recorded. SQA Robot writes messages to the test log for any windows it cannot find on playback.

User Actions and Test Case Commands :

Useraction commands represent the actions you take While recording aGUI procedure. They are not test cases, but represent your actions between test cases. User actions Within a Window are always preceded in the script by a Window Set Context statement establishing the context window within which the actions occur.

Push Buttons Click, “VBName=SelectOK;Visual Text=OK”
User actions statements always begin with an object type (PushButton,Window, EditBox,etc.) followed by the action applied to it (Click,Resize, VScrollTo, etc.).
Result=WindowTC(Compare Menu, “VBName=Main;
VisualText=Mortage Prequalifier”,
“CaseID=QBPRQ01a”)

Test case and wait state functions are preceded by the Result variable.A test case functions corresponds to a test case you insert in to the text procedure While recording. During Playback,if the test case passes, Result equals 1. If the test case fails, Result equals 0.

Closing Commands:

All GUIprocedure scripts must end with the following commands, while terminate the test procedure and reset

SQA Robot:
EndPlay :

(Visual Basic only) This is a subroutine defined by SQA Robot that terminates playback of aVisual Basic test procedure script.

EndSub:
This indicates the end of the Main subroutine.

InitPlay, SetProcID, and EndPlay are required by SQA Robot to initialize and terminate playback of Visual Basic scripts.These commands are not required for SQA Basic scripts

SQA Robot includes the StartSaveWindowPositions and EndSave Window Positions commands only if Save Window Positions is checked in the Recording Options dialog box.

Command Types:

SQA Robot test procedure commands are organized into these these types,based on the tasks they perform in the script:

CONTROL FLOW COMMANDS: Define the Beginning and end ofSQA Robot scripts, and the beginning and end of commands that save and restore windows. Control flow tasks include initialization and termination for test procedure and Playback. Used only in GUI Procedures.

ObJECT SCRIPTING COMMANDS:Acess an application`s objects and object Properties from within a test Procedure script . Object Scripting tasks include retrieving and setting an object`s properties. Used only in GUI Procedures.

TEST CASE COMMANDS:Compare objects captured While playing are same,the test case passes (Result equals 1). If the objects are different, the test case fails back successive builds of the application-under-test against the established base line data. If the objects( Result equals0).Used only in GUI Procedure.

USER ACTION COMMANDS: correspond to user actions on specific objects While recording.Actions include operations like choosing a menu command. Scrolling a list box, clicking a button, or typing text in to an edit box. Used only in GUI Procedures.

UTILITY COMMANDS:Affect the flow of test Procedure playback by setting wait times,calling other test procedures, starting applications, starting applications , starting and stopping timers,and playing back low-level recordings. They also control output to the test log, retrieve results from running test procedures, and set characters used in statements.Used only in GUI Procedures.

If you have Virtual User Recording installed. You can use these SQA Basic commands:

Data Pool commands-Retrieve data from a datapool created with SQA Datapool Manager.
HTTP commands- Issue HTTP or HTTPS requests to aWeb server that is not SSL-protected.
HTTP/HTTPS commands-Issue HTTP or HTTPS requests to aWeb server,depending on whether the server is SSL-protected.

Virtual User commands-Perform operations in virtual user procedures. For example,virtual user commands perform synchronization and timer operations for virtual users, and handle communications between the Master and Agent test stations.Except for control flow commands,which must appear at the beginning and end of every script,the order of recorded commands depends on your recording options and user actions.

NAMING CONVENTIONS OVERVIEW:

SQA Robert requires you to assign an ID for each test procedure and test case (or wait state). These IDs have an eight character limit,imposed by the DOS filename restriction. Because test procedure and test case IDs are referred to in the SQA repository and in test procedure scripts, it is important that you establish and adhere to a naming convention. Having a Meaningful naming convention helps you identify the test item`s purpose.

Because SQA Robot complies Visual Basic test Procedures in to executables with .EXE extensions, aVisual Basic test procedure ID should not be the same as that of another .EXE program.
However,because SQA Robot compiles SQABasic test procedures into .SBX files,this restriction does not apply if SQABasic is your scripting language.

Test Procedure IDs:

A test procedure ID should indicate which application or component the test procedure tests. To achieve this, you canj assign meanings to characters of the filename.

In this diagram two characters identify the testing phase of the application-under-test (for example,function testing,transaction testing,or performance testing), five characters identify the functional area that the procedure tests (the module within the application-under-test),and a different tests on the same application module.

This is only one way in which you can define meaningful test procedure IDS. You can define IDs in whatever way best suits your organization and the application-under-test. The pieces that your naming convention identifies may correspond directly to the test requirements hierarcthy defined using SQA Manager.
Test Case IDs: The ID you define for a test case should help you identify the application-under-test,the test procedure it is in,and the type of test case.

In this diagram,two characters identify the testing phase of the application-under-test,three characters identify the functional area or module that the procedure tests, two characters identify the test case type, and a single character indicates the sequence. You can use either numbers, letters,or both for the sequence character,Which allows up to 36 test cases (0 through 9 plus A through z) in a test procedure. How ever, SQA recommends that you not create such alarge test procedure. Instead of including many test cases, define test procedures that are short and modular.

Autonaming:
SQA Robot can assign IDs for test procedures and test cases with its autonaming feature, which you can set using the preferences dialog box. If Autonaming On is checked,SQA Robot automatically inserts default IDs for test procedures, test cases, or both. You can accept or change these default IDs.

The autonamed test case or wait state IDs consist of the test procedure ID plus a single consecutive alphabetic character. For example, if the test procedure ID is TESTI, the autonamed ID for the first test case (or wait state) inserted in to the procedure is TESTIA, the ID for the second test case is TESTIB,and so on. If you change the default test procedure ID, you should limit it to seven characters to allow the auto naming of test case IDs. Because of the eight character filename limit,SQA Robot cannot automatically generate test case or wait state IDs if a test procedure ID exceeds seven characters.

Managing Test Procedures Overview:

Every time you record a test procedure,SQA Robot automatically adds a test procedure ID and definition to the SQA Repository. SQA Robot also lets you define,modify,and delete test procedure definations in your test planning phase before any actual recording takes place. You can also define test procedures using SQA Manager, Which stores the test procedure IDs,names,and definations in the SQA Repository and makes them available for recording with SQA Robot.

Defining New Test Procedures:
To create a new test procedure ID and definition:
•Choose Admin Test Procedure.
•Choose New.

•Enter the information to define your new test procedure.
This information is displayed in the SQA Test Log Viewer when you examine playback results, and also in SQA Manager when you perform defect tracking and reporting.

•Choose Ok.
•Enter an ID of up to eight alphanumeric characters and choose OK.This becomes the test procedure ID in the selector list.
Test procedures you define in SQA Robot are also available to SQA Manager.
Modifying Test Procedures:
•Choose Admin Test Procedure.
SQA Robot lists all defined test procedures in the dialog box,regardless of whether or not they have been recorded.
•Select a test procedure from the list.
•Choose Open or double-click the test procedure in the list.
•Change the information to redefine your test procedure.
This information is displayed in the SQA Test Log Viewer when you examine playback results,and also in SQA Manager when you perform defect tracking and reporting.
•Choose OK.
Deleting Test Procedures:
•Choose Admin Test Procedure.

SQA Robot lists all defined test procedures in the dialog box, regardless of whether or not they have been recorded.

•Select one or more test procedures from the list, or choose Select All.

•Choose Delete.SQA Robot prompts you to confirm the deletion.

•Choose Yes or Yes to All to delete the the test procedures, choose No to retain the indicated test procedure,or choose Cancel to terminate the entire deletion process.

Test Procedure Recording Overview:

The type of user action you record varies, depending on whether you are recording a GUI procedure or a virtual user procedure.

Recording GUI procedures:

When you record a GUI procedure, you typically record a sequence of user actions required to execute one or more test cases, or verification points, in an application-under-test. When you record a GUI procedure, SQA Robot creates a script that keeps track..[Continue next update]

SQA Planning

Planning is one of the most important aspects of Software Quality Assurance. The entire operation of the SQA team depends on how well their planning is done. In smaller businesses, planning might not really dictate the flow of SQA but in larger businesses, SQA Planning takes on center stage. Without it, each component or department that works on the application will be affected and will never function.

In gist, SQA Planning tackles almost every aspect of SQA’s operation. Through planning, each member and even non-member of the SQA team is clearly defined. The reason for this is very simple: when everyone knows their role and boundaries, there is no overlapping of responsibilities and everyone could concentrate on their roles.

But SQA Planning is not only a document that tells who gets to do the specific task. The stages in are also detailed. The whole SQA team will be very busy once the actual testing starts but with SQA, everyone’s work is clearly laid out. Throughplanning, the actual state of the application testing is known.

Again in smaller businesses, the planning maybe limited to the phase of the application testing but when outlined for corporations, the scenario changes and only throughplanning that everyone will know where they are and where they are going in terms of SQA.

SQA Planning is not just a simple document where objectives are written and stages are clearly stated. Because of the need to standardize software development ensuring the limitation of error, a scientific approach is recommended in developing an SQA plan. Certain standards such as IEEE Std 730 or 983.

SQA Plan Content

An SQA Plan is detailed description of the project and its approach for testing. Going with the standards, an SQA Plan is divided into four sections:

• Software Quality Assurance Plan for Software Requirements;
• Software Quality Assurance Plan for Architectural Design;
• Software Quality Assurance Plan for Detailed Design and Production and;
• Software Quality Assurance Plan for Transfer

In the first phase, the SQA team should write in detail the activities related for software requirements. In this stage, the team will be creating steps and stages on how they will analyze the software requirements. They could refer to additional documents to ensure the plan works out.

The second stage of SQA Plan or the SQAP for AD (Architectural Design) the team should analyze in detail the preparation of the development team for detailed build-up. This stage is a rough representation of the program but it still has to go through rigorous scrutiny before it reaches the next stage.

The third phase which tackles the quality assurance plan for detailed design and actual product is probably the longest among phases. The SQA team should write in detail the tools and approach they will be using to ensure that the produced application is written according to plan. The team should also startplanning on the transfer phase as well.

SQA Software and Tools

In quality assurance, it is always important to get all the help we could get. In other industries, developers could easily check the products manually and discard those that do not meet the standard. The length and the width of the product are checked to maintain standardization of the product. Others use special machines to check the product. With tools and machines, they can easily set a standard with their products.

That also goes the same with software and applications. Although it does not use physical machines, applications go through rigorous testing before they are released to the public even for beta testing.

The tools used in SQA are generally testing tools wherein an application is run through a series of tests to gauge the performance of the application.

The tools used in SQA vary in purpose and performance. These applications range from testing the code or running the application under great stress. These tools are employed to test the application and produce numbers and statistics regarding the actual application. Through these numbers, the SQA team and their developers will know if the application has lived up according to the targeted performance.

Like most developers each SQA team has their preferred tools for application testing. Based on their belief and expertise, the SQA team will usually give the owners or business managers a free hand on what type of testing tool to use.

Notable SQA Tools

The following are some of the renowned SQA tools and applications. There are still hundreds out there but the following tools have been around for years and have been used by thousands or probably millions of testers.

WinRunner – Developed by HP, WinRunner is a user friendly application that can test the applications reaction from the user. But other than measuring the response time, WinRunner can also replay and verify every transaction and interaction the application had with the user. The application works like a simple user and captures and records every response the application does.

LoadRunner – Developed by HP, LoadRunner is one of the simple applications that can test the actual performance of the application. If you are looking for a program to test your application’s tolerance to stress, LoadRunner is your tool. It has the ability to work like thousands of users at the same time – testing the stress of the application.

QuickTest Professional – If you have worked with WinRunner you surely have bumped in with this tool. Built by HP, QuickTest emulates the actions of users and exploits the application depending on the procedure set by testers. It can be used in GUI and non-GUI websites and applications. Thetesting tool could be customized through different plug-ins.

Mercury TestDirector – An all-in-one package, this web-based interface could be used from start to end in testing an application or a website. Every defect will be managed according to their effect to the application. Users will also have the option to use this exclusively for their application or use it together with wide array of testers.

Silktest – Although available in limited operating system, Silktest is a very smart testing tool. Silktest lists all the possible functions and tries to identify the function one by one. It can be implemented in smaller iterations as it translate the available codes into actual objects.

Bugzilla – Developed by Mozilla, this open source testing tool works as the name suggests. Bugzilla specializes in detecting bugs found in the application or website. Since the application is open-source it can be used freely and it is availability in different OS makes it even a viable alternative for error tracking. The only downside is it has a long list of requirements before it could run.

Application Center Test – Also known as ACT, this testing tool was developed by Microsoft using ASP.NET. This application is primarily used for determining the capacity of the servers that handle the application. Testers can test the server by asking constant requests. A customized script either from VB or JS could be used to test the server’s capacity.

OpenSTA – Another open source tool, testers can easily launch the application and use it for testing the application’s stress capacity. The testing process could be recorded and testing times could be scheduled. Great for websites that needs daily maintenance.

SQA Approaches and Methodologies

A scientific approach should have methods. As a scientific process, a stage or a step should be established or used to ensure the final product is according to the user’s specifications. The method is usually determined through the wishes of the clients, the available manpower and circumstances.

It is not that the clients specify the actual method for a scientific approach but the client’s provider takes into consideration the need of the clients. Using the facts and data provided by the client, the method for developing a product will be established. Using the experience and available data a method is determined and executed.

In SQA, the client’s need for a better application is established. But a good application is not the only need of the client. There are metrics that an application should meet and anything below par is not good for business. The SQA team should ensure the metrics are reached by constantly monitoring the development process while giving feedbacks to the developers. The SQA team is there to ensure that the process is done correctly to reach the needed metrics.

To ensure that a proper SQA is done, the SQA team should select a proper methodology. Selecting the proper methodology is quite a challenge. However if the proper facts is laid out, the SQA team should be able to select a good SQA methodology. On the other hand, the factors are also determined before hand so that the methodology will be known. Since SQA is an evaluating process, it reacts to the available information.

Why Not to Use a Methodology

On the other hand, there are SQA and software engineers who have doubts on the importance and use of an SQA methodology. Their reason comes from the fact that the SQA methodologies and approaches are very specific. Since there are very specific and strict, it does not give any room for additional information.

There are also developers and SQA managers who develop their own type of software quality assurance methodology based on their present situation. Again, the reason for that is that the methodologies are to strict that any creativity ofsoftware development is not allowed.

A waste of time is also a major source of disregard from software developers. Instead of determining the methodology, the developers focus their time on other things. The proof of the usability of the methodology of SQA is almost non-existent. There are a few who have tried to prove that having an SQA methodology makes worth more efficient but they are usually associated with the general information and a small part of the text is dedicated to the methodology.

Last but not the least, the methodology for SQA is just a waste of time. This is true especially when you’re trying out a new methodology forsoftware development. It’s always a gamble to try out something new even though they have been tested in simulated environments. It all goes back to the fact that the SQA methodologies are very specific and the possibility of going out of what is written is not a good thing.

Why SQA

There are so many reasons why a company should consider SQA. It is all about business survival and SQA is just one of the many tools the company should effectively use. And just like a tool, it has to be effectively used to its maximum. If the tool is not used to its full extent, the tool will just be a financial burden to the company. SQA should also be utilized at the same way – to its full extent.

One of the reasons why businesses and companies opt out in using SQA is their inability to realize who effective an SQA when properly used. The following are thereasons why SQA should be used by any company before releasing their application to their intended users:

Traceability of Errors

Without using SQA, developers can still locate the errors of the application. Since their familiarity with the application is impeccable, they will have the ability to find a remedy to the problem. SQA on the other hand does not just look for answers to their problems.

SQA looks for the reason why the error occurred. Looking for errors is really easy but tracing the roots of the problem is another thing. The SQA team should be able to tell which practice has started the problem. Most of the time, the errors is not rooted on the execution of the coding but it could be found in the philosophy in developing theapplication.

Cost Efficient

This idea is saving money is very debatable in SQA. Using SQA means you will have to hire another set of people to work with you. Most of the SQA today are 3rd party companies that are only there to work on the specific project. Manpower will always cost money. Manpower does not even cover the cost of the tools the SQA team might use. Unfortunately most of the SQA tools are expensive especially the most efficient tools.

On the other hand, consider the idea of locating the bugs and errors in the application before they are released. Every application that are used in a business setting or sold to the consumers should have an ideal of 0% error. Even though this is rarely attained, SQA should be able to reduce errors to less than 1%.

Considering the effect on productivity together with customer satisfaction, you can always tell that preventing the problem is always better compared to answering them. SQA could practically save your business and boost it to success. Without them, there is a higher possibility that the application will have errors which will discourage your customers from purchasing the application.

In a business setting, an erroneous application will lead to slower productivity since instead of helping their customers; they might end up troubleshooting their application most of the time.

Flexible Solutions

SQA could easily provide solutions to the problem since they look for the root of the problem instead of just answering them. By providing the root of the problem, they have the ability to provide solutions to these problems fast. But instead of one solution only, the SQA team should be able to provide more than one solution.

Testing and Quality Assurance

What is the relationship between testing and Software Quality Assurance (SQA)?

An application that meets its requirements totally can be said to exhibit quality. Quality is not based on a subjective assessment but rather on a clearly demonstrable, and measurable, basis. Quality Assurance and Quality Control are not the same.

Quality Control is a process directed at validating that a specific deliverable meets standards, is error free, and is the best deliverable that can be produced. It is a responsibility internal to the team. QA, on the other hand, is a review with a goal of improving the process as well as the deliverable.

QA is often an external process. QA is an effective approach to producing a high quality product. One aspect is the process of objectively reviewing project deliverables and the processes that produce them (including testing), to identify defects, and then making recommendations for improvement based on the reviews.

The end result is the assurance that the system and application is of high quality, and that the process is working. The achievement of quality goals is well within reach when organizational strategies are used in the testing process. From the client's perspective, an application's quality is high if it meets their expectations

SQA interview questions

3JQSBFACBPZN Readers are welcome to post the answers to the following SQA interview questions. Leave your name for your answer to be published with credit given to you.

1. What is software quality assurance?
2. What is the value of a testing group? How do you justify your work and budget?

Software Testing and Quality Assurance

Software testing can be defined as the process in which the tester aims at checking the software for errors and also, verifying that the functional requirements of a software are met. The system is examined under controlled conditions. The basic orientation is detection of faults within the system.

In this process, a program or an application is executed with the intentions of improving its quality. One strives to find the bugs within the software. As software industry has now grown in both size and complexity, so has its wide area of applications. We see soft wares in banking industry, in gaming industry, in management applications etc. Each has its own target customers and users. Before any software is launched, one must attempt to make an assessment as to its audience, customers, etc. The process of testing aims to aid in this regard.

Verification and Validation

Verification is a thwarting mechanism which helps in detection of all the possible weak points within a software before the process of testing actually begins.

Validation takes place after verification and this is where the concrete testing begins. One aims at finding the defects present in the functional requirements or the software's specifications.

Software Quality

The quality of any software can be determined by the following yardsticks:

- It must be free of bugs
- It must be delivered on the due date
- It must be complete within the budget initially allotted
- It must meet all the requirements and expectations of the client

Software quality assurance

Software Quality Assurance or SQA, as it is popularly known, is a method by which one aims at monitoring the whole process of software engineering and take measures at appropriate durations for ensuring that the quality of the software is not compromised with. This is approached by use of audits of quality management system by which the system was initially built. A few examples of the quality standards are CMMI (Capability Maturity Model Integration) or ISO 9000.

SQA covers the whole process of development of any software and includes the design, coding, control of the code, reviews of the code, managing the configurations and the release of the final product. It is basically a method to control the processes of software development.

Advantages of SQA
- Improved customer satisfaction
- Reduced cost of development
- Reduced cost of maintenance
- Methodology of SQA
- Validation testing
- Data comparison
- Usability testing