The process of the SQA

The aim of the Software Quality Assurance (SQA) process is to develop a highquality software product. Software Quality Assurance is a set of activities designed to evaluate the process by which software is developed and/or maintained.

Quality assurance is a planned and systematic pattern of all actions necessary to provide adequate confidence that the item or product conforms to established technicalrequirements (IEE83).

The purpose of a software quality assurance group is to provide assurance that the procedures, tools, and techniques used during product development and modification are adequate to provide the desired level of confidence in the work products.

The process of the SQA:

1. Defines the requirements for software controlled system fault/failure detection, isolation, and recovery;
2. Reviews the software-development processes and products for softwareerror prevention and/or controlled change to reduced functionality states; and
3. Defines the process for measuring and analyzing defects as well as reliability and maintainability factors.

The process of the SQA

What steps are required to perform statistical SQA?

STATISTICAL SOFTWARE QUALITY ASSURANCE 
Statistical quality assurance reflects a growing trend throughout industry to become more quantitative about quality. For software, statistical quality assurance implies the following steps:

1.Information about software defects is collected and categorized. 
2.An attempt is made to trace each defect to its underlying cause (e.g., non-conformance to specifications, design error, violation of standards, poor communication with the customer). 
3. Using the Pareto principle (80 percent of the defects can be traced to 20 percent of all possible causes), isolate the 20 percent (the "vital few"). 
4. Once the vital few causes have been identified, move to correct the problems that have caused the defects. 


This relatively simple concept represents an important step towards the creation o: an adaptive software engineering process in which changes are made to improve those elements of the process that introduce error. 


To illustrate this, assume that a software engineering organization collects information on defects for a period of one year. Some of the defects are uncovered as software is being developed. Others are encountered after the software has been released to its end-users. 


Although hundreds of different errors are uncovered, all can be tracked to one (or more) of the following causes: 


•incomplete or erroneous specifications (IBS) 
•misinterpretation of customer communication (MCC) 
•intentional deviation from specifications (IDS) 
•violation of programming standards (VPS) 
•error in data representation (EDR) 
•inconsistent component interface (ICI) 
•error in design logic (EDL) 
•incomplete or erroneous testing (IET) 
•inaccurate or incomplete documentation (IID) 
•error in programming language translation of design (PLT) 
•ambiguous or inconsistent human/computer interface (HCI) 
•miscellaneous (MIS) 


To apply statistical SQA, Table 1.1 is built. The table indicates that IES, MCC, and EDR are the vital few causes that account for 53 percent of all errors. It should be noted, however, that IES, EDR, PLT, and EDL would be selected as the vital few causes if only serious errors are considered. Once the vital few causes are determined, the software engineering organization can begin corrective action. 


For example, to correct MCC, the software developer might implement facilitated application specification techniques to improve the quality of customer communication and specifications. 


To improve EDR, the developer might acquire CASE tools for data modeling and perform more stringent data design reviews. It is important to note that corrective action focuses primarily on the vital few. As the vital few causes are corrected, new candidates pop to the top of the stack. 


Statistical quality assurance techniques for software have been shown to provide substantial quality improvement [ART97]. In some cases, software organizations have achieved a 50 percent reduction per year in defects after applying these techniques.


In conjunction with the collection of defect information, software developers can calculate an error index (El) for each major step in the software process [IEE94]. After analysis, design, coding, testing, and release, the following data are gathered: 


 E i = The total number of errors uncovered during the ith step in the software engineering process 


Table 1.1: DATA COLLECTION FOR STATISTICAL SQA

S_i    =    the number of serious errors

M_i   =    the number of moderate errors

T_i    =    the number of minor errors

PS   =    size of the product (LOG, design statements, pages of documentation) at the ith step

W_s, W_m, W_t = weighting factors for serious, moderate, and trivial errors, where recommended values are W_s = 10, W_m = 3, W_t = 1. The weighting factors for each phase should become larger as development progresses. This rewards an organization that finds errors early.

At each step in the software process, a phase index, PI_i, is computed:

Pl_i = W_s (S_i/E_i) + W_m (M_i/E_i) + W_t (T_i/E_i)

The error index is computed by calculating the cumulative effect on each PI_i, weighting errors encountered later in the software engineering process more heavily than those encountered earlier:

EI= Σ(I x PI_i)/PS =  (PI_i + 2PI₂ + 3PI₃ + .. . iPI_i)/PS

The error index can be used in conjunction with information collected.in Table 1.1.To develop an overall indication of improvement in software quality.

The application of the statistical SQA and the Pareto principle can be summarized in a single sentence: Spend your time focusing on things that really matter, but first be sure that you understand what really matters!

A comprehensive discussion of statistical SQA is beyond the scope of this book. Interested readers should see [SCH98], [KAP95], or [KAN95].

What steps are required to perform statistical SQA?

What is the role of an SQA group?

Prepares an SQA plan for a project

The plan is developed during project planning and is reviewed by all interested parties. Quality assurance activities performed by the software engineering team and the SQA group are governed by the plan. The plan identifies

•Evaluations to be performed
•Audits and reviews to be performed
•Standards that are applicable to the project
•Procedures for error reporting and tracking
•Documents to be produced by the SQA group
•Amount of feedback provided to the software project team

 Participates in the development of the project's software process description.

The software team selects a process for the work to be performed. The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.

Reviews software engineering activities to verify compliance with the defined software process.

The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have been made.


Audits designated software work products to verify compliance with those defined as part of the software process.

The SQA group reviews selected work products; identifies, documents, and tracks deviations; verifies that corrections have been made; and periodically reports the results of its work to the project manager.


Ensures that deviations in software work and work products are documented and handled according to a documented procedure.

Deviations may be encountered in the project plan, process description, applicable standards, or technical work, products.


Records any noncompliance and reports to senior management.

Noncompliance items are tracked until they are resolved. In addition to these activities, the SQA group coordinates the control and management of change and helps to collect and analyze software metrics.

What is the role of an SQA group?

Shell Procedure Overview

SQA Robot lets you group several test procedures into a shell procedure that plays back the test procedures in sequence. For example..you may have one test procedure that starts your application a second that searches for and opens a particular file,a third that modifies the file and a fourth that closes the application and returns to the starting point.

Combined into a single shell procedure.Test Procedures can run in unwanted mode and perform comprehensive test coverage. Moreover,the results from all test procedures and test cases are stored in the same test log. Which simplifies results analysis in the SQA Test Log Viewer. For unattended testing,each test procedure should return to a common point in the application under test.This common point could be a main menu.A specific window or dialog box, or even Program Manager.This assures that test procedure playback remains synchronized with the application test.

Shell Procedure Overview

Test Procedure Overview

A test procedure is an SQABasic or Visual Basic script that you record with SQA Robot.When you record a test Procedure,SQA Robot automatically captures user activities and stores them as a series of SQA Basic commands.Test Procedure script files have the same name as the test Procedure ID, but with a .REC extension.Test Procedure files are stored in the \Repository\PROCS sub directory.

If SQA Basic is your scripting language,the test procedure script is displayed in a child Window called the test procedure wi ndow.SQA Robot does not display the test procedure window if Visual Basic is your scripting language.

When you play back the test procedure,SQA Robot executes the activities You recorded.There are two types of test procedures:

•GUI procedure s:This contain the user`srecorded GUI actions,such as Keystrokes and mouse clicks.In regression tests.GUI procedures also contain one or more GUI test cases.

•Virtual user procedures:These contain the user`s recoded HTTP requests to aWeb server.

GUI Procedure Script Struture:

A typical GUI procedure has four major sections:
• Intialization commands
• Window restoration commands (optinal)
• User action and test case commands
• Closing commands

Intialization Commands:

All SQA Robot GUI procedure scripts must begin with the following commands:

SUB mAIN:

This defines the test procedure as a subroutine named Main.this is normally the first command in the script and should not be edited.

Dim result As Integer:

This defines the variable Result as an integer variable.SQA Robot returns the value from each test case function in the variable Result. The value for Result is local to the Main subroutine.
`Intially Recorded: 03/12/96 11:12:02
`Test procedure Name:Mortgage Prequalifier History:

SQA Robot writes two comment lines (which begin with a single quotation mark) in the initialization section of each script. The first tells when the the test procedure was recorded,and the second is the text entered as the name(not the ID)of the procedure.

InitPlay:

(visual Basic only) This is a subroutine defined by SQA Robot that initializes test procedure play back. It must always be the first command in a Visual Basic script.

Set ProcID “QBPRQ01”:

(Visual Basic only) This is a subroutine defined by SQA Robot that identifies the ID of the test Procedure. This always follows Init Play and always Precedes any user action commands in a Visual Basic script.

Window Restoring Commands:

The following commands record the sizes and positions of open windows so that SQA Robot can restore the testing environment during playback. They also indicate the windows within whin user actions occur.

StartSaveWindowPositions
EndSaveWindowPositions

These statements tell SQA Robot that, during plackback,the intervening Window SetContext, Window MoveTo, and Window SetPosition commands are for windows restoration only.All Playback timing defaults are set to zero to process these commands as quickly as possible.if any command fails b/t StartSaveWindowPositions, that failure is reported to the log as a Warning, not a script command failure.

Window SetContext
WindowSetPosition

The Window SetContext statement defines the context window for subsequent test cases and user actions,and Window SetPosition indicates the position,size, and status of that context window.These statements can vary,depending on the window within which user actions occur.During recording,SQA Robot saves the positions of Windows that are normal,minimized, or maximized if Auto Record Window Size is checked in the Recording Options dialog box. It does not save the positions of hidden windows. During playback, SQA Robot restores the Windows to their positions When the test procedure nwas recorded. SQA Robot writes messages to the test log for any windows it cannot find on playback.


User Actions and Test Case Commands :

Useraction commands represent the actions you take While recording aGUI procedure. They are not test cases, but represent your actions between test cases. User actions Within a Window are always preceded in the script by a Window Set Context statement establishing the context window within which the actions occur.

Push Buttons Click, “VBName=SelectOK;Visual Text=OK”
User actions statements always begin with an object type (PushButton,Window, EditBox,etc.) followed by the action applied to it (Click,Resize, VScrollTo, etc.).
Result=WindowTC(Compare Menu, “VBName=Main;
VisualText=Mortage Prequalifier”,
“CaseID=QBPRQ01a”)

Test case and wait state functions are preceded by the Result variable.A test case functions corresponds to a test case you insert in to the text procedure While recording. During Playback,if the test case passes, Result equals 1. If the test case fails, Result equals 0.

Closing Commands:

All GUIprocedure scripts must end with the following commands, while terminate the test procedure and reset

SQA Robot:
EndPlay :

(Visual Basic only) This is a subroutine defined by SQA Robot that terminates playback of aVisual Basic test procedure script.

EndSub:
This indicates the end of the Main subroutine.

InitPlay, SetProcID, and EndPlay are required by SQA Robot to initialize and terminate playback of Visual Basic scripts.These commands are not required for SQA Basic scripts

SQA Robot includes the StartSaveWindowPositions and EndSave Window Positions commands only if Save Window Positions is checked in the Recording Options dialog box. 

Command Types:

SQA Robot test procedure commands are organized into these these types,based on the tasks they perform in the script:

CONTROL FLOW COMMANDS: Define the Beginning and end ofSQA Robot scripts, and the beginning and end of commands that save and restore windows. Control flow tasks include initialization and termination for test procedure and Playback. Used only in GUI Procedures.

ObJECT SCRIPTING COMMANDS:Acess an application`s objects and object Properties from within a test Procedure script . Object Scripting tasks include retrieving and setting an object`s properties. Used only in GUI Procedures.

TEST CASE COMMANDS:Compare objects captured While playing are same,the test case passes (Result equals 1). If the objects are different, the test case fails back successive builds of the application-under-test against the established base line data. If the objects( Result equals0).Used only in GUI Procedure.

USER ACTION COMMANDS: correspond to user actions on specific objects While recording.Actions include operations like choosing a menu command. Scrolling a list box, clicking a button, or typing text in to an edit box. Used only in GUI Procedures.

UTILITY COMMANDS:Affect the flow of test Procedure playback by setting wait times,calling other test procedures, starting applications, starting applications , starting and stopping timers,and playing back low-level recordings. They also control output to the test log, retrieve results from running test procedures, and set characters used in statements.Used only in GUI Procedures.

If you have Virtual User Recording installed. You can use these SQA Basic commands:

Data Pool commands-Retrieve data from a datapool created with SQA Datapool Manager.
HTTP commands- Issue HTTP or HTTPS requests to aWeb server that is not SSL-protected.
HTTP/HTTPS commands-Issue HTTP or HTTPS requests to aWeb server,depending on whether the server is SSL-protected.

Virtual User commands-Perform operations in virtual user procedures. For example,virtual user commands perform synchronization and timer operations for virtual users, and handle communications between the Master and Agent test stations.Except for control flow commands,which must appear at the beginning and end of every script,the order of recorded commands depends on your recording options and user actions.

NAMING CONVENTIONS OVERVIEW:

SQA Robert requires you to assign an ID for each test procedure and test case (or wait state). These IDs have an eight character limit,imposed by the DOS filename restriction. Because test procedure and test case IDs are referred to in the SQA repository and in test procedure scripts, it is important that you establish and adhere to a naming convention. Having a Meaningful naming convention helps you identify the test item`s purpose.

Because SQA Robot complies Visual Basic test Procedures in to executables with .EXE extensions, aVisual Basic test procedure ID should not be the same as that of another .EXE program.
However,because SQA Robot compiles SQABasic test procedures into .SBX files,this restriction does not apply if SQABasic is your scripting language.

Test Procedure IDs: 

A test procedure ID should indicate which application or component the test procedure tests. To achieve this, you canj assign meanings to characters of the filename.

In this diagram two characters identify the testing phase of the application-under-test (for example,function testing,transaction testing,or performance testing), five characters identify the functional area that the procedure tests (the module within the application-under-test),and a different tests on the same application module.

This is only one way in which you can define meaningful test procedure IDS. You can define IDs in whatever way best suits your organization and the application-under-test. The pieces that your naming convention identifies may correspond directly to the test requirements hierarcthy defined using SQA Manager.
Test Case IDs: The ID you define for a test case should help you identify the application-under-test,the test procedure it is in,and the type of test case.

In this diagram,two characters identify the testing phase of the application-under-test,three characters identify the functional area or module that the procedure tests, two characters identify the test case type, and a single character indicates the sequence. You can use either numbers, letters,or both for the sequence character,Which allows up to 36 test cases (0 through 9 plus A through z) in a test procedure. How ever, SQA recommends that you not create such alarge test procedure. Instead of including many test cases, define test procedures that are short and modular.

Autonaming:
SQA Robot can assign IDs for test procedures and test cases with its autonaming feature, which you can set using the preferences dialog box. If Autonaming On is checked,SQA Robot automatically inserts default IDs for test procedures, test cases, or both. You can accept or change these default IDs.

The autonamed test case or wait state IDs consist of the test procedure ID plus a single consecutive alphabetic character. For example, if the test procedure ID is TESTI, the autonamed ID for the first test case (or wait state) inserted in to the procedure is TESTIA, the ID for the second test case is TESTIB,and so on. If you change the default test procedure ID, you should limit it to seven characters to allow the auto naming of test case IDs. Because of the eight character filename limit,SQA Robot cannot automatically generate test case or wait state IDs if a test procedure ID exceeds seven characters.

Managing Test Procedures Overview:

Every time you record a test procedure,SQA Robot automatically adds a test procedure ID and definition to the SQA Repository. SQA Robot also lets you define,modify,and delete test procedure definations in your test planning phase before any actual recording takes place. You can also define test procedures using SQA Manager, Which stores the test procedure IDs,names,and definations in the SQA Repository and makes them available for recording with SQA Robot.

Defining New Test Procedures:
To create a new test procedure ID and definition:
•Choose Admin Test Procedure.
•Choose New.

•Enter the information to define your new test procedure.
This information is displayed in the SQA Test Log Viewer when you examine playback results, and also in SQA Manager when you perform defect tracking and reporting.

•Choose Ok.
•Enter an ID of up to eight alphanumeric characters and choose OK.This becomes the test procedure ID in the selector list.
Test procedures you define in SQA Robot are also available to SQA Manager.
Modifying Test Procedures:
•Choose Admin Test Procedure.
SQA Robot lists all defined test procedures in the dialog box,regardless of whether or not they have been recorded.
•Select a test procedure from the list.
•Choose Open or double-click the test procedure in the list.
•Change the information to redefine your test procedure.
This information is displayed in the SQA Test Log Viewer when you examine playback results,and also in SQA Manager when you perform defect tracking and reporting.
•Choose OK.
Deleting Test Procedures:
•Choose Admin Test Procedure.

SQA Robot lists all defined test procedures in the dialog box, regardless of whether or not they have been recorded.

•Select one or more test procedures from the list, or choose Select All.

•Choose Delete.SQA Robot prompts you to confirm the deletion.

•Choose Yes or Yes to All to delete the the test procedures, choose No to retain the indicated test procedure,or choose Cancel to terminate the entire deletion process.

Test Procedure Recording Overview:

The type of user action you record varies, depending on whether you are recording a GUI procedure or a virtual user procedure. 

Recording GUI procedures:

When you record a GUI procedure, you typically record a sequence of user actions required to execute one or more test cases, or verification points, in an application-under-test. When you record a GUI procedure, SQA Robot creates a script that keeps track..[Continue next update]

Test Procedure Overview

Why SQA

There are so many reasons why a company should consider SQA. It is all about business survival and SQA is just one of the many tools the company should effectively use. And just like a tool, it has to be effectively used to its maximum. If the tool is not used to its full extent, the tool will just be a financial burden to the company. SQA should also be utilized at the same way – to its full extent.

One of the reasons why businesses and companies opt out in using SQA is their inability to realize who effective an SQA when properly used. The following are thereasons why SQA should be used by any company before releasing their application to their intended users:

Traceability of Errors

Without using SQA, developers can still locate the errors of the application. Since their familiarity with the application is impeccable, they will have the ability to find a remedy to the problem. SQA on the other hand does not just look for answers to their problems.

SQA looks for the reason why the error occurred. Looking for errors is really easy but tracing the roots of the problem is another thing. The SQA team should be able to tell which practice has started the problem. Most of the time, the errors is not rooted on the execution of the coding but it could be found in the philosophy in developing theapplication.

Cost Efficient

This idea is saving money is very debatable in SQA. Using SQA means you will have to hire another set of people to work with you. Most of the SQA today are 3rd party companies that are only there to work on the specific project. Manpower will always cost money. Manpower does not even cover the cost of the tools the SQA team might use. Unfortunately most of the SQA tools are expensive especially the most efficient tools.

On the other hand, consider the idea of locating the bugs and errors in the application before they are released. Every application that are used in a business setting or sold to the consumers should have an ideal of 0% error. Even though this is rarely attained, SQA should be able to reduce errors to less than 1%.

Considering the effect on productivity together with customer satisfaction, you can always tell that preventing the problem is always better compared to answering them. SQA could practically save your business and boost it to success. Without them, there is a higher possibility that the application will have errors which will discourage your customers from purchasing the application.

In a business setting, an erroneous application will lead to slower productivity since instead of helping their customers; they might end up troubleshooting their application most of the time.

Flexible Solutions

SQA could easily provide solutions to the problem since they look for the root of the problem instead of just answering them. By providing the root of the problem, they have the ability to provide solutions to these problems fast. But instead of one solution only, the SQA team should be able to provide more than one solution.

Why SQA

Testing and Quality Assurance

What is the relationship between testing and Software Quality Assurance (SQA)?

An application that meets its requirements totally can be said to exhibit quality. Quality is not based on a subjective assessment but rather on a clearly demonstrable, and measurable, basis. Quality Assurance and Quality Control are not the same.

Quality Control is a process directed at validating that a specific deliverable meets standards, is error free, and is the best deliverable that can be produced. It is a responsibility internal to the team. QA, on the other hand, is a review with a goal of improving the process as well as the deliverable.

QA is often an external process. QA is an effective approach to producing a high quality product. One aspect is the process of objectively reviewing project deliverables and the processes that produce them (including testing), to identify defects, and then making recommendations for improvement based on the reviews.

The end result is the assurance that the system and application is of high quality, and that the process is working. The achievement of quality goals is well within reach when organizational strategies are used in the testing process. From the client's perspective, an application's quality is high if it meets their expectations

Testing and Quality Assurance

Introduction SQA

SQA suite is a team-testing environment for client/server applications.It automates test development and execution,and provides team workflow tracking and reporting.

SQA Suite Products:

1)SQA Manager:
Lets you plan,manage,and analyze all aspects of your testing projects.SQA Manager offers integrated,E-Mail-Enabled workflow tracking and deffect management combined with powerful reporting and graphics capabilities.

2)SQA Robot
Allow you to create,modify,and run automated tests for Windows client/server applications.SQA Robot uses specialized object-level integration with Centura,Delphi,Oralce Forms,PoweBuilder,SQLWindows,VisualBasic,and other RAD environment.

3)SQA LoadTest
Lets you run performance,configuration,multi-user,and distributed regression tests on multiple test stations conncected by a TCP/IP,NetBIOS,or IPX/SPX network.With SQA Loadtest,you run,manage,and monitor tests from a central Master test station.SQA Loadtest combines the traditional methods of GUI testing with the ability to impose load on a Web or database server.

4)SQA SiteCheck
Lets you manage your Intract or World Wide Web site.With SQA SiteCheck,you can use graphical capabilities to visualize the structure of your Website.Automatic Web site testing with SQA Robot,and analyze and track,Web site defects using SQA Manager.SQA SiteCheck comes with its own integrated editor,or you can use your favorite HTML,editor to update and repair Web site problems.

5)SQA Process
A formalized methodology for automated testing of Windows client/server applications.SQA Process serves as a framework for creating a world-class quality assurance organization.

6)SQA Repository
The SQA Suite component for storing software testing project information about test planning test execution,and defect tracking. All SQA products are integrated by a shared network repository.

Introduction SQA