Friday, November 26, 2010

HP QuickTest Professional


HP QuickTest Professional software provides functional and regression test automation for software applications and environments.Part of the HP Quality Center tool suite, HP QuickTest Professional can be used for enterprise quality assurance.

HP QuickTest Professional supports keyword and scripting interfaces and features a graphical user interface. It uses the Visual Basic Scripting Edition (VBScript) scripting language to specify a test procedure, and to manipulate the objects and controls of the application under test.

HP QuickTest Professional was originally written by Mercury Interactive.Mercury Interactive was subsequently acquired by Hewlett Packard (HP) in 2006. HP QuickTest Professional 11 is currently available from HP Software & Solutions.


HP QuickTest Professional is automated testing software designed for testing various software applications and environments. It performs functional and regression testing through a user interface such as a native GUI or web interface.[8] It works by identifying the objects in the application user interface or a web page and performing desired operations (such as mouse clicks or keyboard events); it can also capture object properties like name or handler ID. HP QuickTest Professional uses a VBScript scripting language to specify the test procedure and to manipulate the objects and controls of the application under test. To perform more sophisticated actions, users may need to manipulate the underlying VBScript.

Although HP QuickTest Professional is usually used for "UI Based" Test Case Automation, it also can automate some "Non-UI" based Test Cases such as file system operations and database testing.

Verification

Checkpoints verify that an application under test functions as expected. Users can add a checkpoint to check if a particular object, text or a bitmap is present in the automation run. Checkpoints verify that during the course of test execution, the actual application behavior or state is consistent with the expected application behavior or state.

HP QuickTest Professional offers 9 types of checkpoints, enabling users to verify various aspects of an application under test, such as: the properties of an object, data within a table, records within a database, a bitmap image, or the text on an application screen. The types of checkpoints are standard, image, table, page, text, text area, bitmap, database, accessibility and XML checkpoints. Users can also create user-defined checkpoints.

Exception handling

HP QuickTest Professional manages exception handling using recovery scenarios; the goal is to continue running tests if an unexpected failure occurs. For example, if an application crashes and a message dialog appears, HP QuickTest Professional can be instructed to attempt to restart the application and continue with the rest of the test cases from that point. Because HP QuickTest Professional hooks into the memory space of the applications being tested, some exceptions may cause HP QuickTest Professional to terminate and be unrecoverable.

Data-driven testing

HP QuickTest Professional supports data-driven testing. For example, data can be output to a data table for reuse elsewhere. Data-driven testing is implemented as a Microsoft Excel workbook that can be accessed from HP QuickTest Professional. HP QuickTest Professional has two types of data tables: the Global data sheet and Action (local) data sheets. The test steps can read data from these data tables in order to drive variable data into the application under test, and verify the expected result.

Automating custom and complex UI objects

HP QuickTest Professional may not recognize customized user interface objects and other complex objects. Users can define these types of objects as virtual objects. HP QuickTest Professional does not support virtual objects for analog recording or recording in low-level mode.


Extensibility

HP QuickTest Professional can be extended with separate add-ins for a number of development environments that are not supported out-of-the-box. HP QuickTest Professional add-ins include support for Web, .NET, Java, and Delphi. HP QuickTest Professional and the HP QuickTest Professional add-ins are packaged together in HP Functional Testing software.

Test results

At the end of a test, HP QuickTest Professional generates a test result. Using XML schema, the test result indicates whether a test passed or failed, shows error messages, and may provide supporting information that allows users to determine the underlying cause of a failure. Release 10 lets users export HP QuickTest Professional test results into HTML, Microsoft Word or PDF report formats. Reports can include images and screen shots for use in reproducing errors.

User interface

HP QuickTest Professional provides two views--and ways to modify-- a test script: Keyword View and Expert View. These views enable HP QuickTest Professional to act as an Integrated Development Environment (IDE) for the test, and HP QuickTest Professional includes many standard IDE features, such as breakpoints to pause a test at predetermined places.

Keyword view

Keyword View lets users create and view the steps of a test in a modular, table format. Each row in the table represents a step that can be modified. The Keyword View can also contain any of the following columns: Item, Operation, Value, Assignment, Comment, and Documentation. For every step in the Keyword View, HP QuickTest Professional displays a corresponding line of script based on the row and column value. Users can add, delete or modify steps at any pointst.

In Keyword View, users can also view properties for items such as checkpoints, output values, and actions, use conditional and loop statements, and insert breakpoints to assist in debugging a test.[20]

Expert view

VBScript code in Expert View

In Expert View, HP QuickTest Professional lets users display and edit a test's source code using VBScript. Designed for more advanced users, users can edit all test actions except for the root Global action, and changes are synchronized with the Keyword View.

Languages

HP QuickTest Professional uses VBScript as its scripting language. VBScript supports classes but not polymorphism and inheritance. Compared with Visual Basic for Applications (VBA), VBScript lacks the ability to use some Visual Basic keywords, does not come with an integrated debugger, lacks an event handler, and does not have a forms editor. HP has added a debugger, but the functionality is more limited when compared with testing tools that integrate a full-featured IDE, such as those provided with VBA, Java, or VB.NET.

Drawbacks

HP QuickTest Professional cannot be used by a plug-in in non-Windows environments. It fetches objects like ActiveX from the Windows environment but not from other operating systems. QTP cannot be used to test with all browser types and versions

Verification and Validation (software)

In software project management, software testing, and software engineering, Verification and Validation (V&V) is the process of checking that a software system meets specifications and that it fulfils its intended purpose. It is normally part of the software testing process of a project.


Also known as software quality control.

Validation checks that the product design satisfies or fits the intended usage (high-level checking) — i.e., you built the right product. This is done through dynamic testing and other forms of review.

According to the Capability Maturity Model (CMMI-SW v1.1),

  • Verification: The process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. [IEEE-STD-610].
  • Validation: The process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements. [IEEE-STD-610]

In other words, validation ensures that the product actually meets the user's needs, and that the specifications were correct in the first place, while verification is ensuring that the product has been built according to the requirements and design specifications. Validation ensures that 'you built the right thing'. Verification ensures that 'you built it right'. Validation confirms that the product, as provided, will fulfill its intended use.

From testing perspective:

  • Fault - wrong or missing function in the code.
  • Failure - the manifestation of a fault during execution.
  • Malfunction - according to its specification the system does not meet its specified functionality.

Within the modeling and simulation community, the definitions of validation, verification and accreditation are similar:

  • Validation is the process of determining the degree to which a model, simulation, or federation of models and simulations, and their associated data are accurate representations of the real world from the perspective of the intended use(s).[1]
  • Accreditation is the formal certification that a model or simulation is acceptable to be used for a specific purpose.[1]
  • Verification is the process of determining that a computer model, simulation, or federation of models and simulations implementations and their associated data accurately represents the developer's conceptual description and specifications.[1]

Test bench

A test bench is a virtual environment used to verify the correctness or soundness of a design or model (e.g., a software product).

The term has its roots in the testing of electronic devices, where an engineer would sit at a lab bench with tools of measurement and manipulation, such as oscilloscopes, multimeters, soldering irons, wire cutters, and so on, and manually verify the correctness of the device under test.

In the context of software or firmware or hardware engineering, a test bench refers to an environment in which the product under development is tested with the aid of a collection of testing tools. Often, though not always, the suite of testing tools is designed specifically for the product under test.

A test bench or testing workbench has four components.

1.INPUT: The entrance criteria or deliverables needed to perform work

2.PROCEDURES TO DO: The tasks or processes that will transform the input into the output

3.PROCEDURES TO CHECK: The processes that determine that the output meets the standards.

4.OUTPUT: The exit criteria or deliverables produced from the workbench


An example of the software test bench

The tools used to automate the testing process in the above testbench perform the following functions

Test Manager : It manages the running of program tests. The test manager keeps track of test data, expected results and program facilities tested.

Test Data Generator : It generates test data for the program to be tested.

Oracle : It generates predictions of the expected test results. Oracle may be either previous program versions or prototype systems.

File comparator : It compares the results of the program tests with the previous test results and reports the differences between them in a document.

Report generator : It provides report definition and generation facilities for the test results.

Dynamic analyzer : It adds code to a program to count the number of times each statement has been executed. It generates execution profile for the statements to show the number of times they get executed in the program run.

Simulator : It simulates the testing environment where the software product is to be used.

Test harness

In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the Test execution engine and the Test script repository.

Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.

A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.

The typical objectives of a test harness are to:

  • Automate the testing process.
  • Execute test suites of test cases.
  • Generate associated test reports.

A test harness may provide some of the following benefits:

  • Increased productivity due to automation of the testing process.
  • Increased probability that regression testing will occur.
  • Increased quality of software components and application.
  • Ensure that subsequent test runs are exact duplicates of previous ones.
  • Testing can occur at times that the office is not staffed (ie. at night)
  • A test script may include conditions and/or uses that are otherwise difficult to simulate (load, for example)

An alternative definition of a test harness is software constructed to facilitate integration testing. Where test stubs are typically components of the application under development and are replaced by working component as the application is developed (top-down design), test harnesses are external to the application being tested and simulate services or functionality not available in a test environment. For example, if you're building an application that needs to interface with an application on a mainframe computer but none is available during development, a test harness maybe built to use as a substitute. A test harness maybe part of a project deliverable. It's kept outside of the application source code and maybe reused on multiple projects. Because a test harness simulates application functionality - it has no knowledge of test suites, test cases or test reports. Those things are provided by a testing framework and associated automated testing tools.

Test automation framework

A test automation framework is a set of assumptions, concepts and tools that provide support for automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application.

Choosing right framework/scripting technique helps in maintaining the costs and maintaining Costs associated with test scripting are due to the development effort and the maintenance efforts. The approach of scripting used during test automation has effect on the costs Various framework/scripting techniques are generally used:

  1. Linear (simple record and playback)
  2. Structured (uses control structures - typically 'if-else', 'switch', 'for', 'while' conditions/ statements)
  3. Hybrid
  4. Data-driven (data is separated and stored in external excel sheets)
  5. Keyword-driven

The Testing framework is responsible for:

  1. defining the format in which to express expectations
  2. creating a mechanism to hook into or drive the application under test
  3. executing the tests
  4. reporting results

Decision tables

Decision tables are a precise yet compact way to model complicated logic.

Decision tables, like flowcharts and if-then-else and switch-case statements, associate conditions with actions to perform, but in many cases do so in a more elegant way.


A decision table is typically divided into four quadrants, as shown below.

The four quadrants
Conditions Condition alternatives
Actions Action entries

Each decision corresponds to a variable, relation or predicate whose possible values are listed among the condition alternatives. Each action is a procedure or operation to perform, and the entries specify whether (or in what order) the action is to be performed for the set of condition alternatives the entry corresponds to. Many decision tables include in their condition alternatives the don't care symbol, a hyphen. Using don't cares can simplify decision tables, especially when a given condition has little influence on the actions to be performed. In some cases, entire conditions thought to be important initially are found to be irrelevant when none of the conditions influence which actions are performed.

Aside from the basic four quadrant structure, decision tables vary widely in the way the condition alternatives and action entries are represented. Some decision tables use simple true/false values to represent the alternatives to a condition (akin to if-then-else), other tables may use numbered alternatives (akin to switch-case), and some tables even use fuzzy logic or probabilistic representations for condition alternatives[citation needed]. In a similar way, action entries can simply represent whether an action is to be performed (check the actions to perform), or in more advanced decision tables, the sequencing of actions to perform (number the actions to perform).

Decision table can be used if the combination of conditions are given. In decision table conditions are known as causes and serinal numbers of conditions are known as business rule.


he limited-entry decision table is the simplest to describe. The condition alternatives are simple Boolean values, and the action entries are check-marks, representing which of the actions in a given column are to be performed.

A technical support company writes a decision table to diagnose printer problems based upon symptoms described to them over the phone from their clients.

The following is a balanced decision table.

Printer troubleshooter


Rules
Conditions Printer does not print Y Y Y Y N N N N
A red light is flashing Y Y N N Y Y N N
Printer is unrecognised Y N Y N Y N Y N
Actions Check the power cable     X          
Check the printer-computer cable X   X          
Ensure printer software is installed X   X   X   X  
Check/replace ink X X     X X    
Check for paper jam   X   X        

Of course, this is just a simple example (and it does not necessarily correspond to the reality of printer troubleshooting), but even so, it demonstrates how decision tables can scale to several conditions with many possibilities.