Seven Principles of Software testing

Software Testing focuses on ensuring the quality of the software. Software not tested properly can cause many problems including financial losses, reputation loss and at times injury or even death. How much testing is required depends on the risks associated with the software as well as time constraints. There are few principles of testing that can help to better understand the process of testing and define testing process

The principles of testing are as follows:

1.  Testing shows presence of defect  

Testing helps in uncovering defects although we can never be sure that that there are no defects in the software even after testing. So the focus of testing is to find as many defects in the system.

2.  Exhaustive Testing is not feasible

 Going by the first principle, tester can never be sure that there are no defects in the system, One of the reason for first principle is exhaustive testing is impossible. We can test all the scenarios related to software but the time and resource cost for testing exhaustively will be so huge, it will not bring any value and will not be able to deliver software. Due to this proper risk analysis and priorities of scenarios should be taken care of in testing for balanced and timely testing completion.  Test cases should be prepared based on the functional specification, as test cases for high priority scenario should be executed first.

3.  Testing should start as early as possible

 Catching defects early in the Software development life cycle reduces the cost of fixing of defect. For e.g. Testing should start as early as possible, Reviews should take place to identify gaps in requirements understanding, and identify code issues early.  Also test designing should start as soon as possible.

4.  Most of the Defects are confined to small module

This has been noticed that most of the defects are confined to small module; the reason of defects can be understanding gaps in the particular set of modules and the technical skill of the resources.

5.  Pesticide Paradox

 Executing the same test cases again and again eventually results in no defects being identified by test cases. Due to this reason test cases should be reviewed from time to time, with new test cases added and redundant test cases removed from the regression pack to keep the test suite fresh, up to date and effective. Test cases must include both manual and automated test cases for higher coverage of test scenarios.

6.  Testing requirement varies from software to software 

Testing is done differently for different software. A social networking site will have different requirement of content and performance testing compared to banking software that will require functionality and security as main factors compared to the social networking site. The testing requirements should be well defined in the scope of testing.

7.  The software developed should be usable by user

and should meet user’s need and expectation. It does not matter if there were no defects in testing the software if it does not meet the need of end users.Software 

Understanding Quality Assurance and Quality Control

Whenever a product is delivered to the client, client expects the quality of the end product to be high. To assure high quality of end products, there must be stress on quality. Quality has different meaning for different users. For developer of a product, it means quality of product meets the requirements mentioned in the specifications whereas for end user, it means the product is fit for use and works as expected. A quality Software should possess desired features,should be fit for use and usable at the desired cost

To ensure quality, there are quality Assurance and quality control groups in the organization. In this article, we will discuss the role and difference between quality assurance and quality control.

Quality Assurance

Quality assurance is a planned and systematic set of activities necessary to provide adequate confidence that products and services will conform to specified requirements and meet user needs. The purpose of quality Assurance is to ensure quality policies are defined. The project Quality Assurance must ensure process are implemented for:

•      System development methodologies
•      Estimation processes
•      System maintenance processes
•      Requirements definition processes
•      Testing processes and standards

Quality assurance is a continuous improvement activity in processes uncovering weakness in the process and strengthening the process followed. Normally the quality assurance activities are defined at organization level and are uniformly followed across the project. There are regular audits by an independent QA audit team within organization which audits across different project to ensure Quality assurance activities are followed across the organization.

Quality Control

Quality control is the process in which developed product is compared with requirement at different phases and discrepancy is logged as defects. The process of quality control begins with the development phase. This includes Static testing, and dynamic testing.

Static testing includes reviews and analysis of the code using tools. Reviews include manual examination of the code as well as documentation. Reviews are important to identify gaps early in the development process. Defects detected during reviews early in the life cycle are often much cheaper to remove than those detected while running tests Reviews can vary from informal reviews to formal reviews. Informal reviews, Walkthrough, Technical review, and inspection are all examples of review with varying degree of documentation and formal approach. Static analysis also includes verifying the code effectiveness through code analysis tools.

Once the development of the requirement is completed, the product is passed on to testing team for complete testing. The purpose of testing is to uncover as many defects in the system as possible. The focus of testing is look at the product from different views. A system tester or functional tester will verify the system/functionality is working as per functional requirement. Performance tester will need to verify the performance of system under stress and load conditions and so on. How much testing and what testing is required is normally defined in the scope of testing. There are various test management tools like Quality center that helps to manage and map various test artifacts and defects with test cases.

Quality Assurance and Quality Control activities are required to ensure quality deliverables to client and should be followed religiously to ensure high quality to the customer

Understanding Insight Objects in UFT 11.5

With Unified Functional Testing, there were a lot of features like integrating GUI testing and API testing, introducing File Content Checkpoint, and create insight object or recording in Insight mode. 

In this article I will try to explain what this feature of Insight recording is and how to implement same.

Insight is an image-based identification ability, to recognize objects in application based on what they look like instead of using properties that are part of design. UFT stores an image of the object with insight test objects and uses this image as main description property to identify object in the application.

Together with insight property, UFT can use ordinal identifier to uniquely identify the object. Another property we can use is Similarity - Specifies how similar a control in the application has to be to the test object image for it to be considered a match. And Visual relation identifier can also be used to identify the object based on its relative position with respect to other objects in the application. 

Low points of insight object are we cannot add insight object from object spy and snapshots occupy a lot of disk space.

Adding Insight Objects :

UFT allow adding Insight object either in recording Mode or manually in object repository:

1.  Insight Recording

 When we record in UFT, There is option to select recording mode. The recording mode in UFT is default, analog, Low Level recording, and Insight recording. When we select recording mode as Insight recording, UFT records object as insight object for images and identifies rest of objects like edit box as window object WinObject and perform action on them.

Recording bar in UFT 11.5

2.   Insight Objects can be added through Object Repository.

Go to Resources>Object Repository. This will open Object Repository window. In Object Repository window, Navigate to Object>Add Insight objects to Local. This can also be added by icon as shown below.

Once we click on Add Insight objects to local, UFT asks to select Learn Mode which can be manual and automatic. Manual Mode provides magnifier to select minute objects in the application.

Adding object from object repository

Insight object is always added to the object repository as a child of the test object that represents its containing application, such as a Window or Browser object.

In the Editor, test object image is displayed in the step instead of the test object name. Enlarge Image is displayed on hovering over the object.

Options available to work with Insight in UFT 11.5:

UFT 11.5 provides following options for working with insight objects. Options for Insight Object are at Tools>Options>GUI Testing>Insight. This pane enables you to define options that customize how UFT handles Insight test objects when creating test object, and during record and run sessions.

Options to customize the insight object are as follows:

a.   Save the clicked coordinates as the test object’s Click Point when recording a test object.

b.   When recording a test, display mouse operations.

options for insight 

c.   When editing shows test object image in steps and displaying Select learn mode dialog box

d.  Snapshots for insight object takes a lot of disk space, we can provide option to limit test object image, maximum pixels around image, and maximum number of snapshots to save when recording a test object

 

option for insight

Unified Functional testing: Understanding New Features

Unified functional testing 11.5 is latest from HP on functional automation testing. Previously I worked with QTP 11.0, and there are many changes in QTP 11 and UFT 11.5. Let me share the changes that I encountered and new features in UFT 11.5. Unified functional testing helps to work with API  as well as GUI together and the IDE has changed a lot from QTP 11.

1. UI Interface: 

Interface for UFT 11.5 has changed a lot from QTP 11 and it seems as if I was working on a completely new tool.

a.   Creating a new test:

Now we can create an API Test as well as GUI Test. API Test is for testing API Interface whereas GUI Test is the normal test or QTP Test that we have been creating in the past.

creating a new test in QTP

b.   Keyword View: 

We cannot  view Keyword view and expert view in UFT 11.5 together. To view Keyword view, We have to select View>Keyword View. To go back from keyword view to expert view, select View>Editor.

How a test looks in UFT

c.   UFT 11.5 provides easy navigation to functions/classes in the editor as shown in the below screenshot.

d.   Menu Option have changed a lot In UFT 11.5, with separate menu item for ALM, and redesign and menu options shuffled from one tab to another.

e.   Provides creating solution in which we can manage test documents including tests, function libraries, business components and test flows.

2.Using Solution explorer

we can work with multiple tests in the solutions and can edit, manage multiple tests at the same time.

3.   One of the new features of UFT 11.5 is Insight for object Identification

, an image-based identification ability to recognize objects in your application based on what they look like, instead of using properties that are part of their design. This can be useful if you are working with an application whose technology is not supported by UFT, or with an application running on a remote computer. This is a new feature and will be discussed in details in next articles.

4.   Using File content checkpoint

we can verify the content of file and can be useful for comparing Pdf files or contents of PDF file. PDF, HTML, Word, TXT, and RTF file types are supported.

5.  UFT supports testing for QT and Adobe flex Applications.

6.  Supports Microsoft Office and plug-in controls.

7.  In API Testing, Using the soapUI to API Test conversion tool, you can convert soapUI tests to UFT API tests.

8.  With UFT, new versions in different environment are supported that were not supported in QTP 11

9.  Start Page of Unified function testing provide links to new features, communities, useful links, and support. Using this we can connect to other users for forum support and information.

10. We can set preference to automatically export run results in HTML or PDF format after every run session through Tools > Options > General tab > Run Sessions

11. Bitmap checkpoint now enables to specify multiple areas to compare or ignore within the bitmap checking. It also supports checking whether a specific image appears anywhere within the runtime bitmap. 

Test Estimation : Understanding Test Efforts Calculation

Estimates plays very important role in successful projects as incorrect estimates can lead to time crunches , or improper resource utilization in case of excessive time in estimates.
In this article we will discuss various factors in estimation and how to create good estimates in a testing project.

Before providing estimates, we need to know the requirements of the project. First step before any testing task including test plans and estimates is to know what we are delivering to the client. Once we are clear on the requirements, we need to define the tasks that need to be done as exit criteria for the project. 

We have to answer following points before progressing with estimates:

1.   What is the time required to set-up the basic infrastructure. This includes acquiring resources, testing tools, setting up environment and training requirements of resources.

2.   What are the testing types we are performing in project? Does it include automation testing, API Testing, and other testing types?

3.   What is the Software Development Life cycle followed in the project, i.e. it is agile, waterfall or V-model, if iterative, what is the scope of each iteration.

4.   Is this project new phase of an existing project or a version of existing application to released in the current project.

5.  Techni Experience of resources working in Project. This helps to determine the time for preparing test artifacts.

So Once we know answers to above questions, we need to create a WBS document most preferably in Microsoft Project which helps in defining the tasks with time estimates, resources overloading  and defining milestones in project. Below points should be considered while creating the Work Breakdown Sheet.

1.  First of all create the high level milestone for the project. This can be various phases of the project or iterations of project in and agile methodology.

2.  Now create subtask for each milestones of the project. This will include tasks like test planning, test review, test execution, test data preparation for various forms of testing and regular team meetings.

3.  Task test planning, test review, test data preparation and test execution requires further breakdown of requirements into test scenarios and further into Test cases. Once the requirements are broken down into test scenarios/ test cases, we require estimates for each of the tasks.

Example: 

• Suppose we have 250 test cases in an application for a particular milestone. We need to define time for test planning, test review, test execution, and test data preparation for each of the test artifact, can be test case for manual execution and test script in automation testing. Automation testing will also include task for framework development.

• Now divide each of the test cases based on test complexity into high, medium and low complexity. Dividing and allocating time for each test case using test case complexity is very useful in case of large number of test cases where estimates for a particular test case in not feasible and time required for each test case is more or less uniform based on complexity. In this approach, time estimates are provided for test cases based on complexity. For e.g.  10 hours for test case preparation for high complexity, 6 hours for medium complexity, and 4 hours for low complexity.

• In Point b), we provide test estimates based on complexity and was useful since time for test cases was uniform and there was large set of test cases. 

• In case the project has 10 requirement of varying complexity, like 1 requirement will take 100 hours and other 2 hours, So Rather providing estimates based on complexity, we can define the estimate based on past experience of users, and provide below time:
• Min Time to complete the task
• Worst Scenario Time to complete the task.
•  Average time to complete the task

• We can calculate the average of above time to calculate time to complete the task. While providing the estimate as calculated in step c) we must take into consideration resources experience in the application and technology.

4.   We can provide time for each of the tasks in the work breakdown sheet. Summing time for all the activities will give us the test efforts required in the project.

5.   We further need to assign task to different types of resources based on the tasks and can provide resourcing estimates  for the project. we can define the resources utilization during different phases of project .

I will try to cover further in future articles on estimates things I might have missed here.

Creating First Test in Selenium IDE using FireFox

Starting working with Selenium IDE:

1. Before Working with Selenium IDE, we will need to have mozilla firefox installed in the machine.
2. Also plugins and add-ons for using Selenium needs to be installed from selenium official webSite.
3. Once the add-ons are properly installed, To Open Selenium IDE, go to Web Developer>Selenium IDE or Press "Ctrl+Alt+S"

Understanding Selenium IDE

Below Figure explains various components of Selenium IDE

File Menu - File Menu provides the options to create a new test case, test suite, and the test suite can be exported to Ruby/Java/c# Web Driver or Remote Control.
Recording - Recording starts automatically on launching the IDE.
Test Run - Icon to run  all tests or individual tests is available on Selenium IDE
Test Execution Speed - Test execution script can be set as fast or slow as shown  in figure 1.

Creating First Test in Selenium IDE

Recording starts automatically once we launch Selenium IDE, and perform some action on the Firefox browser.Once some action is performed, the action is displayed is command , e.g : Click , Open, etc. Target displays the property of object on which command is to executed and value is value to be set in input.

Assertions and Verification can not be done during recording and needs to be added manually.Difference between assert and verify Commands is test execution stops once assert statement fails but test execution continues to next step if we execute the Verify command, although the test status fails in both cases.

Tips for adding command: 
User is provided with all the possible commands on a object when Selenium IDE is open. 

Steps:
1. Right Click on an object in Firefox browser.
2. Click on Show All Available Commands>Select the required Command.
3. Command will be added in the test.

Adding command to selenium test

How to compare 2 dictionary object and report the differences in QTP results

This post discusses how to compare two dictionary objects and display the difference between the values in two dictionary objects in table format in QTP results using Reporter.logevent.

''function to compare two Dictionaries 

Public Function Comfunct_CompareDictionary(ObjDictBase, ObjDictGen)

Comfunct_CompareDictionary = True 
''Check if both dictionary objects have the same number of items
If ObjDictBase.Count <> ObjDictGen.Count Then
Reporter.reportevent micFail,"Data in Dictionary 1 and 2","Data mismatch in the dictionary. there are " & objDictBase.Count & "records in dictionary 1 compared to " & objDictGen.Count & "records in dictionary 2" 
Comfunct_CompareDictionary = False
Exit Function
End If
''Compare keys and values
arrKeys = ObjDictBase.Keys
''Create a table for the differences
strTable = "<table border=""""2""""><tr><td>Key Name</td><td><Value in Dictionary 1</td><td><Value in Dictionary 2</td></tr>"
For i = 0 to uBound(arrKeys)
''Compare key names in both dictionaries and validate keys in one dictionary exists in other dictionary
  If Not ObjDictGen.Exists(arrKeys(i)) Then
''ObjDictBase has a key which ObjDictGen doesn't have, hence exiting the function
Comfunct_CompareDictionary = False
Reporter.ReportEvent micFail, "Key " & arrKeys(i) & " exists in dictionary 1 but not in dictionary 2, " Key difference in both dictionaries"
Exit Function 
  End If
  ''Compare value of each keys in both dictionary and display the difference between 2 dictionary in Qtp results 
  If ObjDictBase(arrKeys(i)) <> ObjDictGen(arrKeys(i)) Then
''ObjDictBase value for arrKeys(i) differs from ObjDictGen, then log the difference in the table created above
Comfunct_CompareDictionary = False
strTable = strTable & "<tr><td>" & arrkeys(i) & "</td><td>" & ObjDictBase(arrKeys(i)) & "</td><td>" & ObjDictGen(arrKeys(i)) & "/td></tr>" 
  End If
Next
If Comfunct_CompareDictionary = True Then
Reporter.ReportEvent micPass, "Comparision of dictionary 1 and dictionary 2","Both dictionary matches"
Else
strTable = strTable & "</Table>"
''Compare value of each keys in both dictionary and display the difference between 2 dictionary in Qtp results 
Call reportinQTPhtmlContent(strTable)
End If 
End Function

''Function to report in QTP result in table format

Public Function reportinQTPhtmlContent(strTable)

Set objdict = CreateObject("Scripting.dictionary")

objdict.Add "Status", micdone

objdict.Add "NodeName", "Table Data"

objdict.Add ("StepHtmlInfo"), strTable

Reporter.LogEvent "User", objdict, Reporter.GetContext

End function

How to Configure Eclipse to work with Selenium

Before starting working with Selenium using Eclipse IDE, we did some configuration that needs to be done for starting and understanding Selenium. This post will explain step wise, how to start working with selenium using Eclipse:

Pre- Conditions: We will require following before beginning the configuration:

1. Java needs to be installed in the machine.

2. Eclipse IDE needs to be installed .

3. Standalone libraries of selenium, at the time of writing this post, I am using selenium-server-standalone-2.33.0.jar

4. Verify in Path variable in User defined variable for machine, is set to the path where standalone libraries is placed as shown below

Configuration Steps: 

Step 1: In eclipse, create a new java project as shown below from File>New Java Project

Create a new Java Project

Step 2: Once a java project is created, right click on project and select option "Build Path>Configure build path

Configure build path of project

Step 3: Click on Add External JAR's and add the selenium standalone jar as shown below.

 Step 4: Now in src for the project, Add a new package, and add a new class as shown below.

Select methods stubs to create as public static void main(String[] args)

                                         

Step 5: Write the code in the class created above and execute. This will open Google in internet explorer . Write selenium in search and click on search.

Code to be placed in void main:

WebDriver ieDriver = new InternetExplorerDriver(); ieDriver.get("http://www.google.com"); WebElement element = ieDriver.findElement(By.name("q")); element.sendKeys("selenium"); element.submit();

Manipulating Array with examples using vbscript in QTP

In this post we discusses examples working with arrays and manipulating them in vbscript:

1.How to count number of elements in an array

intcount = ubound(arrayname)

2. How to join all the values in an array to a string and display the value in messagebox

dl=Array("Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday")
daysinWeek = join(dl, " ")
msgbox "days in weeks " & daysinWeek

3. How to split values in a string in an array based on delimiter

dl="Sunday>Monday>Tuesday>Wednesday>Thursday>Friday>Saturday"
daysinWeek = split(dl, ">")

This method can be used on looping as shown in example below:

On error resume next

Set objDesc= Description.create

'Set of Property values are delimited by "," 

If (instr(1,strDescObj,",") >0) Then

''Split multiple links based on delimiter.

arrDesc = split(strDescObj,",")

intPropSet = Ubound(arrDesc)

''Loop through each of the link and click the required link

for i = 0 to intPropSet

strDesc = split(arrDesc(i),"|")

objDesc(strDesc(0)).value = strDesc(1)

Next

End If

Understanding Date time function: Using VBScript in QTP

Below are some of the useful function for working with date/time in vb script. We will use the functions and illustrate where the functions can be used.

1. Date - This returns the current date of the system.

2. Now - This returns the current Date and time of the system.So now contains time part as well as date part while date function returns only date.

3. Time - This returns the current time of the system. So in terms of relation between date , time, and now function 

Now = date + time

4. Year, Month, Day, hour, minute, second - These functions return year, month,day,hour,minute,second part for the expression.

example : day(now) or  minute(now)

Above functions discussed above are useful for timestamp , and also for generating a unique number based on the function used.

5. Timer - Timer returns the number of seconds from 12:00a.m. This function is very useful for evaluating time spent in performing an activity by taking timer at the start and end of the activity and subtracting the time between two.

6. DatePart - returns specific part of the date .Possible values or part of date , that can be extracted as shown below:

• yyyy - Year
• q - Quarter
• m - Month
• y - Day of year
• d - Day
• w - Weekday
• ww - Week of year
• h - Hour
• n - Minute
• s - Second

Syntax of the function is : DatePart("m",d))where d is the date as "2013-06-09" and m will return the month of the date.

7. DateDiff - Returns the difference between two dates: 

Syntax : DateDiff(interval,date1,date2). Interval can be yyyy,q,m, y,d,.... as discussed earlier.

8. DateAdd - Add specified date to the date.

Syntax : DateAdd(interval,number,date)

VB Script functions to manipulate strings in QTP

We should know the useful vb script function to manipulate strings. In this post, we will discuss some of the vb script function to manipulate strings and how they can be used in codes:

1. Instr searches for a string within another another string and returns the location of occurrence of one string within another string.It returns the position of first occurrence of the string.This function is used if we need to verify if a particular text appears in a string.

Syntax : InStr([start,]string1,string2[,compare])

Example - Report and loop in case a particular text in not found in the string :

If (Instr(1, string1, string2)>0) Then
    Reporter.ReportEvent micpass , "string 2 is found in string 1","As expected"
Else
    Reporter.ReportEvent micfail , "string 2 is  found in string 1","string 2 is not found in string1"
End If

2. InstrRev is similar to instr and can be used for similar purposes, with only difference search starts from the end of string, although count returned is from start of string.

InStrRev(string1,string2[,start[,compare]])

3. Trim , lTrim, Rtrim removes characters from both left and right, only left and only right respectively blank spaces.

4. Left , Right, and Mid returns a specified character from left, right, and middle of the string. 

Mid(string,start[,length])

5. len returns number of characters in the string.

6. Split splits a string based on the delimiter in the function.

Example - How to find the total number of occurrence of a string in another string.By default, string2 has value space if not provided.

arrString = split(string1,string2)
totOccurence = ubound(arrString)

Example- Total number of words in a string.

arrString = split(string1)
totOccurence = ubound(arrString)