Understanding User Acceptance Testing

In software development, the application developed is tested by different teams. Developers perform Unit testing and Integration testing before passing the product to QA for their validations. QA Performs System Testing, SIT and regression testing on the product. Once the QA certifies the product, the Product is passed on to the client/UAT Team for User Acceptance Testi

Client/UAT Team before passing the product to the end users needs to verify if the product delivered to him is the same as requested or a different product is developed. The basis for testing is the requirements of client and product is developed as expected in the requirement. The product is passed on to client once the exit criteria between client and supplier is met which includes conditions of no major issues in the developed product by supplier.

Client on receiving the developed product tests the product internally if it is as per expectation and then passes the same to a subset of end users (beta users) to validate the product from end user point of view. Once it passes both internal customers testing and external customer testing (by beta users and can be called as beta testing), it is released to market to be used by customers or end users.

Entry Criteria for UAT testing is usually the exit criteria for System Testing Phase. Entry criteria for UAT testing includes in general:

• No Major open defects in the product

• Test Execution completed and sign off communication provided by System Testing team

• List of Known issue provided.

• Environment for UAT deployed.

Once Entry Criteria for UAT is met, the UAT team starts testing on the product. UAT can happen in two stages. In First stage, the testing is done by the UAT Team of the client. The focus or key aspect of testing is to validate the product is developed as per the requirement agreed between the stakeholders. To Test the requirements, business scenario needs to be identified and testing should focus on business scenario validations. Once UAT Team feels the product is as per agreed in requirements and there are no major defects in the product, the product can be provided to subset of targeted end users to know their views and feedback on the usability of the products before releasing it officially to the end users. It is important to know either of the two stages i.e. testing by UAT Team of client or subset of targeted end users can be skipped based on the product.

To understand this, beta testing by subset of the end users is very important for a shopping or social networking website. As it is very critical to know the actual end user needs and before releasing it to market. While if the application is an internal tool to be used by client itself, he can skip the beta testing.

Next Question is who the UAT testers are. UAT Testers are usually the subject matter expert with good in-depth knowledge of domain. UAT testers usually focus on real life scenario that can happen. For e.g. for a banking Lending application, it is important to have someone with prior experience in banking and specifically in Lending domain to drive the UAT testing and preparing plan for execution.

Once the UAT team completes User Acceptance testing on the product, it provides sign-off for the product to be released to production/Go-Live. And then it’s party time for all stakeholders involved in the application with finger-crossed for no defect identified in production. 

Signing off this post and publishing to end users of this blog. Feedback/suggestion must welcome.

Understanding System Testing

In this article, we will discuss on System Testing. To Understanding System Testing, we need to understand what a system is. Let us take our self away from software testing and go to Politics. In Politics, when we say System is corrupt, it means our political system, Administrative, and support system is corrupt. If a project comes with requirement to test this system as a whole, We need to test political, administrative and support system are working correctly and we need to validate also the integration between them is also working fine.

So now we have identified system and sub-systems. It is important to identify what all needs to be tested in system, i.e. we need to identify testing types required to test the system. Testing Types needs to be identified based on the requirement in hand.Testing type for a financial site and can be different from an e-commerce website.

While a financial domain will stress more on functional testing – The requirement is completely tested and the calculations and functionality is complete is utmost priority as a small mistake can lead to huge losses and impact the business value. Next priority can be security testing as to avoid loss to customer due to security functionality not matching. Performance Testing can or cannot be priority to the financial project. E.g. for a front –end system (e.g. Market Order Placement Website), Performance testing can be priority so as to ensure the user transaction is completed in time and issues like crashing do not take place, Whereas for a back-end system (where accounting and settlement happens in back-end for the transactions), Performance testing can be lower priority compared to various other testing types.

So now we know which testing needs to be done depends upon the system requirements and on basis of testing type, we identify the tools and resources required for the testing type.

• GUI Testing – Testing the graphical user interface.

• Usability testing – Testing whether the software developed is usable for the end users

• Performance testing – Testing how system performs in terms of responsiveness and stability

• Compatibility testing – Tests the compatibility of software with different environment conditions

• Exception handling – Tests different error conditions are properly handled.

• Load testing- tests the behavior of system under both normal and peak load conditions

• Volume testing – tests the system under varying volume of data.

• Stress testing - Tests robustness of system under stress condition.

• Security testing   - Test for Security features of the application.

• Scalability testing – tests whether the software is scalable.

• Sanity and Smoke testing – initial tests to identify basic tests are working and the software is good to continue further testing.

• Exploratory/ Ad-hoc testing – Performed mostly by experience tester without any proper documentation.

• Regression testing – Performed in the final phase of software testing to identify any defect fixes has not resulted in residual effect in rest of software.

• Installation testing – Testing the installing of software on different environment.

• Accessibility testing - Test the application meets the accessibility requirement

Also based on application type, the testing can vary. E.g.: Web Services testing for SOA Application, Localization testing for same application used across different geographies and so on. All of above testing types are a topic in itself.

Right, Once we are done with System testing, It means the software developer team( and this includes testers too) have developed and tested the application and feels the system is good enough to be delivered to the client once all the open defects are closed, or some minor issues exists which can be shared with client. Once system testing is completed, it is passed to the client to perform acceptance testing on the same to provide itself and release it to the required audience. So next topic will be around acceptance testing. Perhaps next weekend. Will keep you posted.

Understanding Integration Testing with example

In this article we will discuss on integration testing. Stage two in testing after component testing. We start integration testing once the individual components are tested and it is the time to validate if they fit in with each other.

Best example to understand importance of integration testing is the jigsaw puzzle, and you will know the importance far better, if you put in lot of effort in solving the puzzle and due to few incorrect pieces, you failed to complete the picture.

The small pieces of jigsaw puzzle can be considered as the module or component. When all the pieces in the puzzles fit in together perfectly, it creates the image. In software terms, it can be considered as the software system. Integration of different modules is important to fit in the pieces together. Below diagram shows the flow in testing as discussed in above explanation.

So now we have the basic knowledge of Integration testing. Let us discuss on the different standard approach for the Integration testing. Below are the different approaches for integration testing and will try to explain the same in plain language.

• Top down Approach – In this approach, the top modules is tested and integrated first. When we say the top modules are tested first. Two questions come in mind immediately.

• How to test the top module when the individual modules that will be integrated to form the top module are not developed yet?

• Answer to this, we use stubs for flow of data from top level to bottom level modules. Stubs acts as simulator to simulate behavior of lower level module which are not developed or integrated as yet. Expected Output from stub is the same as expected in case module would have been developed and integrated.

• What is the purpose of integration testing using top down approach? 

• Answer:  Using Top down approach allows high level logic and data flow to be tested early in the testing process. Testing the top module earlier helps to flow the logic from system to sub-module so that end result is in focus and information flow from top level to bottom.

 

• Bottom up Approach –If we represent Bottom up and top down approach in a pictorial representation, Bottom up approach is just reverse of Top down approach. In this approach, we test the bottom modules first and move to the top modules. Till the time top modules are ready, we use driver to simulate the system flow.

• Big Bang Approach – In this all the sub-modules are developed first and are integrated once all the module are developed instead of incremental approach as in above examples. Big Bang Approach can be used in case of smaller system. In Large system, the big bang approach probability of failing is very high.

We have discussed on what is integration testing, types of approaches in integration testing. Let us now summarize on integration testing.

• What is integration testing –Testing Integrating and Interaction of different modules or Interfaces in the system.
• Who Performs integration tests – Integration testing should be performed by developers before releasing the system to QA. Testers should also test the interaction between subsystems.
• Bottom up and top down are example of incremental approach to integration testing.

Are there any references for further study? While studying for integration testing and advanced concept on same. I find following two articles interesting. Please read through for further details on the topic

http://www.eecs.yorku.ca/course_archive/2009-10/W/4313/slides/14-IntegrationTesting.pdf

http://www.iai.uni-bonn.de/III/lehre/vorlesungen/SWT/OOSC05/slides/15%20-%20Testing%20(2-2).pdf