Understanding System Testing

In this article, we will discuss on System Testing. To Understanding System Testing, we need to understand what a system is. Let us take our self away from software testing and go to Politics. In Politics, when we say System is corrupt, it means our political system, Administrative, and support system is corrupt. If a project comes with requirement to test this system as a whole, We need to test political, administrative and support system are working correctly and we need to validate also the integration between them is also working fine.

So now we have identified system and sub-systems. It is important to identify what all needs to be tested in system, i.e. we need to identify testing types required to test the system. Testing Types needs to be identified based on the requirement in hand.Testing type for a financial site and can be different from an e-commerce website.

While a financial domain will stress more on functional testing – The requirement is completely tested and the calculations and functionality is complete is utmost priority as a small mistake can lead to huge losses and impact the business value. Next priority can be security testing as to avoid loss to customer due to security functionality not matching. Performance Testing can or cannot be priority to the financial project. E.g. for a front –end system (e.g. Market Order Placement Website), Performance testing can be priority so as to ensure the user transaction is completed in time and issues like crashing do not take place, Whereas for a back-end system (where accounting and settlement happens in back-end for the transactions), Performance testing can be lower priority compared to various other testing types.

So now we know which testing needs to be done depends upon the system requirements and on basis of testing type, we identify the tools and resources required for the testing type.

• GUI Testing – Testing the graphical user interface.

• Usability testing – Testing whether the software developed is usable for the end users

• Performance testing – Testing how system performs in terms of responsiveness and stability

• Compatibility testing – Tests the compatibility of software with different environment conditions

• Exception handling – Tests different error conditions are properly handled.

• Load testing- tests the behavior of system under both normal and peak load conditions

• Volume testing – tests the system under varying volume of data.

• Stress testing - Tests robustness of system under stress condition.

• Security testing   - Test for Security features of the application.

• Scalability testing – tests whether the software is scalable.

• Sanity and Smoke testing – initial tests to identify basic tests are working and the software is good to continue further testing.

• Exploratory/ Ad-hoc testing – Performed mostly by experience tester without any proper documentation.

• Regression testing – Performed in the final phase of software testing to identify any defect fixes has not resulted in residual effect in rest of software.

• Installation testing – Testing the installing of software on different environment.

• Accessibility testing - Test the application meets the accessibility requirement

Also based on application type, the testing can vary. E.g.: Web Services testing for SOA Application, Localization testing for same application used across different geographies and so on. All of above testing types are a topic in itself.

Right, Once we are done with System testing, It means the software developer team( and this includes testers too) have developed and tested the application and feels the system is good enough to be delivered to the client once all the open defects are closed, or some minor issues exists which can be shared with client. Once system testing is completed, it is passed to the client to perform acceptance testing on the same to provide itself and release it to the required audience. So next topic will be around acceptance testing. Perhaps next weekend. Will keep you posted.

Understanding Integration Testing with example

In this article we will discuss on integration testing. Stage two in testing after component testing. We start integration testing once the individual components are tested and it is the time to validate if they fit in with each other.

Best example to understand importance of integration testing is the jigsaw puzzle, and you will know the importance far better, if you put in lot of effort in solving the puzzle and due to few incorrect pieces, you failed to complete the picture.

The small pieces of jigsaw puzzle can be considered as the module or component. When all the pieces in the puzzles fit in together perfectly, it creates the image. In software terms, it can be considered as the software system. Integration of different modules is important to fit in the pieces together. Below diagram shows the flow in testing as discussed in above explanation.

So now we have the basic knowledge of Integration testing. Let us discuss on the different standard approach for the Integration testing. Below are the different approaches for integration testing and will try to explain the same in plain language.

• Top down Approach – In this approach, the top modules is tested and integrated first. When we say the top modules are tested first. Two questions come in mind immediately.

• How to test the top module when the individual modules that will be integrated to form the top module are not developed yet?

• Answer to this, we use stubs for flow of data from top level to bottom level modules. Stubs acts as simulator to simulate behavior of lower level module which are not developed or integrated as yet. Expected Output from stub is the same as expected in case module would have been developed and integrated.

• What is the purpose of integration testing using top down approach? 

• Answer:  Using Top down approach allows high level logic and data flow to be tested early in the testing process. Testing the top module earlier helps to flow the logic from system to sub-module so that end result is in focus and information flow from top level to bottom.

 

• Bottom up Approach –If we represent Bottom up and top down approach in a pictorial representation, Bottom up approach is just reverse of Top down approach. In this approach, we test the bottom modules first and move to the top modules. Till the time top modules are ready, we use driver to simulate the system flow.

• Big Bang Approach – In this all the sub-modules are developed first and are integrated once all the module are developed instead of incremental approach as in above examples. Big Bang Approach can be used in case of smaller system. In Large system, the big bang approach probability of failing is very high.

We have discussed on what is integration testing, types of approaches in integration testing. Let us now summarize on integration testing.

• What is integration testing –Testing Integrating and Interaction of different modules or Interfaces in the system.
• Who Performs integration tests – Integration testing should be performed by developers before releasing the system to QA. Testers should also test the interaction between subsystems.
• Bottom up and top down are example of incremental approach to integration testing.

Are there any references for further study? While studying for integration testing and advanced concept on same. I find following two articles interesting. Please read through for further details on the topic

http://www.eecs.yorku.ca/course_archive/2009-10/W/4313/slides/14-IntegrationTesting.pdf

http://www.iai.uni-bonn.de/III/lehre/vorlesungen/SWT/OOSC05/slides/15%20-%20Testing%20(2-2).pdf

Component Testing – First Testing in the Project

The first phase is testing the smallest testing unit developed in the Project. A unit can be defined as an isolated piece of code which can be tested independently.Let us take an example, suppose we have to develop an application for an e-commerce site with following requirement.

• Login into System.
• Home Page is displayed.
• Selecting any product and adding to my cart.
• Paying for the Product.

So In this example, we can divide the project development into following components or units.

• Login Module.
• Home Page Module.
• Transaction Page.
• View Cart Module.
• Payment Module.

Before discussing on different testing type. Let us discuss how the development flow takes place. 

• Development team will develop different modules or units of code.
• The modules should be integrated successfully with rest of the modules
• The System developed should be integrated with third party components or existing components.
• The system should work as per requirement document.

As one of the principle of software testing, defects should be captured as early as possible in the software development life cycle. Since the first step in testing is testing the individual module developed. If the individual module has defect, fixing the same can be painful if captured in later stages of SDLC.Unit testing searches for defects in the testable unit or module as discussed. We need to focus on testing the functional features as well as non-functional characteristics.

Let us take the example of unit testing the login module. A login Page needs to be tested as shown below.

The functional test case for the unit will include:

• Error Message on providing invalid username or Password.
• Error Message for minimum and maximum allowed characters.
• On providing successful username/Password, Home Page is displayed. 
• Since Home Page is not yet created, we will use stubs and drivers to dummy for home Page.

The above are examples of functional test cases for components.

.

Component testing also includes non-functional characteristics including memory leaks, and structural testing for code coverage. Component testing is done mainly by Developers and is mostly white box testing specially for verification of non-functional validation. Defects are fixed as they are found as is mostly development defect uncovered during dev testing itself.

There are various tools and framework available for unit testing. Junit, testNG are example of framework available for unit testing. Another framework is Test Driven Framework, in which test scripts are prepared before coding. The tests are expected to fail before coding phase. Once Coding is done, all the unit tests are expected to pass. In case, test failing post development, code needs to be fixed until all the tests created for component passed.

Summarizing, all we discussed so far.

Component testing is testing the unit or component or module as an independent unit.
Component Testing is the first testing performed in the project.Component testing is performed mainly by developers.Component testing is more of white box testing on the code.Different Unit testing frameworks are available in the project.

Some Excellent reference article for detailed understanding of Component testing are as follows:

http://www.testingstandards.co.uk/Component%20Testing.pdf

http://people.cs.clemson.edu/~johnmc/joop/col3/column3.html