Friday, 30 November 2012

7 Major Challenges of Successful User Acceptance Testing (UAT) and How You Can Overcome These

What is User Acceptance Testing (UAT)?

UAT is final testing performed when functional, system and regression testing is completed. The main purpose of UAT is to validate the software against the business requirements. This validation is carried out by end users who are familiar with the business requirements. UAT, alpha and beta testing are different types of acceptance testing.

As user acceptance testing is the last testing carried out before the software goes live, obviously this is the last chance for the customer to test the software and measure if it's fit for the purpose.

UAT testing

Need of User Acceptance Testing

Developers and functional testers are technical people who validate the software against the functional specifications. They interpret the requirements according to their knowledge and develop/test the software (here is the importance of domain knowledge). This software is complete according to the functional specifications but there are some business requirements and processes which are known to end users only are either missed to communicate or misinterpreted. UAT plays an important role in validating if all the business requirements are fulfilled or not before releasing software for the market use. Use of live data and real use cases make UAT testing an important part of the release cycle.

Many businesses who suffered big losses due to post-release issues know the importance of successful UAT. The cost of fixing defects after release is many times greater than fixing it before.

Who should be part of the UAT team?

Mainly end users of the software should be involved in UAT. The team can be comprised of beta testers or customer should select UAT members internally from every group of organization so that each and every user role can be tested.

7 major challenges of UAT and mitigation plan to overcome these

It doesn't matter if you are part of a billion dollar release or a startup team, you should overcome these UAT challenges for delivering successful software for the end user.

1) UAT environment setup and deployment process

Carrying out UAT on same environment used by functional test team will certainly end up overlooking real world use cases. Also crucial testing activities like performance testing can't be carried out on test environment with incomplete test data. Separate production like environment should be setup for UAT.

Once the UAT environment is separated from test environment you need to control the release cycle effectively. Uncontrolled release cycle may leads to different software versions on test and UAT environment. Valuable UAT time is wasted by not testing software on latest version. Not to mention the time required for issue tracking on incorrect software version.

2) UAT test planning

UAT should be planned with clear acceptance test plan in requirement analysis and design phase. In strategy planning, set of real world use cases should be identified for execution. It is very important to define test objectives for UAT testing as complete test execution for large application is not possible in UAT phase. Testing should be carried out by prioritizing critical business objectives first.

UAT is carried out at the end of the testing cycle. Obviously it is the most critical period for the software release. Delay in any of the previous stages of development and testing eat up UAT time. Improper test planning, in worst cases, leads to overlap between system testing and UAT. Due to less time for UAT and pressure to meet deadlines, software is deployed to UAT environment even though functional testing is not completed. UAT goals can't be achieved in such situations.

UAT test plan (template) should be prepared and communicated to team well before beginning UAT. This will help them for test planning, writing test cases and test scripts and creating UAT environment.

3) Handling new business requirements as incidents/defects

Ambiguities in requirements get caught in UAT phase. UAT testers find issues arising due to ambiguous requirements (by looking at the complete UI which wasn't available during requirement gathering phase) and log it as a defect. Customer expects these to be fixed in current release without considering the time for change requests. If timely decision is not taken by project management on these last-minute changes this could lead to the release failure.

4) Unskilled testers or testers without business knowledge

If there is no permanent UAT test team, company select UAT staff from various internal departments. Even if this staff is well familiar with business needs, if they are not trained for the new requirements being developed they can't perform effective UAT. Also non-technical business team might face many technical difficulties executing UAT test cases. Also assigning testers at the end of the UAT cycle do not add any value to the project. Little time to train UAT staff can significantly increase the chances of UAT success.

5) Improper Communication channel

Communication between remote development, testing and UAT team is more difficult. Email communication is often very difficult when you have an offshore tech team. A small ambiguity in incident reports can delay its fix for a day. Proper planning and effective communication are critical to effective team collaboration. Project teams should use web based tool to log defects and questions. This will help to distribute work load evenly and avoid reporting duplicate issues.

6) Asking functional test team to perform UAT

There is no worse situation than asking functional test team to perform UAT. Customers offload their responsibility to test team due to lack of resources. The whole purpose of UAT gets compromised in such cases. Once the software goes live, end users will quickly spot the issues which are not considered as real world scenarios by functional testers. Solution to this is to assign UAT to dedicated and skilled testers having business knowledge.

7) The blame game

Sometimes business users just try to find reasons to reject the software. It might be their selfdom to show how superior they are or blame the development and testing team to get respect in business team. This is very rare but happens in teams with internal politics. It's very difficult to deal with such situations. Building positive relationship with business team would definitely help to avoid blame game.

What to be automated and what to be not

What to be automated and what to be not

Automation of whole application, product or process is not possible or one can say it is near to impossible to achieve 100% automation. So it is important to decide which test cases should be automated on High priority.

The main idea of Automating the test is to save the time of manual resource and how to increase the test coverage. So one should determine the test cases which are repeated very often and taking large amount of data on the same set of actions like login of a screen. There could be many combination of testing Login screen with different set of data, which takes very much time of manual tester.
One should not automate the Test cases which take most of the Automation test engineer effort and time and in future that test script usage is very less.

One can get the most benefit out of your automated testing efforts by automating:

• Repetitive tests that run for multiple builds.
• Tests that tend to cause human error.
• Tests that require multiple data sets.
• Frequently used functionality that introduces high risk conditions.
• Tests that is impossible to perform manually.
• Tests that run on several different hardware or software platforms and configurations.
• Tests that take a lot of effort and time when manual testing
When trying to automate the test cases, always try to make the modules independent and focused on an objective. By using this strategy, one can reuse the independent modules to run the test repeatedly

When to be automated:

Testing of a product or application is not sufficient in a single round as developer may change their code due to some previous bug fix or due to new enhancements to application. In each and every build we need to test the same functionality number of times. Lot of manual effort is involved. To save the time and effort of manual effort, we will automate the test cases.

If we encounter the following question, then it's perfect time that we automated the test cases:

What if a programmer then changes the code and introduces a bug? What if I don't catch that bug because I didn't rerun the test after the change?

There are some scenarios in which one should think of automating the test:

• Both automation and manual testing are plausible. That's not always the case. For example, load testing often requires the creation of heavy user workloads. Even if it were possible to arrange for 300 testers to use the product simultaneously, it's surely not cost-effective. Load tests need to be automated.

What Technology should be used for Automation?
For some Automation tools they used their own Scripting language like Rational Robot use SQABasic. Now a days there are so many tools they used global Scripting languages like RFT use .NET and Java.
Selenium used Ruby, Python, java,.NET etc.

Choice of Automation tool?

Selecting an automated testing tool is essential for test automation. There are a lot of automated testing tools on the market, and it is important to choose the right automated testing tool that best suits your overall requirements.

Consider these key points when selecting an automated testing tool:

• Support for your platforms and technology. Are you testing .Net, C# or WPF applications and on what operating systems?
• Flexibility for testers of all skill levels. Can your QA department write automated test scripts or is there a need for keyword testing?
• Feature rich but also easy to create automated tests. Does the automated testing tool support record-and-playback test creation as well as manual creation of automated tests; does it include features for implementing checkpoints to verify values, databases, or key functionality of your application?
• Create automated tests that are reusable, maintainable and resistant to changes in the applications UI. Will my automated tests break if my UI changes?
• Support for Cross Platform or cross OS test support of tool
• Good Support with the AUT.
• Testing Multi-Language Applications
• Automated Testing Tool: Support for Team Work
• Command Line and OLE Automation Support
• Integration with Team Systems and Build Software
• Technical Support
• Pricing Policy
• Free Trial Versions

Now once the tool gets selected, the primary focus of Automation test Engineer is to setup the Automation framework. It should be Tool Independent. One can create Automation framework in ASP.Net or java. Framework should be user friendly and independent of AUT. One Automation framework could be used for multiple Applications. As in our product Eloquence we are using ASP.Net for our Automation framework Webpage. This is completely independent of Automation Tool.

ETL Testing – Basics

ETL Testing – Basics


ETL basically stands for Extract Transform Load – which simply implies the process where you extract data from Source Tables, transform them in to the desired format based on certain rules and finally load them onto Target tables. There are numerous tools that help you with ETL process – Informatica, Control-M being a few notable ones.

In ETL Testing, the following are validated:

  • Data File loads from Source System on to Source Tables
  • Transform Process that is designed to extract data from Source tables and move them to Staging tables
  • Data Validation of all Mapping Rules/Transformation Rules within the Staging tables
  • Data Validation within Target tables to ensure data is present in required format and there is no data loss from Source to Target tables

So ETL Testing implies – Testing this entire process using a tool or at table level with the help of test cases and Rules Mapping document.

Typically, data that is loaded into a data warehouse is derived from diverse sources of operational data, which may consist of data from databases, feeds, application files or flat files. The data must be extracted from these diverse sources, transformed to a common format, and loaded into the data warehouse. Typically, it is further aggregated into a data mart for efficient reporting. The ETL (Extract, transform and load) process is a critical step in any data warehouse implementation, and continues to be an area of major significance whenever the ETL code is updated.

An effective data warehouse testing strategy focuses on the main structures within the data warehouse architecture:

1. The ETL layer

2. The data warehouse itself

3. Associated data marts

4. The front-end business intelligence/reporting applications

ETL Testing is categorized into four different engagements:

  • New Data Warehouse Testing- a new data warehouse is built from ground up, gathering inputs from customer, extracting different data sources.  This is verified with the help of ETL tools
  • Migration Testing – In this type of engagement, migrating from the current ETL tool to a better option to improve efficiency
  • Change Request – In this type of project new data is added from different sources to an existing DW. Also, there might be a condition where customer needs to change their existing business rule or they might integrate the new rule.
  • Report Testing- Validating report layout, data in the report and calculation

 

ETL Testing Challenges

Some of the ETL Testing Challenges include:

  • Environment Instability
  •  Response time from the query executed, the failure of the jobs, the data set up required for the FIT testing, volume testing
  • Data selection from multiple source systems and analysis that follows pose great challenge
  • Volume and the complexity of the data
  • Inconsistent and redundant data in a data warehouse
  • Inconsistent and Inaccurate reports
  • Non-availability of History data

Saturday, 17 November 2012

Security Testing

Security testing is performed to ensure that the software is safe and secure by checking whether there is any information leakage in the sense by encrypting the application or using wide range of software's and hardware's and firewall etc. This testing makes sure that the software is not easily hacked by malicious codes and helps software developers to identify and remove loopholes in the software and to ensure that the system will not be attacked by hackers and other third party intruders.


Objective of Security Testing

The objective of security testing is to ensure that adequate attention is provided to identify the security risks, to ensure that a realistic mechanism to define and enforce access to the system is in place, to ensure that sufficient expertise exists to perform adequate security testing and to conduct reasonable tests to confirm the proper functioning of the implemented security measures.

Who should do the Security Testing?

During security testing, testers may need to use a risk-based approach and can easily focus on software security assurance by finding out risks in the systems and designing tests by those risks. Irrespective of the type of testing, the testing engineers plan and conduct security testing who should have significant security and networking related knowledge including expertise in network security, firewalls, intrusion detection system, operating systems, and programming and networking protocols like TCP/IP.

Types of Security Testing

There are different types of Security Testing. They are as follows:

Security Auditing: 


Security Auditing means direct inspection of the application developed and Operating Systems and any system on which it is being developed. This also involves code walk-through. This test just looks at the overall hardware security and software security of the system.

Security Scanning: 


Security scanning involves scanning and verification of the system and applications. 

Vulnerability Scanning: 


Vulnerability scanning involves scanning of the application for all known vulnerabilities including hosts and open ports and it also provides information on the associated vulnerabilities. This scanning is generally done through various vulnerability scanning software. 

Risk Assessment:
 


Risk assessment is a method of analyzing and deciding the risk, which depends upon the type of loss and the possibility/probability of loss occurrence. It helps in finding out and preparing possible backup-plan for any type of potential risk, hence contributing towards the security conformance.

Ethical Hacking:


 It is a forced intrusion of an external element into the system and applications that are under Security Testing. Ethical hacking involves number of penetration tests over the wide network on the system under test.

Posture Assessment and Security Testing: This is a combination of Security Scanning, Risk Assessment, and Ethical Hacking in order to reach a conclusive point and help your organization know its stand in context with Security.

Penetration Testing: 


Penetration testing will give you an opportunity to make valid assessments to find out if the software is effective or not especially against hacking attempts. In this type of testing, a tester tries to forcibly access and enter the application under test. In the penetration testing, a tester may try to enter into the application/system with the help of some other application or with the help of combinations of loopholes that the application has kept open unknowingly. Penetration test is highly important as it is the most effective way to practically find out potential loopholes in the application. Penetration testing can be either black hat testing, internal pen test, or external pen test.
Software Security Concerns

Creating software is not similar to securing it with a firewall at a later stage. We must build it along with the development of software and inside it. 
Building software safe and secure depends on three issues: People, technology, and process.
People should have sufficient skills and knowledge about how to test the software application and compose a report.
The people involved in the Process should know the procedures they must adapt to keep the software safe and effective.
Technology will ensure the effectiveness of the software's implementation. It will also use the necessary security features in the development framework.
The development framework should have the following activities to help thesoftware become more secure: 



•    Authentication. 
•    Session management.
•    Authorization.
•    Data validation.
•    Data protection.



The conclusion is that securing applications can ensure system safety and security, which can impede attacks by hackers. This testing is the most important tests that we should conduct before introducing it to the commercial domain.