Defect Life Cycle which is also called as Bug Life Cycle. Below is what we cover in this post:
When the bug is posted for the first time, its state will be “NEW”. This means that the bug is not yet approved.
After a tester has posted a bug, the lead of the tester approves that the bug is genuine and he changes the state as “OPEN”.
Once the lead changes the state as “OPEN”, he assigns the bug to corresponding developer or developer team. The state of the bug now is changed to “ASSIGN”.
Once the developer fixes the bug, he has to assign the bug to the testing team for next round of testing. Before he releases the software with bug fixed, he changes the state of bug to “TEST”. It specifies that the bug has been fixed and is released to testing team.
The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software.
If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “REJECTED”.
If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”.
Once the bug is fixed and the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “VERIFIED”.
If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle once again.
Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”. This state means that the bug is fixed, tested and approved.
In this article, we will be covering following:
— Importance of the bug is known as Priority/Urgency to fix a fault.
— Impact of the bug is known as Severity /Impact of a failure caused by this fault.
— An item that prevents further testing of the product or function under test can be classified as Critical bug. No workaround is possible for such bugs. Examples of this include a missing menu option or security permission required to access a function under test.
— A defect that does not function as expected/designed or cause other functionality to fail to meet requirements can be classified as Major Bug. The workaround can be provided for such bugs. Examples of this include inaccurate calculations; the wrong field being updated, etc.
— The defects which do not conform to standards and conventions can be classified as Medium Bugs. Easy workarounds exists to achieve functionality objectives. Examples include matching visual and text links which lead to different end points.
— Cosmetic defects which does not affect the functionality of the system can be classified as Minor Bugs.
— The defect is an irritant which should be repaired, but repair can be deferred until after more serious defect have been fixed.
— The defect should be resolved in the normal course of development activities. It can wait until a new build or version is created.
— The defect must be resolved as soon as possible because the defect is affecting the application or the product severely. The system cannot be used until the repair has been done.
— An error which occurs on the basic functionality of the application and will not allow the user to use the system. (Eg. A site maintaining the student details, on saving record, if it doesn’t allow to save the record then this is high priority and high severity bug.)
— The spelling mistakes that happens on the cover page or heading or title of an application.
— If there is an application that crashes after multiple use of any functionality (example–save Button use 200 times then that application will crash). Means High Severity because application crashed but Low Priority because no need to debug right now you can debug it after some days.
In this article, we will be covering following:
QA Team start writing Test Scenarios & eventually Test Cases based on some input documents – Business Requirements Document (BRD), Functional Specification Document (FSD) and Technical Design Document (optional).
Ex:- BRD & FSD
Based on the two input documents, below is the list of high-level scenarios to test:
For each Test Scenario, you are going to have at least 1 or more Test Cases. So, include another column and write the Test Case IDs as shows below:
At this stage, Traceability Matrix can be used to find gaps. For example, you see that there are no test cases written for FSD section 1.2 in the above Traceability Matrix.
As a general rule, any empty spaces in the Traceability Matrix are potential areas for investigation. So a gap like this can mean one of the two things:
If it is scenario 1, it will indicate the places where test team needs to work more to ensure 100% coverage. In scenario 2, TM not just show gaps it also points to incorrect documentation that needs immediate correction.
Let us now expand the TM to include test case execution status and defects.
The below version of the Traceability Matrix is generally prepared during or after test execution:
This article isn’t only going to cover Test Cases Vs Test Scenarios but it will cover following:
Few years back, when I was working in an MNC and dealing with a testing project, I suggested my colleagues to document the test Scenarios instead of test cases. Can you imagine, their reaction amazed me as they all were staring at me all together, I could not argue. It was like I have made a big mistake in my life. Everyone’s opinion was same except mine. They prefer to follow the traditional method of writing Test Cases and not Test Scenarios. Those who are newbies in this field might get confuse with the word Test Scenarios and Test Cases. This article will clear all doubts about it by giving real time examples and explanation.
Few years later, I Switched to another company and handled a testing project which was based on searching and generating different reports with the help of different menu options. We in a team were discussing about the project and how we can proceed with that? Actually, client wanted that project early and we were having deadline to complete the project. Considering my last experience about suggesting the documentation of Test Scenarios, I kept quiet and thinking of other idea. Suddenly, one of my colleagues raised his voice that “We should prepare Test Scenarios in a document”. This time, everyone was quite satisfied with his idea and we further started preparing the Test Scenarios. The conclusion of both cases suggests that preparing the Test Scenarios and Test Cases depends on the urgency and requirement of the project. Generally, in companies, test cases are prepared rather than documenting test scenarios.
As we all know nothing is permanent and with this concept, the software industries and their processes are also changing with time and cost constrains.
V-models and waterfall models which are considered to be traditional models are being replaced by iterative and agile models. In every testing project, document is indeed essential but to complete the project fast within the deadline and to make the process transparent and easy, the way of documentation is changed these days.
One of my experience will give you clear picture.
I was handling one of the testing projects from fortune 500 company and they didn’t give us any deadline to complete the project. That mean we were having flexible timelines to complete the project. We were already having template of test cases, and with the help of that we prepared the test cases and it got approved from client as well.
Once the development team finished their module, they were giving us to test it and most of our duty was to follow 100 test cases in a day. We used to document with pass/fail result, further sending it to the client at the end of the day. The project was really big and had different modules within it. To do the same process of documenting the test cases was a monotonous work for some of my colleagues. But at the end of the day, the company was generating revenue from it.
During the project, we had one break where we were not having any task of documenting the test cases. We were having a good time and I was discussing about new ideas and techniques so as make improvement in the existing test cases. I have found during that discussion that none of my colleague was interested in implementing those new ideas and techniques as they thought all the test scenarios is been covered already while making huge amount of test cases. No one was really interested in putting the efforts for new techniques as it a human mentality that we do not want to rework once the work is done. Our mind automatically stops making any efforts for any new ways and techniques for the past work. Same had happened with my colleagues and they just wanted to relax on that certain day.
It is also a fact that in IT companies, most of the testers follow the mechanical process of making the test case and no one really makes effort to make additional test case for the existing one? The same might have also happened with you, if I am not wrong.
While extensively involved in a team challenge activity, we were having a discussion and asked the team members to prepare the test scenarios for the product. We have all started to make the test scenarios and there was not a specific document where we would literally fill expected result and pre-condition for each test case. At the end of the day, we have collected almost 50 test scenarios and it was indeed a great experience while proceeding with that project.
Assume that you are having your website and it has features like username, password, and login page along with cancel button. If you were asked to write the test cases for the above features then your test case may exceed 50 and if you would ask to write the test scenarios, then it will be just a matter of lines. Below explanation will give you the exact idea about test scenario.
High Level Scenario: Login Functionality
Low Level Scenarios:
As we all are at short of time, test scenarios just acts as pain killer spray rather than that old time IODEX. And still the effect is same in both the case.
Finally, I would like to summarize difference as below:
|What it is =>||A concept which offers complete info what to test, steps to be taken and expected result of the same||A concept which offers one-line info about what to test.|
|It’s about =>||It’s more about documenting particulars.||It’s more about rational and conversing particulars.|
|Importance =>||It’s significant when testing is off shored and development is onsite. Writing test cases with particulars will aid both dev and QA team in sync.||It’s significant when time is less and most of the team members can agree / appreciate the details from one-liner scenario.|
|Advantage =>||One time documentation of all the test cases is helpful to track 1000s rounds of regression testing in future.
Most of the time, it is helpful while bug reporting. Tester just need to give reference of test case ID and does not require mentioning each and every minute detail.
|A time saver and idea generation activity, favoured by new generation software testing community.
Modification and addition is simple and not specific to a person.
For a huge project, where group of people know specific modules only, this activity gives a chance to everybody to look into other modules and brain storm and deliberate.
|Beneficial to =>||A full-proof test case document is a life line for new tester.||Good test coverage can be attained by dividing application in test scenarios and it decreases repeatability and complexity of product|
|Disadvantage =>||Time and money consuming as it requires more resources to detail out everything about what to test and how to test||If created by exact person, the reviewer or the other user might not sync the precise idea behind it. Need more discussions and team efforts.|
Test cases are most significant part of Software Testing Life Cycle which is also referred as STLC. With its absence, it’s tough to understand, track, follow and reason out something. But in the era of Agile, test cases are being substituted fast with test scenarios.
Database testing, GUI testing, functionality testing has the common test checklist which is coupled with modern artillery and test scenarios. With participation in trainings, Discussions, questions and practice one can definitely learn and change the final graph of Bug report matrix and your productivity.
Following sample bug/defect report will give you precise clue of how to log bugs in a bug tracking tool. You can use any bug tracking tool for tracking the bug for your reference.
Let’s say you have built a new application. Now, you want to create a new user with new information in the application. To achieve this, you should login and then navigate to USERS menu>New User and under that fill all the required details such as First Name, Last Name, Address, Age, Phone etc. After entering all the details you should save that info by clicking SAVE button. After this, you will get the message “New User has been created successfully”. This is the exact process that you should follow for creating new user in your website.
Log in and then navigating to USERS menu > New User you save all the info by clicking the SAVE button and BANG! You get an error message and system suddenly crashes down. At that moment, you should save that error message by capturing it and saving it into Microsoft paint file or wherever you feel comfortable.
This is actually a Bug scenario for a tester and he now needs to log that BUG into bug tracking tool. Different tools are used for tracking the BUG and some of them are JIRA, Bugzilla, IBM Rational Clear Quest etc.
Below is the sample bug report for above stated example :
(Note : ‘bug report’ fields vary depending on Bug Tracking Tool)
Bug Name: Application crashed while clicking the SAVE button for creating a new user.
Bug ID: (It will be automatically created by the BUG Tracking tool once you save this bug)
Area Path: USERS menu > New Users
Build Number: Version Number 3.0.4
Severity: HIGH (High/Medium/Low) or 1
Priority: HIGH (High/Medium/Low) or 1
Assigned to: Developer-X
Reported By: Your Name
Reported On: Date
Status: New/Open/Active (Depends on the Tool you are using)
Environment: Windows 10/SQL Server 2008
Application crashed on clicking SAVE button for creating a new user, henceforth not able to create a new user in the application.
Steps To Reproduce:
1) Logon into the application
2) Navigate to the Users Menu > New User
3) Filled all the user information fields
4) Clicked on ‘Save’ button
5) Seen an error page “OWX847 Exception: Insert values Error…”
6) See the attached logs for more information (Attach more logs related to bug.. IF any)
7) And also see the attached screenshot of the error page.
Expected result: On clicking SAVE button, it should give a success message “New User has been created successfully”.
(Attach ‘application crash’ screen shot. IF any)
Save the defect/bug in the BUG TRACKING TOOL which you are using. There, you will get a bug id that can be used further as a Bug reference.
Default ‘New bug’ mail will go to particular developer and the default module owner (Team leader or manager) for further action.
This article is all about opting Automation Testing. But every tester who hasn’t yet learnt automation testing or never faced any need of automation might have a question that Why to Learn Automation Testing?. So, here we go:
In today’s rapid and changing day to day life, technology is going so far and challenging human mind to take risks so as to improve the quality and efficiency of software systems development. Considering the time and cost constraints, testing is neglected most of the time in an IT industry. This eventually affects the quality of the product which is further followed by customer disappointment and dissatisfaction. This also affects the overall quality costs of the project. For this reason, it is extremely important that testing should be carried out so as to improve the quality of the project along with customer satisfaction.
We can do testing of the application manually or by automation. If you are preferring manual testing then you have to do the monotonous work of writing test cases daily and maintain it as per the changing requirements. It is really a tedious work and requires lot of efforts to sit in front of computer continuously while getting bored at the same process. If the project is big and requires more test cases then manual tester’s life is hell as he fails at maintaining proper test cases. Here comes the role of Automation testing where you can actually automate your testing process hence increasing the efficiency of the complete testing process.
With Test Automation, you can improve the testing process of any software application. Initially, you may need to pay attention to the automation process but later you can thoroughly enjoy its results. Main benefit of automation testing is you can execute your test cases during night so that till morning you are done with the testing process.
Automated tests re-use modules within different tests and aids in fast running of the tests with great performance. If requirements are frequently changing in the project then you need to do the testing again and again to ensure the quality of the software product. Automation testing plays key role in this and provides great results on automation of regression tests. This saves time and benefits the company while quickly out-gaining the initial costs. This is the main reason that automation testing has overcome manual testing over past few years.
Another benefit with automation testing is that it increases the test coverage in wide scale. Manually covering the test case is very laborious and time consuming. With automation tests, you can achieve sufficient test coverage of your software projects.
With Test automation, you can actually perform different kinds of testing effectively and efficiently.
If you want long maintenance of your software products then you should automate your tests as it is very fast and frequent. It also decreases cost while enhancing quality of your software product. Most of the companies now work in agile environment where requirements change frequently and automation testing is better option to react with these ever changing requirements. The best part is new test cases are continuously generated which can be further added to existing automation in parallel to the development of the software.
Your test cases need to be extended for longer periods of time whether in manual or automation testing environments. This is indeed essential because software project continuously progresses and maintaining test cases is extremely important. You can achieve complete coverage of your test cases with automation as it is very reliable and realistic. Value of the test case vs the effort to create them should be considered while deciding which tests to automate first. Test cases which have low effort and high value should be considered first for automating. Moreover, frequently used test cases, regression test cases; along with test cases having low to moderate effort in setting up the development and test environment of the automation project are best matched for automation.
Software requirements are indeed essential for test estimation as it is based completely on what would be tested. The development team has much involvement in the software requirements as compared to the testing team and the requirements are mostly established by the development team. When all the estimation are done related to the project cost, duration of the project then the development team asks the testing team that how much time will be required by them in order to complete the testing of the application. The answer given by the testing team is considered for the SDLC process.
Testing team plays a vital role in estimation of the project because testing is an essential part of SDLC. Testing application, one can deliver a bug free application to the client.
The testing team classifies the requirements before estimating them in the following categories:
The expert opinion in each requirement suggests how long it would take for testing them. The categories would eventually help the experts in estimating the effort for testing the requirements.
Previous project knowledge and experience are used for the estimation of the project. If new project is introduced then its estimation is based on similar projects which have been executed previously.
Organization should create an Organization Process Database which is also referred as OPD, where the project metrics are recorded and referred for future projects.
Spreadsheets and Old process is still used by the testing team in order to estimate the project completely. If new rules are introduced during the process then the testing team follows that and at the same time compares the old process too so as to get complete idea in test Estimation.
The results which come from the new estimate process should be faster and cheaper as compared to the old process and in percentage of 20 to 20%. If different percentage are measured by the testing team then they go back to the old process to check whether they are missing something during the process or not.
During testing of the process all the results should be recorded in order to refer it for future projects. It is also beneficial when requirements of the projects change and hence the recording will eventually help the testing members in estimating the project. This will help testing team to not return all the steps again and will further help in taking same decisions. It also gives the opportunity to test the estimation before making any changes to it.
A spreadsheet containing metrics eventually helps reaching the estimation as quickly as possible. The spreadsheet will calculate it automatically hence giving ease to the team members. It also helps in tracking the duration and time of the projects.
Risk Table, and free notes should be included in the template and need to be filled in the tool. These tools also help in showing different options for testing which actually aids the customer in deciding which types of test he needs practically.
All estimation should be verified in a certain period of time. Along with this, another spreadsheet for recording the estimations should be created. The estimation is compared to the previous ones recorded in a spreadsheet so as to see if they have similar trend. If the estimation has any deviance from the recorded ones, then it is indeed essential to a re-estimation of the entire project.
Product down time, resource unavailability, skill improvement and learning capability are some of the risks that should be covered in test estimation.
Assume you have a project and you have estimated that it will take a 10 person team 300 days to complete. (3000 workdays – Sunny day scenario)
Knowing that there are 365 working days in a year (52 weeks x 5 days), without holidays, means that the project, at best, will take 1.13 years or 13.6 months to complete.
Suppose the average year includes 12 days company holidays (total impact = 12×10 people = 120 daysx1.13years = 136 days) and average employee takes 4 sick days and 3 comp days off a year (total impact = 7 x 10 people = 70 x 1.13 = 79)
Average employee takes 7 work days of vacation a year (total impact = 7 x 10 people = 70 x 1.13 = 79)
Suppose also that each employee drops 3 days out of each year for training (30 days)
Total duration change due to employee holidays, sick time, vacation and training = 136+79+79+30=324
Total workdays for the project is now 3000+324 = 3324 or a 10.8% increase in duration (300 day schedule is now 333 days or 15.1 months.)
If you want to be successful in Project Test Estimation then you should have execution knowledge which is eventually very significant in Software Testing Life Cycle. Software Test Estimation Techniques are indeed essential to make good reputation with your client when you bid for testing projects. It also gives you idea that how your approach should be during the execution of the project.
In case, you have experience in various software testing life cycle then chances are higher that you estimate your project very well as your experience may help you in providing various ideas & techniques for that testing project. It is practically not possible to put some number of days or estimate with old time formula of one third of the development effort. The requirement keeps on changing from client side & we have to adjust the same accordingly. Some companies which offer testing services use this formula which isn’t correct as it is not based on any scientific techniques or principles.
Following factors are indeed essential for any software test estimation and they are highly recommended in any software testing organization.
Many techniques are used in recent years that have been developed for test estimation of the software.
Use-Case Point Method is completely based on the use cases that we calculate the un-adjusted actor weights to determine the software test estimation.
Use case is a document which well specifies systems, different users or other stakeholders that interact with the concerned application. They are certainly named as ‘Actors’. The interactions achieve some defined goals hence protecting the interest of all stakeholders via flow termed or different behavior as scenarios.
The formula used for this technique is:
3-Point Software Testing Estimation Technique is completely based on statistical methods where each testing task is broken down into sub tasks and further into three types on each tasks.
The formula used by this technique is:
Test Estimate = P + (4*N) + E / 6
Whereas P = Positive Scenarios or Optimistic Estimate (Best case scenario in which nothing goes wrong and all conditions are optimal.)
N = Negative Scenarios or Most Likely Estimate (most likely duration and there may be some problem but most of the things will go right.)
E = Exceptional Scenarios or Pessimistic Estimate (worst case scenario which everything goes wrong.)
Standard deviation for the technique is calculated as,
Standard Deviation (SD) = (N – E)/6
This method is created by breaking down the test project into small pieces. Modules are separated into sub-modules. Sub modules are then further separated into functionalities and functionalities are then separated in sub-functionalities.
You should review all the requirements from Requirement Document so as to make sure that they are added in WBS. Later, figure out the number of tasks your team needs to complete. Estimate properly the duration of each task.
Following the same procedure as above in WBS. In this, work breakdown structure is disintegrated for each task and further distributed to a team comprising of say 3-7 members so as to re-estimate the task. The final estimate is the consequence of the brief estimates that are based on the team consensus. This method actually speaks more on experience instead of any statistical formula. Moreover, this method is initially popularized by Barry Boehm to emphasize on the group iteration so as to reach to a consensus where the team visualize on the different aspects of problems whereas estimating the test effort.
The Function Point technique is a direct pointer of the functionality of software application from the user’s viewpoint. This technique is the most accepted technique that is used to estimate the size of a software project.
This technique is also a part of TMap. In function point technique, function points are converted into test points. In Test Point analysis, we carry out following:
In Software Testing, estimation is generally based on previously created application prototype or requirement specification document. To calculate FP for a project, some of the major components are required.
The major components required are:
Unadjusted Data Function Points:
Unadjusted Transaction Function Points:
Number of Test cases = [Number of Function Points] x 1.2
Total Actual Effort, TAE = (Number of Test cases) * (Percentage of development effort /100)
This method is used when requirement document or a detailed low level design document is available. Previous data for development and testing & measure of function point is available.