Test Management

14. Defining and using Test Metrics

We are going to talk about how to define and how to use the test metrics in this particular section. So first of all, let's try to understand the different types of metrics and their content. So, the classification of test metrics can be like this. It is divided into four categories: project metrics, product metrics, process metrics, and people metrics. Metrics for the project: it measures progress, established exit criteria, percentage of testcases executed, passed, failed, and so on. When it comes to product metrics, they measure attributes of products like the extent to which a product has been tested or defect density. Process Metrics each measure the capability of process-like testing or development, percentage of defects detected, defect density, et cetera. People Metrics measures the capability of individual or group implementation of testcases within a given schedule. At any given time, a metric may belong to more than one or all categories. Metrics help testers to report results in a consistent way and enable coherent tracking of progress over time. So, definition of metrics: what are these metrics? There is a limited set of usefulmetrics that should be defined. Metrics should be defined based on specific objectives for the project, process, and product. Metrics should be defined for balance, as a single metric may give a misleading impression of status or trends. Once metrics have been defined, their interpretation must be agreed upon by all stakeholders in order to avoid confusion when the measurements are discussed. There is often a tendency to define too many metrics instead of concentrating on the most persistent areas. So this particular point should be taken care of. Track Metrics reporting and merging metrics should be as automated as possible to reduce the time spent on taking and processing measurements. Variations of measurements over time for a specific metric may reflect information other than the interpretation agreed upon. The test manager should be prepared to carefully analyse possible divergence in measurements from expectations and the reasons for the divergence when it comes to reporting of metrics. The particular objective of reporting of metrics is to provide an immediate understanding of the information. For management purposes, presentations may show a snapshot of a metric at a certain time, or they can show the evolution of a metric over time so that trends can be evaluated. Then it comes to the validity of metrics. What is the validity? A tasting manager must also verify the information that is being reported. The measurements taken for a metric may not reflect the true status of a project or may convey an overly positive or negative trend. Before any data is presented, the test manager must review it for both accuracy and the message that it is likely to convey. Now, let's talk about how to measure test progress. The first one is product risks. Product risk, that is, the percentage of risks completely covered by passing tests, can be considered for this particularfactor, that is product risk are percentage ofrisks completely covered by passing tests.% of risks for which some or all tests fail, % of risks for which testing has not yet been completed, % of risks covered, sorted by risk category, % of risks identified after the initial quality risk analysis As a result, all of these testmetrics will discuss product risks. The second one is defects. Here, the metrics to be considered are the cumulative number reported versus the cumulative number resolved. In the mean time, between failure arrival rate trends in the lake, time from defect reporting to resolution, and the number of defect fixes that introduce new defects, sometimes called dotted bugs. Breakdown of the number or percentage of defects categorised by the following particular testitems or components: root causes, source of defects, test releases, phase introduced, detected, and removed priority severity reports rejected or duplicated number three, it's about this.At this point, metrics to consider include total number of tests planned, specified, rented, and skipped; regression and confirmation test status, including trends and totals for regression test and confirmation test failure; hours of testing planned per day versus actual hours achieved; and taste environment availability, such as percentage of planttaste or when the test environment is usable by the tasting. Then it comes to coverage. Over here, metrics to be considered are requirements and design elements; coverage; risk coverage; environment or configuration coverage; and core coverage; to interpret and use coverage metrics for higher level testing, system testing, system integrationtesting, acceptance testing, then requirement specifications, designspecifications, use cases, user stories, product risks,supported environments, and supported configurations for lower level testing like unit testing and component integrationtesting. The metrics to be used are structural core coverage metrics. So now it's about metrics related to different test activities. Up till now, we talked about product-based testing like this requirements coverage. Now we are going to talk about testing activities. So first of all, it's about test planning and control activities for this particular testing phase. The metrics that can be defined or that can be covered are risk requirements and other test element coverage defects that are planned versus actual to develop test work and execute test cases. Number two is test analysis activities, that is, the number of test conditions identified, the number of defects found during test analysis, and then test design activities. For this particular phase, the percentage of test conditions covered by test cases and the number of defects found during test design are the metrics to be considered test implementation activities for this particular phase of testing. The metrics to be defined and tracked are the percentage of test environments configured, percentage of test data records loaded, and percentage of test cases automated. Then it comes to test execution activities, percentage of planned test cases executed, passed and failed,percentage of test conditions covered by executed test cases, defects reported or resolved, and planned versus actual coverage achieved. And the last one is monitoring of test progress and completion activities. There are a number of metrics to be considered for this particular phase. Because ultimately, this phase defines the overall testing and it helps in exceeding testing and releasing the product. So, what are the metrics being considered for this particular phase? Our number of test conditions Test cases or test specifications are planned and executed based on whether they died or not, and are frequently broken down by severity. Priority current state Affected classification or subsystem The number of changes required Accepted. Built and the test plan versus the actual cost plan versus the actual duration plan versus the actual testing dates Product risk status is frequently divided into mitigated versus unmitigated major risk areas, and new risks are discovered following test analysis. Etc. Test effort loss as a percentage Cost or time due to blocked events or plans changes confirmation and regression test status, then it comes to monitoring test closure activities in this phase. The following kinds of test metrics are considered: percentage of test cases executed. passed Failed blocked and skipped during test execution percentage of test cases checked into reusable test case repository percentage of automated or planned test cases versus actual test cases automated percentage of test cases included in regression Test the percentage of defect reports that have been resolved or have not been resolved. What exactly is the metric? Usage metrics are used for analysis to discover trends and causes that may be discriminated against via test result reporting to communicate test findings to interested project participants and stakeholders control to change the course of testing or the project as a whole and monitor the outcome of that change As this likes defines, there are three major applications for test metrics. They are analysis, reporting, and control. Factors to Consider: A test manager needs proper information to guide the test efforts towards successful completion of test mission strategies and objectives. Planning must consider all the mentioned information needs. Monitoring must include gathering any work product metrics, the volume of information required, and the effort required to collect it, depending on the project's size, complexity, and risk. Case control must respond to information generated by testing as well as to the changing conditions in which a project or endeavour exists. If divergence from the type test plan is discovered via the test progress report,test control should be performed. Test control is aimed at redirecting the project and the testing in a more successful direction. When using test results to influence or measure control efforts on the project, The following options should be considered for revising the quality. Risk analysis Test priorities and test plans adding resources or otherwise increasing the project or test effort and delaying the release date Relaxing or strengthening the testexit criteria Changing the scope of all the information delivered in a test report should depend largely on the information needs of the task audience, like project management or business management. So that's it about test metrics.

15. Business Value of Testing

So let's talk about the business value of testing. Whenever testing happens, it should be optimised testing. So what is optimised testing? Let's understand what is excessive testing and what is too little testing and we will be able to define what is optimised testing. So, excessive testing is something which does not deliver good business value because testing will impose unreasonable delays and cost more than it saves too little testing.It does not deliver good business value because too many defects will be delivered to users, and ultimately it's not a qualitative product. So the optimum lies between these two extremes. I mean, optimal testing is something which is not too excessive but not too little, and at the same time, it should be able to deliver a qualitative product. So, what are the quantitative and qualitative values of testing? Testing delivers value to the organization, project, and operation in both quantitative and qualitative ways. Finding defects that are prevented or fixed prior to release; finding defects that are known prior to release; reducing risk by running tests; and delivering information on project, process, and product status. Qualitative values include improved reputation for quality,smoother and more predictable releases, increased confidence,protection from legal liability, reducing the risk of loss of whole missions or even lies. And finally, the cost of quality. What is the cost of quality? A well-established method for measuring the quantitative value and efficiency It involves classifying projects and operational costs into the below mentioned categories related to product defect costs. So first of all, the cost of prevention. This can be reduced by training developers to write more maintainable or secure code. Then comes the cost of detection. This can be reduced by writing testcases, configuring test environments, and reviewing requirements. The cost of internal failure These costs can be reduced by fixing defects detected during testing or reviews prior to delivery. And finally, the cost of external failure. That means a customer finds a bug. How to prevent this kind of cost? Support costs associated with defective software are delivered to customers. A portion of the testing budget is the cost of detection, while the remainder is the cost of internal failure. The total cost of detection plus internal failure should be less than the cost of external failure because we all know that once the end user or customer finds a bug, it's very expensive to fix that and win the trust again. So that's all about nice. value of testing.

16. Distributed, Ousourced and Insourced Testing

In this chapter we are going to talk about distributed outsourced and in-source testing. So what is distributed testing? If the test effort occurs at multiple locations,that test effort is distributed through outsourced testing. If the test effort is carried out at one or more locations by people who are not employees of the company and who are not co-located with the project team, the test effort is outsourced and ultimately in source testing. If the test effort is carried out by people who are colocated with the project team but who are not fellow employees, the test effort is insourced. So, factors to consider for testing include number one, clear channels of communication. defined ways of communication, including addressing topics such as escalation of problems,types of information to be communicated, and methods of communication to be used. Problems in communication occur mostly due to location, time zone, cultural differences, language differences, well-defined expectations for mission tasks and deliverables are required mostly when it comes to source or outsource testing. As the definition says, it occurs in multiple locations by multiple teams. So, as we all must have worked on some kind of outsourced project in one way or the other, we know how client communication happens, what kinds of glitches can be expected, what kind of struggle is there and to avoid those kinds of struggles, clear communication is the key point. Number two is the alignment of methodologies. For example, two groups are using different methodologies. The testing team employs different methodologies than the development or project management teams. If a client is using agile methodology for development and testing services, the provider has a predefined test strategy just like in the waterfall model. In this kind of situation, testing efforts are not fruitful for 100% because the alignment of methodologies to be followed are not the same. Number three is task distribution for distributed teams over multiple locations. It should be intelligently defined. If not done properly, the most competent team cannot work to their best capability. If each team is not clear about what they are responsible for, they may not do what they are supposed to do. Expectations from each team should be very clearly defined. If work distribution is not proper, gaps and overlaps make the whole testing process difficult for us. And finally, the last factor is the trust factor. As the teams are not sitting in the same location and the teams are not employees of the same company, there might be trust issues. The entire project team must develop and maintain trust. Inefficiency, unwelcome delays, internal politics, and blaming issues result from a lack of trust, and testing efforts suffer as a result. So for distributed in-source or outsourced testing, if some of the points are considered earlier, then it can be fruitful and be very beneficial.

17. Managing the application of Industry Standards

And the final topic of that particular chapter, called Testing Content, ismanaging the application of industry standards. There are different standards. Standards can come from different sources, such as international or with international objectives. domain specific, such as when international or national standards are adapted to or developed for specific domains national, such as national applications of international standards For example, international standards are ISO and IAA. So the standards for software testing when it comes to ISO are ISO 250, ISO 12,207, and ISO 15,504. When it comes to AEEE standards for software testing, IEEE eight two nine and IEEE 100:28. National Standards So what are the national standards? National standards are standards developed by a specificnation are called national standards.For example, UK standard BS 79252 provides information related to test design techniques. Domain-specific norms Some standards are specific to domains. For example, in the Avenue domain, the US Bo 178 of the Federal Aviation Administration applies to software used in civilian aircraft. This standard prescribes a certain level of structural coverage criteria based on the level of criticality of the software being tested. medical systems in the US. The Food and Drug Administration's Title 21 CFRPart Eighty standard recommends certain structural and functional test techniques. The standard also recommends testing strategies and principles that are consistent with the Istqbc Levy. Other frameworks that are supposed to be understood are PMI's PMBOK Principle and PMI's PRINCE2. These frameworks are commonly used project management frameworks in North America and Europe, respectively. And the second one is iTile? That is the Information Technology Infrastructure Library. This framework is for ensuring that an IT group delivers valuable services to the organisation in which it exists. So that's all about checking. The two lengthiest chapters of Istqbcsyllables for advanced liver taste managers Let's move ahead to Chapter Three.

ExamSnap's ISTQB ATM Practice Test Questions and Exam Dumps, study guide, and video training course are complicated in premium bundle. The Exam Updated are monitored by Industry Leading IT Trainers with over 15 years of experience, ISTQB ATM Exam Dumps and Practice Test Questions cover all the Exam Objectives to make sure you pass your exam easily.