The authors are themselves developers of the ISTQB syllabus and are highly respected international authorities and teachers within the field of software testing. It offers an internationally recognized qualification that ensures there is an international, common understanding of software and system testing issues. The authors are themselves developers of the ISTQB syllabus and are highly respected international authorities, teachers and authors within the field of software testing.
This book adopts a practical and hands-on approach, covering the fundamental principles that every system and software tester should know. Each of the six sections of the syllabus is cove. This thoroughly revised and updated fifth edition covers the Foundation Level entry level and teaches the most important methods of software testing.
Instead, it answers the top 47 questions that we are asked and those we come across in forums, our consultancy and education programs. It tells you exactly how to deal with those questions, with tips that have never before been offered in print.
This book teaches test managers what they need to know to achieve advanced skills in test estimation, test planning, test monitoring, and test control. Readers will learn how to define the overall testing goals and strategies for the systems being tested. This hands-on, exercise-rich book provides experience with planning, scheduling, and tracking these tasks.
You'll be able to describe and organize the necessary activities as well as learn to select, acquire, and assign adequate resources for testing tasks. You'll learn how to form, organize, and lead testing teams, and master the organizing of communication among the members of the testing teams, and between the testing teams and all the other stakeholders.
Additionally, you'll learn how to justify decisions and provide adequate reporting information where applicable. With over thirty years of software and systems engineering experience, author Rex Black is President of RBCS, is a leader in software, hardware, and systems testing, and is the most prolific author practicing in the field of software testing today.
He has published a dozen books on testing that have sold tens of thousands of copies worldwide. Included are sample exam questions, at the appropriate level of difficulty, for most of the learning objectives covered by the ISTQB Advanced Level Syllabus. Our vision is to be a world-class organisation for IT. Our 70, strong membership includes practitioners, businesses, academics and students in the UK and internationally. We deliver a range of professional development tools for practitioners and employees.
As leading IT qualification body, we offer a range of widely recognised qualifications. All rights reserved. Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted by the Copyright Designs and Patents Act , no part of this publication may be reproduced, stored or transmitted in any form or by any means, except with the prior permission in writing of the publisher, or in the case of reprographic reproduction, in accordance with the terms of the licences issued by the Copyright Licensing Agency.
Enquiries for permission to reproduce material outside those terms should be directed to the publisher. All trade marks, registered names etc. Disclaimer: The views expressed in this book are of the authors and do not necessarily reflect the views of BCS or BISL except where explicitly stated as such.
Although every care has been taken by the authors and BISL in the preparation of the publication, no warranty is given by the authors or BISL as publisher as to the accuracy or completeness of the information contained within it and neither the authors nor BISL shall be responsible or liable for any loss or damage whatsoever arising by virtue of such information or any instructions or advice contained within this publication or by any of the aforementioned.
He has worked in areas as diverse as real-time avionics, legacy systems maintenance and e-business strategies. He contributed to the development of software quality standards while at the Ministry of Defence and later became the head of systems and software engineering at The University of Greenwich. He was technical director of ImagoQA and general manager of Microgen IQA, a specialist company providing consultancy in software testing and quality assurance primarily to the financial services sector.
He is now concentrating on writing. Peter Morgan is a freelance testing practitioner. He has been working as a hands-on tester for a number of years, often on projects with over 30 testers. Early in her career she took an interest in developing staff, managing the training of new engineers across the company, to the standards laid down by the IEE now the IET. She has also instructed delegates in other aspects of testing, such as unit testing, user acceptance testing and managing testing projects, in the UK, Europe, North America and Australia.
Geoff Thompson has been involved in testing for nearly 25 years, specialising in test strategy, test management and process improvement.
He is currently consultancy director of the consulting organisation Experimentus Ltd. He has been a self-employed contract test manager or consultant in both financial services and the public sector. He has evaluated test processes and subsequently implemented improvements, at various organisations, including test management and execution tools as appropriate.
An intermediate level qualification was introduced in as a step towards the more advanced Practitioner qualification. The Certified Tester Foundation Level Syllabus has been updated and released in a version, and this book relates to the version of the syllabus.
The book is therefore structured to support learning of the key ideas in the syllabus quickly and efficiently for those who do not plan to attend a course, and to support structured revision for anyone preparing for the exam. In this introductory chapter we will explain the nature and purpose of the Foundation Level and provide an insight into the way the syllabus is structured and the way the book is structured to support learning in the various syllabus areas.
Finally we offer guidance on the best way to use this book, either as a learning resource or as a revision resource. As a result coverage of topics is variable, with some only briefly mentioned and others studied in some detail.
The arrangement of the syllabus and the required levels of understanding are explained in the next section. The authors of the syllabus have aimed it at people with varying levels of experience in testing, including those with no experience at all. This makes the certificate accessible to those who are or who aim to be specialist testers, but also to those who require a more general understanding of testing, such as project managers and software development managers.
One specific aim of this qualification is to prepare certificate holders for the next level of certification, but the Foundation Level has sufficient breadth and depth of coverage to stand alone. These timings are further broken down for each topic within a section.
Each section of the syllabus also includes a list of learning objectives that provides candidates with a guide to what they should know when they have completed their study of a section and a guide to what can be expected to be asked in an examination.
The learning objectives can be used to check that learning or revision is adequate for each topic. In the book, which is structured around the syllabus sections, we have presented the learning objectives for each section at the beginning of the relevant chapter, and the summary at the end of each chapter confirms how those learning objectives have been addressed.
Finally, each topic in the syllabus has associated with it a level of understanding, represented by the legend K1, K2, K3 or K4: Level of understanding K1 is associated with recall, so that a topic labelled K1 contains information that a candidate should be able to remember but not necessarily use or explain.
Level of understanding K3 is associated with the ability to apply a topic in a practical setting. Level of understanding K4 is associated with the ability to analyse a situation or a set of information to determine what action to take.
The level of understanding influences the level and type of questions that can be expected to be asked about that topic in the examination. More detail about the question style and about the examination is given in Chapter 7.
Example questions, written to the level and in the formats used in the examination, are included within each chapter to provide generous examination practice.
Syllabus map The syllabus can usefully be viewed as a mind map, as shown in Figure 0. In this representation the main sections of the syllabus, corresponding to chapters in the book, provide the first level of ordering. The next level provides the breakdown into topics within each section.
In most cases the syllabus breaks topics down even further, but this level of breakdown is omitted from the diagram for clarity. Figure 0. By recognising the relative strengths and weaknesses by topic within sections it is easier to understand the nature and extent of the weakness. For example, problems with certain black-box techniques that are not also associated with white-box techniques and experience-based techniques should give confidence in the overall section on test case design techniques.
The structure enables you to go straight to the place you need, with confidence either that what you need to know will be covered there and nowhere else, or that relevant cross references will be provided.
Each chapter of the book incorporates the learning objectives from the syllabus and identifies the required level of understanding for each topic. Each chapter also includes examples of typical examination questions to enable you to assess your current knowledge of a topic before you read the chapter, and further questions at the end of each chapter to provide practice in answering typical examination questions. Topics requiring K3 level of understanding are presented with worked examples as a model for the level of application expected from real 3 -ba tion se or w g st te ns itio co nd an d tic Sta es iqu hn tec an he dt tes ess roc wp vie Re is by tools Static analys Test life levels odels nt m pme evelo are d Softw cle cy Maintenance testing Test types: the targets of testing Test progress monitoring and control ol into an g a to Introducin n organisatio re wa oft y?
Answers are provided for all questions, and the rationale for the correct answer is discussed for all practice questions. A final chapter explains the Foundation Level examination strategy and provides guidance on how to prepare for the examination and how to manage the examination experience to maximise your own performance.
If you are using the book as an alternative to attending an accredited course you will probably find the first method of using the book described below to be of greatest value.
If you are using the book as a revision aid you may find the second approach more appropriate. In either case you would be well advised to acquire a copy of the syllabus available from www. Using the book as a learning aid For those of you using the book as an alternative to attending an accredited course the first step is to familiarise yourself with the syllabus structure and content by skim reading the opening sections of each chapter where the learning objectives are identified for each topic.
You may then find it helpful to turn to Chapter 7 and become familiar with the structure of the examination and the types and levels of questions that you can expect in the examination. From here you can then work through each of the six main chapters in any sequence before returning to Chapter 7 to remind yourself of the main elements of the examination.
For each chapter begin by attempting the self-assessment questions at the beginning to get initial confirmation of your level of confidence in the topics covered by that chapter. This may help you to prioritise how you spend your time. Work first through the chapters where your knowledge is weakest, attempting all the exercises and following through all the worked examples.
Read carefully through the chapters where your knowledge is less weak but still not good enough to pass the exam. You can be more selective with exercises and examples here, but make sure you attempt the practice questions at the end of the chapters. For the areas where you feel strong you can use the chapter for revision, but remember to attempt the practice questions to confirm positively your initial assessment of your level of knowledge.
Every chapter contains a summary section that reiterates the learning objectives, so reading the first and last sections of a chapter will help you to understand how your current level of knowledge relates to the level required to pass the examination.
The best confirmation of this is to attempt questions at the appropriate K level for each topic; these are provided in the book. The information in Chapter 7 will enable you to construct a properly balanced mock exam of your own. Your mock exam will provide some experience of answering typical questions under the same time pressures that you will experience in the real examination, and this will provide you with a fairly reliable guide to your current state of readiness to take the real examination.
You can also discover which areas most need revision from your performance in the mock exam, and this will guide you as you plan your revision. Revise first where you feel weakest. You can use the opening sections of each chapter, containing the learning objectives and the self-assessment questions, together with the summary at the end of each chapter to refine further your awareness of your own weaknesses.
From here you can target your studies very accurately. Remember that every K3 topic will have at least one worked example and some exercises to help you build your confidence before tackling questions at the level set in the real examination. You can get final confirmation of your readiness to take the real examination by taking the sample examination paper provided by ISEB. The car should have five wheels, a steering wheel, an engine and all the other essential components, and it should come with appropriate documentation, with all pre-sales checks completed and passed satisfactorily.
The car you receive should be the car described in the sales literature; it should have the correct engine size, the correct colour scheme, and whatever extras you have ordered, and performance in areas such as fuel consumption and maximum speed should match published figures.
In short, a level of expectation is set by brochures, by your experience of sitting in the driving seat, and probably by a test drive. If your expectations are not met you will feel justifiably aggrieved. This kind of expectation seems not to apply to new software installations; examples of software being delivered not working as expected, or not working at all, are common.
Why is this? There is no single cause that can be rectified to solve the problem, but one important contributing factor is the inadequacy of the testing to which software applications are exposed. Software testing is neither complex nor difficult to implement, yet it is a discipline that is seldom applied with anything approaching the necessary rigour to provide confidence in delivered software. Software testing is costly in human effort or in the technology that can multiply the effect of human effort, yet is seldom implemented at a level that will provide any assurance that software will operate effectively, efficiently or even correctly.
This book explores the fundamentals of this important but neglected discipline to provide a basis on which a practical and cost-effective software testing regime can be constructed. Secondly, we will establish the structure that we have used throughout the book to help you to use the book as a learning and revision aid. Thirdly, we will use this chapter to point forward to the content of later chapters.
We begin by defining what we expect you to get from reading this chapter. The learning objectives below are based on those defined in the Software Foundation Certificate syllabus ISTQB, , so you need to ensure that you have achieved all of these objectives before attempting the examination. Learning objectives The learning objectives for this chapter are listed below.
The chapter summary will remind you of the key ideas. The sections are allocated a K number to represent the level of understanding required for that section; where an individual topic has a lower K number than the section as a whole, this is indicated for that topic; for an explanation of the K numbers see the Introduction. Why is testing necessary? K2 Describe, with examples, the way in which a defect in software can cause harm to a person, to the environment or to a company.
Distinguish between the root cause of a defect and its effects. Give reasons why testing is necessary by giving examples. Describe why testing is part of quality assurance and give examples of how testing contributes to higher quality. Recall the terms error, defect, fault, failure and the corresponding terms mistake and bug.
K1 What is testing? K2 Recall the common objectives of testing. K1 Provide examples for the objectives of testing in different phases of the software life cycle. Differentiate testing from debugging. General testing principles K2 Explain the fundamental principles in testing. The psychology of testing K2 Recall the psychological factors that influence the success of testing.
K1 Contrast the mindset of a tester and of a developer. Self-assessment questions The following questions have been designed to enable you to check your current level of understanding for the topics in this chapter. The answers are given at the end of the chapter. Question SA1 K1 A bug or defect is: a. Question SA2 K1 The effect of testing is to: a. Question SA3 K1 What is retesting? Running the same test again in the same circumstances to reproduce the problem.
A cursory run through a test pack to see if any new errors have been introduced. Checking that the predetermined exit criteria for the test phase have been met. A software error caused the rocket to deviate from its vertical ascent, and the self-destruct capabilities were enacted before the then unpredictable flight path resulted in a bigger problem.
When the UK Government introduced online filing of tax returns, a user could sometimes see the amount that a previous user earned. This was regardless of the physical location of the two applicants.
The publication of this information was described in newspapers and on morning radio and television and, as a result, many people attempted to access the site. The performance of the website proved inadequate under this load and the website had to be taken offline.
The publicity created performance peaks beyond the capacity of the website. Development staff had not anticipated that anyone would attempt to purchase a negative number of books. Code was developed to allow refunds to customers to be made by administrative staff — but self-requested refunds are not valid.
A small, one-line, change in the billing system of an electrical provider blacked out the whole of a major US city. What is it about these examples that make them so startling? Is it a sense that something fairly obvious was missed? Is it the feeling that, expensive and important as they were, the systems were allowed to enter service before they were ready?
Do you think these systems were adequately tested? Obviously they were not, but in this book we want to explore why this was the case and why these kinds of failure continue to plague us. To understand what is going on we need to start at the beginning, with the people who design systems.
Do they make mistakes? Of course they do. People make mistakes because they are fallible, but there are also many pressures that make mistakes more likely. Pressures such as deadlines, complexity of systems and organisations, and changing technology all bear down on designers of systems and increase the likelihood of errors in specifications, in designs and in software code.
These errors are where major system failures usually begin. If a document with an error in it is used to specify a component the component will be faulty and will probably exhibit incorrect behaviour. If this faulty component is built into a system the system may fail.
While failure is not always guaranteed, it is likely that errors in specifications will lead to faulty components and faulty components will cause system failure. Figure 1. Environmental conditions such as the presence of radiation, magnetism, electronic fields or pollution can affect the operation of hardware and firmware and lead to system failure.
If we want to avoid failure we must either avoid errors and faults or find them and rectify them. Testing can contribute to both avoidance and rectification, as we will see when we have looked at the testing process in a little more detail.
One thing is clear: if we wish to influence errors with testing we need to begin testing as soon as we begin making errors — right at the beginning of the development process — and we need to continue testing until we are confident that there will be no serious system failures — right at the end of the development process.
Before we move on, let us just remind ourselves of the importance of what we are considering. Incorrect software can harm: people e.
Software failures can sometimes cause all three of these at once. The scenario of a train carrying nuclear waste being involved in a crash has been explored to help build public confidence in the safety of transporting nuclear waste by train. This may not be likely we hope it is not but it is a possibility that could be linked with software failure. There may be several themes that we could draw out of the examples, but one theme is clear: either insufficient testing or the wrong type of testing was done.
More and better software testing seems a reasonable aim, but that aim is not quite as simple to achieve as we might expect. Exhaustive testing of complex systems is not possible With the Ariane 5 rocket launch, a particular software module was reused from the Ariane 4 programme. Only part of the functionality of the module was required, but the module was incorporated without changes. The unused functionality of the reused module indirectly caused a directional nozzle to move in an uncontrolled way because certain variables were incorrectly updated.
In an Ariane 4 rocket the module would have performed as required, but in the Ariane 5 environment this malfunction in an area of software not even in use caused a catastrophic failure. The failure is well documented, but what is clear is that conditions were encountered in the first few seconds after the launch that were not expected, and therefore had not been tested. If every possible test had been run, the problem would have been detected. However, if every test had been run, the testing would still be running now, and the ill-fated launch would never have taken place; this illustrates one of the general principles of software testing, which are explained below.
With large and complex systems it will never be possible to test everything exhaustively; in fact it is impossible to test even moderately complex systems exhaustively. In the Ariane 5 case it would be unhelpful to say that not enough testing was done; for this particular project, and for many others of similar complexity, that would certainly always be the case.
In the Ariane 5 case the problem was that the right sort of testing was not done because the problem had not been detected. Testing and risk Risk is inherent in all software development.
The system may not work or the project to build it may not be completed on time, for example. These uncertainties become more significant as the system complexity and the implications of failure increase. Intuitively, we would expect to test an automatic flight control system more than we would test a video game system. Because the risk is greater. What we test, and how much we test it, must be related in some way to the risk. Greater risk implies more and better testing. There is much more on risk and risk management in Chapter 5.
Testing and quality Quality is notoriously hard to define. While the user community for such a system is potentially large and disparate, it is hard to imagine any user that would find that situation anything other than unacceptable. In the top 10 criminals example the problem was slightly different. There was no failure of functionality in this case; the system was simply swamped by requests for access.
This is an example of a non-functional failure, in that the system was not able to deliver its services to its users because it was not designed to handle the peak load that materialised after radio and TV coverage. Of course the software development process, like any other, must balance competing demands for resources. If we need to deliver a system faster i.
The items at the corners or vertices of the triangle of resources in Figure 1. These three affect one another, and also influence the features that are or are not included in the delivered software.
Testing cannot directly remove defects, nor can it directly enhance quality. By reporting defects it makes their removal possible and so contributes to the enhanced quality of the system.
In addition, the systematic coverage of a software product in testing allows at least some aspects of the quality of the software to be measured. Testing is one component in the overall quality assurance activity that seeks to ensure that systems enter service without defects that can lead to serious failures.
We have so far decided that we cannot test everything, even if we would wish to. We also know that every system is subject to risk of one kind or another and that there is a level of quality that is acceptable for a given system. These are the factors we will use to decide how much testing to do. The most important aspect of achieving an acceptable result from a finite and limited amount of testing is prioritisation.
Do the most important tests first so that at any time you can be certain that the tests that have been done are more important than the ones still to be done.
Even if the testing activity is cut in half it will still be true that the most important testing has been done. The most important tests will be those that test the most important aspects of the system: they will test the most important functions as defined by the users or sponsors of the system, and the most important non-functional behaviour, and they will address the most significant risks.
The next most important aspect is setting criteria that will give you an objective test of whether it is safe to stop testing, so that time and all the other pressures do not confuse the outcome. These criteria, usually known as completion criteria, set the standards for the testing activity by defining areas such as how much of the software is to be tested this is covered in more detail in Chapter 4 and what levels of defects can be tolerated in a delivered product which is covered in more detail in Chapter 5.
Priorities and completion criteria provide a basis for planning which will be covered in Chapter 2 and Chapter 5 but the triangle of resources in Figure 1. In the end, the desired level of quality and risk may have to be compromised, but our approach ensures that we can still determine how much testing is required to achieve the agreed levels and we can still be certain that any reduction in the time or effort available for testing will not affect the balance — the most important tests will still be those that have already been done whenever we stop.
Give three consequences of software failures. We have recognised that it is an activity used to reduce risk and improve quality by finding defects, which is all true. Testing and debugging Testing and debugging are different kinds of activity, both of which are very important. Debugging is the process that developers go through to identify the cause of bugs or defects in code and undertake corrections.
Ideally some check of the correction is made, but this may not extend to checking that other areas of the system have not been inadvertently affected by the correction. Testing, on the other hand, is a systematic exploration of a component or system with the main aim of finding and reporting defects. Testing does not include correction of defects — these are passed on to the developer to correct.
Testing does, however, ensure that changes and corrections are checked for their effect on other parts of the component or system. Effective debugging is essential before testing begins to raise the level of quality of the component or system to a level that is worth testing, i.
Debugging does not give confidence that the component or system meets its requirements completely. Testing makes a rigorous examination of the behaviour of a component or system and reports all defects found for the development team to correct. Testing then repeats enough tests to ensure that defect corrections have been effective. So both are needed to achieve a quality result. Static testing and dynamic testing Static testing is the term used for testing where the code is not exercised.
This may sound strange, but remember that failures often begin with a human error, namely a mistake in a document such as a specification. We need to test these because errors are much cheaper to fix than defects or failures as you will see. That is why testing should start as early as possible, another basic principle explained in more detail later in this chapter. Static testing involves techniques such as reviews, which can be effective in preventing defects, e.
Dynamic testing is the kind that exercises the program under test with some test data, so we speak of test execution in this context. The discipline of software testing encompasses both static and dynamic testing. Testing as a process We have already seen that there is much more to testing than test execution. Before test execution there is some preparatory work to do to design the tests and set them up; after test execution there is some work needed to record the results and check whether the tests are complete.
Even more important than this is deciding what we are trying to achieve with the testing and setting clear objectives for each test. A test designed to give confidence that a program functions according to its specification, for example, will be quite different from one designed to find as many defects as possible. We define a test process to ensure that we do not miss critical steps and that we do things in the right order.
We will return to this important topic later, where we explain the fundamental test process in detail. It might seem paradoxical, but a good test is one that finds a defect if there is one present. A test that finds no defect has consumed resources but added no value; a test that finds a defect has created an opportunity to improve the quality of the product. How do we design tests that find defects?
We actually do two things to maximise the effectiveness of the tests. First we use well-proven test design techniques, and a selection of the most important of these is explained in detail in Chapter 4.
The techniques are all based on certain testing principles that have been discovered and documented over the years, and these principles are the second mechanism we use to ensure that tests are effective. Even when we cannot apply rigorous test design for some reason such as time pressures we can still apply the general principles to guide our testing.
We turn to these next. We now describe some general testing principles that help testers, principles that have been developed over the years from a variety of sources. These are not all obvious, but their purpose is to guide testers, and prevent the types of problems described previously.
Testers use these principles with the test techniques described in Chapter 4. Testing shows the presence of bugs Running a test through a software system can only show that one or more defects exist.
Advertisement Hide. Front Matter Pages i-xii. Your email address will not be published. Completely updated to comprehensively reflect the most recent changes to the ISTQB Foundation Syllabus, the book adopts a practical, hands-on approach, covering the fundamental topics that every system and software tester should know. The authors are themselves developers of the ISTQB syllabus and are highly respected international authorities, teachers and authors within the field of software testing.
Today, software engineers need to know not only how to program effectively but also how to …. Skip to main content. Start your free trial. Book description This best-selling software testing title explains the basic steps of software testing and how to perform effective tests.
0コメント