CS2103/T Aug '18
  • Flat (current format)
  •     Nested
  • Schedule
  • Textbook
  • Admin Info
  • Report Bugs
  • Slack
  • Forum
  • Instructors
  • Announcements
  • File Submissions
  • Tutorial Schedule
  • Team IDs
  • Java Coding Standard
  • samplerepo-things
  • Addressbook-level1
  • Addressbook-level2
  • Addressbook-level3
  • Addressbook-level4
  • Projects List
  • config.json templates for Reposense
  • Project Code Dashboard (BETA)
  • Week 10 [Oct 22]

    Todo

    Admin info to read:

    Admin Project: mid-v1.3 [week 10]

    Continue to enhance features. Make code RepoSense-compatible. Try doing a proper release.

    Project Management:

    Ensure your code is RepoSense-compatible, as explained below:

    In previous semesters we asked students to annotate all their code using special @@author tags so that we can extract each student's code for grading. This semester, we are trying out a new tool called RepoSense that is expected to reduce the need for such tagging, and also make it easier for you to see (and learn from) code written by others.

    Figure: RepoSense Report Features

    1. View the current status of code authorship data:

    • The report generated by the tool is available at Project Code Dashboard (BETA). The feature that is most relevant to you is the Code Panel (shown on the right side of the screenshot above). It shows the code attributed to a given author. You are welcome to play around with the other features (they are still under development and will not be used for grading this semester).
    • Click on your name to load the code attributed to you (based on Git blame/log data) onto the code panel on the right.
    • If the code shown roughly matches the code you wrote, all is fine and there is nothing for you to do.

    2. If the code does not match:

    • Here are the possible reasons for the code shown not to match the code you wrote:

      • the git username in some of your commits does not match your GitHub username (perhaps you missed our instructions to set your Git username to match GitHub username earlier in the project, or GitHub did not honor your Git username for some reason)
      • the actual authorship does not match the authorship determined by git blame/log e.g., another student touched your code after you wrote it, and Git log attributed the code to that student instead
    • In those cases,

      • Install RepoSense (see the Getting Started section of the RepoSense User Guide)
      • Use the two methods described in the RepoSense User Guide section Configuring a Repo to Provide Additional Data to RepoSense to provide additional data to the authorship analysis to make it more accurate.
      • If you add a config.json file to your repo (as specified by one of the two methods),
        • Please use the template json file given in the module website so that your display name matches the name we expect it to be.
        • If your commits have multiple author names, specify all of them e.g., "authorNames": ["theMyth", "theLegend", "theGary"]
        • Update the line config.json in the .gitignore file of your repo as /config.json so that it ignores the config.json produced by the app but not the _reposense/config.json.
      • If you add @@author annotations, please follow the guidelines below:

    Adding @@author tags indicate authorship

    • Mark your code with a //@@author {yourGithubUsername}. Note the double @.
      The //@@author tag should indicates the beginning of the code you wrote. The code up to the next //@@author tag or the end of the file (whichever comes first) will be considered as was written by that author. Here is a sample code file:

      //@@author johndoe
      method 1 ...
      method 2 ...
      //@@author sarahkhoo
      method 3 ...
      //@@author johndoe
      method 4 ...
      
    • If you don't know who wrote the code segment below yours, you may put an empty //@@author (i.e. no GitHub username) to indicate the end of the code segment you wrote. The author of code below yours can add the GitHub username to the empty tag later. Here is a sample code with an empty author tag:

      method 0 ...
      //@@author johndoe
      method 1 ...
      method 2 ...
      //@@author
      method 3 ...
      method 4 ...
      
    • The author tag syntax varies based on file type e.g. for java, css, fxml. Use the corresponding comment syntax for non-Java files.
      Here is an example code from an xml/fxml file.

      <!-- @@author sereneWong -->
      <textbox>
        <label>...</label>
        <input>...</input>
      </textbox>
      ...
      
    • Do not put the //@@author inside java header comments.
      👎

      /**
        * Returns true if ...
        * @@author johndoe
        */
      

      👍

      //@@author johndoe
      /**
        * Returns true if ...
        */
      

    What to and what not to annotate

    • Annotate both functional and test code There is no need to annotate documentation files.

    • Annotate only significant size code blocks that can be reviewed on its own  e.g., a class, a sequence of methods, a method.
      Claiming credit for code blocks smaller than a method is discouraged but allowed. If you do, do it sparingly and only claim meaningful blocks of code such as a block of statements, a loop, or an if-else statement.

      • If an enhancement required you to do tiny changes in many places, there is no need to annotate all those tiny changes; you can describe those changes in the Project Portfolio page instead.
      • If a code block was touched by more than one person, either let the person who wrote most of it (e.g. more than 80%) take credit for the entire block, or leave it as 'unclaimed' (i.e., no author tags).
      • Related to the above point, if you claim a code block as your own, more than 80% of the code in that block should have been written by yourself. For example, no more than 20% of it can be code you reused from somewhere.
      • 💡 GitHub has a blame feature and a history feature that can help you determine who wrote a piece of code.
    • Do not try to boost the quantity of your contribution using unethical means such as duplicating the same code in multiple places. In particular, do not copy-paste test cases to create redundant tests. Even repetitive code blocks within test methods should be extracted out as utility methods to reduce code duplication. Individual members are responsible for making sure code attributed to them are correct. If you notice a team member claiming credit for code that he/she did not write or use other questionable tactics, you can email us (after the final submission) to let us know.

    • If you wrote a significant amount of code that was not used in the final product,

      • Create a folder called {project root}/unused
      • Move unused files (or copies of files containing unused code) to that folder
      • use //@@author {yourGithubUsername}-unused to mark unused code in those files (note the suffix unused) e.g.
      //@@author johndoe-unused
      method 1 ...
      method 2 ...
      

      Please put a comment in the code to explain why it was not used.

    • If you reused code from elsewhere, mark such code as //@@author {yourGithubUsername}-reused (note the suffix reused) e.g.

      //@@author johndoe-reused
      method 1 ...
      method 2 ...
      
    • You can use empty @@author tags to mark code as not yours when RepoSense attribute the to you incorrectly.

      • Code generated by the IDE/framework, should not be annotated as your own.

      • Code you modified in minor ways e.g. adding a parameter. These should not be claimed as yours but you can mention these additional contributions in the Project Portfolio page if you want to claim credit for them.

    • After you are satisfied with the new results (i.e., results produced by running RepoSense locally), push the config.json file you added and/or the annotated code to your repo. We'll use that information the next time we run RepoSense (we run it at least once a week).
    • If you choose to annotate code, please annotate code chunks not smaller than a method. We do not grade code snippets smaller than a method.
    • If you encounter any problem when doing the above or if you have questions, please post in the forum.

    We recommend you ensure your code is RepoSense-compatible by v1.3

    Product:

    • Do a proper product release as described in the Developer Guide. You can name it something like v1.2.1. Ensure that the jar file works as expected by doing some manual testing. Reason: You are required to do a proper product release for v1.3. Doing a trial at this point will help you iron out any problems in advance. It may take additional effort to get the jar working especially if you use third party libraries or additional assets such as images.

    Documentation:

    • User Guide: Update where the document does not match the current product.
    • Developer Guide: Similar to the User Guide.

    Outcomes

    Requirements

    W10.1 Can explain object oriented domain models W10.1a Can explain object oriented domain models

    Design → Modelling → Modelling Structure

    Object Oriented Domain Models

    The analysis process for identifying objects and object classes is recognized as one of the most difficult areas of object-oriented development. --Ian Sommerville, in the book Software Engineering

    Class diagrams can also be used to model objects in the problem domain (i.e. to model how objects actually interact in the real world, before emulating them in the solution). Class diagrams are used to model the problem domain are called conceptual class diagrams or OO domain models (OODMs).

    OO domain model of a snakes and ladders game is given below.

    Description: Snakes and ladders game is played by two or more players using a board and a die. The board has 100 squares marked 1 to 100. Each player owns one piece. Players take turns to throw the die and advance their piece by the number of squares they earned from the die throw. The board has a number of snakes. If a player’s piece lands on a square with a snake head, the piece is automatically moved to the square containing the snake’s tail. Similarly, a piece can automatically move from a ladder foot to the ladder top. The player whose piece is the first to reach the 100th square wins.

    The above OO domain model omits the ladder class for simplicity. It can be included in a similar fashion to the Snake class.

    OODMs do not contain solution-specific classes (i.e. classes that are used in the solution domain but do not exist in the problem domain). For example, a class called DatabaseConnection could appear in a class diagram but not usually in an OO domain model because DatabaseConnection is something related to a software solution but not an entity in the problem domain.

    OODMs represents the class structure of the problem domain and not their behavior, just like class diagrams. To show behavior, use other diagrams such as sequence diagrams.

    OODM notation is similar to class diagram notation but typically omit methods and navigability.

    This diagram is,

    • a. A class diagram.
    • b. An object diagram.
    • c. An OO domain model, also known as a conceptual class diagram.
    • d. Can be either a class diagram or an OO domain model.

    (a)

    Explanation: The diagram shows navigability which is not shown in an OO domain model. Hence, it has to be a class diagram.

    What is the main difference between a class diagram and and an OO domain model?

    (a)

    Explanation: Both are UML diagrams, and use the class diagram notation. While it is true that often a class diagram may have more classes and more details, the main difference is that the OO domain model describes the problem domain while the class diagram describes the solution.

    Design

    W10.2 Can use some basic design patterns

    Introduction

    W10.2a Can explain design patterns

    Design → Design Patterns → Introduction →

    What

    Design Pattern : An elegant reusable solution to a commonly recurring problem within a given context in software design.

    In software development, there are certain problems that recur in a certain context.

    Some examples of recurring design problems:

    Design Context Recurring Problem
    Assembling a system that makes use of other existing systems implemented using different technologies What is the best architecture?
    UI needs to be updated when the data in application backend changes How to initiate an update to the UI when data changes without coupling the backend to the UI?

    After repeated attempts at solving such problems, better solutions are discovered and refined over time. These solutions are known as design patterns, a term popularized by the seminal book Design Patterns: Elements of Reusable Object-Oriented Software by the so-called "Gang of Four" (GoF) written by Eric Gamma, Richard Helm, Ralph Johnson,and John Vlissides.

    Which one of these describes the ‘software design patterns’ concept best?

    (b)

    W10.2b Can explain design patterns format

    Design → Design Patterns → Introduction →

    Format

    The common format to describe a pattern consists of the following components:

    • Context: The situation or scenario where the design problem is encountered.
    • Problem: The main difficulty to be resolved.
    • Solution: The core of the solution. It is important to note that the solution presented only includes the most general details, which may need further refinement for a specific context.
    • Anti-patterns (optional): Commonly used solutions, which are usually incorrect and/or inferior to the Design Pattern.
    • Consequences (optional): Identifying the pros and cons of applying the pattern.
    • Other useful information (optional): Code examples, known uses, other related patterns, etc.

    When we describe a pattern, we must also specify anti-patterns.

    False.

    Explanation: Anti-patterns are related to patterns, but they are not a ‘must have’ component of a pattern description.


    Singleton pattern

    W10.2c Can explain the Singleton design pattern

    Design → Design Patterns → Singleton →

    What

    Context

    A certain classes should have no more than just one instance (e.g. the main controller class of the system). These single instances are commonly known as singletons.

    Problem

    A normal class can be instantiated multiple times by invoking the constructor.

    Solution

    Make the constructor of the singleton class private,  because a public constructor will allow others to instantiate the class at will. Provide a public class-level method to access the single instance.

    Example:

    We use the Singleton pattern when

    (c)


    Evidence:

    Identify where in the project Singleton pattern can be used.

    W10.2d Can apply the Singleton design pattern

    Design → Design Patterns → Singleton →

    Implementation

    Here is the typical implementation of how the Singleton pattern is applied to a class:

    class Logic {
        private static Logic theOne = null;
    
        private Logic() {
            ...
        }
    
        public static Logic getInstance() {
            if (theOne == null) {
                theOne = new Logic();
            }
            return theOne;
        }
    }
    

    Notes:

    • The constructor is private, which prevents instantiation from outside the class.
    • The single instance of the singleton class is maintained by a private class-level variable.
    • Access to this object is provided by a public class-level operation getInstance() which instantiates a single copy of the singleton class when it is executed for the first time. Subsequent calls to this operation return the single instance of the class.

    If Logic was not a Singleton class, an object is created like this:

    Logic m = new Logic();
    

    But now, the Logic object needs to be accessed like this:

    Logic m = Logic.getInstance();
    

    Evidence:

    Identify where in the project Singleton pattern has been used. Apply singleton pattern somewhere (in a toy example or in a project)

    W10.2e Can decide when to apply Singleton design pattern

    Design → Design Patterns → Singleton →

    Evaluation

    Pros:

    • easy to apply
    • effective in achieving its goal with minimal extra work
    • provides an easy way to access the singleton object from anywhere in the code base

    Cons:

    • The singleton object acts like a global variable that increases coupling across the code base.
    • In testing, it is difficult to replace Singleton objects with stubs (static methods cannot be overridden)
    • In testing, singleton objects carry data from one test to another even when we want each test to be independent of the others.

    Given there are some significant cons, it is recommended that you apply the Singleton pattern when, in addition to requiring only one instance of a class, there is a risk of creating multiple objects by mistake, and creating such multiple objects has real negative consequences.


    Evidence:

    Identify places in the project where only a single object of a class is needed but there is no need to apply the Singleton pattern.


    Facade pattern

    W10.2f Can explain the Facade design pattern

    Design → Design Patterns → Facade Pattern →

    What

    Context

    Components need to access functionality deep inside other components.

    The UI component of a Library system might want to access functionality of the Book class contained inside the Logic component.

    Problem

    Access to the component should be allowed without exposing its internal details.  e.g. the UI component should access the functionality of the Logic component without knowing that it contained a Book class within it.

    Solution

    Include a Façade class that sits between the component internals and users of the component such that all access to the component happens through the Facade class.

    The following class diagram applies the Façade pattern to the Library System example. The LibraryLogic class is the Facade class.

    Does the design below likely to use the Facade pattern?

    True.

    Facade is clearly visible (Storage is the << Facade >> class).


    Evidence:

    Discuss the current/potential applications of the pattern in your project.


    Command pattern

    W10.2g Can explain the Command design pattern

    Design → Design Patterns → Command Pattern →

    What

    Context

    A system is required to execute a number of commands, each doing a different task. For example, a system might have to support Sort, List, Reset commands.

    Problem

    It is preferable that some part of the code executes these commands without having to know each command type.  e.g., there can be a CommandQueue object that is responsible for queuing commands and executing them without knowledge of what each command does.

    Solution

    The essential element of this pattern is to have a general <<Command>> object that can be passed around, stored, executed, etc without knowing the type of command (i.e. via polymorphism).

    Let us examine an example application of the pattern first:

    In the example solution below, the CommandCreator creates List, Sort, and Reset Command objects and adds them to the CommandQueue object. The CommandQueue object treats them all as Command objects and performs the execute/undo operation on each of them without knowledge of the specific Command type. When executed, each Command object will access the DataStore object to carry out its task. The Command class can also be an abstract class or an interface.

    The general form of the solution is as follows.

    The <<Client>> creates a <<ConcreteCommand>> object, and passes it to the <<Invoker>>. The <<Invoker>> object treats all commands as a general <<Command>> type. <<Invoker>> issues a request by calling execute() on the command. If a command is undoable, <<ConcreteCommand>> will store the state for undoing the command prior to invoking execute(). In addition, the <<ConcreteCommand>> object may have to be linked to any <<Receiver>> of the command (?) before it is passed to the <<Invoker>>. Note that an application of the command pattern does not have to follow the structure given above.


    Evidence:

    Discuss the current/potential applications of the pattern in your project.


    Abstraction Occurrence pattern

    W10.2h Can explain the Abstraction Occurrence design pattern

    Design → Design Patterns → Abstraction Occurrence Pattern →

    What

    Context

    There is a group of similar entities that appears to be ‘occurrences’ (or ‘copies’) of the same thing, sharing lots of common information, but also differing in significant ways.

    In a library, there can be multiple copies of same book title. Each copy shares common information such as book title, author, ISBN etc. However, there are also significant differences like purchase date and barcode number (assumed to be unique for each copy of the book).

    Other examples:

    • Episodes of the same TV series
    • Stock items of the same product model (e.g. TV sets of the same model).

    Problem

    Representing the objects mentioned previously as a single class would be problematic because it results in duplication of data which can lead to inconsistencies in data (if some of the duplicates are not updated consistently).

    Take for example the problem of representing books in a library. Assume that there could be multiple copies of the same title, bearing the same ISBN number, but different serial numbers.

    The above solution requires common information to be duplicated by all instances. This will not only waste storage space, but also creates a consistency problem. Suppose that after creating several copies of the same title, the librarian realized that the author name was wrongly spelt. To correct this mistake, the system needs to go through every copy of the same title to make the correction. Also, if a new copy of the title is added later on, the librarian (or the system) has to make sure that all information entered is the same as the existing copies to avoid inconsistency.

    Anti-pattern

    Refer to the same Library example given above.

    The design above segregates the common and unique information into a class hierarchy. Each book title is represented by a separate class with common data (i.e. Name, Author, ISBN) hard-coded in the class itself. This solution is problematic because each book title is represented as a class, resulting in thousands of classes (one for each title). Every time the library buys new books, the source code of the system will have to be updated with new classes.

    Solution

    Let a copy of an entity (e.g. a copy of a book)be represented by two objects instead of one, separating the common and unique information into two classes to avoid duplication.

    Given below is how the pattern is applied to the Library example:

    Here's a more generic example:

    The general solution:

    The << Abstraction >> class should hold all common information, and the unique information should be kept by the << Occurrence >> class. Note that ‘Abstraction’ and ‘Occurrence’ are not class names, but roles played by each class. Think of this diagram as a meta-model (i.e. a ‘model of a model’) of the BookTitle-BookCopy class diagram given above.

    Which pairs of classes are likely to be the << Abstraction >> and the << Occurrence >> of the abstraction occurrence pattern?

    1. CarModel, Car. (Here CarModel represents a particular model of a car produced by the car manufacturer. E.g. BMW R4300)
    2. Car, Wheel
    3. Club, Member
    4. TeamLeader, TeamMember
    5. Magazine (E.g. ReadersDigest, PCWorld), MagazineIssue

    One of the key things to keep in mind is that the << Abstraction >> does not represent a real entity. Rather, it represents some information common to a set of objects. A single real entity is represented by an object of << Abstraction >> type and << Occurrence >> type.

    Before applying the pattern, some attributes have the same values for multiple objects. For example, w.r.t. the BookTitle-BookCopy example given in this handout, values of attributes such as book_title, ISBN are exactly the same for copies of the same book.

    After applying the pattern, the Abstraction and the Occurrence classes together represent one entity. It is like one class has been split into two. For example, a BookTitle object and a BookCopy object combines to represent an actual Book.

    1. CarModel, Car: Yes
    2. Car, Wheel: No. Wheel is a ‘part of’ Car. A wheel is not an occurrence of Car.
    3. Club, Member: No. this is a ‘part of’ relationship.
    4. TeamLeader, TeamMember: No. A TeamMember is not an occurrence of a TeamLeader or vice versa.
    5. Magazine, MagazineIssue: Yes.

    Which one of these is most suited for an application of the Abstraction Occurrence pattern?

    (a)

    Explanation:

    (a) Stagings of a drama are ‘occurrences’ of the drama. They have many attributes common (e.g., Drama name, producer, cast, etc.) but some attributes are different (e.g., venue, time).

    (b) Students are not occurrences of a Teacher or vice versa

    (c) Module, Exam, Assignment are distinct entities with associations among them. But none of them can be considered an occurrence of another.


    Evidence:

    Explain relevance of the pattern to the project.

    Quality Assurance

    W10.3 Can explain test case design W10.3a Can explain the need for deliberate test case design

    Quality Assurance → Test Case Design → Introduction →

    What

    Except for trivial SUTs, exhaustive testing is not practical because such testing often requires a massive/infinite number of test cases.

    Consider the test cases for adding a string object to a collection:

    • Add an item to an empty collection.
    • Add an item when there is one item in the collection.
    • Add an item when there are 2, 3, .... n items in the collection.
    • Add an item that has an English, a French, a Spanish, ... word.
    • Add an item that is the same as an existing item.
    • Add an item immediately after adding another item.
    • Add an item immediately after system startup.
    • ...

    Exhaustive testing of this operation can take many more test cases.

    Program testing can be used to show the presence of bugs, but never to show their absence!
    --Edsger Dijkstra

    Every test case adds to the cost of testing. In some systems, a single test case can cost thousands of dollars  e.g. on-field testing of flight-control software. Therefore, test cases need to be designed to make the best use of testing resources. In particular:

    • Testing should be effective i.e., it finds a high percentage of existing bugs e.g., a set of test cases that finds 60 defects is more effective than a set that finds only 30 defects in the same system.

    • Testing should be efficient i.e., it has a high rate of success (bugs found/test cases) a set of 20 test cases that finds 8 defects is more efficient than another set of 40 test cases that finds the same 8 defects.

    For testing to be E&E, each new test we add should be targeting a potential fault that is not already targeted by existing test cases. There are test case design techniques that can help us improve E&E of testing.

    Given below is the sample output from a text-based program TriangleDetector ithat determines whether the three input numbers make up the three sides of a valid triangle. List test cases you would use to test this software. Two sample test cases are given below.

    C:\> java TriangleDetector
    Enter side 1: 34
    Enter side 2: 34
    Enter side 3: 32
    Can this be a triangle?:  Yes
    Enter side 1:
    

    Sample test cases,

    34,34,34: Yes
    0, any valid, any valid: No
    

    In addition to obvious test cases such as

    • sum of two sides == third,
    • sum of two sides < third ...

    We may also devise some interesting test cases such as the ones depicted below.

    Note that their applicability depends on the context in which the software is operating.

    • Non-integer number, negative numbers, 0, numbers formatted differently (e.g. 13F), very large numbers (e.g. MAX_INT), numbers with many decimal places, empty string, ...
    • Check many triangles one after the other (will the system run out of memory?)
    • Backspace, tab, CTRL+C , …
    • Introduce a long delay between entering data (will the program be affected by, say the screensaver?), minimize and restore window during the operation, hibernate the system in the middle of a calculation, start with invalid inputs (the system may perform error handling differently for the very first test case), …
    • Test on different locale.

    The main point to note is how difficult it is to test exhaustively, even on a trivial system.

    Explain the why exhaustive testing is not practical using the example of testing newGame() operation in the Logic class of a Minesweeper game.

    Consider this sequence of test cases:

    • Test case 1. Start Minesweeper. Activate newGame() and see if it works.
    • Test case 2. Start Minesweeper. Activate newGame(). Activate newGame() again and see if it works.
    • Test case 3. Start Minesweeper. Activate newGame() three times consecutively and see if it works.
    • Test case 267. Start Minesweeper. Activate newGame() 267 times consecutively and see if it works.

    Well, you get the idea. Exhaustive testing of newGame() is not practical.

    Improving efficiency and effectiveness of test case design can,

    • a. improve the quality of the SUT.
    • b. save money.
    • c. save time spent on test execution.
    • d. save effort on writing and maintaining tests.
    • e. minimize redundant test cases.
    • f. forces us to understand the SUT better.

    (a)(b)(c)(d)(e)(f)


    Evidence:

    Explain why testing needs to be E&E, using examples from your project.

    W10.3b Can explain exploratory testing and scripted testing

    Quality Assurance → Testing → Exploratory and Scripted Testing →

    What

    Here are two alternative approaches to testing a software: Scripted testing and Exploratory testing

    1. Scripted testing: First write a set of test cases based on the expected behavior of the SUT, and then perform testing based on that set of test cases.

    2. Exploratory testing: Devise test cases on-the-fly, creating new test cases based on the results of the past test cases.

    Exploratory testing is ‘the simultaneous learning, test design, and test execution’ [source: bach-et-explained] whereby the nature of the follow-up test case is decided based on the behavior of the previous test cases. In other words, running the system and trying out various operations. It is called exploratory testing because testing is driven by observations during testing. Exploratory testing usually starts with areas identified as error-prone, based on the tester’s past experience with similar systems. One tends to conduct more tests for those operations where more faults are found.

    Here is an example thought process behind a segment of an exploratory testing session:

    “Hmm... looks like feature x is broken. This usually means feature n and k could be broken too; we need to look at them soon. But before that, let us give a good test run to feature y because users can still use the product if feature y works, even if x doesn’t work. Now, if feature y doesn’t work 100%, we have a major problem and this has to be made known to the development team sooner rather than later...”

    💡 Exploratory testing is also known as reactive testing, error guessing technique, attack-based testing, and bug hunting.

    Exploratory Testing Explained, an online article by James Bach -- James Bach is an industry thought leader in software testing).

    Scripted testing requires tests to be written in a scripting language; Manual testing is called exploratory testing.

    A) False

    Explanation: “Scripted” means test cases are predetermined. They need not be an executable script. However, exploratory testing is usually manual.

    Which testing technique is better?

    (e)

    Explain the concept of exploratory testing using Minesweeper as an example.

    When we test the Minesweeper by simply playing it in various ways, especially trying out those that are likely to be buggy, that would be exploratory testing.


    Evidence:

    Explain exploratory and scripted testing in the context of your project.

    W10.3c Can explain the choice between exploratory testing and scripted testing

    Quality Assurance → Testing → Exploratory and Scripted Testing →

    When

    Which approach is better – scripted or exploratory? A mix is better.

    The success of exploratory testing depends on the tester’s prior experience and intuition. Exploratory testing should be done by experienced testers, using a clear strategy/plan/framework. Ad-hoc exploratory testing by unskilled or inexperienced testers without a clear strategy is not recommended for real-world non-trivial systems. While exploratory testing may allow us to detect some problems in a relatively short time, it is not prudent to use exploratory testing as the sole means of testing a critical system.

    Scripted testing is more systematic, and hence, likely to discover more bugs given sufficient time, while exploratory testing would aid in quick error discovery, especially if the tester has a lot of experience in testing similar systems.

    In some contexts, you will achieve your testing mission better through a more scripted approach; in other contexts, your mission will benefit more from the ability to create and improve tests as you execute them. I find that most situations benefit from a mix of scripted and exploratory approaches. --[source: bach-et-explained]

    Exploratory Testing Explained, an online article by James Bach -- James Bach is an industry thought leader in software testing).

    Scripted testing is better than exploratory testing.

    B) False

    Explanation: Each has pros and cons. Relying on only one is not recommended. A combination is better.

    W10.3d Can explain positive and negative test cases

    Quality Assurance → Test Case Design → Introduction →

    Positive vs Negative Test Cases

    A positive test case is when the test is designed to produce an expected/valid behavior. A negative test case is designed to produce a behavior that indicates an invalid/unexpected situation, such as an error message.

    Consider testing of the method print(Integer i) which prints the value of i.

    • A positive test case: i == new Integer(50)
    • A negative test case: i == null;

    Evidence:

    Give examples of positive and negative test cases from your project.

    W10.3e Can explain black box and glass box test case design

    Quality Assurance → Test Case Design → Introduction →

    Black Box vs Glass Box

    Test case design can be of three types, based on how much of SUT internal details are considered when designing test cases:

    • Black-box (aka specification-based or responsibility-based) approach: test cases are designed exclusively based on the SUT’s specified external behavior.

    • White-box (aka glass-box or structured or implementation-based) approach: test cases are designed based on what is known about the SUT’s implementation, i.e. the code.

    • Gray-box approach: test case design uses some important information about the implementation. For example, if the implementation of a sort operation uses different algorithms to sort lists shorter than 1000 items and lists longer than 1000 items, more meaningful test cases can then be added to verify the correctness of both algorithms.

    Note: these videos are from the Udacity course Software Development Process by Georgia Tech

     


     


     



    Evidence:

    Explain black/white/grey box testing using examples from your project.

    W10.3f Can explain test case design for use case based testing

    Quality Assurance → Test Case Design →

    Testing Based on Use Cases

    Use cases can be used for system testing and acceptance testing. For example, the main success scenario can be one test case while each variation (due to extensions) can form another test case. However, note that use cases do not specify the exact data entered into the system. Instead, it might say something like user enters his personal data into the system. Therefore, the tester has to choose data by considering equivalence partitions and boundary values. The combinations of these could result in one use case producing many test cases.

    To increase E&E of testing, high-priority use cases are given more attention. For example, a scripted approach can be used to test high priority test cases, while an exploratory approach is used to test other areas of concern that could emerge during testing.

     

    Every test case adds to the cost of testing. In some systems, a single test case can cost thousands of dollars  e.g. on-field testing of flight-control software. Therefore, test cases need to be designed to make the best use of testing resources. In particular:

    • Testing should be effective i.e., it finds a high percentage of existing bugs e.g., a set of test cases that finds 60 defects is more effective than a set that finds only 30 defects in the same system.

    • Testing should be efficient i.e., it has a high rate of success (bugs found/test cases) a set of 20 test cases that finds 8 defects is more efficient than another set of 40 test cases that finds the same 8 defects.

    For testing to be E&E, each new test we add should be targeting a potential fault that is not already targeted by existing test cases. There are test case design techniques that can help us improve E&E of testing.

     

    Quality Assurance → Testing → Exploratory and Scripted Testing →

    What

    Here are two alternative approaches to testing a software: Scripted testing and Exploratory testing

    1. Scripted testing: First write a set of test cases based on the expected behavior of the SUT, and then perform testing based on that set of test cases.

    2. Exploratory testing: Devise test cases on-the-fly, creating new test cases based on the results of the past test cases.

    Exploratory testing is ‘the simultaneous learning, test design, and test execution’ [source: bach-et-explained] whereby the nature of the follow-up test case is decided based on the behavior of the previous test cases. In other words, running the system and trying out various operations. It is called exploratory testing because testing is driven by observations during testing. Exploratory testing usually starts with areas identified as error-prone, based on the tester’s past experience with similar systems. One tends to conduct more tests for those operations where more faults are found.

    Here is an example thought process behind a segment of an exploratory testing session:

    “Hmm... looks like feature x is broken. This usually means feature n and k could be broken too; we need to look at them soon. But before that, let us give a good test run to feature y because users can still use the product if feature y works, even if x doesn’t work. Now, if feature y doesn’t work 100%, we have a major problem and this has to be made known to the development team sooner rather than later...”

    💡 Exploratory testing is also known as reactive testing, error guessing technique, attack-based testing, and bug hunting.

    Exploratory Testing Explained, an online article by James Bach -- James Bach is an industry thought leader in software testing).

    Scripted testing requires tests to be written in a scripting language; Manual testing is called exploratory testing.

    A) False

    Explanation: “Scripted” means test cases are predetermined. They need not be an executable script. However, exploratory testing is usually manual.

    Which testing technique is better?

    (e)

    Explain the concept of exploratory testing using Minesweeper as an example.

    When we test the Minesweeper by simply playing it in various ways, especially trying out those that are likely to be buggy, that would be exploratory testing.


    Evidence:

    Explain use case based test case design using examples from your project.

    W10.4 Can use EP for test case design W10.4a Can explain equivalence partitions

    Quality Assurance → Test Case Design → Equivalence Partitions →

    What

    Consider the testing of the following operation.

    isValidMonth(m) : returns true if m (and int) is in the range [1..12]

    It is inefficient and impractical to test this method for all integer values [-MIN_INT to MAX_INT]. Fortunately, there is no need to test all possible input values. For example, if the input value 233 failed to produce the correct result, the input 234 is likely to fail too; there is no need to test both.

    In general, most SUTs do not treat each input in a unique way. Instead, they process all possible inputs in a small number of distinct ways. That means a range of inputs is treated the same way inside the SUT. Equivalence partitioning (EP) is a test case design technique that uses the above observation to improve the E&E of testing.

    Equivalence partition (aka equivalence class): A group of test inputs that are likely to be processed by the SUT in the same way.

    By dividing possible inputs into equivalence partitions we can,

    • avoid testing too many inputs from one partition. Testing too many inputs from the same partition is unlikely to find new bugs. This increases the efficiency of testing by reducing redundant test cases.
    • ensure all partitions are tested. Missing partitions can result in bugs going unnoticed. This increases the effectiveness of testing by increasing the chance of finding bugs.
    W10.4b Can apply EP for pure functions

    Quality Assurance → Test Case Design → Equivalence Partitions →

    Basic

    Equivalence partitions (EPs) are usually derived from the specifications of the SUT.

    These could be EPs for the isValidMonth example:

    • [MIN_INT ... 0] : below the range that produces true (produces false)
    • [1 … 12] : the range that produces true
    • [13 … MAX_INT] : above the range that produces true (produces false)
     

    isValidMonth(m) : returns true if m (and int) is in the range [1..12]

    When the SUT has multiple inputs, you should identify EPs for each input.

    Consider the method duplicate(String s, int n): String which returns a String that contains s repeated n times.

    Example EPs for s:

    • zero-length strings
    • string containing whitespaces
    • ...

    Example EPs for n:

    • 0
    • negative values
    • ...

    An EP may not have adjacent values.

    Consider the method isPrime(int i): boolean that returns true if i is a prime number.

    EPs for i:

    • prime numbers
    • non-prime numbers

    Some inputs have only a small number of possible values and a potentially unique behavior for each value. In those cases we have to consider each value as a partition by itself.

    Consider the method showStatusMessage(GameStatus s): String that returns a unique String for each of the possible value of s (GameStatus is an enum). In this case, each possible value for s will have to be considered as a partition.

    Note that the EP technique is merely a heuristic and not an exact science, especially when applied manually (as opposed to using an automated program analysis tool to derive EPs). The partitions derived depend on how one ‘speculates’ the SUT to behave internally. Applying EP under a glass-box or gray-box approach can yield more precise partitions.

    Consider the method EPs given above for the isValidMonth. A different tester might use these EPs instead:

    • [1 … 12] : the range that produces true
    • [all other integers] : the range that produces false

    Some more examples:

    Specification Equivalence partitions

    isValidFlag(String s): boolean
    Returns true if s is one of ["F", "T", "D"]. The comparison is case-sensitive.

    ["F"] ["T"] ["D"] ["f", "t", "d"] [any other string][null]

    squareRoot(String s): int
    Pre-conditions: s represents a positive integer
    Returns the square root of s if the square root is an integer; returns 0 otherwise.

    [s is not a valid number] [s is a negative integer] [s has an integer square root] [s does not have an integer square root]

    Consider this SUT:

    isValidName (String s): boolean

    Description: returns true if s is not null and not longer than 50 characters.

    A. Which one of these is least likely to be an equivalence partition for the parameter s of the isValidName method given below?

    B. If you had to choose 3 test cases from the 4 given below, which one will you leave out based on the EP technique?

    A. (d)

    Explanation: The description does not mention anything about the content of the string. Therefore, the method is unlikely to behave differently for strings consisting of numbers.

    B. (a) or (c)

    Explanation: both belong to the same EP

    W10.4c Can apply EP for OOP methods

    Quality Assurance → Test Case Design → Equivalence Partitions →

    Intermediate

    When deciding EPs of OOP methods, we need to identify EPs of all data participants that can potentially influence the behaviour of the method, such as,

    • the target object of the method call
    • input parameters of the method call
    • other data/objects accessed by the method such as global variables. This category may not be applicable if using the black box approach (because the test case designer using the black box approach will not know how the method is implemented)

    Consider this method in the DataStack class: push(Object o): boolean

    • Adds o to the top of the stack if the stack is not full.
    • returns true if the push operation was a success.
    • throws
      • MutabilityException if the global flag FREEZE==true.
      • InvalidValueException if o is null.

    EPs:

    • DataStack object: [full] [not full]
    • o: [null] [not null]
    • FREEZE: [true][false]

    Consider a simple Minesweeper app. What are the EPs for the newGame() method of the Logic component?

    As newGame() does not have any parameters, the only obvious participant is the Logic object itself.

    Note that if the glass-box or the grey-box approach is used, other associated objects that are involved in the method might also be included as participants. For example, Minefield object can be considered as another participant of the newGame() method. Here, the black-box approach is assumed.

    Next, let us identify equivalence partitions for each participant. Will the newGame() method behave differently for different Logic objects? If yes, how will it differ? In this case, yes, it might behave differently based on the game state. Therefore, the equivalence partitions are:

    • PRE_GAME : before the game starts, minefield does not exist yet
    • READY : a new minefield has been created and waiting for player’s first move
    • IN_PLAY : the current minefield is already in use
    • WON, LOST : let us assume the newGame behaves the same way for these two values

    Consider the Logic component of the Minesweeper application. What are the EPs for the markCellAt(int x, int y) method?. The partitions in bold represent valid inputs.

    • Logic: PRE_GAME, READY, IN_PLAY, WON, LOST
    • x: [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] (we assume a minefield size of WxH)
    • y: [MIN_INT..-1] [0..(H-1)] [H..MAX_INT]
    • Cell at (x,y): HIDDEN, MARKED, CLEARED

    Evidence:

    Explain equivalence partitioning using examples from the project.

    W10.5 Can use Boundary Value Analysis for test case design W10.5a Can explain boundary value analysis

    Quality Assurance → Test Case Design → Boundary Value Analysis →

    What

    Boundary Value Analysis (BVA) is test case design heuristic that is based on the observation that bugs often result from incorrect handling of boundaries of equivalence partitions. This is not surprising, as the end points of the boundary are often used in branching instructions etc. where the programmer can make mistakes.

    markCellAt(int x, int y) operation could contain code such as if (x > 0 && x <= (W-1)) which involves boundaries of x’s equivalence partitions.

    BVA suggests that when picking test inputs from an equivalence partition, values near boundaries (i.e. boundary values) are more likely to find bugs.

    Boundary values are sometimes called corner cases.

    Boundary value analysis recommends testing only values that reside on the equivalence class boundary.

    False

    Explanation: It does not recommend testing only those values on the boundary. It merely suggests that values on and around a boundary are more likely to cause errors.

    W10.5b Can apply boundary value analysis

    Quality Assurance → Test Case Design → Boundary Value Analysis →

    How

    Typically, we choose three values around the boundary to test: one value from the boundary, one value just below the boundary, and one value just above the boundary. The number of values to pick depends on other factors, such as the cost of each test case.

    Some examples:

    Equivalence partition Some possible boundary values

    [1-12]

    0,1,2, 11,12,13

    [MIN_INT, 0]
    (MIN_INT is the minimum possible integer value allowed by the environment)

    MIN_INT, MIN_INT+1, -1, 0 , 1

    [any non-null String]

    Empty String, a String of maximum possible length

    [prime numbers]
    [“F”]
    [“A”, “D”, “X”]

    No specific boundary
    No specific boundary
    No specific boundary

    [non-empty Stack]
    (we assume a fixed size stack)

    Stack with: one element, two elements, no empty spaces, only one empty space

    🅿️ Project

    W10.6 Can evolve a product incrementally

    Covered by:

    Tutorial 10

    Questions to discuss during tutorial:

    Whole team: do this question together, using the whiteboard:

    Draw an OODM for the description below, about how courses work in a certain university:

    A course has a name and a code. A course is read by 10 or more students, and taught by one instructor. A course can have a number tasks which can be assignments or tests. Some assignments are compulsory. When a student attempts a task, a grade and an optional feedback is given.


    Divide these questions among team members and get ready to answer them.

    Q1

    1. How does an OO Domain Model differ from a class diagram?
    2. What's the relationship between an OODM and a class diagram for a software? e.g., exactly the same?, one is a sub-set of the other?
    3. Which diagram can we use to show classobject structures in a problem domain?

    Q2

    1. Explain and justify: testing should be efficient and effective
    2. Explain how exploratory and scripted testing is used in AB4/project
    3. Give an example of a negative test case in AB4/project
    4. Explain grey-box test case design

    Q3

    1. Explain: Equivalence Partition improve E&E of testing
    2. What are the EPs for the parameter day of this method
      /**
       * Returns true if the three values represent a valid day
       */
      boolean isValidDay(int year, int month, int day){
      
      } 
      

    Q4

    1. Explain: Boundary Value Analysis improves E&E of testing
    2. What are the boundary values for the parameter day in the question above?
    3. How can EP and BVA heuristics be used in AB4/project? Hint: See [AB4 Learning Outcomes: LO-TestCaseDesignHeuristics]

    Q5

    1. What’s a design pattern?
    2. When do you use the Singleton pattern, Facade pattern, Command pattern?
    3. Which of the three patterns are used in AB4? Hint: See [AB4 Learning Outcomes: LO-DesignPatterns]

    For W10.2c Can explain the Singleton design pattern
    Details of the LO

    Design → Design Patterns → Singleton →

    What

    Context

    A certain classes should have no more than just one instance (e.g. the main controller class of the system). These single instances are commonly known as singletons.

    Problem

    A normal class can be instantiated multiple times by invoking the constructor.

    Solution

    Make the constructor of the singleton class private,  because a public constructor will allow others to instantiate the class at will. Provide a public class-level method to access the single instance.

    Example:

    We use the Singleton pattern when

    (c)



    Evidence:

    Identify where in the project Singleton pattern can be used.

    For W10.2d Can apply the Singleton design pattern
    Details of the LO

    Design → Design Patterns → Singleton →

    Implementation

    Here is the typical implementation of how the Singleton pattern is applied to a class:

    class Logic {
        private static Logic theOne = null;
    
        private Logic() {
            ...
        }
    
        public static Logic getInstance() {
            if (theOne == null) {
                theOne = new Logic();
            }
            return theOne;
        }
    }
    

    Notes:

    • The constructor is private, which prevents instantiation from outside the class.
    • The single instance of the singleton class is maintained by a private class-level variable.
    • Access to this object is provided by a public class-level operation getInstance() which instantiates a single copy of the singleton class when it is executed for the first time. Subsequent calls to this operation return the single instance of the class.

    If Logic was not a Singleton class, an object is created like this:

    Logic m = new Logic();
    

    But now, the Logic object needs to be accessed like this:

    Logic m = Logic.getInstance();
    


    Evidence:

    Identify where in the project Singleton pattern has been used. Apply singleton pattern somewhere (in a toy example or in a project)

    For W10.2e Can decide when to apply Singleton design pattern
    Details of the LO

    Design → Design Patterns → Singleton →

    Evaluation

    Pros:

    • easy to apply
    • effective in achieving its goal with minimal extra work
    • provides an easy way to access the singleton object from anywhere in the code base

    Cons:

    • The singleton object acts like a global variable that increases coupling across the code base.
    • In testing, it is difficult to replace Singleton objects with stubs (static methods cannot be overridden)
    • In testing, singleton objects carry data from one test to another even when we want each test to be independent of the others.

    Given there are some significant cons, it is recommended that you apply the Singleton pattern when, in addition to requiring only one instance of a class, there is a risk of creating multiple objects by mistake, and creating such multiple objects has real negative consequences.



    Evidence:

    Identify places in the project where only a single object of a class is needed but there is no need to apply the Singleton pattern.

    For W10.2f Can explain the Facade design pattern
    Details of the LO

    Design → Design Patterns → Facade Pattern →

    What

    Context

    Components need to access functionality deep inside other components.

    The UI component of a Library system might want to access functionality of the Book class contained inside the Logic component.

    Problem

    Access to the component should be allowed without exposing its internal details.  e.g. the UI component should access the functionality of the Logic component without knowing that it contained a Book class within it.

    Solution

    Include a Façade class that sits between the component internals and users of the component such that all access to the component happens through the Facade class.

    The following class diagram applies the Façade pattern to the Library System example. The LibraryLogic class is the Facade class.

    Does the design below likely to use the Facade pattern?

    True.

    Facade is clearly visible (Storage is the << Facade >> class).



    Evidence:

    Discuss the current/potential applications of the pattern in your project.

    For W10.2g Can explain the Command design pattern
    Details of the LO

    Design → Design Patterns → Command Pattern →

    What

    Context

    A system is required to execute a number of commands, each doing a different task. For example, a system might have to support Sort, List, Reset commands.

    Problem

    It is preferable that some part of the code executes these commands without having to know each command type.  e.g., there can be a CommandQueue object that is responsible for queuing commands and executing them without knowledge of what each command does.

    Solution

    The essential element of this pattern is to have a general <<Command>> object that can be passed around, stored, executed, etc without knowing the type of command (i.e. via polymorphism).

    Let us examine an example application of the pattern first:

    In the example solution below, the CommandCreator creates List, Sort, and Reset Command objects and adds them to the CommandQueue object. The CommandQueue object treats them all as Command objects and performs the execute/undo operation on each of them without knowledge of the specific Command type. When executed, each Command object will access the DataStore object to carry out its task. The Command class can also be an abstract class or an interface.

    The general form of the solution is as follows.

    The <<Client>> creates a <<ConcreteCommand>> object, and passes it to the <<Invoker>>. The <<Invoker>> object treats all commands as a general <<Command>> type. <<Invoker>> issues a request by calling execute() on the command. If a command is undoable, <<ConcreteCommand>> will store the state for undoing the command prior to invoking execute(). In addition, the <<ConcreteCommand>> object may have to be linked to any <<Receiver>> of the command (?) before it is passed to the <<Invoker>>. Note that an application of the command pattern does not have to follow the structure given above.



    Evidence:

    Discuss the current/potential applications of the pattern in your project.

    For W10.2h Can explain the Abstraction Occurrence design pattern
    Details of the LO

    Design → Design Patterns → Abstraction Occurrence Pattern →

    What

    Context

    There is a group of similar entities that appears to be ‘occurrences’ (or ‘copies’) of the same thing, sharing lots of common information, but also differing in significant ways.

    In a library, there can be multiple copies of same book title. Each copy shares common information such as book title, author, ISBN etc. However, there are also significant differences like purchase date and barcode number (assumed to be unique for each copy of the book).

    Other examples:

    • Episodes of the same TV series
    • Stock items of the same product model (e.g. TV sets of the same model).

    Problem

    Representing the objects mentioned previously as a single class would be problematic because it results in duplication of data which can lead to inconsistencies in data (if some of the duplicates are not updated consistently).

    Take for example the problem of representing books in a library. Assume that there could be multiple copies of the same title, bearing the same ISBN number, but different serial numbers.

    The above solution requires common information to be duplicated by all instances. This will not only waste storage space, but also creates a consistency problem. Suppose that after creating several copies of the same title, the librarian realized that the author name was wrongly spelt. To correct this mistake, the system needs to go through every copy of the same title to make the correction. Also, if a new copy of the title is added later on, the librarian (or the system) has to make sure that all information entered is the same as the existing copies to avoid inconsistency.

    Anti-pattern

    Refer to the same Library example given above.

    The design above segregates the common and unique information into a class hierarchy. Each book title is represented by a separate class with common data (i.e. Name, Author, ISBN) hard-coded in the class itself. This solution is problematic because each book title is represented as a class, resulting in thousands of classes (one for each title). Every time the library buys new books, the source code of the system will have to be updated with new classes.

    Solution

    Let a copy of an entity (e.g. a copy of a book)be represented by two objects instead of one, separating the common and unique information into two classes to avoid duplication.

    Given below is how the pattern is applied to the Library example:

    Here's a more generic example:

    The general solution:

    The << Abstraction >> class should hold all common information, and the unique information should be kept by the << Occurrence >> class. Note that ‘Abstraction’ and ‘Occurrence’ are not class names, but roles played by each class. Think of this diagram as a meta-model (i.e. a ‘model of a model’) of the BookTitle-BookCopy class diagram given above.

    Which pairs of classes are likely to be the << Abstraction >> and the << Occurrence >> of the abstraction occurrence pattern?

    1. CarModel, Car. (Here CarModel represents a particular model of a car produced by the car manufacturer. E.g. BMW R4300)
    2. Car, Wheel
    3. Club, Member
    4. TeamLeader, TeamMember
    5. Magazine (E.g. ReadersDigest, PCWorld), MagazineIssue

    One of the key things to keep in mind is that the << Abstraction >> does not represent a real entity. Rather, it represents some information common to a set of objects. A single real entity is represented by an object of << Abstraction >> type and << Occurrence >> type.

    Before applying the pattern, some attributes have the same values for multiple objects. For example, w.r.t. the BookTitle-BookCopy example given in this handout, values of attributes such as book_title, ISBN are exactly the same for copies of the same book.

    After applying the pattern, the Abstraction and the Occurrence classes together represent one entity. It is like one class has been split into two. For example, a BookTitle object and a BookCopy object combines to represent an actual Book.

    1. CarModel, Car: Yes
    2. Car, Wheel: No. Wheel is a ‘part of’ Car. A wheel is not an occurrence of Car.
    3. Club, Member: No. this is a ‘part of’ relationship.
    4. TeamLeader, TeamMember: No. A TeamMember is not an occurrence of a TeamLeader or vice versa.
    5. Magazine, MagazineIssue: Yes.

    Which one of these is most suited for an application of the Abstraction Occurrence pattern?

    (a)

    Explanation:

    (a) Stagings of a drama are ‘occurrences’ of the drama. They have many attributes common (e.g., Drama name, producer, cast, etc.) but some attributes are different (e.g., venue, time).

    (b) Students are not occurrences of a Teacher or vice versa

    (c) Module, Exam, Assignment are distinct entities with associations among them. But none of them can be considered an occurrence of another.



    Evidence:

    Explain relevance of the pattern to the project.

    For W10.3a Can explain the need for deliberate test case design
    Details of the LO

    Quality Assurance → Test Case Design → Introduction →

    What

    Except for trivial SUTs, exhaustive testing is not practical because such testing often requires a massive/infinite number of test cases.

    Consider the test cases for adding a string object to a collection:

    • Add an item to an empty collection.
    • Add an item when there is one item in the collection.
    • Add an item when there are 2, 3, .... n items in the collection.
    • Add an item that has an English, a French, a Spanish, ... word.
    • Add an item that is the same as an existing item.
    • Add an item immediately after adding another item.
    • Add an item immediately after system startup.
    • ...

    Exhaustive testing of this operation can take many more test cases.

    Program testing can be used to show the presence of bugs, but never to show their absence!
    --Edsger Dijkstra

    Every test case adds to the cost of testing. In some systems, a single test case can cost thousands of dollars  e.g. on-field testing of flight-control software. Therefore, test cases need to be designed to make the best use of testing resources. In particular:

    • Testing should be effective i.e., it finds a high percentage of existing bugs e.g., a set of test cases that finds 60 defects is more effective than a set that finds only 30 defects in the same system.

    • Testing should be efficient i.e., it has a high rate of success (bugs found/test cases) a set of 20 test cases that finds 8 defects is more efficient than another set of 40 test cases that finds the same 8 defects.

    For testing to be E&E, each new test we add should be targeting a potential fault that is not already targeted by existing test cases. There are test case design techniques that can help us improve E&E of testing.

    Given below is the sample output from a text-based program TriangleDetector ithat determines whether the three input numbers make up the three sides of a valid triangle. List test cases you would use to test this software. Two sample test cases are given below.

    C:\> java TriangleDetector
    Enter side 1: 34
    Enter side 2: 34
    Enter side 3: 32
    Can this be a triangle?:  Yes
    Enter side 1:
    

    Sample test cases,

    34,34,34: Yes
    0, any valid, any valid: No
    

    In addition to obvious test cases such as

    • sum of two sides == third,
    • sum of two sides < third ...

    We may also devise some interesting test cases such as the ones depicted below.

    Note that their applicability depends on the context in which the software is operating.

    • Non-integer number, negative numbers, 0, numbers formatted differently (e.g. 13F), very large numbers (e.g. MAX_INT), numbers with many decimal places, empty string, ...
    • Check many triangles one after the other (will the system run out of memory?)
    • Backspace, tab, CTRL+C , …
    • Introduce a long delay between entering data (will the program be affected by, say the screensaver?), minimize and restore window during the operation, hibernate the system in the middle of a calculation, start with invalid inputs (the system may perform error handling differently for the very first test case), …
    • Test on different locale.

    The main point to note is how difficult it is to test exhaustively, even on a trivial system.

    Explain the why exhaustive testing is not practical using the example of testing newGame() operation in the Logic class of a Minesweeper game.

    Consider this sequence of test cases:

    • Test case 1. Start Minesweeper. Activate newGame() and see if it works.
    • Test case 2. Start Minesweeper. Activate newGame(). Activate newGame() again and see if it works.
    • Test case 3. Start Minesweeper. Activate newGame() three times consecutively and see if it works.
    • Test case 267. Start Minesweeper. Activate newGame() 267 times consecutively and see if it works.

    Well, you get the idea. Exhaustive testing of newGame() is not practical.

    Improving efficiency and effectiveness of test case design can,

    • a. improve the quality of the SUT.
    • b. save money.
    • c. save time spent on test execution.
    • d. save effort on writing and maintaining tests.
    • e. minimize redundant test cases.
    • f. forces us to understand the SUT better.

    (a)(b)(c)(d)(e)(f)



    Evidence:

    Explain why testing needs to be E&E, using examples from your project.

    For W10.3b Can explain exploratory testing and scripted testing
    Details of the LO

    Quality Assurance → Testing → Exploratory and Scripted Testing →

    What

    Here are two alternative approaches to testing a software: Scripted testing and Exploratory testing

    1. Scripted testing: First write a set of test cases based on the expected behavior of the SUT, and then perform testing based on that set of test cases.

    2. Exploratory testing: Devise test cases on-the-fly, creating new test cases based on the results of the past test cases.

    Exploratory testing is ‘the simultaneous learning, test design, and test execution’ [source: bach-et-explained] whereby the nature of the follow-up test case is decided based on the behavior of the previous test cases. In other words, running the system and trying out various operations. It is called exploratory testing because testing is driven by observations during testing. Exploratory testing usually starts with areas identified as error-prone, based on the tester’s past experience with similar systems. One tends to conduct more tests for those operations where more faults are found.

    Here is an example thought process behind a segment of an exploratory testing session:

    “Hmm... looks like feature x is broken. This usually means feature n and k could be broken too; we need to look at them soon. But before that, let us give a good test run to feature y because users can still use the product if feature y works, even if x doesn’t work. Now, if feature y doesn’t work 100%, we have a major problem and this has to be made known to the development team sooner rather than later...”

    💡 Exploratory testing is also known as reactive testing, error guessing technique, attack-based testing, and bug hunting.

    Exploratory Testing Explained, an online article by James Bach -- James Bach is an industry thought leader in software testing).

    Scripted testing requires tests to be written in a scripting language; Manual testing is called exploratory testing.

    A) False

    Explanation: “Scripted” means test cases are predetermined. They need not be an executable script. However, exploratory testing is usually manual.

    Which testing technique is better?

    (e)

    Explain the concept of exploratory testing using Minesweeper as an example.

    When we test the Minesweeper by simply playing it in various ways, especially trying out those that are likely to be buggy, that would be exploratory testing.



    Evidence:

    Explain exploratory and scripted testing in the context of your project.

    For W10.3d Can explain positive and negative test cases
    Details of the LO

    Quality Assurance → Test Case Design → Introduction →

    Positive vs Negative Test Cases

    A positive test case is when the test is designed to produce an expected/valid behavior. A negative test case is designed to produce a behavior that indicates an invalid/unexpected situation, such as an error message.

    Consider testing of the method print(Integer i) which prints the value of i.

    • A positive test case: i == new Integer(50)
    • A negative test case: i == null;


    Evidence:

    Give examples of positive and negative test cases from your project.

    For W10.3e Can explain black box and glass box test case design
    Details of the LO

    Quality Assurance → Test Case Design → Introduction →

    Black Box vs Glass Box

    Test case design can be of three types, based on how much of SUT internal details are considered when designing test cases:

    • Black-box (aka specification-based or responsibility-based) approach: test cases are designed exclusively based on the SUT’s specified external behavior.

    • White-box (aka glass-box or structured or implementation-based) approach: test cases are designed based on what is known about the SUT’s implementation, i.e. the code.

    • Gray-box approach: test case design uses some important information about the implementation. For example, if the implementation of a sort operation uses different algorithms to sort lists shorter than 1000 items and lists longer than 1000 items, more meaningful test cases can then be added to verify the correctness of both algorithms.

    Note: these videos are from the Udacity course Software Development Process by Georgia Tech

     


     


     




    Evidence:

    Explain black/white/grey box testing using examples from your project.

    For W10.3f Can explain test case design for use case based testing
    Details of the LO

    Quality Assurance → Test Case Design →

    Testing Based on Use Cases

    Use cases can be used for system testing and acceptance testing. For example, the main success scenario can be one test case while each variation (due to extensions) can form another test case. However, note that use cases do not specify the exact data entered into the system. Instead, it might say something like user enters his personal data into the system. Therefore, the tester has to choose data by considering equivalence partitions and boundary values. The combinations of these could result in one use case producing many test cases.

    To increase E&E of testing, high-priority use cases are given more attention. For example, a scripted approach can be used to test high priority test cases, while an exploratory approach is used to test other areas of concern that could emerge during testing.

     

    Every test case adds to the cost of testing. In some systems, a single test case can cost thousands of dollars  e.g. on-field testing of flight-control software. Therefore, test cases need to be designed to make the best use of testing resources. In particular:

    • Testing should be effective i.e., it finds a high percentage of existing bugs e.g., a set of test cases that finds 60 defects is more effective than a set that finds only 30 defects in the same system.

    • Testing should be efficient i.e., it has a high rate of success (bugs found/test cases) a set of 20 test cases that finds 8 defects is more efficient than another set of 40 test cases that finds the same 8 defects.

    For testing to be E&E, each new test we add should be targeting a potential fault that is not already targeted by existing test cases. There are test case design techniques that can help us improve E&E of testing.

     

    Quality Assurance → Testing → Exploratory and Scripted Testing →

    What

    Here are two alternative approaches to testing a software: Scripted testing and Exploratory testing

    1. Scripted testing: First write a set of test cases based on the expected behavior of the SUT, and then perform testing based on that set of test cases.

    2. Exploratory testing: Devise test cases on-the-fly, creating new test cases based on the results of the past test cases.

    Exploratory testing is ‘the simultaneous learning, test design, and test execution’ [source: bach-et-explained] whereby the nature of the follow-up test case is decided based on the behavior of the previous test cases. In other words, running the system and trying out various operations. It is called exploratory testing because testing is driven by observations during testing. Exploratory testing usually starts with areas identified as error-prone, based on the tester’s past experience with similar systems. One tends to conduct more tests for those operations where more faults are found.

    Here is an example thought process behind a segment of an exploratory testing session:

    “Hmm... looks like feature x is broken. This usually means feature n and k could be broken too; we need to look at them soon. But before that, let us give a good test run to feature y because users can still use the product if feature y works, even if x doesn’t work. Now, if feature y doesn’t work 100%, we have a major problem and this has to be made known to the development team sooner rather than later...”

    💡 Exploratory testing is also known as reactive testing, error guessing technique, attack-based testing, and bug hunting.

    Exploratory Testing Explained, an online article by James Bach -- James Bach is an industry thought leader in software testing).

    Scripted testing requires tests to be written in a scripting language; Manual testing is called exploratory testing.

    A) False

    Explanation: “Scripted” means test cases are predetermined. They need not be an executable script. However, exploratory testing is usually manual.

    Which testing technique is better?

    (e)

    Explain the concept of exploratory testing using Minesweeper as an example.

    When we test the Minesweeper by simply playing it in various ways, especially trying out those that are likely to be buggy, that would be exploratory testing.



    Evidence:

    Explain use case based test case design using examples from your project.

    For W10.4c Can apply EP for OOP methods
    Details of the LO

    Quality Assurance → Test Case Design → Equivalence Partitions →

    Intermediate

    When deciding EPs of OOP methods, we need to identify EPs of all data participants that can potentially influence the behaviour of the method, such as,

    • the target object of the method call
    • input parameters of the method call
    • other data/objects accessed by the method such as global variables. This category may not be applicable if using the black box approach (because the test case designer using the black box approach will not know how the method is implemented)

    Consider this method in the DataStack class: push(Object o): boolean

    • Adds o to the top of the stack if the stack is not full.
    • returns true if the push operation was a success.
    • throws
      • MutabilityException if the global flag FREEZE==true.
      • InvalidValueException if o is null.

    EPs:

    • DataStack object: [full] [not full]
    • o: [null] [not null]
    • FREEZE: [true][false]

    Consider a simple Minesweeper app. What are the EPs for the newGame() method of the Logic component?

    As newGame() does not have any parameters, the only obvious participant is the Logic object itself.

    Note that if the glass-box or the grey-box approach is used, other associated objects that are involved in the method might also be included as participants. For example, Minefield object can be considered as another participant of the newGame() method. Here, the black-box approach is assumed.

    Next, let us identify equivalence partitions for each participant. Will the newGame() method behave differently for different Logic objects? If yes, how will it differ? In this case, yes, it might behave differently based on the game state. Therefore, the equivalence partitions are:

    • PRE_GAME : before the game starts, minefield does not exist yet
    • READY : a new minefield has been created and waiting for player’s first move
    • IN_PLAY : the current minefield is already in use
    • WON, LOST : let us assume the newGame behaves the same way for these two values

    Consider the Logic component of the Minesweeper application. What are the EPs for the markCellAt(int x, int y) method?. The partitions in bold represent valid inputs.

    • Logic: PRE_GAME, READY, IN_PLAY, WON, LOST
    • x: [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] (we assume a minefield size of WxH)
    • y: [MIN_INT..-1] [0..(H-1)] [H..MAX_INT]
    • Cell at (x,y): HIDDEN, MARKED, CLEARED


    Evidence:

    Explain equivalence partitioning using examples from the project.

    For W10.6 Can evolve a product incrementally

    Covered by:

    Lecture 10