cs5351-testing2-v3.pptx - Program Testing Part 2 Dr W.K Chan Department of Computer Science Email [email protected] Website

cs5351-testing2-v3.pptx - Program Testing Part 2 Dr W.K...

This preview shows page 1 - 12 out of 41 pages.

1 Program Testing Part 2 Dr W.K. Chan Department of Computer Science Email: [email protected] Website:
Image of page 1
Outline Automated test case generation and Evosuite Test Oracle and Metamorphic Testing 2
Image of page 2
Limitation of Fuzzing Recalling that Fuzzing can generate system-level test cases as they generate inputs accepted by UI or main() as arguments. Fuzzing does not help developers when developing an individual module (e.g., Java class) or while developing it (knowing the module is incomplete or faulty). Apart from manually crafting unit test cases, what kinds of facilities help developers to deliver higher productivity and quality? 3
Image of page 3
Automated JUnit Test Case Generation These tools automatically generate JUnit test cases. Evosuite [1] is the state-of-the-art tool. Up to 2019, all such automated test case generation tools are immature to be used in industry. Most of test cases are executable but quite human- incomprehensible . They can be used to detect crashes in the code. They cannot validate whether the program runs produce incorrect outputs or return erroneous objects or values. Debugging based on such human-incomprehensible test cases is difficult and ineffective. 4
Image of page 4
EvoSuite: Example Program 5
Image of page 5
EvoSuite: Test Cases Generated by EvoSuite 6
Image of page 6
EvoSuite: Experimental Results 7 Results from a Java Unit Testing Tool Competition held in 2017 (see Ref [1].(e)) 480 seconds on average to generate a test suite Branch coverage Evosuite: 50.8% Manual test suites: 53.8% Mutation score (the higher is the better) Evosuite: 36.9% Manual test suites: 54.3% In 2 out of 69 projects, Evosuite cannot generate test cases.
Image of page 7
Experimental Results Results from a Java Unit Testing Tool Competition held in 2019 (see Ref [1].(f)) Reference of t3 is Ref [8] 8
Image of page 8
Evosuite: A Case Study on Automated Test Case Generation A common approach to isolate a class from its dependencies is by mocking. In general, mocking refers to the use of replacement classes during testing rather than real classes. For manual mocking objects in unit testing, developers may write their unit test cases on top of a mocking framework, see Ref [3] for the list of such frameworks. Evosuite [1d] is a state-of-the-art tool to complement manual JUnit test cases. Read Ref [1].(d) (4 pages only) to get the brief idea of Evosuite 9
Image of page 9
Basic Ideas of Evosuite Evosuite consists of a number of stages Seeding, Mutation, Evolution, Whole Test Suite Optimization, Assertion Generation 10 Aim to test the following Java class: public class Foo { public static boolean isOK(int x) if (x * 2 == 42) return true; return false; } Seeding, Mutation, and Evolution is also used in our fuzzing algorithm presented in Week 3 lecture
Image of page 10
Basic Ideas of Evosuite: Seeding Scan all bytecode files (e.g., via the ASM instrumentation framework [4]) of the Java project and collect every constant (1, 2, 3, or “Hello” etc) of every primitive data type (integer, double, string) from the file into a constant
Image of page 11
Image of page 12

You've reached the end of your free preview.

Want to read all 41 pages?

  • Fall '17
  • Ms MONG, Yu

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors