The notes of How We Test Software At Microsoft(1)

略过1－3章，直接从第四章开始读。

Chapter4：A Practical Approach to Test Case Design

Practicing good software design and test design

Test design has many parallels with good software design and with good design in general. Test design requires planning and problem solving to determine which kinds of tests to run and which types of testing will be most effective at verifying functionality and confirming that failure path are handled gracefully. One of the most important aspects of test design is anticipating customer needs and expectations and then creating tests that address these needs appropriately .Good test design often starts with a review, or critique, of the software design.

测试设计需要做的是找出需要做哪些测试以及哪些测试能更有效的验证产品的功能性和错误处理能力等。测试设计中其中最重要的部分是预见客户对产品的期望及需要，并创建测试用例来验证这些客户的期望。测试设计要从客户的角度上去理解认识产品，并找出产品的不足。

Using Test patterns

如果design pattern一样，test patterns也是用来共通问题，并为测试人员设计测试的时候提供指导方针。

在Testing Object-Oriented Systems: Models, Patterns and Tools, Robert Binder提出了37个test design patterns.

Scope	Pattern Name	Intent
Method Scope	Category-Partition	Design a test suite based on input/output analysis.
	Combinational Function Test	Design a test suite for behaviors selected by combinational logic.
	Recursive Function Test	Design a test suite for a recursive method.
	Polymorphic Message Test	Design a test suite for a client of polymorphic server.
Class Scope	Invariant Boundary Test	Identify test vectors based on the class invariant.
	Non-modal Class Test	Design a test suite for a class without sequential constraints.
	Modal Class Test	Design a test suite for a class with sequential constraints.
	Quasi-Modal Class Test	Design a test suite for a class with content-determined sequential constraints.
Class Scope Integration	Small Pop	Order of code/test at method/class scope.
	Alpha-Omega Cycle	Order of code/test at method/class scope.
Flattened Class Scope	Polymorphic Server Test	Design a test suite to check LSP compliance of a polymorphic server hierarchy.
	Modal Hierarchy Test	Design a test suite for a hierarchy of modal classes.
Reusable Components	Abstract Class Test	Develop and test an implementation of an interface.
	Generic Class Test	Develop and test an implementation of a parameterized class.
	New Framework Test	Develop and test a demo application of new framework.
	Popular Framework Test	Test changes to widely used framework.
Subsystem	Class Association Test	Design a test suite based on class associations.
	Round-trip Scenario Test	Design a test suite for aggregate state-based behavior.
	Controlled Exception Test	Design a test suite to verify exception handling.
	Mode Machine Test	Design a test suite based on sequentially-constrained stimulus-response scenarios.
Integration	Big Bang Integration	Try everything at the same time.
	Bottom up Integration	Integration by dependencies.
	Top Down Integration	Integration by control hierarchy.
	Collaboration Integration	Integration by cluster scenarios.
	Backbone Integration	Hybrid integration of subsystems.
	Layer Integration	Integration for layered architecture.
	Client/Server Integration	Integration for client/server architecture.
	Distributed Services Integration	Integration for distributed architecture.
	High Frequency Integration	Build and test at frequent, regular intervals.
Application Scope	Extended Use Case Test (pdf)	Develop testable use cases, design a test suite to cover application input-output relationships.
	Covered in CRUD	Exercise all basic operations.
	Allocate Tests by Frequency	Allocate system test effort to maximize operational reliability.
Regression Test	Retest All	Rerun all tests.
	Retest Risky Use Cases	Rerun tests of risky code.
	Retest Profile	Rerun tests by frequency of use.
	Retest Changed Code	Rerun tests for code that depends on changes.
	Retest Within Firewall	Rerun tests for code that is impacted by changes.

附加17种自动测试的test patterns。

Capability	Pattern Name	Intent
Built-in Test	Percolation	Perform automatic verification of super/subclass contracts.
Test Cases	Test Case/TestSuite Method	Implement a test case or a test suite as a method.
	Catch All Exceptions	Test driver generates and catches IUT's exceptions.
	Test Case /Test Suite Class	Implement test case or test suite as an object of class TestCase.
Test Control	Server Stub	Use a stub implementation of a server object for greater control.
	Server Proxy	Use a proxy implementation of a server object for greater control.
API/Class Drivers	TestDriver Super Class	Use an abstract superclass for all test drivers.
	Percolate the Object Under Test	Pass the object under test to driver.
	Symmetric Driver	Driver hierarchy is symmetric to hierarchy of classes under test.
	Subclass Driver	Driver is a subclass.
	Private Access Driver	Driver uses encapsulation avoiding features.
	Test Control Interface	Driver uses interface extension features.
	Drone	Driver is a mixin.
	Built-in Self Test	Driver is implemented as part of an application class.
Test Execution Control	Command Line Test Bundle	Code and build a test executable to be run from a command line or console.
	Incremental Testing Framework	Test suites are based on a simple framework that supports incremental development.
	Fresh Objects	Test environment with registration by and interface to built-in test in all application objects.

 

最常用的共享test patterns的方式就是构建模板。

Robert Binder’s Test design pattern 包含10个属性：

1)      Name

2)      Problem:简要介绍pattern要解决的问题

3)      Analysis：介绍问题的区域，并说明为什么采用这个pattern的技术去设计测试更有效

4)      Design：说明如何根据pattern设计测试

5)      Oracle：说明预期的结果

6)      Examples：举例说明使用pattern是如何找bug

7)      Pitfalls and Limitations: 说明在某些情况下应该避免使用这个pattern

8)      Related Patterns：列出相关联的patterns