运行pytest-命令行参数

运行pytest-命令行参数

使用pytest框架编写的测试用例是可以直接在命令行运行的，下面以简单的测试用例来说明命令行运行下的不同参数的含义。

-h | --help

显示帮助信息，以及参数列表
常用参数列表

  -k EXPRESSION         only run tests which match the given substring
                        expression. An expression is a python evaluatable
                        expression where all names are substring-matched
                        against test names and their parent classes. Example:
                        -k 'test_method or test_other' matches all test
                        functions and classes whose name contains
                        'test_method' or 'test_other'. Additionally keywords
                        are matched to classes and functions containing extra
                        names in their 'extra_keyword_matches' set, as well as
                        functions which have names assigned directly to them.
  -m MARKEXPR           only run tests matching given mark expression.
                        example: -m 'mark1 and not mark2'.
  --markers             show markers (builtin, plugin and per-project ones).
  -x, --exitfirst       exit instantly on first error or failed test.
  --maxfail=num         exit after first num failures or errors.
  --strict              marks not registered in configuration file raise
                        errors.
  -c file               load configuration from `file` instead of trying to
                        locate one of the implicit configuration files.
  -s                    shortcut for --capture=no.
  --lf, --last-failed   rerun only the tests that failed at the last run (or
                        all if none failed)
  --ff, --failed-first  run all tests but run the last failures first. This
                        may re-order tests and thus lead to repeated fixture
                        setup/teardown
  -v, --verbose         increase verbosity.
  -q, --quiet           decrease verbosity.
  -r chars              show extra test summary info as specified by chars
                        (f)ailed, (E)error, (s)skipped, (x)failed, (X)passed,
                        (p)passed, (P)passed with output, (a)all except pP.
                        Warnings are displayed at all times except when
                        --disable-warnings is set
  --disable-warnings, --disable-pytest-warnings
                        disable warnings summary

test_demo.py

“”“Tesing demo”“”

from collections import nametuple		# 命名元祖

Task = nametuple('Task', ['summary', 'owner', 'done', 'id'])
Task.__new__.__defaults__ = (None, None, False, None)

def test_defaults():
	t1 = Task()
    t2 = Task(None, None, False, None)
    assert t1 == t2

no parameters

最简单执行测试用例的命令行就是：在当前目录下直接运行 pytest``

$ cd ~/Documents/my_code/aaaTest/blog 
$ pytest
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 1 item

test_demo.py .

============================================================================ 1 passed in 0.01 seconds ============================================================================

当前目录下只有 test_demo.py 一个测试文件，运行 pytest 时可以指定目录和文件，如果没有指定， pytest 命令会默认搜索当前目录及其子目录中以 test_ 开头或以 _test 结尾的测试函数。

–collect-only

使用 --collect-only 选项可以真实在给定的配置下哪些测试用例会被运行。

$ cd ~/Documents/my_code/aaaTest/blog
$ pytest --collect-only
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 1 item
<Module 'test_demo.py'>
  <Function 'test_default'>

========================================================================== no tests ran in 0.02 seconds ==========================================================================

--collect-only 选项可以让你非常方便在测试运行之前，检查选中的测试用例是否符合预期。

-k

-k EXPRESSION
-k 选项允许你使用表达式指定希望运行的测试用例。
此选项在多个测试函数名称前缀或后缀是相同的时候，是非常适用的，首先在 test_demo.py 中添加测试用例

...
def test_defaults_add():
    assert 1+2 == 2+1

接着使用 -k 参数来结合上面的 --collect-only 参数来检查测试用例

$ pytest -k 'defaults' --collect-only
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 2 items
<Module 'test_demo.py'>
  <Function 'test_defaults'>
  <Function 'test_defaults_add'>

========================================================================== no tests ran in 0.02 seconds ==========================================================================

符合预期。让它正常运行

$ pytest -k 'defaults'
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 2 items

test_demo.py ..

============================================================================ 2 passed in 0.01 seconds ============================================================================

正常执行。

-m

-m MARKERPR
-m 标记（marker）用于标记测试用例并分组，方便我们快速选择要运行的测试用例。
添加测试文件 test_demo2.py

import pytest

@pytest.mark.run_this
def test_tuple():
    
    assert (1, 2, 3) == (3, 2, 1)

我们将 test_demo.py 中的测试用例也加上标记

import pytest
...

@pytest.mark.run_this
def test_defaults():
	t1 = Task()
    t2 = Task(None, None, False, None)
    assert t1 == t2

使用 -m 运行测试用例。可以看到，包含标记 run_this 的测试函数都被成功执行。

$ pytest -m 'run_this'
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items

test_demo.py .
test_demo2.py F

==================================================================================== FAILURES ====================================================================================
___________________________________________________________________________________ test_tuple ___________________________________________________________________________________

    @pytest.mark.run_this
    def test_tuple():
>       assert (1, 2, 3) == (3, 2, 1)
E       assert (1, 2, 3) == (3, 2, 1)
E         At index 0 diff: 1 != 3
E         Use -v to get the full diff

test_demo2.py:12: AssertionError
=============================================================================== 1 tests deselected ===============================================================================
================================================================ 1 failed, 1 passed, 1 deselected in 0.09 seconds ================================================================

-m 参数还支持使用表达式筛选多个标记，例如 'mark1 and mark2' 'mark1 or mark2' 'mark1 and not mark2'

-x

-x, --exitfirst
在正常情况下， pytest 会运行搜索到的所有测试用例，如果某个测试用例执行结果为失败，会被标记为 F 并继续执行下个测试用例。但是有的时候我们在 debug 的时候希望遇到失败的用例立即停止，那么 -x 就派上用场了

$ pytest -x
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items

test_demo.py ..
test_demo2.py F

==================================================================================== FAILURES ====================================================================================
___________________________________________________________________________________ test_tuple ___________________________________________________________________________________

    @pytest.mark.run_this
    def test_tuple():
>       assert (1, 2, 3) == (3, 2, 1)
E       assert (1, 2, 3) == (3, 2, 1)
E         At index 0 diff: 1 != 3
E         Use -v to get the full diff

test_demo2.py:12: AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================= 1 failed, 2 passed in 0.21 seconds =======================================================================

这样看到的结果不明显，让我们把失败的用例放在前面。 test_demo.py rename为 test_demo_opt.py ，测试完成后会再改回去

$ pytest -x
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items

test_demo2.py F

==================================================================================== FAILURES ====================================================================================
___________________________________________________________________________________ test_tuple ___________________________________________________________________________________

    @pytest.mark.run_this
    def test_tuple():
>       assert (1, 2, 3) == (3, 2, 1)
E       assert (1, 2, 3) == (3, 2, 1)
E         At index 0 diff: 1 != 3
E         Use -v to get the full diff

test_demo2.py:12: AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 1 failed in 0.21 seconds ============================================================================

这下可以很明显的看到 -x 参数的作用

–maxfail=num

–maxfail=num
-x 选项是一遇到失败的用例就会立即停止，而 --maxfail=num 可以允许我们设置失败几次后再停止。

$ pytest --maxfail=1      # 失败1次后就停止，和-x作用相同
...
$ pytest --maxfail=2			# 失败2次后停止
...

-s | --capture=method

-s 和 --capture=method
-s 选项允许终端在测试运行时输出用例中的某些结果，包含任何符合标准的输出流信息，例如 print() 的输出。
-st | --capture=no 选项为关闭输出

–lf | --last-failed

-lf 和 --last-failed
当一个或者多个测试用例失败的时候，我们通常希望定位到最后一个失败的测试用例去debug，这时候需要使用参数 -lf | --last-failed

$ pytest --lf
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items
run-last-failure: rerun previous 1 failure

test_demo2.py F

==================================================================================== FAILURES ====================================================================================
___________________________________________________________________________________ test_tuple ___________________________________________________________________________________

    @pytest.mark.run_this
    def test_tuple():
>       assert (1, 2, 3) == (3, 2, 1)
E       assert (1, 2, 3) == (3, 2, 1)
E         At index 0 diff: 1 != 3
E         Use -v to get the full diff

test_demo2.py:12: AssertionError
=============================================================================== 2 tests deselected ===============================================================================
===================================================================== 1 failed, 2 deselected in 0.11 seconds =====================================================================

如果你觉得输出信息很繁琐，可以使用 --tb=no 参数来隐藏部分信息

$ pytest --lf --tb=no
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items
run-last-failure: rerun previous 1 failure

test_demo2.py F

=============================================================================== 2 tests deselected ===============================================================================
===================================================================== 1 failed, 2 deselected in 0.08 seconds =====================================================================

-v | --verbose

-v ，–verbose
顾名思义， -v 参数会使得输出信息更为详细。最明显的区别就是每个文件中的每个测试用例都会占一行。测试函数名字和结果都会显示完整，而不是 . | F

$ pytest -v --tb=no
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.7.8, pytest-3.2.1, py-1.10.0, pluggy-0.4.0 -- /Users/roy/Documents/my_code/aaaTest/my_env/bin/python
cachedir: .cache
rootdir: /Users/roy/Documents/my_code/aaaTest/blog, inifile:
collected 3 items

test_demo.py::test_defaults PASSED
test_demo.py::test_defaults_add PASSED
test_demo2.py::test_tuple FAILED

======================================================================= 1 failed, 2 passed in 0.07 seconds =======================================================================

而且在有颜色命令行中我们还可以看到 FAILED 被标记为红色，而 PASSED 标记为绿色

-q | --quiet

-q , --quiet
该参数选项与 -v 整好相反，是简化输出
与 --tb=no 配合输出极简信息

$ pytest -q --tb=no
..F
1 failed, 2 passed in 0.10 seconds

–tb=style

–tb=style
--tb=style 选项会决定有失败用例时候捕捉到的失败信息的显示方式，这样的信息我们一般称为“信息回溯”。
--tb=style 可选参数有 short 、line、no

• short : 仅输出 assert 的一行以及系统判定的内容
• line : 只使用一行输出显示所有的错误信息
• no : 屏蔽所有的回溯信息

-n

分布式执行测试用例

pytest xxx -n NUM

–reruns

用例失败后重新执行

pytest xxx --reruns NUM