[TIP] Digging into pytest's history

[Florian Bruhin]

Hey,

I'm not sure how many of you follow GitHub Discussions in the pytest-dev
repository, so I'm crossposting this (lightly edited) from there:
https://github.com/pytest-dev/pytest/discussions/8667

I've always wondered when pytest actually really was born - the first
commit in the current pytest repository (5992a8ef21424d7571305a8d7e2a3431ee7e1e23) is
from January 2007, and even that commit alone already tells a lot: This
repository originally was from the py library[1] (later split off to
pytest), and it originally was a SVN revision, migrated to Mercurial,
and finally migrated to git.

However, the commit says "create the new development trunk" and is
already quite big: 435 files changed, 58640 insertions(+).

So this week, I dug around in various other repositories and
mailinglists to find out how it all started. As you might or might not
know, pytest originally was born as part of PyPy[2], to make it easier
to write tests for it. Here's what I found it with a bit of digging
around:

[1] https://pypi.org/project/py/
[2] https://www.pypy.org/

---

- Late 2002 / early 2003, PyPy was born:
  https://morepypy.blogspot.com/2018/09/the-first-15-years-of-pypy.html

- Like that blog post mentioned, from very early on, there was a big
  focus on testing. There were various `testsupport` files on top of
  unittest.py, and as early as June 2003, Holger Krekel (@hpk42)
  refactored its test framework to clean things up (`pypy.tool.test`,
  but still on top of `unittest.py`, with nothing pytest-like yet, as
  far as I can tell):
  https://mail.python.org/pipermail/pypy-dev/2003-June/000787.html

- In December 2003, there was another iteration at improving their
  testing situation, by Stefan Schwarzer, called `pypy.tool.newtest`:
  https://foss.heptapod.net/pypy/pypy/-/commit/02752373e1b29d89c6bb0a97e5f940caa22bdd63

- However, it didn't seem to be around for long, as around June/July
  2004, efforts started on a thing called `utest`, offering plain
  assertions. This seems like the start of something pytest-like, but
  unfortunately, I didn't actually find the test runner's code anywhere.
  The best I could find is one file, but that doesn't seem like a
  complete test runner at all. What I can see is that there were various
  efforts by Laura Creighton and Samuele Pedroni (@pedronis) at
  automatically converting existing tests to the new `utest` framework:
  https://foss.heptapod.net/pypy/pypy/-/commit/0735f9ed287ec20950a7dd0a16fc10810d4f6847
  https://foss.heptapod.net/pypy/pypy/-/commits/branch/default?utf8=%E2%9C%93&search=utest

- Around the same time, for Europython 2004, @hpk42 started a project
  originally called "std", intended to be a "complementary standard
  library":
  http://web.archive.org/web/20041020215353/http://codespeak.net/svn/user/hpk/talks/std-talk.txt

  In that talk, he was already laying out the principles behind what later became pytest:

  > - current "batteries included" are very useful, but
  >   * some of them are written in a pretty much java-like style,
  >     especially the unittest-framework
  >   * [...]
  >   * the best API is one that doesn't exist
  >
  > [...]
  >
  > - a testing package should require as few boilerplate code as
  >   possible and offer much flexibility
  > - it should provide premium quality tracebacks and debugging aid
  >
  > [...]
  >
  > - first of all ...  forget about limited "assertXYZ APIs"
  >   and use the real thing, e.g.:
  >
  >             assert x == y
  >
  > - this works with plain python but you get unhelpful "assertion
  >   failed" errors with no information
  >
  > - std.utest (magic!) actually reinterprets the assertion expression
  >   and offers detailed information about underlying values

- In September 2004, the `py-dev` mailinglist gets born, which is now
  pytest-dev, but thankfully with all the original archives still
  intact: https://mail.python.org/pipermail/pytest-dev/

- Around September/October 2004, the `std` project was renamed to `py`:
  https://mail.python.org/pipermail/pypy-dev/2004-September/001565.html
  with that move, `std.utest` became `py.test`. This is also the first
  time I actually found the entire source code:
  https://foss.heptapod.net/pypy/pypy/-/commit/42cf50c412026028e20acd23d518bd92e623ac11

  I was surprised how much of the API still seems around today:

  * `py.path.local`, which we're trying to get rid of (in favour of
    pathlib) some 16-17 years later
  * The idea of the collection tree, including `Collector`,
    `FSCollector`, `Directory`, `PyCollector`, `Module`, `Class`
  * Arguments like `-x` / `--exitfirst`, `-l` / `--showlocals`,
    `--fulltrace`, `--pdb`, `-S` / `--nocapture` (`-s` / `--capture=off`
    today), `--collectonly` (`--collect-only` today)

- In the same month, the `py` library gets split off from `PyPy`:
  https://foss.heptapod.net/pypy/pypy/-/commit/6bdafe9203ad92eb259270b267189141c53bce33

- It seemed to get rather quiet for a while, and I can't really find
  much between October 2004 (removing `py` from PyPy) and January 2007
  (first commit in the now-pytest repository). However, there were
  various discussions about features/ideas on the mailinglist, and a
  couple of releases every couple of months:
  https://pypi.org/project/py/0.8.0-alpha2/#history

  * March 2006: py 0.8.0-alpha2
  * May 2007: py 0.9.0
  * March 2008: py 0.9.1
    (first release to be found in the pytest changelog:
    https://github.com/pytest-dev/pytest/blob/main/doc/en/changelog.rst#091)
  * August 2008: py 0.9.2

- In August 2009, py 1.0.0 was released, introducing a lot of fundamental features:
  https://holgerkrekel.net/2009/08/04/pylib-1-0-0-released-the-testing-with-python-innovations-continue/

  * funcargs/fixtures
  * A plugin architecture which still looks very much the same today!
    http://web.archive.org/web/20090629032718/https://codespeak.net/py/dist/test/extend.html
  * Various default plugins, including monkeypatch
    http://web.archive.org/web/20091005181132/https://codespeak.net/py/dist/test/plugin/index.html
    http://web.archive.org/web/20091012022829/http://codespeak.net/py/dist/test/plugin/monkeypatch.html

- Even back there, the FAQ said:
  http://web.archive.org/web/20091005222413/http://codespeak.net/py/dist/faq.html

  > Clearly, [a second standard library] was ambitious and the naming
  > has maybe haunted the project rather than helping it. There may be a
  > project name change and possibly a split up into different projects
  > sometime.

  and that finally happened in November 2010, when pytest 2.0.0 was
  released as a package separate from `py` (but still called `py.test`):
  https://mail.python.org/pipermail/pytest-dev/2010-November/001687.html

- In August 2016, pytest 3.0.0 was released, which adds `pytest` (rather
  than `py.test`) as the recommended command-line entry point:
  https://docs.pytest.org/en/latest/changelog.html#id1313

---

So, do I have an answer to my question "when was pytest started" now?
Well, yes and no. It depends what point you really regard as the start
of it all - I'd pick Europython 2004, i.e. around June/July 2004 - which
means pytest is turning 17 soon, with many of its core concepts still
being around!

If anyone finds something I've missed (e.g. about what happened between
mid-2004 and mid-2007), I'd be more than happy to hear about it!

Florian

-- 
            me at the-compiler.org | https://www.qutebrowser.org 
       https://bruhin.software/ | https://github.com/sponsors/The-Compiler/
       GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc
             I love long mails! | https://email.is-not-s.ms/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20210513/4d7dcace/attachment.pgp>