[TIP] Why does tox use "pip install --pre" by default?

Ned Batchelder ned at nedbatchelder.com
Mon Sep 29 04:41:05 PDT 2014

On 9/29/14 3:32 AM, holger krekel wrote:
> On Sat, Sep 27, 2014 at 16:00 -0400, Ned Batchelder wrote:
>> On 9/27/14 2:24 PM, holger krekel wrote:
>>> On Sat, Sep 27, 2014 at 13:46 -0400, Ned Batchelder wrote:
>>>> I tried to release an alpha version of coverage.py 4.0 today, and
>>>> immediately got complaints about Travis builds failing.
>>>> It turns out that tox's default install_command is "pip install --pre
>>>> {opts} {packages}".
>>> Did you you release the alpha to pypi.python.org and then
>>> other packages using coverage via tox started failing?
>> Yes.  coveralls.io (unwisely) uses internals of coverage.py which
>> have changed in the major refactor that was part of 4.0.  So
>> coveralls breaks when it tries to report statistics.  Anyone using
>> tox and coveralls.io would have seen a failure.
> Chances coveralls.io would have broken when 4.0 proper is released
> so they should use a version specifier especially if they use internal APIs.
Yes, we've gotten coveralls.io updated to not allow 4.0 (with "<3.9999", 
since "<4.0" allowed "4.0a1"  :(  )
>>> When i release beta/devs of my packages i typically release
>>> them to a public index other than pypi.python.org
>>> (e.g. https://devpi.net/hpk/dev/ ).
>> I don't have another index I can use, unless they are out there and
>> I don't know they exist.
> FWIW I can arrange for you to have a https://devpi.net/ned/dev
> where you can upload things and let people install from.
I appreciate the offer, but the Python packaging people want people to 
be able to put pre-releases on PyPI.
>>>> Why the --pre ?  Do most people want to use beta
>>>> versions of dependencies in their tests?
>>> a) I (and i think many others) often use tox to test against
>>>     other packages development packages or release candidates.
>> The real question is, "what does a naive user of tox expect?"
> Well, for one there are all kinds of different users and over time it
> also changes.  So "a naive user" is not as easy to pinpoint.  Probably
> none of the people here on the list are "naive users".
>> You want to test against development packages, so you will make sure
>> that happens.  But if someone hasn't put in the effort to learn
>> and/or change the defaults, what do they expect to happen?  I would
>> have thought that tox by default should use pip's defaults.  I see
>> no reason why running tests in a number of scenarios (which is tox's
>> job) should mean that I want to install software in an unusual way.
>> Certainly it's good to be able to test against development packages,
>> and overriding the tox defaults would be a way to do that.
>> But by default, I think tox should use pip's defaults.
> That's a good point in principle.
>>> b) when we introduced "install_command" the newest version of pip
>>>     was just changing defaults to not install dev/rc/beta packages
>>>     so the "install whatever is there" was still what most people
>>>     expected.
>>> In any case, I wouldn't like to change defaults at this stage now
>>> unless there is a strong reason for it.  It would break my own setups
>>> and probably many others.
>> I certainly understand that mindset.  But I don't agree that it
>> would "break" a setup.
> I am pretty sure i know if my setups would break.  Giving trainings
> to companies wrt to their tox usage and development process i also
> have some views on how it might play out there.
My understanding is that you added the "--pre" at a time when pip would 
refuse to install even a specifically requested prerelease if --pre 
weren't specified.  That is no longer true.  So perhaps you no longer 
need to specify --pre ?
>> Your code would continue to test against all
>> the same packages, but it would use the latest released package
>> rather than the latest pre-release package.  If your code will only
>> run against the latest pre-release package, instead of the latest
>> released package, then you need a version specifier in your
>> requirements.
>> People who preferred to always use pre-release packages would be
>> able to change their tox.ini to continue doing so.
>> Changing defaults to make them safer is a time-honored tradition.
>> Pip did it, as you point out, and we were all fine.
> We were not all fine.  Thing broke on my side and for some other people.
> Even today i sometimes i forget to tell people to use "pip install --pre"
> instead of just "pip install" to test out release files of mine.  And I
> guarantee that changing tox' current default for "install_command" will
> break things for some people.  Question is only how many and if the
> consistency with pip is worth it in the long run.
I agree this is the right approach: first decide how it should behave, 
then assess what might break.
> best,
> holger

More information about the testing-in-python mailing list