[TIP] Why does tox use "pip install --pre" by default?

[Ian Cordasco]

On Sat, Sep 27, 2014 at 3:00 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 9/27/14 2:24 PM, holger krekel wrote:
>>
>> On Sat, Sep 27, 2014 at 13:46 -0400, Ned Batchelder wrote:
>>>
>>> I tried to release an alpha version of coverage.py 4.0 today, and
>>> immediately got complaints about Travis builds failing.
>>> It turns out that tox's default install_command is "pip install --pre
>>> {opts} {packages}".
>>
>> Did you you release the alpha to pypi.python.org and then
>> other packages using coverage via tox started failing?
>
> Yes.  coveralls.io (unwisely) uses internals of coverage.py which have
> changed in the major refactor that was part of 4.0.  So coveralls breaks
> when it tries to report statistics.  Anyone using tox and coveralls.io would
> have seen a failure.
>>
>>
>> When i release beta/devs of my packages i typically release
>> them to a public index other than pypi.python.org
>> (e.g. https://devpi.net/hpk/dev/ ).
>
> I don't have another index I can use, unless they are out there and I don't
> know they exist.
>>
>>
>>> Why the --pre ?  Do most people want to use beta
>>> versions of dependencies in their tests?
>>
>> a) I (and i think many others) often use tox to test against
>>     other packages development packages or release candidates.
>
> The real question is, "what does a naive user of tox expect?"  You want to
> test against development packages, so you will make sure that happens.  But
> if someone hasn't put in the effort to learn and/or change the defaults,
> what do they expect to happen?  I would have thought that tox by default
> should use pip's defaults.  I see no reason why running tests in a number of
> scenarios (which is tox's job) should mean that I want to install software
> in an unusual way. Certainly it's good to be able to test against
> development packages, and overriding the tox defaults would be a way to do
> that.
>
> But by default, I think tox should use pip's defaults.
>>
>>
>> b) when we introduced "install_command" the newest version of pip
>>     was just changing defaults to not install dev/rc/beta packages
>>     so the "install whatever is there" was still what most people
>>     expected.
>>
>> In any case, I wouldn't like to change defaults at this stage now
>> unless there is a strong reason for it.  It would break my own setups
>> and probably many others.
>
> I certainly understand that mindset.  But I don't agree that it would
> "break" a setup.  Your code would continue to test against all the same
> packages, but it would use the latest released package rather than the
> latest pre-release package.  If your code will only run against the latest
> pre-release package, instead of the latest released package, then you need a
> version specifier in your requirements.
>
> People who preferred to always use pre-release packages would be able to
> change their tox.ini to continue doing so.
>
> Changing defaults to make them safer is a time-honored tradition. Pip did
> it, as you point out, and we were all fine.
>
> --Ned.
>
>>
>> cheers,
>> holger
>>
>>> Of course, there's a slew of things that had to go wrong for the
>>> failures, notably that coveralls.io is using internal things from
>>> coverage that are no longer there, which is what actually caused the
>>> failures.  Also, "pip install --pre" will install even versions that
>>> are hidden on PyPI.  :(
>>>
>>> --Ned.

I thoroughly agree with Ned. If packages I depend on started making
pre-releases that broke my testing, I'd be more confused that this was
happening because of tox than because of pip. I would never expect a
*testing tool* to be making this decision for me. For people who don't
pin, but only care to support released versions this sounds more like
a footgun than a feature. Further, beyond this, there's no reason for
developers not to put pre-release versions on PyPI along with final
versions. That's what PyPI is there for and that's why pip can install
pre-release versions of packages. It's also why pip decided to make
the behaviour explicit and opt-in with the `--pre` flag. I'm surprised
that this is the default.