[TIP] Why does tox use "pip install --pre" by default?

[holger krekel]

On Sat, Sep 27, 2014 at 16:00 -0400, Ned Batchelder wrote:
> On 9/27/14 2:24 PM, holger krekel wrote:
> >On Sat, Sep 27, 2014 at 13:46 -0400, Ned Batchelder wrote:
> >>I tried to release an alpha version of coverage.py 4.0 today, and
> >>immediately got complaints about Travis builds failing.
> >>It turns out that tox's default install_command is "pip install --pre
> >>{opts} {packages}".
> >Did you you release the alpha to pypi.python.org and then
> >other packages using coverage via tox started failing?
> Yes.  coveralls.io (unwisely) uses internals of coverage.py which
> have changed in the major refactor that was part of 4.0.  So
> coveralls breaks when it tries to report statistics.  Anyone using
> tox and coveralls.io would have seen a failure.

Chances coveralls.io would have broken when 4.0 proper is released
so they should use a version specifier especially if they use internal APIs.

> >When i release beta/devs of my packages i typically release
> >them to a public index other than pypi.python.org
> >(e.g. https://devpi.net/hpk/dev/ ).
> I don't have another index I can use, unless they are out there and
> I don't know they exist.

FWIW I can arrange for you to have a https://devpi.net/ned/dev 
where you can upload things and let people install from.

> >>Why the --pre ?  Do most people want to use beta
> >>versions of dependencies in their tests?
> >a) I (and i think many others) often use tox to test against
> >    other packages development packages or release candidates.
> The real question is, "what does a naive user of tox expect?"

Well, for one there are all kinds of different users and over time it
also changes.  So "a naive user" is not as easy to pinpoint.  Probably
none of the people here on the list are "naive users".

> You want to test against development packages, so you will make sure
> that happens.  But if someone hasn't put in the effort to learn
> and/or change the defaults, what do they expect to happen?  I would
> have thought that tox by default should use pip's defaults.  I see
> no reason why running tests in a number of scenarios (which is tox's
> job) should mean that I want to install software in an unusual way.
> Certainly it's good to be able to test against development packages,
> and overriding the tox defaults would be a way to do that.
>
> But by default, I think tox should use pip's defaults.

That's a good point in principle.

> >
> >b) when we introduced "install_command" the newest version of pip
> >    was just changing defaults to not install dev/rc/beta packages
> >    so the "install whatever is there" was still what most people
> >    expected.
> >
> >In any case, I wouldn't like to change defaults at this stage now
> >unless there is a strong reason for it.  It would break my own setups
> >and probably many others.
> I certainly understand that mindset.  But I don't agree that it
> would "break" a setup.

I am pretty sure i know if my setups would break.  Giving trainings
to companies wrt to their tox usage and development process i also 
have some views on how it might play out there.

> Your code would continue to test against all
> the same packages, but it would use the latest released package
> rather than the latest pre-release package.  If your code will only
> run against the latest pre-release package, instead of the latest
> released package, then you need a version specifier in your
> requirements.
> 
> People who preferred to always use pre-release packages would be
> able to change their tox.ini to continue doing so.
> 
> Changing defaults to make them safer is a time-honored tradition.
> Pip did it, as you point out, and we were all fine.

We were not all fine.  Thing broke on my side and for some other people.
Even today i sometimes i forget to tell people to use "pip install --pre"
instead of just "pip install" to test out release files of mine.  And I
guarantee that changing tox' current default for "install_command" will
break things for some people.  Question is only how many and if the
consistency with pip is worth it in the long run.

best,
holger