[TIP] [tox] Capturing output from commands

André Caron andre.l.caron at gmail.com
Mon Jun 13 14:43:21 PDT 2016


Hi,

Thanks for the quick response :-)

I'm already in the process of changing my program because I'll get the
change faster, but some of the things I want to run using this method are
not under my control and I'll need another solution for those.  I guess I
can work my way around this using a shell script, but then I have to
maintain a pair of them (Linux and Windows) in some cases.  Maybe I'll end
up writing a Python wrapper instead, but it's still boilerplate that I
would prefer avoiding.

Concerning shell redirection... I would be ready to live with explicit
"stdout=..." and "stderr=..." options in my testenv sections.  I simply
said I tried it and it didn't work, not that I really want this built into
Tox.  Anyways, I'm pretty confident that I can hack up a way to parse the
string to extract the redirection operators and then use a safer technique
(can't be that hard), but I'm wondering what the security concern is.  The
subprocess module warns against using this technique to run a command that
comes from untrusted user input, which I doubt is the case here.  Besides,
if I work around this issue by delegating the task to a shell script, how
am I any safer?

André

On Mon, Jun 13, 2016 at 4:32 PM, Ian Cordasco <graffatcolmingov at gmail.com>
wrote:

> So, let's look at how we might take a command from tox and use
> subprocess to run it:
>
> import shlex
> import subprocess
>
> command_string = 'echo "foo" > ex.txt'
> p = subprocess.Popen(shlex.split(command_string),
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> (out, _) = p.communicate()
> print(out)
>
> What you'll see printed is:
>
> "foo" > ex.txt
>
> In other words, subprocess is limited to being secure by default and
> doesn't allow shell redirection for good reasons. What you can do
> instead is write a tiny bash/bat script that does the redirection for
> you and then call that. Alternatively you could teach the program to
> put its output into a file. (Flake8, for example, added --output-file
> for this exact use case quite a while back.)
>
> In other words, to add redirection into tox you have to teach it about
> shell semantics and potentially weaken its security stance.
>
>
>
> On Mon, Jun 13, 2016 at 3:14 PM, André Caron <andre.l.caron at gmail.com>
> wrote:
> > Thanks, I'll take a crack at that!
> >
> > I hadn't noticed before, but by fiddling around a bit, but it looks like:
> > - Tox merges stdout and stderr of the commands into Tox's stdout.
> > - shell redirection inside the commands doesn't work (at least on
> Windows).
> >
> > Intuitively, I would expect stderr from my commands to be forwarded to
> > stderr.  Since this is not the case, I was willing to fall back onto
> > redirecting stdout from my process directly to a file, but then I can't
> do
> > that inside my Tox.ini file as my command gets the ">foo.log" as a CLI
> > argument.
> >
> > Is there any non-obvious rationale for this?  If not, would you consider
> > changes in this behavior?
> >
> > Thanks!
> >
> > André
> >
> >
> > On Mon, Jun 13, 2016 at 12:54 PM, Florian Bruhin <me at the-compiler.org>
> > wrote:
> >>
> >> Hey André,
> >>
> >> * André Caron <andre.l.caron at gmail.com> [2016-06-13 12:07:37 -0400]:
> >> > I looked through Tox' docs and CLI help, but I can't find a "silent"
> >> > mode
> >> > or a way to force (only) Tox to write to stderr.
> >> >
> >> > Do you have anything to recommend?  If not, any chance you would
> >> > consider a
> >> > patch to introduce a silent mode or a config change to send Tox' logs
> to
> >> > stederr?
> >>
> >> There's an issue for a -q flag, but nobody worked on it so far:
> >> https://bitbucket.org/hpk42/tox/issues/256/
> >>
> >> I still want to work on that, but never got to it with all the other
> >> FOSS stuff I do :) So if you want to work on a patch, please do!
> >>
> >> Florian
> >>
> >> --
> >> http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP)
> >>    GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc
> >>          I love long mails! | http://email.is-not-s.ms/
> >
> >
> >
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20160613/2c611cea/attachment-0001.htm>


More information about the testing-in-python mailing list