[twill] twill-fork: troubles are deepest than I expected

[Timur Hairullin]

Hello.

Titus, first of all, let me give you many thanks for twill: it's 
excellent for doing quick-and-dirty web testing work! :)

And now, about twill-fork. I've experienced several strange problems 
while trying to use it for heavy stressing. And it seems that most of it 
are buried inside twill guts, but not in twill-fork.
First cause: I've just running out of open file descriptors. ulimit -n 
16384 helps me a bit, but twill-fork explodes before it, leavig all its 
childs in unconscious state. Let me guess: open a temporary file for 
each thread and spare precious time for diskio in twill-fork isn't 
really good idea when we trying hit the limits in stressing an AUT.
After manually removing (quick and dirty, just by commenting out) all 
possible measurement stuff from twill-fork, I've got even worse results:

=====
EXCEPTION raised at line 6 of 'xe1.twill'
        go $addr
'NoneType' object has no attribute 'get_url'
Traceback (most recent call last):
  File "./twill-fork", line 91, in ?
    execute_file(filename, initial_url=options.url)
  File "/opt/home/th/twill/twill/parse.py", line 142, in execute_file
    execute_command(cmd, args, globals_dict, locals_dict)
  File "/opt/home/th/twill/twill/parse.py", line 80, in execute_command
    return eval(eval_str, globals_dict, locals_dict)
  File "<string>", line 0, in ?
  File "/opt/home/th/twill/twill/commands.py", line 374, in go
    state.go(url)
  File "/opt/home/th/twill/twill/commands.py", line 109, in go
    print '==> at', self.get_url()
  File "/opt/home/th/twill/twill/commands.py", line 148, in get_url
    return self._last_result.get_url()
=====

what I am doing:
$ ./twill-fork -n 5000 -p 500 xe1.twill

xe1.twill:
=====
setlocal addr http://some.address.here
run "from random import *"
run "cookie='cookies/' + str(randint(1,10000)) + '.cookie'"
#get a random cookie from file
echo $cookie
load_cookies $cookie
go $addr
go $addr
go $addr
go $addr
go $addr
go $addr
go $addr
go $addr
go $addr
go $addr
=====

So, I can give a hand with digging out the trouble origins, but I cannot 
locate the point where it cracks yet.

-- 
Timur Hairullin
Yandex QA Team
http://www.yandex.ru