[twill] Threads and Twill

[Barry Hart]

I'm not sure that Twill is thread safe. I have used Twill in concurrent processes (rather than threads) with no problems.

Barry

----- Original Message -----
From: "Carlos Marcelo Cabrera" <point_to_null at esdebian.org>
To: twill at lists.idyll.org
Sent: Monday, July 5, 2010 7:34:29 PM
Subject: [twill] Threads and Twill

I'm trying to write a program based on Twill that takes advantage of parallel processing of queries.

Using the module threading just as in http://pastebin.com/zHapMYeE I get the following output:

<output>
Singlethread try:
sex: 826.000.000
family: 1.340.000.000
windows: 955.000.000
linux: 268.000.000
religion: 305.000.000
science: 795.000.000

Multithreads try:
Exception in thread Thread-7:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "threadtwil.py", line 25, in run
    self.result = self.func(*self.args, **self.kw)
  File "threadtwil.py", line 48, in count_pages
    browser.go("http://google.com")
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 114, in go
    self._journey('open', u)
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 543, in _journey
    self.result = ResultWrapper(code, r.geturl(), r.read())
AttributeError: 'NoneType' object has no attribute 'geturl'

Exception in thread Thread-8:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "threadtwil.py", line 25, in run
    self.result = self.func(*self.args, **self.kw)
  File "threadtwil.py", line 49, in count_pages
    tc.formvalue(2, "q", query)
  File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
    form = browser.get_form(formname)
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
    forms = self.get_all_forms()
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
    forms = list(self._browser.forms())
  File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
    if not self.viewing_html():
  File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
    raise BrowserStateError("not viewing any document")
BrowserStateError: not viewing any document

Exception in thread Thread-12:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "threadtwil.py", line 25, in run
    self.result = self.func(*self.args, **self.kw)
  File "threadtwil.py", line 49, in count_pages
    tc.formvalue(2, "q", query)
  File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
    form = browser.get_form(formname)
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
    forms = self.get_all_forms()
  File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
    forms = list(self._browser.forms())
  File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
    if not self.viewing_html():
  File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
    raise BrowserStateError("not viewing any document")
BrowserStateError: not viewing any document

sex: None
family: None
windows: 268.000.000
linux: 268.000.000
religion: 305.000.000
science: None
</output>

Is there a way to run parallel Twill?

Regards

-- 
/* ************************************************************
Carlos Marcelo Cabrera, alias "Point to null"
Medios de contacto adicionales:
    Weblog: http://pointtonull.esdebian.org
    Jabber: point_to_null at esdebian.org
    Yahoo: dxm84ar at yahoo.com.ar
    ICQ: 303014677 
************************************************************ */

_______________________________________________
twill mailing list
twill at lists.idyll.org
http://lists.idyll.org/listinfo/twill