[twill] Threads and Twill

[C. Titus Brown]

mechanize (the main library that twill is built on) is not threadsafe.  so
twill can't be, either.

best,
--titus

On Mon, Jul 05, 2010 at 08:34:29PM -0300, Carlos Marcelo Cabrera wrote:
> I'm trying to write a program based on Twill that takes advantage of parallel processing of queries.
> 
> Using the module threading just as in http://pastebin.com/zHapMYeE I get the following output:
> 
> <output>
> Singlethread try:
> sex: 826.000.000
> family: 1.340.000.000
> windows: 955.000.000
> linux: 268.000.000
> religion: 305.000.000
> science: 795.000.000
> 
> Multithreads try:
> Exception in thread Thread-7:
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
>     self.run()
>   File "threadtwil.py", line 25, in run
>     self.result = self.func(*self.args, **self.kw)
>   File "threadtwil.py", line 48, in count_pages
>     browser.go("http://google.com")
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 114, in go
>     self._journey('open', u)
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 543, in _journey
>     self.result = ResultWrapper(code, r.geturl(), r.read())
> AttributeError: 'NoneType' object has no attribute 'geturl'
> 
> Exception in thread Thread-8:
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
>     self.run()
>   File "threadtwil.py", line 25, in run
>     self.result = self.func(*self.args, **self.kw)
>   File "threadtwil.py", line 49, in count_pages
>     tc.formvalue(2, "q", query)
>   File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
>     form = browser.get_form(formname)
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
>     forms = self.get_all_forms()
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
>     forms = list(self._browser.forms())
>   File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
>     if not self.viewing_html():
>   File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
>     raise BrowserStateError("not viewing any document")
> BrowserStateError: not viewing any document
> 
> Exception in thread Thread-12:
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
>     self.run()
>   File "threadtwil.py", line 25, in run
>     self.result = self.func(*self.args, **self.kw)
>   File "threadtwil.py", line 49, in count_pages
>     tc.formvalue(2, "q", query)
>   File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
>     form = browser.get_form(formname)
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
>     forms = self.get_all_forms()
>   File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
>     forms = list(self._browser.forms())
>   File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
>     if not self.viewing_html():
>   File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
>     raise BrowserStateError("not viewing any document")
> BrowserStateError: not viewing any document
> 
> sex: None
> family: None
> windows: 268.000.000
> linux: 268.000.000
> religion: 305.000.000
> science: None
> </output>
> 
> Is there a way to run parallel Twill?
> 
> Regards
> 
> -- 
> /* ************************************************************
> Carlos Marcelo Cabrera, alias "Point to null"
> Medios de contacto adicionales:
>     Weblog: http://pointtonull.esdebian.org
>     Jabber: point_to_null at esdebian.org
>     Yahoo: dxm84ar at yahoo.com.ar
>     ICQ: 303014677 
> ************************************************************ */



> _______________________________________________
> twill mailing list
> twill at lists.idyll.org
> http://lists.idyll.org/listinfo/twill


-- 
C. Titus Brown, ctb at msu.edu