I'm trying to write a program based on Twill that takes advantage of parallel processing of queries.
Using the module threading just as in http://pastebin.com/zHapMYeE I get the following output:
<output>
Singlethread try:
sex: 826.000.000
family: 1.340.000.000
windows: 955.000.000
linux: 268.000.000
religion: 305.000.000
science: 795.000.000
Multithreads try:
Exception in thread Thread-7:
Traceback (most recent call last):
File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "threadtwil.py", line 25, in run
self.result = self.func(*self.args, **self.kw)
File "threadtwil.py", line 48, in count_pages
browser.go("http://google.com")
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 114, in go
self._journey('open', u)
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 543, in _journey
self.result = ResultWrapper(code, r.geturl(), r.read())
AttributeError: 'NoneType' object has no attribute 'geturl'
Exception in thread Thread-8:
Traceback (most recent call last):
File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "threadtwil.py", line 25, in run
self.result = self.func(*self.args, **self.kw)
File "threadtwil.py", line 49, in count_pages
tc.formvalue(2, "q", query)
File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
form = browser.get_form(formname)
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
forms = self.get_all_forms()
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
forms = list(self._browser.forms())
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
if not self.viewing_html():
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
raise BrowserStateError("not viewing any document")
BrowserStateError: not viewing any document
Exception in thread Thread-12:
Traceback (most recent call last):
File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "threadtwil.py", line 25, in run
self.result = self.func(*self.args, **self.kw)
File "threadtwil.py", line 49, in count_pages
tc.formvalue(2, "q", query)
File "/usr/lib/pymodules/python2.6/twill/commands.py", line 431, in formvalue
form = browser.get_form(formname)
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 274, in get_form
forms = self.get_all_forms()
File "/usr/lib/pymodules/python2.6/twill/browser.py", line 261, in get_all_forms
forms = list(self._browser.forms())
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 424, in forms
if not self.viewing_html():
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 449, in viewing_html
raise BrowserStateError("not viewing any document")
BrowserStateError: not viewing any document
sex: None
family: None
windows: 268.000.000
linux: 268.000.000
religion: 305.000.000
science: None
</output>
Is there a way to run parallel Twill?
Regards
--
/* ************************************************************
Carlos Marcelo Cabrera, alias "Point to null"
Medios de contacto adicionales:
Weblog: http://pointtonull.esdebian.org
Jabber: point_to_null at esdebian.org
Yahoo: dxm84ar at yahoo.com.ar
ICQ: 303014677
************************************************************ */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/twill/attachments/20100705/70484510/attachment.pgp>