[twill] A fix, a fix, to BeautifulSoup 3.0 parsing

Titus Brown titus at caltech.edu
Tue Oct 17 09:57:01 PDT 2006

Hi all,

I committed a heinous fix to twill last night.  I spent some time trying
to figure out the source of this error:

File "/disk/u/t/dev/twill/twill/other_packages/BeautifulSoup.py", line
1057, in endData
    currentData = ''.join(self.currentData)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 0: ordinal not in range(128)

and ended up changing the BeautifulSoup code to to do a

    currentData = ''.join(str(self.currentData))

I don't understand unicode well enough to know whether or not this is
going to cause huge problems, but it was the only way to get mechanize
and BS 3.0 to play nice.

I'm interested in putting together a test suite for unicode issues.
Does anyone have suggestions for a variety of pages to test?


More information about the twill mailing list