[twill] twill and pages with YUI (and other javascript)

[Jason Stevens]

> That being the case, is there some reason twill just doesn't  
> completely ignore everything in script tags? It would seem that if  
> it doesn't understand it, then it should completely ignore it (i.e.  
> see the page rendered as if there were no javascript in the page at  
> all).

I've been looking at this and have a little more info.  The problem  
occurs when you have javascript strings that look like html tags.  A  
pseudo-example:

<script>
var myJavascriptString = "abc <select> def <\/select> ghi";
</script>
<select name="whatever">
</select>

Trying to do anything form-related in twill with this page throws a  
nested selects error.  I suspect the parser is complaining because it  
sees a select tag inside myJavascriptString but it doesn't see the  
close tag in that string because of the escaped slash.

As a really quick fix I tried added adding another entry to  
BeautifulStoneSoup.PARSER_MASSAGE to strip out the script tags before  
processing, but that didn't seem to do the trick.

Anyone have any thoughts on how to get around this situation?  I'd be  
happy if I could get twill/mechanize/BS to just totally throw out  
anything within script tags.

Thanks
-jason