[twill] Twill Headers vs. Rails

[Kevin Boers]

Hiya,

I've run into an interesting little problem in using twill with an  
application written in Ruby on Rails. For whatever reason, I was not  
able to load the landing page until I used clear_extra_headers to  
turn off all of the default headers. As you can see in the debug  
output below, I get a 500 from the server when headers are sent, and  
a 200 when they're not:

-= Welcome to twill! =-

current page:  *empty page*
 >> debug http 1
DEBUG: setting http debugging to level 1
current page:  *empty page*
 >> go http://qa.xxxxx.com/xxxxx
connect: (qa.xxxxx.com, 80)
send: 'GET /xxxxx HTTP/1.1\r\nAccept-Encoding: identity\r\nHost:  
qa.xxxxx.com\r\nConnection: close\r\nAccept: text/html; */*\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date: Mon, 01 Oct 2007 22:10:13 GMT
header: Server: Mongrel 1.0.1
header: Status: 500 Internal Error
header: Cache-Control: no-cache
header: Content-Type: text/html; charset=utf-8
header: Content-Length: 9742
header: Set-Cookie: _session_id=23e1a01fb82c5356cda6cb34fadfabea; path=/
header: Connection: close
==> at http://qa.xxxxx.com/xxxxx
current page: http://qa.xxxxx.com/xxxxx


 >> clear_extra_headers
current page: http://qa.xxxxx.com/xxxxx
 >> go http://qa.xxxxx.com/xxxxx
connect: (qa.xxxxx.com, 80)
send: 'GET /xxxxx HTTP/1.1\r\nAccept-Encoding: identity\r\nHost:  
qa.xxxxx.com\r\nCookie2: $Version="1"\r\nCookie:  
_session_id=23e1a01fb82c5356cda6cb34fadfabea\r\nConnection: close\r\n 
\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Mon, 01 Oct 2007 22:10:31 GMT
header: Server: Mongrel 1.0.1
header: Status: 200 OK
header: Cache-Control: no-cache
header: Content-Type: text/html; charset=utf-8
header: Content-Length: 20121
header: Set-Cookie: _session_id=23e1a01fb82c5356cda6cb34fadfabea; path=/
header: Connection: close
==> at http://qa.xxxxx.com/xxxxx
current page: http://qa.xxxxx.com/xxxxx

Rails is routing incoming traffic based on these headers, so this is  
obviously the problem. What I don't get is why curl, which sends a  
similar Accept header, works but twill does not.

Using http://gallery.future-i.com/about/test-accepts to show the  
headers the server is receiving, I get the following responses.

curl:
Accept: */*
Accept-Encoding:

twill:
Accept: text/html; */*
Accept-Encoding: identity

twill with clear_extra_headers:
Accept:
Accept-Encoding: identity

Has anyone come across an issue like this before? If so, I'd greatly  
appreciate any advice or direction that might help resolve this.

Thanks much,
Kevin