Internationalized GET parameters with Tomcat

One tiny, oft overlooked detail when working with internationalized web apps is that the Content-Type header specifies the type and encoding of the body of the http request.

Say you want to add searching to your site, which is implemented as a form that submits using method="GET", and your target audience is Japanese people. How do you deal with internationalized query parameters, considering that GET requests do not have a body?

Historically, query strings were expected to contain characters in the ISO-8859-1 character set, with anything but a subset of ASCII encoded using the %xx notation. In discussions a few years ago it was agreed that using UTF-8 encoding for international characters was the best alternative. This is reflected in RFC 3986 (which is backed by the W3C):

When a new URI scheme defines a component that represents textual data consisting of characters from the Universal Character Set [UCS], the data should first be encoded as octets according to the UTF-8 character encoding [STD63]; then only those octets that do not correspond to characters in the unreserved set should be percent-encoded. For example, the character A would be represented as "A", the character LATIN CAPITAL LETTER A WITH GRAVE would be represented as "%C3%80", and the character KATAKANA LETTER A would be represented as "%E3%82%A2".

If you're running Tomcat, watch out! If your URL contains unicode characters, the browser will properly encode them as UTF-8 and then do the %xx magic for each byte. Tomcat will treat each encoded byte in the UTF-8 representation as a single ISO-8859-1 character and make your Japanese users very unhappy.

Unfortunately, this is Tomcat's default behavior. Fortunately, there is server configuration option that will force it to treat URLs as per RFC 3986. To enable it, edit server.xml and add the following parameter to the connector element:

<Server ...>
  <Service ...>
    <Connector ... URIEncoding="UTF-8"/>
      ...
    </Connector>
  </Service>
</Server>

BTW, I've tested this in Tomcat 5.5.12. I don't know whether it will work in previous versions.