December 21, 2005

Internationalized GET parameters with Tomcat

One tiny, oft overlooked detail when working with internationalized web apps is that the Content-Type header specifies the type and encoding of the body of the http request.

Say you want to add searching to your site, which is implemented as a form that submits using method="GET", and your target audience is Japanese people. How do you deal with internationalized query parameters, considering that GET requests do not have a body?

Historically, query strings were expected to contain characters in the ISO-8859-1 character set, with anything but a subset of ASCII encoded using the %xx notation. In discussions a few years ago it was agreed that using UTF-8 encoding for international characters was the best alternative. This is reflected in RFC 3986 (which is backed by the W3C):


When a new URI scheme defines a component that represents textual data consisting of characters from the Universal Character Set [UCS], the data should first be encoded as octets according to the UTF-8 character encoding [STD63]; then only those octets that do not correspond to characters in the unreserved set should be percent-encoded. For example, the character A would be represented as "A", the character LATIN CAPITAL LETTER A WITH GRAVE would be represented as "%C3%80", and the character KATAKANA LETTER A would be represented as "%E3%82%A2".



If you're running Tomcat, watch out! If your URL contains unicode characters, the browser will properly encode them as UTF-8 and then do the %xx magic for each byte. Tomcat will treat each encoded byte in the UTF-8 representation as a single ISO-8859-1 character and make your Japanese users very unhappy.

Unfortunately, this is Tomcat's default behavior. Fortunately, there is server configuration option that will force it to treat URLs as per RFC 3986. To enable it, edit server.xml and add the following parameter to the connector element:


<Server ...>
  <Service ...>
    <Connector ... URIEncoding="UTF-8"/>
      ...
    </Connector>
  </Service>
</Server>


BTW, I've tested this in Tomcat 5.5.12. I don't know whether it will work in previous versions.

13 Comments:

Blogger Tayfun Yüğrük said...

This is an old but very helpful post for me, thanks.

November 14, 2007 5:53 PM  
Blogger shea said...

Very helpful, even years after the post. You rock!

February 27, 2008 11:36 AM  
Blogger SimonOx said...

Thank you, helped a lot (even today in 2009!).

March 16, 2009 4:09 AM  
Blogger nasha said...

Why was there no follow on bankruptcy then? The bailout of AIG FP went to (wow power leveling) hedge funds that bound credit swaps on Lehman failing or others betting on rating (wow power leveling) declines. AIG has drained over 100 billion from the government. Which had to go to (wow power leveling) those who bet on failures and downgrades. Many of whom (power leveling)were hedge funds. I-banks that had offsetting swaps needed the money from the AIG bailout or they would have been caught. Its an (wow powerleveling) insiders game and it takes just a little bit too much time for most people to think (wow gold) through where the AIG 100 billion bailout money went to, hedge funds and players, many of whom hire from the top ranks of DOJ, Fed, Treasury, etc. ZHANG XIAO CHEN

April 19, 2009 10:00 PM  
Blogger param said...

Hi Guys,

This is regarding the issue I am facing while sending UTF-8 characters using GET method to a servlet directly from browser.
I have done the following settings:
1. Created a CharsetFilter, which sets encoding type for each request as UTF-8
2. Applied this filter in web.xml before all the requests
3. In my servlet, while writing the response, I have set response.setContentType to text/html;charset=utf-8

For the above mentioned settings accented characters like ÀÁÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ works correctly. But Chinese characters, Arabic characters etc does not work.

How ever if along with above settings, I change the server.xml settings to have useBodyEncodingForURI="true" OR/AND URIEncoding="UTF-8" in connector tag, the Chinese & Arabic characters works fine but now accented characters do not work :(.

I have tried all the combination of the settings mentioned but some how only one of the above two situations work.

Has anybody come across this problem? Any pointers will be great.

Thanks
Param

August 03, 2009 2:13 AM  
Blogger Kayla said...

Cell phones from simple communication became a full-featured entertainment terminal . Can be said that cell phone china new milestone in the industry. This fully demonstrates that, cheap cell phones entertainment has become the most sought after consumer applications and has become the focus of the mobile phones market.

September 26, 2009 8:31 PM  
Blogger Kayla said...

Present-day society, the rapid development of science and technology makes cell phones wholesale is no longer a simple phone communications equipment only. Wholesale cell phones have many features, entertainment, scalability, and so have a good expression of the discount cell phones will be to maximize the effectiveness of multimedia technology.

September 26, 2009 8:31 PM  
Blogger Adi said...

Oes Tsetnoc | Mengembalikan Jati Diri Bangsa | Kenali dan Kunjungi Objek Wisata di Pandeglang | Oes tsetnoc | Online Marketing | Electronics Gadgets | etips solution | Travel Guide

October 27, 2009 12:40 AM  
Blogger crazyloko said...

buy products wholesaleThis phenomenom is typified by China Wholesalersthe rise ofbusiness. Incredible range of products available with China Wholesale “Low Price and High Quality” not only reaches directly to their target clients worldwide but also ensures that wholesale from china from China means margins you cannot find elsewhere and China Wholesale will skyroket your profits.china wholesale productsbuy china wholesalewholesale chinawholesale productsbuy products

October 27, 2009 12:53 AM  
Blogger The Jack said...

Thanks ever so much, very useful article. If you do not mind, please visit my article related to pandeglang district in Banten, Indonesia at Kenali dan Kunjungi Objek Wisata di Pandeglang or Kenali dan Kunjungi Objek Wisata di Pandeglang second and also Kenali dan Kunjungi Objek Wisata di Pandeglang Objek Wisata Air Terjun Curug Gendang or related to a leadership at Mengembalikan Jati Diri Bangsa and Oes Tsetnoc or Oes Tsetnoc the second and our hard work at Kerja Keras Adalah Energi Kita that's right Kerja Keras Adalah Energi Kita, and Kenali dan Kunjungi Objek Wisata di Pandeglang Memasuki Babak Akhir also Objek Wisata Taman Wisata Alam Carita, Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, also Kenali dan Kunjungi Objek Wisata di Pandeglang, or Kenali dan Kunjungi Objek Wisata di Pandeglang, very smart thank you!

October 28, 2009 11:01 PM  
Blogger Daniela said...

Find your great Travel News and sing the songs at Free Song Lyric or you can watch the drama at Korea Drama Online one of great korea drama is A Love to Kill if you go to travel to Indonesia learn Learn Indonesia Language first! And find your home cari rumah or make a blog Belajar membuat Blog find your home again rumah dijual and again at jual rumah then if you want buy a new laptop see the Laptop Price List or you can buy a New Blackberry and then take care your Health & Jewerly.

November 14, 2009 7:48 PM  
Blogger nike said...

Merry Christmas, my dear friends:
Nike shoes
nike shox torch
Cheap nike shoes
Discount nike shoes
Nike shox r4
nike shox shoes
puma cat
cheap sport shoes
cheap nike shox
cheap nike max
nike tn dollar
nike running shoes
nike air max tn
puma shoes
discount puma shoes
puma mens shoes
puma running shoes
puma shoes
ed hardy clothes
ed hardy shirts
ed hardy jackets
ed hardy hoodies
ed hardy boots
ed hardy polo shirts
ed hardy shoes
ed hardy jeans
ed hardy outerwear
ed hardy long sleeve shirts
ed hardy bags
ed hardy winter boots
ed hardy handbags
ed hardy love kills slowly shirts
ed hardy love kills slowly shoes
ed hardy love kills slowly boots
ed hardy trousers
ed hardy mens
ed hardy womens
ed hardy t shirts
ed hardy sunglasses
ghd hair straighteners mk4
hair straightners
ghd iv styler hair straightener
ghd hair straightners
cheap ghd hair straighteners

December 10, 2009 5:09 PM  
Blogger leto-fm said...

Thank you!
You saved my day!

April 14, 2010 12:26 PM  

Post a Comment

Links to this post:

Create a Link

<< Home