Yesterday, I stumbled upon a blog that made me laugh. The truth be told, I have a snarky side—my evil twin, if you will. I keep it in check. Mostly. Some of you who will understand. Some of you won’t. It’s a Gemini thing. My twin is fun but good luck putting it back in its cage. And believe me, for every dollar of fun you get, you’ll pay ten dollars in social disaster. Best to keep the twin in check. ...
I stumbled across an article on lostlaowai.com www.lostlaowai.com/survival-chinese which lead me to poke around the site a bit. At the above URL, I noticed that some of the combining diacritical marks (tone marks) used in writing pinyin were not rendering properly. I had not seen this problem before. It didn’t make sense. Things that don’t make sense bug me. And being something of a character geek, I couldn’t let it go. So I tried to reproduce the problem in a test example. I couldn’t. That’s when I discovered a quirky Mac OS X copy+paste issue. I sensed there was a problem but the truth was elusive. You can’t see that copy+paste changes the string characters unless you look at a binary dump of the file (which I did). ...
You’d think in this day and age that modern operating systems, especially OS X, would be set for UTF8 handling by default. Not so. My previous post, centos l10n problem, showed that CentOS defaults to set its locale LANG as POSIX rather than UTF8. Mac takes the lunacy one step further. Or should I say one step backwards in time. I use maven2 as my build manager. Normally, I ignore the stream of info at the beginning of a build, Either it succeeds (yeah) or it fails. Either way, I’ve been more interested in seeing the end result; You know, those last few lines rather than the first few lines. ...
I’ve had opportunity to once again visit the UTF8 chain of failure and thought I’d write about it. If for no other reason, it’s easier for me to find my notes when I shove them into a blog entry. I previously wrote about UTF8 on Tomcat. I pointed out that I needed to add an attribute to the connector element so that the mod_jk connection would be UTF8-ified. I neglected to also point out that I needed to UTF8-ified the database connection. ...
I needed to look up the various strings Apple uses to name the iTunes Library. First I tried to get name from the iTunes resource bundle echo "this won't work..." echo "so don't even try it" cd /Applications/iTunes.app/Contents/Resources/English.lproj cat Localizable.strings | grep ‘PrimaryPlaylistName’ But I quickly learned that grep doesn’t work on the strings file. Why? Because Apple string files are not UTF-8. They are UTF-16. Usually. But in this case they are. I wanted to iterate over the set of resource strings and extract just the string I wanted. ...
Just about the time I believe the UTF-8 beast is in the cage, it escapes and runs amok. This AM, I started to deploy an update to the webapp on EC2. Seems that some of the static strings in the app contained UTF-8 encoded non-ascii characters. The java compiler barfed. “The heck?”, I thought. I just compiled the app on my MacBook. I checked the usual suspects (tomcat’s server.xml, JAVA_OPTS) but everything looked fine. However, when I looked at the code, it was indeed mangled. ...
I use Apache httpd + mod_jk with tomcat and connect to port 8009. Be sure to tell the tomcat connection that you are using UTF-8. # vi /usr/local/tomcat/conf/server.xml <Connector port="8009" URIEncoding="UTF-8" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> Add two parameters to the list of JAVA_OPTS (at least for tomcat) -Djavax.servlet.request.encoding=UTF-8 -Dfile.encoding=UTF-8"
Character codes and encoding In the beginning, there was ASCII. (There were others, but we begin here with ASCII). 7-bit ASCII in an 8-bit package. Using only the first seven bits of a byte, standard ASCII could not deal with diacritical marks (accents and funny dots in the vulgar vernacular). Therefore, the German word for later, später, would sometimes be transliterated into spaeter. I suspect this started in the days before computers when typewriters had only so many keys. You had to make do. ...