tomcat error: clearThreadLocalMap

kelly — Fri, 12 Mar 2010 00:43:57 +0000

I’m trying to move from Tomcat 5.5.28 to Tomcat 6.0.26 for my Struts2-based webapp. I previously tried–unsuccessfully–to make the move to Tomcat 6.0.20 but had to roll back. There were problems with the underlying connection to mail that I didn’t have time to track down.

I use a standard stack.

$ cat /etc/redhat-release
CentOS release 5.4 (Final)

$ /usr/sbin/httpd -v
Server version: Apache/2.2.3
Server built:   Nov 12 2009 18:43:41

$ java -version
java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode)

$ ls -l /usr/local | grep tomcat
… tomcat -> ./src/apache-tomcat-6.0.26

Be forewarned: I’m jotting down my notes and this seems to be the easiest place for me to find them later. If you find something unclear or missing, leave a comment and I will look into it.

I habitually edit and format log data to cut down on the noise and improve readability. This torques the purists. I’m not a purist. I won’t change the essentials but I will eliminate—what I consider to be—unimportant details. For example, I will replace unimportant date stamps

Mar 11, 2010 4:30:33 PM

with a marker that’s less distracting

$TIME

I also will wrap output so it will fit nicely in my blog. When I wrap, I will use the marker “↩”.

As an example, the $CATALINA_HOME/logs/catalina.out log entry produced upon startup:

$TIME org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.20.

$TIME org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true],↩
accept filters [false], random [true].

$TIME org.apache.coyote.ajp.AjpAprProtocol init
INFO: Initializing Coyote AJP/1.3 on ajp-8009

$TIME org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 544 ms

$TIME org.apache.catalina.core.StandardService start
INFO: Starting service Catalina

$TIME org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.26

$TIME org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor host-manager.xml

$TIME org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor manager.xml

$TIME org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ws

$TIME org.apache.coyote.ajp.AjpAprProtocol start
INFO: Starting Coyote AJP/1.3 on ajp-8009

$TIME org.apache.catalina.startup.Catalina start
INFO: Server startup in 2131 ms

So far, so good.

You can see from the startup entry that I am using the Tomcat Native Library. There is a nice section in Tomcat: The Definitive Guide that explains why not to use the native library but I use it anyway.

You’ll also notice that I don’t listen on port 8080; Tomcat connects only to Apache httpd via mod_jk.

#  grep -E "8[0-9]{3}" /usr/local/tomcat/conf/server.xml

port="8009" 

# cat /etc/httpd/conf/workers.properties
workers.tomcat_home=/usr/local/tomcat
workers.java_home=/usr/java/default
ps=/

worker.list=router

worker.worker1.port=8009
worker.worker1.host=localhost
worker.worker1.type=ajp13
worker.worker1.lbfactor=1

worker.worker1.redirect=worker2

worker.worker2.port=8009
worker.worker2.host=localhost
worker.worker2.type=ajp13
worker.worker2.lbfactor=1

worker.worker2.activation=disabled

worker.router.type=lb
worker.router.balance_workers=worker1,worker2

Everything else is pretty standard.

When I go to shut down, Tomcat 6.0.26 gives me a few surprise errors.

$TIME org.apache.coyote.ajp.AjpAprProtocol pause
INFO: Pausing Coyote AJP/1.3 on ajp-8009

$TIME org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina

$TIME org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal↩
  with key of type [null]↩
  (value [com.opensymphony.xwork2.inject.ContainerImpl$10@5a9b8ff9])↩
  and a value of type [java.lang.Object[]]↩
  (value [[Ljava.lang.Object;@1b0952e8])↩
  but failed to remove it when the web application was stopped.↩
  To prevent a memory leak, the ThreadLocal has been forcibly removed.

$TIME org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal↩
  with key of type [null]↩
  (value [com.opensymphony.xwork2.inject.ContainerImpl$10@1220b36])↩
  and a value of type [java.lang.Object[]]↩
  (value [[Ljava.lang.Object;@620e06ce])↩
  but failed to remove it when the web application was stopped.↩
  To prevent a memory leak, the ThreadLocal has been forcibly removed.

$TIME org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal↩
  with key of type [null]
  (value [com.opensymphony.xwork2.inject.ContainerImpl$10@1b1402c4])↩
  and a value of type [java.lang.Object[]]↩
  (value [[Ljava.lang.Object;@3ec19fbf])↩
  but failed to remove it when the web application was stopped.↩
  To prevent a memory leak, the ThreadLocal has been forcibly removed.

$TIME org.apache.coyote.ajp.AjpAprProtocol destroy
INFO: Stopping Coyote AJP/1.3 on ajp-8009

The question I have is, “Is this new or is the new listener included in the Tomcat 6 server.xml file reporting a previously missed problem?”

It turns out that Tomcat 6 has new listeners that detect this kind of problem.

“The new Tomcat 6 adds a series of listeners to prevent against memory leaks and one of them, is against ThreadLocals.”

“The new memory leak detection code in Tomcat has found an issue with an application that needs to be fixed.”

This seems to be a case of an existing problem which is just now being reported.

I should mention that I started tomcat and immediately shut it down. My webapp did not process any http requests between startup and shutdown. That doesn’t preclude my webapp from being the culprit, though.

I would be happier if Tomcat and/or Struts2 (Spring?) in a future release could eliminate the SEVERE notice. There’s something unsettling about ignoring serious warnings.

Failed to create poller with specified size

kelly — Wed, 08 Apr 2009 23:35:04 +0000

It had always bugged me that I got this warning on my OS X, MacBook Pro development system. (Well, it’s really an INFO, not a WARN but it bugged me nonetheless.)

Apr 8, 2009 4:10:55 PM org.apache.coyote.ajp.AjpAprProtocol init
INFO: Initializing Coyote AJP/1.3 on ajp-8009

	

Apr 8, 2009 4:10:58 PM org.apache.tomcat.util.net.AprEndpoint allocatePoller
INFO: Failed to create poller with specified size of 8192

	

Apr 8, 2009 4:10:58 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 2703 ms

I never saw this issue on my CentOS production servers so I figured it must be a BSD thing. It any rate, I recently tracked down a solution: add a pollerSize attribute to Tomcat’s connector element in server.xml.

For a full listing of my server.xml, see UTF8 JDBC on Tomcat.

I suppose I could have mucked with OS X’s limits but somehow this solution seemed less problematic. I’m not serving to the web from my laptop so I feel comfortable lowering the pollerSize. Besides, I had two years of history in which a problem never arose even though tomcat failed to create the poller… I’ve had a couple of months now in which the poller has been created and works fine.

Red Leopard » tomcat

tomcat error: clearThreadLocalMap

Failed to create poller with specified size