At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Some days ago I posted a blog entry about using Glassfish v2 EJBs with the Tomcat. At this moment I thought that I solved the problems, but last week we were setting our testing evironment on a Debian Linux box and the problems reoccured.

At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Enough with apologies at this moment, we were late setting up the testing environment too and there was work to do. So I decided to take a different approach and tried to find out some more about the classloading of Tomcat 6. One of my primary questions was: "Why did the Tomcat 6 developers removed the different classloading directories?"

The answer suprised me more, than I expected! After using Google for a while, trying to find out more about the differences in classloading between Tomcat 5.5 and Tomcat 6, I stumbled across a small file: The catalina.properties. This file revealed the secrets of the differnces in classloading between these two Tomcat version. The simple and quite beautyful answer is: There aren't any real differences. The distributions have different default settings, but the classloading mechanism is quite the same. At this point I am getting some more cautious than the last time. The two mechanisms are the same, as far I can judge it from my appication developer/deployer point of view

So what is this catalina.properties all about? The catalina.properties - as I understand it - controls some major classloading and security issues while bootstrapping the Tomcat web container. It has three (for my problem) relevant entries:

common.loader=${catalina.home}/lib,${catalina.home}/lib/*.jar
[...]
server.loader=
[...]
shared.loader=

These three entries are controlling three classloader in Tomcats classloader hierachy. The server classloader, the common classloader and the shared classloader. The server classloader is resopnsible for loading the libraries of the tomcat server. It is not set in the Tomcat 6 default distribution and will default to the common classloader which will load the server related classes now. The common classloader is responsible for loading classes common to the tomcat installation. Which are in the Tomcat 6 the server classes as well as addiontional classes like the JavaMail API oder some database driver classes (if you are configuring them with JNDI i.e.). The last classloader is the shared classloader which may load classes which may be shared between different web applications. This is the place where the Amis blog entry suggests the glassfish JARs to be placed, but which I hadn't found in the Tomcat 6 distribution.

After I discovered the secrets of the catalina.properties it took me half an hour to create a shared/lib directory in my TOMCAT_BASE and adapting the catalina.properties file. Now I had the proper place for my Glassfish files and it worked nearly instantly. Good to know that you can make the integration of Glassfish EJB and the Tomcat 6 in a proper way and also good to know about this tiny little classloading tricks of the Tomcat 6

As you say in german: "Kaum macht man es richtig, dann geht's!" (If you starting to do it the right way it works)

The last days I had some fun trying to access some Glassfish EJB from a Tomcat web container. Some of you will now start to suggest to use only the Glassfish, so that is the whole story: There is a service layer which provides access to services on the different backend systems. This service layer is provided by our customer using the Glassfish J2EE container. Our application is planned to run on a Tomcat 6 web container - which is some kind of work horse in the company.

I was quite busy in the last weeks, we are starting couple of new projects for a long time customer so I had lot's of work to do and very few time for blogging. On the other hand side I have some new topics to blog about, so it was quite a fair deal.

The last days I had some fun trying to access some Glassfish EJB from a Tomcat web container. Some of you will now start to suggest to use only the Glassfish, so that is the whole story: There is a service layer which provides access to services on the different backend systems. This service layer is provided by our customer using the Glassfish J2EE container. Our application is planned to run on a Tomcat 6 web container - which is some kind of work horse in the company.

I would like to say this is a quit typical setup for EJB usage in the last years (EJB 1.x / EJB 2.x) so I didn't expect any problems. I confess it was quit a time ago I used EJB, but I fooled around with EJB in BEA and JBoss a couple of years ago.

So we started very naive deploying the appserv-rt.jar and the dependencies into WEB-INF/lib directory of our web application automatically with the maven build. It turned out to be a bad idea. The Tomcat kind of started but the web application could not access any JNDI resources.
As a first fix we moved the Glassfish jars to the $TOMCAT_HOME/lib with basically the same result. So where is the glitch?

I decided to take a closer look to the appserv-rt.jar of the glassfish. I first thought this jar contains some kind of client runtime. You may imagine my ... suprise ... when I realized that this jar contains the application server runtime, which is almost the whole application server. At this point I decided to have a break and go home.

Some times the best ideas come, when you are distracted. So this morning under the shower I thougt that I have an idea who the bad guy is. I suspected the jndi.properties in the appserv-rt.jar to confuse the JNDI initialisation of the Tomcat.
The first thing I tried in the office, was to remove the file from the appserv-rt.jar but it didn't work out. So I really stated to wonder. So consulted to web again and found a nice blog entry concerning the topic, the only difference was the Tomcat version (6 vs. 5.5). It proved my theory on one hand but it doesn't work with the Tomcat 6. So I had to compare the two setups and elaborate the key differences.

In the Tomcat 5.5 the Glassfish jar files had to be put into the $TOMCAT_HOME/shared/lib folder. The Tomcat 6 only provides one library folder at $TOMCAT_HOME/lib. A very imporant difference to the classloading behaviuor between the two Tomcat versions. The Tomcat 5.5 classloader hierachy is to load from the tomcat common lib classloader first and use the shared lib classloader "afterwards". In this configuration the classloader which is responsible for loading the Tomcat system classes has a higher priority to the classloader which loads the glassfish classes, so the JNDI startup could be performed by the Tomcat properly.
The Tomcat 6 doesn't seem to have this mechanism an longer, but why are the glassfish classes are preferred? The answer to this question is as simple as it is painflul. The Glassfish libraries are beginning the the letter a (i.e. appserv-rt.jar) the tomcat libraries are starting the the letters c or t so that the glassfish libraries are earlier in the classpath and preffered to the Tomcat libraries.
This insight was the key to solving the problem. I renamed the appserv-rt.jar (and the dependencies) to xappserv-rt.jar and voilá it worked.

Edit: Sometimes later I discovered a better way to combine the Glassfish EJBs and the Tomcat Webcontainer.

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

Stating with this whole website performance an ETag topic opened pandoras box. It's a real mess and lame compromises all over the place. But I have to admit that the whole thing is not as easy as I expected it to be.

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

First I wanted to check if the Apache was sending two different ETags if you have to Apache web servers delivering the content of a website. A simple test with wget verified it:

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x17|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "1041e6-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

When requesting the first web server the ETag "1041e6-7b2c8-6c9fc900" war returned ...

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x33|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "b824e-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

... after requesting the second web server the ETag "b824e-7b2c8-6c9fc900". So if you do not disable the INode usage for the ETag calculation the whole mechanism will not work. But does this really matter. As you can see the DNS caching was disabled for the test. In a real life situation the DNS caching we be activated and my browser will talk only to one web server and the INode issue won't matter at all.
But if we have multiple web server and the internet browser will only talk to one of them, why do I need the ETag? The Last-Modified header will be enough, won't it?

So my conclusion is use either the Last-Modifed header or the ETag header but using both header makes no really sense at all. This should be right in any case for the Apache HTTPD, the Microsoft IIS or any other web server delivering static content. I also recommend using the Last-Modified because the ETag calculation requires more computing time. There is only one scenario I can provide where the ETag will work and the Last-Modifed Header will not. If someone is changing content and does not modify the last modified time stamp. But this should be rather uncommon.

But how is the situation at dynamic content creation? My favorite web framework is Java Servlets and all related technologies (JSP, JSF, Tapestry, and so on) and after that very disappointing chapter with the ETag I decided to take a closer look at the facilities the servlet and JSP APIs are offering. Another sad very sad story at my actual point of view right now. But for tonight it is enough. Later more about it ...