I already made some experiences with the Flying Saucer on another job and it is still a very nice framework for PDF creation. For me, a major advantage of "The Flying Saucer" over other PDF creation frameworks, like XSLT and FOP, in the Web world is the homogenous technology mix.

"The Flying Saucer Project" create PDF from XML/XHTML pages using CSS style sheets. All technologies known to the web designer that gives your web site this pretty look. I guess it's okay when you say, that XSLT and FOP is the more powerful technology mix, but styling pages on this technology mix is not a simple task to do.

So back to the initial topic: A basic ingredient which is needed to create PDFs with the "The Flying Saucer Project" is some XHTML. In our project at scoyo we wanted to use Apache Tapestry to create this XHTML.

After taking a closer look to the Apache Tapestry rendering pipeline I found out, that the PageResponseRenderer is the perfect place to hook up the PDF creation into the rendering process. On the one hand the PageResponseRenderer uses a MarkupWriter to create the (X)HTML content from the current Page instance. On the other hand the PageResponseRenderer is responsible for writing the generated content into the server response.

Using the Apache Tapestry own IoC mechanism it's quite an easy task to decorate the Tapestry PageResponseRenderer implementation. It's a beautiful mechanism when you understood how it works. Okay I guess using the 5.1 version of tapestry it would be better to advice the renderPageResponse(...) method, but I had not enough time to get familiar with it yet.

One real big plus on Apache Tapestry is the implementation. Every function is well encapsulated into separate services. And the service implementing classes are very short and almost always on consistent level of abstraction. So you easily understand how a service works and you are able to copy and extend the service functions. So I extended PageResponseRenderer to determine if a page creates XHTML output which should be transformed to PDF. If no PDF should be created my PageResponseRenderer simply delegates to the original Tapestry instance.

When the page wants to create PDF my PageResponseRenderer buffers the created content and redirects it into the ITextRenderer of the a href="https://xhtmlrenderer.dev.java.net/">"The Flying Saucer". Okay this sounds more nice, than it actually is. The content is stored into a ByteArrayOutputStream and re-parsed by the standard JDK XML parser for further processing by the ITextRenderer. But I guess the we will live with the performance impact until it gets a real issue.

So long explanation short story. The whole thing worked out great! It took me like two to three hours to implement a first prototype. In out web application. This also includes the embedding of custom fonts into the PDF and a nice discovery of pages to render into PDF, via custom annotations using the component model meta-data facilities and a ClassTransformationWorker.

Web development can be so easy when you are using Apache Tapestry. On an earlier project we used a similar architacture with JSP and the "The Flying Saucer". I guess it took us ten times the effort to establish an equal infrastructure using JSF and the Java Servlet API ...

So this weekend I read the article about some new Features of the 5.1 Version of the Tapestry framework ...

One of the blogs I am reading on a regular basis is the Tapestry Central blog from Howard Lewis Ship the founder and lead developer of the Tapestry framework. It already provided some really nice ideas and introduced me into some really interesting topics.

So this weekend I read the article about some new Features of the 5.1 Version of the Tapestry framework. And I was really exited to read about the plans to reduce/minimize the traffic over the wire. I am studying this topic for quite a while, since I stumbled across it during a project. The trigger was the Best practices article from the Yahoo developer team.

Since then I was using different approaches and tools to improve the performance of the websites of my clients. One very useful tool is JAWR a Javascript and CSS bundling and compression tool. JAWR offers facilities to join multiple Javascript or CSS files in bundles. Each bundle will be delivered compressed (if possible), in addition very strong cache HTTP headers will be set to the HTTP response, so that the internet infrastructure may cache these files. So the traffic the originating server will be reduced. To prevent nasty caching problems the JAWR framework provides facilities to generate URLs to these bundles which contain a version information, so when the bundle changes, a new URL will be generated.

The very exciting thing about the planned Features in Tapestry 5.1 is, that Tapestry is the first content generating web framework which will provide out the box facilities for this kind of issues.
A great weakness of the JSP and JSF approaches to generate content is, that there is no fast/cheap way to determine a last modified date for a page or a component. So the application can't use the "If-Modified-Since" and "Not Modified" (HTTP status 304) mechanism to provide a lightweight answer for requested resources which have not changed. Maybe Howard Lewis Ship is able to provide this with his Tapestry framework. This would be a great performance boost for any kind of web application.

May very dynamic application don't address the content delivery issues at all. Its quite sad to see, but on the other hand side it's good to see that some frameworks are beginning to address this kind of features.

At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Some days ago I posted a blog entry about using Glassfish v2 EJBs with the Tomcat. At this moment I thought that I solved the problems, but last week we were setting our testing evironment on a Debian Linux box and the problems reoccured.

At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Enough with apologies at this moment, we were late setting up the testing environment too and there was work to do. So I decided to take a different approach and tried to find out some more about the classloading of Tomcat 6. One of my primary questions was: "Why did the Tomcat 6 developers removed the different classloading directories?"

The answer suprised me more, than I expected! After using Google for a while, trying to find out more about the differences in classloading between Tomcat 5.5 and Tomcat 6, I stumbled across a small file: The catalina.properties. This file revealed the secrets of the differnces in classloading between these two Tomcat version. The simple and quite beautyful answer is: There aren't any real differences. The distributions have different default settings, but the classloading mechanism is quite the same. At this point I am getting some more cautious than the last time. The two mechanisms are the same, as far I can judge it from my appication developer/deployer point of view

So what is this catalina.properties all about? The catalina.properties - as I understand it - controls some major classloading and security issues while bootstrapping the Tomcat web container. It has three (for my problem) relevant entries:

common.loader=${catalina.home}/lib,${catalina.home}/lib/*.jar
[...]
server.loader=
[...]
shared.loader=

These three entries are controlling three classloader in Tomcats classloader hierachy. The server classloader, the common classloader and the shared classloader. The server classloader is resopnsible for loading the libraries of the tomcat server. It is not set in the Tomcat 6 default distribution and will default to the common classloader which will load the server related classes now. The common classloader is responsible for loading classes common to the tomcat installation. Which are in the Tomcat 6 the server classes as well as addiontional classes like the JavaMail API oder some database driver classes (if you are configuring them with JNDI i.e.). The last classloader is the shared classloader which may load classes which may be shared between different web applications. This is the place where the Amis blog entry suggests the glassfish JARs to be placed, but which I hadn't found in the Tomcat 6 distribution.

After I discovered the secrets of the catalina.properties it took me half an hour to create a shared/lib directory in my TOMCAT_BASE and adapting the catalina.properties file. Now I had the proper place for my Glassfish files and it worked nearly instantly. Good to know that you can make the integration of Glassfish EJB and the Tomcat 6 in a proper way and also good to know about this tiny little classloading tricks of the Tomcat 6

As you say in german: "Kaum macht man es richtig, dann geht's!" (If you starting to do it the right way it works)

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

Stating with this whole website performance an ETag topic opened pandoras box. It's a real mess and lame compromises all over the place. But I have to admit that the whole thing is not as easy as I expected it to be.

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

First I wanted to check if the Apache was sending two different ETags if you have to Apache web servers delivering the content of a website. A simple test with wget verified it:

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x17|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "1041e6-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

When requesting the first web server the ETag "1041e6-7b2c8-6c9fc900" war returned ...

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x33|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "b824e-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

... after requesting the second web server the ETag "b824e-7b2c8-6c9fc900". So if you do not disable the INode usage for the ETag calculation the whole mechanism will not work. But does this really matter. As you can see the DNS caching was disabled for the test. In a real life situation the DNS caching we be activated and my browser will talk only to one web server and the INode issue won't matter at all.
But if we have multiple web server and the internet browser will only talk to one of them, why do I need the ETag? The Last-Modified header will be enough, won't it?

So my conclusion is use either the Last-Modifed header or the ETag header but using both header makes no really sense at all. This should be right in any case for the Apache HTTPD, the Microsoft IIS or any other web server delivering static content. I also recommend using the Last-Modified because the ETag calculation requires more computing time. There is only one scenario I can provide where the ETag will work and the Last-Modifed Header will not. If someone is changing content and does not modify the last modified time stamp. But this should be rather uncommon.

But how is the situation at dynamic content creation? My favorite web framework is Java Servlets and all related technologies (JSP, JSF, Tapestry, and so on) and after that very disappointing chapter with the ETag I decided to take a closer look at the facilities the servlet and JSP APIs are offering. Another sad very sad story at my actual point of view right now. But for tonight it is enough. Later more about it ...

So well, my first blog post with some contentent. I am quite excited and somewhat terrified, I really hope my english will be good enough.

So what is this ETag thing. I am working since 1998 in the web business but I came across ETags some weeks ago when some of my customers starting to worry about their perfomace. Don't get me wrong, out apllication worked - as always - fine, but some pages where slower as the equivalent pages at the websites of their competitors. We checked our monitoring data, it's always good to know of the webstite performed on yesterday, yesterweek or yestermonth, but were unable to find some issues. So we went from the server to the protocol and the client and checked some static and dynamic pages. As I was taking a closer look at a static webpage I stumbled over a HTTP header which I didn't know: ETag.

What is this ETag thingy? Why do only static pages have it? Okay to get this clear, I have to reveal some secrets about our customers setup. It runs on an Apache webserver who delegates the dynamic webpages to a Tomcat servlet container. They dynamic content will be generated there by some servlets and JSP. But this is also the solution. The static page was deliviered by the Apache HTTPD and the server adds the ETag header to all files.

So again what is this ETag thingy? What is it good for? Something I learned some years ago: It's always a good Idea to go to the source. In this case its the RFC 2616 in Section 14.19 the ETag HTTP header field is defined. But nowadays it's quite okay if you check Wikipedia first and at least a Wikipedia you will get a link to the RFC.

I confess it took a little while for me to understand what the ETag is good for. The web server may create an entity tag for the requested entity. This tag acts like some kind of content identifier or a kind of version number of the content. The Browser can use this information at a later time, when the entity (URL) is requested again send the ETag value in the "If-Match" or more likely the "If-None-Match" header field. If the browser does so, the server may evaluate the header and send a simple 304 (Not Modified) response instead of a 200 (OK) response and the data of the entity. The whole thing works basically equally to the "Last-Modified" and "If-Modified-Since" or "If-Unmodified-Since" mechanism. So why to we need this redundancy?

I am not that deep into the specification process of the HTTP protocol, but I am working a while with distrubuted web applications. Sometimes you simply need to mess around with the latest modification date of a file to get some rsync-ing right or something like that. In general the bad news is: The last modified date is not that reliable if you want to determine if a file is still the same. So this ETag thingy seems to be a good solution to this.

In real life nothing is this easy. The Apache web server calculates the ETag value out of the INode of the file, the last modified date and the size of the file. This is quite awkward if you have more than one HTTPD instance delivering the same file (i.e. if you use DNS round robin load balancing). It would be a small miracle if the file gets an INode with the same id on the different instances. So the ETag mechanism may get some kind of useless, so that Yahoo! recommends in their Best Practices for Speeding Up Your Web Site to deactivate the mechanism. The logic behind this recommendation is: A useless check done is a waste of time. I am not totally lucky with this but Yahoo! has some kind of point with it.

Luckily you can configure the ETag creation in the Apache web server with the FileETag directive. So may use only the modification time (MTime) and the file size (Size) for the ETag creation. This should work out quite well and if you can keep the last modified date in sync between the your web server instances the mechanism should work properly.

Okay I hope someone else reads this and may find it useful. At the moment I am investigating some ETag and last modified header issues with servlets and JSP. I guess I will try to blog about it soon ...