I already made some experiences with the Flying Saucer on another job and it is still a very nice framework for PDF creation. For me, a major advantage of "The Flying Saucer" over other PDF creation frameworks, like XSLT and FOP, in the Web world is the homogenous technology mix.

"The Flying Saucer Project" create PDF from XML/XHTML pages using CSS style sheets. All technologies known to the web designer that gives your web site this pretty look. I guess it's okay when you say, that XSLT and FOP is the more powerful technology mix, but styling pages on this technology mix is not a simple task to do.

So back to the initial topic: A basic ingredient which is needed to create PDFs with the "The Flying Saucer Project" is some XHTML. In our project at scoyo we wanted to use Apache Tapestry to create this XHTML.

After taking a closer look to the Apache Tapestry rendering pipeline I found out, that the PageResponseRenderer is the perfect place to hook up the PDF creation into the rendering process. On the one hand the PageResponseRenderer uses a MarkupWriter to create the (X)HTML content from the current Page instance. On the other hand the PageResponseRenderer is responsible for writing the generated content into the server response.

Using the Apache Tapestry own IoC mechanism it's quite an easy task to decorate the Tapestry PageResponseRenderer implementation. It's a beautiful mechanism when you understood how it works. Okay I guess using the 5.1 version of tapestry it would be better to advice the renderPageResponse(...) method, but I had not enough time to get familiar with it yet.

One real big plus on Apache Tapestry is the implementation. Every function is well encapsulated into separate services. And the service implementing classes are very short and almost always on consistent level of abstraction. So you easily understand how a service works and you are able to copy and extend the service functions. So I extended PageResponseRenderer to determine if a page creates XHTML output which should be transformed to PDF. If no PDF should be created my PageResponseRenderer simply delegates to the original Tapestry instance.

When the page wants to create PDF my PageResponseRenderer buffers the created content and redirects it into the ITextRenderer of the a href="https://xhtmlrenderer.dev.java.net/">"The Flying Saucer". Okay this sounds more nice, than it actually is. The content is stored into a ByteArrayOutputStream and re-parsed by the standard JDK XML parser for further processing by the ITextRenderer. But I guess the we will live with the performance impact until it gets a real issue.

So long explanation short story. The whole thing worked out great! It took me like two to three hours to implement a first prototype. In out web application. This also includes the embedding of custom fonts into the PDF and a nice discovery of pages to render into PDF, via custom annotations using the component model meta-data facilities and a ClassTransformationWorker.

Web development can be so easy when you are using Apache Tapestry. On an earlier project we used a similar architacture with JSP and the "The Flying Saucer". I guess it took us ten times the effort to establish an equal infrastructure using JSF and the Java Servlet API ...

Due to the generosity of Google I got a nice Google wave test account. Now I am looking desperately to some spare time to evaluate the integration of jaev to the wave. One idea is to expand the validation to wave addresses via the wave protocol. Maybe an online validation as wave bot is another exciting idea.

The jaev framework is in productional use at least one installation and no major problems where filed so far. I'm keeping my fingers crossed

I am working already on some new features:

• Caching negative responses of mail servers
  When a mail server refuses communication due to suspected spam, the result will be cached to reduce network traffic.
• Suggest e-mail addresses
  When the validation fails suggest a list e-mail addresses. This algorithm may use the previous request and a default list of domains to compute some suggestions.

If you have any good ideas, file them on the issue tracking system or the google group.

Maybe I am applying for a google wave developer sandbox access to provide a google wave robot for online e-mail suggestion. Bah, that sounds like some work

At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Some days ago I posted a blog entry about using Glassfish v2 EJBs with the Tomcat. At this moment I thought that I solved the problems, but last week we were setting our testing evironment on a Debian Linux box and the problems reoccured.

At this moment I thought of a glitch in the server setup but some nagging hours later I realized that my so clever solution was a dirty hack. Which works in a Microsoft Windows environment, but refuses to work on in a Linux environment.
Ok, I have to confess, that I wasn't really happy with my first solution in the end. Renaming JARs to provide the correct order of class loading always leaves a bad taste. But at that moment I was happy, no other solution was in sight and I had absolutly no time.

Enough with apologies at this moment, we were late setting up the testing environment too and there was work to do. So I decided to take a different approach and tried to find out some more about the classloading of Tomcat 6. One of my primary questions was: "Why did the Tomcat 6 developers removed the different classloading directories?"

The answer suprised me more, than I expected! After using Google for a while, trying to find out more about the differences in classloading between Tomcat 5.5 and Tomcat 6, I stumbled across a small file: The catalina.properties. This file revealed the secrets of the differnces in classloading between these two Tomcat version. The simple and quite beautyful answer is: There aren't any real differences. The distributions have different default settings, but the classloading mechanism is quite the same. At this point I am getting some more cautious than the last time. The two mechanisms are the same, as far I can judge it from my appication developer/deployer point of view

So what is this catalina.properties all about? The catalina.properties - as I understand it - controls some major classloading and security issues while bootstrapping the Tomcat web container. It has three (for my problem) relevant entries:

common.loader=${catalina.home}/lib,${catalina.home}/lib/*.jar
[...]
server.loader=
[...]
shared.loader=

These three entries are controlling three classloader in Tomcats classloader hierachy. The server classloader, the common classloader and the shared classloader. The server classloader is resopnsible for loading the libraries of the tomcat server. It is not set in the Tomcat 6 default distribution and will default to the common classloader which will load the server related classes now. The common classloader is responsible for loading classes common to the tomcat installation. Which are in the Tomcat 6 the server classes as well as addiontional classes like the JavaMail API oder some database driver classes (if you are configuring them with JNDI i.e.). The last classloader is the shared classloader which may load classes which may be shared between different web applications. This is the place where the Amis blog entry suggests the glassfish JARs to be placed, but which I hadn't found in the Tomcat 6 distribution.

After I discovered the secrets of the catalina.properties it took me half an hour to create a shared/lib directory in my TOMCAT_BASE and adapting the catalina.properties file. Now I had the proper place for my Glassfish files and it worked nearly instantly. Good to know that you can make the integration of Glassfish EJB and the Tomcat 6 in a proper way and also good to know about this tiny little classloading tricks of the Tomcat 6

As you say in german: "Kaum macht man es richtig, dann geht's!" (If you starting to do it the right way it works)

Friday evening I read about the new version of JSF. The reference implementation (RI) Mojarra made an early draft review (EDR) of Mojarra 2.0 which implements the lastest specification of the JSR-314 which features JavaServer Faces (JSF) 2.0.

I don't want so summarize alle the new little gizmos in the JSF 2.0 spec or the EDR, Ryan Lubke does this in his blog already. There will be a long way to go to finish all the issues of the JSR but I can't stand to comment the one or other thing.

Like i.e. the project stage feature. Damn I am doing this like this for years in my application. It's a wonderful way to tell the application how many secrets have to be kept in this environment.
In a test environment I like the application to be very verbose and tell my of every little affliction, but in productive envorinments the application has to be very reticent. I don't want to tell the bad guys out there all over the internet about the flaws in the application.
One thing I am still missing, which goes hand in hand with project stages in my appliations it the version number. The application provides facilities to access the version number of the application and display it in any page. A wonderful tools to communicate with our QA department. They can check the version history if an issues was solved in the current version deployed on the test and QA system and they don't have to recheck about the version with me every time.

Another nice feature I noticed is, that in JSF 2.0 the component tree may be aware of the head and body areas in a HTML page. So it now should be quite easy to add CSS and Java-Scipts to a page. This may end some nasty implementations to re-parse the rendered output before it's send to the client to add CSS or Java-Scripts to the header. Or worse the tons of inline Java-Script code.
Let's be honest most of the HTML code generated by JSF is quite a horror and the frontend engineers are hating it a long time. Maybe we'll get some really nice HTML code now, so we don't need any more perfomace hacks to accellerate the bad HTML code.

I will keep an eye on the JSF 2.0 spec, it may solve much of the pain I have to today using JSF ...

The last days I had some fun trying to access some Glassfish EJB from a Tomcat web container. Some of you will now start to suggest to use only the Glassfish, so that is the whole story: There is a service layer which provides access to services on the different backend systems. This service layer is provided by our customer using the Glassfish J2EE container. Our application is planned to run on a Tomcat 6 web container - which is some kind of work horse in the company.

I was quite busy in the last weeks, we are starting couple of new projects for a long time customer so I had lot's of work to do and very few time for blogging. On the other hand side I have some new topics to blog about, so it was quite a fair deal.

The last days I had some fun trying to access some Glassfish EJB from a Tomcat web container. Some of you will now start to suggest to use only the Glassfish, so that is the whole story: There is a service layer which provides access to services on the different backend systems. This service layer is provided by our customer using the Glassfish J2EE container. Our application is planned to run on a Tomcat 6 web container - which is some kind of work horse in the company.

I would like to say this is a quit typical setup for EJB usage in the last years (EJB 1.x / EJB 2.x) so I didn't expect any problems. I confess it was quit a time ago I used EJB, but I fooled around with EJB in BEA and JBoss a couple of years ago.

So we started very naive deploying the appserv-rt.jar and the dependencies into WEB-INF/lib directory of our web application automatically with the maven build. It turned out to be a bad idea. The Tomcat kind of started but the web application could not access any JNDI resources.
As a first fix we moved the Glassfish jars to the $TOMCAT_HOME/lib with basically the same result. So where is the glitch?

I decided to take a closer look to the appserv-rt.jar of the glassfish. I first thought this jar contains some kind of client runtime. You may imagine my ... suprise ... when I realized that this jar contains the application server runtime, which is almost the whole application server. At this point I decided to have a break and go home.

Some times the best ideas come, when you are distracted. So this morning under the shower I thougt that I have an idea who the bad guy is. I suspected the jndi.properties in the appserv-rt.jar to confuse the JNDI initialisation of the Tomcat.
The first thing I tried in the office, was to remove the file from the appserv-rt.jar but it didn't work out. So I really stated to wonder. So consulted to web again and found a nice blog entry concerning the topic, the only difference was the Tomcat version (6 vs. 5.5). It proved my theory on one hand but it doesn't work with the Tomcat 6. So I had to compare the two setups and elaborate the key differences.

In the Tomcat 5.5 the Glassfish jar files had to be put into the $TOMCAT_HOME/shared/lib folder. The Tomcat 6 only provides one library folder at $TOMCAT_HOME/lib. A very imporant difference to the classloading behaviuor between the two Tomcat versions. The Tomcat 5.5 classloader hierachy is to load from the tomcat common lib classloader first and use the shared lib classloader "afterwards". In this configuration the classloader which is responsible for loading the Tomcat system classes has a higher priority to the classloader which loads the glassfish classes, so the JNDI startup could be performed by the Tomcat properly.
The Tomcat 6 doesn't seem to have this mechanism an longer, but why are the glassfish classes are preferred? The answer to this question is as simple as it is painflul. The Glassfish libraries are beginning the the letter a (i.e. appserv-rt.jar) the tomcat libraries are starting the the letters c or t so that the glassfish libraries are earlier in the classpath and preffered to the Tomcat libraries.
This insight was the key to solving the problem. I renamed the appserv-rt.jar (and the dependencies) to xappserv-rt.jar and voilá it worked.

Edit: Sometimes later I discovered a better way to combine the Glassfish EJBs and the Tomcat Webcontainer.

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

Stating with this whole website performance an ETag topic opened pandoras box. It's a real mess and lame compromises all over the place. But I have to admit that the whole thing is not as easy as I expected it to be.

Anyway ... today I had a nice discussion with a colleague about this topic. He's a senior front-end developer. This means he is a real pro in HTML, CSS, Java-Script and stuff like this and he's the one who showed me the whole Yahoo! performance stuff. After this discussion I wanted to check some things out in detail.

First I wanted to check if the Apache was sending two different ETags if you have to Apache web servers delivering the content of a website. A simple test with wget verified it:

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x17|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "1041e6-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

When requesting the first web server the ETag "1041e6-7b2c8-6c9fc900" war returned ...

wget --no-dns-cache -S http://www.a-domain.de/flash/ballons_main.swf
--09:14:26--  http://www.a-domain.de/flash/ballons_main.swf
           => `ballons_main.swf'
Resolving www.a-domain.de... 84.xxx.xxx.x17, 84.xxx.xxx.x33
Connecting to www.a-domain.de|84.xxx.xxx.x33|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Thu, 22 May 2008 07:14:26 GMT
  Last-Modified: Tue, 20 May 2008 15:49:24 GMT
  ETag: "b824e-7b2c8-6c9fc900"
  Accept-Ranges: bytes
  Content-Length: 504520
  Keep-Alive: timeout=15, max=45
  Connection: Keep-Alive
  Content-Type: application/x-shockwave-flash
Length: 504,520 (493K) [application/x-shockwave-flash]

... after requesting the second web server the ETag "b824e-7b2c8-6c9fc900". So if you do not disable the INode usage for the ETag calculation the whole mechanism will not work. But does this really matter. As you can see the DNS caching was disabled for the test. In a real life situation the DNS caching we be activated and my browser will talk only to one web server and the INode issue won't matter at all.
But if we have multiple web server and the internet browser will only talk to one of them, why do I need the ETag? The Last-Modified header will be enough, won't it?

So my conclusion is use either the Last-Modifed header or the ETag header but using both header makes no really sense at all. This should be right in any case for the Apache HTTPD, the Microsoft IIS or any other web server delivering static content. I also recommend using the Last-Modified because the ETag calculation requires more computing time. There is only one scenario I can provide where the ETag will work and the Last-Modifed Header will not. If someone is changing content and does not modify the last modified time stamp. But this should be rather uncommon.

But how is the situation at dynamic content creation? My favorite web framework is Java Servlets and all related technologies (JSP, JSF, Tapestry, and so on) and after that very disappointing chapter with the ETag I decided to take a closer look at the facilities the servlet and JSP APIs are offering. Another sad very sad story at my actual point of view right now. But for tonight it is enough. Later more about it ...

So well, my first blog post with some contentent. I am quite excited and somewhat terrified, I really hope my english will be good enough.

So what is this ETag thing. I am working since 1998 in the web business but I came across ETags some weeks ago when some of my customers starting to worry about their perfomace. Don't get me wrong, out apllication worked - as always - fine, but some pages where slower as the equivalent pages at the websites of their competitors. We checked our monitoring data, it's always good to know of the webstite performed on yesterday, yesterweek or yestermonth, but were unable to find some issues. So we went from the server to the protocol and the client and checked some static and dynamic pages. As I was taking a closer look at a static webpage I stumbled over a HTTP header which I didn't know: ETag.

What is this ETag thingy? Why do only static pages have it? Okay to get this clear, I have to reveal some secrets about our customers setup. It runs on an Apache webserver who delegates the dynamic webpages to a Tomcat servlet container. They dynamic content will be generated there by some servlets and JSP. But this is also the solution. The static page was deliviered by the Apache HTTPD and the server adds the ETag header to all files.

So again what is this ETag thingy? What is it good for? Something I learned some years ago: It's always a good Idea to go to the source. In this case its the RFC 2616 in Section 14.19 the ETag HTTP header field is defined. But nowadays it's quite okay if you check Wikipedia first and at least a Wikipedia you will get a link to the RFC.

I confess it took a little while for me to understand what the ETag is good for. The web server may create an entity tag for the requested entity. This tag acts like some kind of content identifier or a kind of version number of the content. The Browser can use this information at a later time, when the entity (URL) is requested again send the ETag value in the "If-Match" or more likely the "If-None-Match" header field. If the browser does so, the server may evaluate the header and send a simple 304 (Not Modified) response instead of a 200 (OK) response and the data of the entity. The whole thing works basically equally to the "Last-Modified" and "If-Modified-Since" or "If-Unmodified-Since" mechanism. So why to we need this redundancy?

I am not that deep into the specification process of the HTTP protocol, but I am working a while with distrubuted web applications. Sometimes you simply need to mess around with the latest modification date of a file to get some rsync-ing right or something like that. In general the bad news is: The last modified date is not that reliable if you want to determine if a file is still the same. So this ETag thingy seems to be a good solution to this.

In real life nothing is this easy. The Apache web server calculates the ETag value out of the INode of the file, the last modified date and the size of the file. This is quite awkward if you have more than one HTTPD instance delivering the same file (i.e. if you use DNS round robin load balancing). It would be a small miracle if the file gets an INode with the same id on the different instances. So the ETag mechanism may get some kind of useless, so that Yahoo! recommends in their Best Practices for Speeding Up Your Web Site to deactivate the mechanism. The logic behind this recommendation is: A useless check done is a waste of time. I am not totally lucky with this but Yahoo! has some kind of point with it.

Luckily you can configure the ETag creation in the Apache web server with the FileETag directive. So may use only the modification time (MTime) and the file size (Size) for the ETag creation. This should work out quite well and if you can keep the last modified date in sync between the your web server instances the mechanism should work properly.

Okay I hope someone else reads this and may find it useful. At the moment I am investigating some ETag and last modified header issues with servlets and JSP. I guess I will try to blog about it soon ...