It’s quite sad, but if you are honest even with your source code quality is far below average the project may be a success by the given business goals. On the other hand side your source code may be excellent, your code metrics are outstanding and your architecture is state of the art but you are missing the needs of your stakeholders by miles.Programming is the most known part of software engineering to non software types. A couple of years ago I improved my skills in programming with the hope to improve my software engineering skills. But it took not very long for me to realize, that it takes a lot more to be successful in software engineering.

This is quite sad to say, but if you are honest: Even with your source code quality is far below average the project may be a success by the given business goals. On the other hand side your source code may be excellent, your code metrics are outstanding and your architecture is state of the art but you are missing the needs of your stakeholders by miles.

Realizing this made me a kind of sad, because learning to be a good programmer is task you can achieve by practicing. But learning how to e a good software engineer is a lot more difficult. It’s about communication, understanding the needs of your stakeholders, making good decisions, being flexible and able to incorporating short-term requirements.

In the last years I was able to learn some important lessons, which I believe, made me a better software engineer. Working on some medium to large sized software development projects from 2001 to 2009 I learned what is important and works fine but I also had the chance to make some quite big mistakes.

During the last years I read a couple of books about different software development approaches. Starring the classics from Tom DeMarco (e.g. The Deadline) which are quite excellent. My problem with the DeMarco/Lister books is, that they soon start to repeat them self.

The next great books where from the extreme programming (e.g. from Kent Beck) and agile development guys. I love this concepts. They are great! I guess anyone dealing with waterfall style processes and monster MS Project plans will agree.

One of the highlights of the last year where the books of Mary and Tom Poppendiek:

• Lean Software Development: An Agile Toolkit”
• Implementing Lean Software Development: From Concept to Cash

The Poppendieks are bearing some resemblance between agile software development and lean production methods and offering some very easy to understand principles. I am quite curious and exited how I will be able to incorporate this principles to my daily work and our organization at scoyo.

One thing I learned the hard way. No software development process will work, if your organization is not willing to accept the consequences.

The jaev framework is in productional use at least one installation and no major problems where filed so far. I’m keeping my fingers crossed
I am working already on some new features:

• Caching negative responses of mail servers
  When a mail server refuses communication due to suspected spam, the result will be cached to reduce network traffic.
• Suggest e-mail addresses
  When the validation fails suggest a list e-mail addresses. This algorithm may use the previous request and a default list of domains to compute some suggestions.

If you have any good ideas, file them on the issue tracking system or the google group.

Maybe I am applying for a google wave developer sandbox access to provide a google wave robot for online e-mail suggestion. Bah, that sounds like some work

The last time when I blogged about the Flex mojos I was still quite frustrated and thought I had a better solution. Now some time passed and on April the 3rd the new version (3.1) of the flex mojos was released and me and some of my colleges made some interesting discoveries.

Well, but in the last few weeks I often thought about the meaning and the source of the word mojo. Lets take a closer look at wikipedia.

Mojo (pronounced /ˈmoʊdʒoʊ/) is a term commonly encountered in the African-American folk belief called hoodoo. A mojo is a type of magic charm, often of red flannel cloth and tied with a drawstring, containing botanical, zoological, and/or mineral curios, petition papers, and the like. It is typically worn under clothing.

Okay, hoodoo! That explains quite a lot. Sometimes maven and the flex-mojos are a kind of magic.

But let’s get back on topic The boys from the Flex mojos where quite busy and created with the 3.x release very usable platform for flex builds in the maven world. It took us (mainly a freelance colleague) two more weeks to adapt our build process form various ant scripts into a multi-module maven2 build. But now it looks quite nice, the build is even faster and we were able to fix a problem with our Flex Builder configuration.

Due to my position as head of client development I am very curios about the performance of our website when TV broadcasting starts. We have lots of good people in the operations and software development team but we never tested our system in peak usage scenarios. I’ll guess you can’t simulate the load generated by TV broadcast anyway. This reminds me on the 30th of may 2005, when we launched the simyo Website with prime time TV spots. Damn, I was nervous. And I really was relieved when called your customer to tell that the website was fine (gimme more!) and they already sold over 1,000 units. A highlight on their release party, which I missed. But I was a scary chicken that day and didn’t want to travel half around Germany while nobody takes care about a brand new website

So, this is the scoyo TV spot:

I’ll keep my fingers crossed when the first TV spot goes over the air …

After I heard the F-5 fighter jets yesterday twice in the city I decided to got to the banks of the Elbe to see, if I can fetch a look. It was a sunny day, so I was in no hurry. But after 45 minutes waiting I decided to leave, without spotting the planes. On the news I discovered that the plan was stripped-down due to some complaints. Maybe tomorrow I’ll have some luck
But the time was not wasted at all. I took some pictures of boats sailing to the http://87.106.3.237/?p=25 I used the weekend and moved the blog onto a new technical infrastructure. The blog system is still Wordpress, but a more recent version. This enabled the switch to a more decent theme, which is now fusion by digital|nature. This is a good start for further customization I guess.

The platform build process is realized with Apache Maven, which is, in humble my opinion, still the first choice for Java based software projects, but integrating other project types into maven is a kind of challenging due to the nature of Maven. You may summarize this in: If you do it the maven way it’s very easy, if you don’t it is up to you …

Luckily there was already an existing build with Ant and a dysfunctional Maven variant with the Maven Flex-mojos plugin. So it stated my journey to insanity an back …

In the last week the release 1.0 platform release of scoyo came close and we were tying up some loose ends. One of these ends was the integration of the scoyo Flash/Flex portal build process into the platform build process.

The platform build process is realized with Apache Maven, which is, in humble my opinion, still the first choice for Java based software projects, but integrating other project types into maven is a kind of challenging due to the nature of Maven. You may summarize this in: If you do it the maven way it’s very easy, if you don’t it is up to you …

Luckily there was already an existing build with Ant and a dysfunctional Maven variant with the Maven Flex-mojos plugin. So it stated my journey to insanity an back.

My first attempts started with reactivating the Flex-mojos maven build. I already had some experience with Flex-mojos when preparing the build for the jaev Flex example. This example Project for the jaev framework also uses the Flex-mojos maven plugin to create the example .swf and uses the infrastructure of maven to assemble the web application with the HTML and the BlazeDS services. Took me some time and a quite nice tutorial from Sébastien Arbogast but finally it worked!

So I thought I was prepared for the task and nothing could stop me. There is no law against being wrong, at least in Germany .

The main problems was finding all the required libraries in various maven repositories. Additionally there were some foul libraries. Some yahoo uploaded HTML files as .pom and .jar file in a repository. It took me quite a moment to realize this. Unfortunately the current Flex SDK 3.3 was not available at all in any maven repository. At least I did not find any.

So after wasting some hours I changed the strategy: My new Idea was to call the Flex Ant tasks provided by Adobe from the maven build using the maven-antrun-plugin. The antrun-plugin is quite nice if you have to perform some tasks not covered by maven. So you can execute some Ant tasks to enrich your build.

But things where going to be worse. A couple of very “unsatisfying” hours later I discovered a bug or incompatibility of the maven maven-antrun-plugin and the Adobe Ant tasks. So it was time to rethink my strategy again. Should I switch back to the Flex-mojo approach? I went for a minor adjustment first: Instead of using the the Ant tasks I went for the compilers (mxmlc for Flex applications and compc for Flex libraries) directly. Calling them with the Ant <exec /> task, which executes a command on the command line. Then use the maven-assembly-plugin to create a ZIP file of the compiled artifact and install/deploy it into the repository.

Let’s sum this up:

1. start JVM and execute maven
2. execute the maven-antrun-plugin to execute ant tasks
3. use some Ant tasks to assemble the Flex build path (external and included libraries)
4. use the Ant <exec /> task to execute the flex compiler (let’s share a secret: The Flex compilers are written in Java. Welcome back home )
5. use the maven-assembly-plugin

to create a ZIP file

6. install the ZIP file into the local repository
7. (optionally deploy the artifact into a remote repository)

That does not sound like a simple build process. And let me amend something: If you a using shared libraries in the build (yepp we have a couple of them) you may add another step before invoking the maven-antrun plugin. You’ll have to use the maven dependency plugin to copy and uncompress the libraries from the local (remote) repository to your build environment, so they can be accessed/found by the build. You’ll imagine I was not satisfied at all with this build variant …

Again my old friend came along, the question:”What to do?”

The Flex-mojo approach obliged my to upload the whole Flex SDK into our local archiva installation which hosts out company maven repository. This includes the minimum 54 libraries supplied by the Flex SDK. Okay most of them are quite common, but when a new Flex SDK is released (which happens on a quite regular basis) you’ll have to recheck all dependencies. No thanks …
The maven/Ant/Flex compiler variant emerged as juggernaut with bad performance and a loads of catches. No improvement at all …

So I decided to take a closer look at the Flex-mojo plugin itself. Why do I need to have all this dependencies? The major advantages to the Flex-mojo plugin are the special packaging types swf and swc for Flash movies and libraries. Using this packaging types maven will use a special life cycle and offer the possibility to install/deploy the artifacts directly into the repository. To my relief only a quite simple XML file (a plexus component definition) was necessary to achieve this.

So desperate as I was, I decided to write a maven plugin on my own. I must confess it was not my first time. I already wrote a simple maven plugin for the SinnerSchrader commerce framework, a library developed by my former employer. I really do not like to reinvent software on my own. But I was curious and I want to give it a shot. My goal was to combine the two approaches I tried before and use the best of both. So the plugin should be able to manage the swc dependencies, the flex compiler configuration (via -load-config) and the direct deployment of swc and swf artifacts.

The approach was simple:

1. check the maven dependencies for libraries to reference or to include
2. check for a configured compiler configuration
3. create the compiler command line and all options like -load-config, -compile.debug, -compiler.include-libraries or -compiler.external-library-path, etc.
4. invoke the compiler via thejava.lang.ProcessBuilder
5. install the artifact into the local repository

It took me not an hour to build a working proof of concept and only approx. four more hours to mavenize almost the entire build. This was really fun after two days of trying and failing. There are still some minor adjustments to make on monday, but I am really looking forward to finish the build. And if all works out as I want, there might be a 15-20% speed improvements for a complete build. But this also reduces the necessity to perform complete builds, because some shared libraries are rarely changing at all, so we can deploy them into our company maven repository.

As I went to bed last night, I was wondering why the Flex-mojo approach failed and the home-grown maven plugin approach succeeded. Under the shower this morning it was quite obvious to me. The Flex-mojo plugin stores the whole Flex SDK in maven repositories, so you don’t even have to download the Flex SDK to compile your Flex application. My home-grown plugin needs the Flex SDK installed on the build computer and the FLEX_HOME environment variable set. But that is okay, because the maven-compiler-plugin does the same. It does not work without the JAVA_HOME set and it does not try to deploy the Java SDK into maven repositories. Maybe the Flex-mojo guys will also realize the difference. It seems to be a quite nice piece of software but with too much unnecessary complexity and very “clustered” documentation.

After tying up this loose end me and a very experienced Flex development colleague already discovered some further TODOs relating to the maven and Flex Builder integration. There is still lots of work to do, so I am exited. My new job is different as I expected but a am very confident that it will offer some exiting further challenges for me …

The second release candidate was built this afternoon. It contains some fixes and some improvements suggested by PMD and Findbugs. Another major focus was to improve the performance of jaev. This was achieved in reducing the number DNS lookups by introducing a cache for DNS MX record resolution.

This may lead to another nifty feature in the next release candidate, final or 1.1 release. I am planning to introduce a domain suggest function. This may be achieved by using a fix/initial suggestion list and the queries against the DNS lookup cache. If everything works out well the i.e. the Tapestry 5 example may offer a suggest function for e-mail addresses.

I’am using log4j for quite a while. I am not sure when the framework was created by Ceki Gülcü, but log4j was the straight edge for java logging in the recent years. Unfortunately it became very quite around the log4j project recently.

Recently I discovered something new about a thing I am using on an every day basis like most of the software engineers. It’s about logging. You may ask: “What is so exiting about logging?”
In my opinion logging is one of the underestimated aspects of software development. Everyone uses it, but only very few are using it right.
Logging is the bridge between the development phase and the operational phase in the software life cycle. If you do it right it may provide valuable informations about the inner workings of an application.

I’am using log4j for quite a while. I am not sure when the framework was created by Ceki Gülcü, but log4j was the straight edge for java logging for the recent years. Unfortunately it became very quite around the log4j project recently.

In my new open source project I broke some new grounds. It started with the simple logging for java (slf4j), which I used for logging. The motivation was clear, I wanted to get rid of a dedicated logger implementation and offer the possibility to integrate the output of jaev logging into the application log files (streams).
The sl4j is not the first “meta” logging library. The commons logging framework was one of the first of these frameworks. At least it’s the one I came across most. The sl4j framework is quite new. It offers a similar functionality like the commons logging framework, but slf4j uses a static binding against the executing logging framework. It also supports various logger implementations and offers bridges from i.e. the JUL (java.util.logging) logging into the sl4j world. So you can integrate all logging informations in the sl4j infrastructure.

This is quite handy if you are combining frameworks which rely on a specific logger implementation (i.e. facelets which relies on JUL). When you use the JUL to slf4j bridge you may merge all logging streams and handle the events on you logger implementations. The slf4j framework by the way was created by Ceki Gülcü.

One other word to the logger implementation. Durning my first steps with slf4j I used the log4j implementation. Which works pretty well. But in last weeks I stumbled about the logback project, an offspring of Ceki Gülcü too. The logback project is a native implementation of the slf4j API and it offers a state of the art logger implementation. If you are using log4j you feel familiar in no time. But it also offers quite some interesting new features and continues the evolution which stalled at the log4j project for quite some years. Here are some highlights:

• Configuration fully in XML via logback.xml and read automatically on logging initialisation.
• Different configuration files for productive and test environment (logback.xml and logback-test.xml).
• Configuration via JMX possible.
• Quite handy reduction of the logger name length.
• Provides position informations of the caller (class, line, method).
• Shortened stack traces of exceptions

It took not very long to convince me to switch my logging implementation from log4j to logback for the jaev project.

What does jaev do? The name jaev is the abbreviation for java (advanced) email validation. So the framework does validate e-mails. Sounds quite boring, isn’t it? Of course not!

The name of who/what is jaev? The jaev framework is my first open source project! Jepp, I joined the community and giving something back after years of using the frameworks of other people.

What does jaev do? The name jaev is the abbreviation for java (advanced) email validation. So the framework does validate e-mails. Sounds quite boring, isn’t it? Of course not! There are some quite good tools for e-mail validation like the one in the Apache commons validator project (I “borrowed” some test cases there). But the e-mail validation in java was quite shallow. If you want to have a more advanced validation, i.e. of the domain, you’ll had to do it yourself. The jaev framework tries to fill this gap.

It provides facilities to validate the syntax of an e-mail address with including addresses with names like Niclas Meier <niclas.meier@example.com> (I confess the handling of comments in e-mail addresses has still some flaws, but it’s not so important I guess). This is obligatory but the jaev framework offers quite more.

With the jaev framework you’ll may check the domain of the e-mail address. The framework uses a local (or server) copy of the IANA top level domain database (http://data.iana.org/TLD/tlds-alpha-by-domain.txt) to validate the top level domain. If the top level domain is okay the DNS database will be queries if the domain exists and has MX entries. If a domain has no MX entries you’ll have to assume that you will could not relay an e-mail message to the e-mail address. The mail server uses the same mechanism.

A very important feature of the jaev framework is: You have the choice! You are receiving a Result object as outcome of the validation process. The Result contains a code, which specifies the details, and a validity level which specifies the level of validity of an e-mail address. I.e. SYNTAX suggests that an e-mail address is syntactically valid but the domain and account may be invalid. The maximum validity is ACCESSIBLE. With ACCESSIBLE validity the jaev framework was able to verify the existence and accessibility of the e-mail address on the MTA of the e-mail domain.

This is the last big feature of the jaev framework. The framework can connect to the MTA specified in the MX entry of the domain and tries to send an e-mail to the e-mail address. If the mail transfer can be initiated the framework will abort the process, no mail will be send, but we can assure that the e-mail account exits.

Around the three step validation mechanism the framework offers quite simple and extendible construction and configuration mechanisms, examples for JSF, Tapestry 5 and Flex & BlazeD and different acceptance strategies. Timeouts may be applied on different levels and operations, so you can assure that you’ll get a Result in a defined amount of time.

I think this is quite a handy package. And I spent lots of my spare time the last four weeks to bring this to life. Maybe someone out there agrees with me and uses the framework in his application.

If you encounter some problems or you would like to contribute/request some features you may post them to the google group or to the issue management system. Maybe there is someone out there who is firm in ANTLR which the jeav framework uses for e-mail parsing.