Category: Uncategorized

I am working with a customer where customer took the decision to go with Dropwizard instead of Spring Boot(or Spring ecosystem). I initially respected their decision and decided to give Dropwizard a fair chance. Now after spending a couple of months building a system that uses Dropwizard I don’t recommend it. I wrote about my initial thoughts here.

There are three main advantages of a battle tested and highly used framework like Spring:

• Good APIs
• Solution to common problems
• Good searchable documentation

Let me help you understand that by taking an example of integrating Kakfa in a Dropwizard application. The Dropwizard official organization provides a Kafka bundle[1] so we decided to use it for adding Kafka support. I found following issues with it:

Poor API: When you create the KafkaConsumerBundle in the *Application class you are forced to provide an implementation of ConsumerRebalanceListener. KafkaConsumerBundle does not do anything with it but it forces you to provide it [2]. If you read Kafka documentation you need to provide ConsumerRebalanceListener not at the time of creation but when you subscribe it. ConsumerRebalanceListener is used to commit the offsets in case of partitions. There is also an open issue [3] in Github repo on the same without any answer from the maintainers.

Incomplete configuration: The Dropwizard Kafka bundle does not support all the Kafka producer and consumer configuration properties. For example, it is often recommended to set number of retries in producer to Integer.MAX_VALUE and rely on delivery.timeout.ms to limit the number of retries. Since it is not a configuration option in Dropwizard bundle you have to hardcode it during the bundle creation.

Missing solution to common problems: Any real world Kafka application need to solve for these three common problems. Spring Kafka part of Spring ecosystem provides solution to these problems.

1. It does not provide serializer/deserialzer for JSON or any other format. You have to write one yourself or find a library that implements it.
2. Handling of Poison pill messages using ErrorHandlingDeserializer
3. Publishing of Poison pill messages to a dead letter topic

Conclusion

Yes, you can write your own bundle which fixes all these issues. But, then you are doing the undifferentiated work. Your team can spend time on writing business logic rather than writing and maintain this code. There is no right or wrong answer here. There is a cost that has to be paid when you take these decisions. You should keep that in mind. There is no free lunch.

Resources

1. https://github.com/dropwizard/dropwizard-kafka
2. https://github.com/dropwizard/dropwizard-kafka/blob/master/src/main/java/io/dropwizard/kafka/KafkaConsumerBundle.java#L33
3. https://github.com/dropwizard/dropwizard-kafka/issues/179

This is how you can get process id of a Java process.

import java.lang.management.ManagementFactory;

public class JavaProcessId {

    public static void main(String[] args) {
        String vmName = ManagementFactory.getRuntimeMXBean().getName();
        int p = vmName.indexOf("@");
        String pid = vmName.substring(0, p);
        System.out.println(pid);
    }
}

1. Create an OpenShift application using following command
$ rhc app-create xwiki jbosseap –gear large

2. Downloaded the xwiki war file from the official web site.

3. Extracted the war file using $ jar xfv xwiki.war

4. Downloaded three jars — guice-servlet, guice, and h2 from http://mvnrepository.com/ and placed the jars in WEB-INF/lib directory. The application expects these jars but don’t bundle them.

5. Update the hibernate.cfg.xml. Basically, we have commented out hsql and uncommented h2. The NullPointerException that you were seeing was because xwiki does not work with JNDI datasource so you have to bind the url manually. If you want to use postgres or mysql then use proper connection url, username, password etc.

6. Pack the war file again using $jar cfv ROOT.war .

7. Copy the artifact to deployments folder in your app source code. Please delete src/ and pom.xml as you are deploying war file.

8. Git commit and push the war.

9. Check the logs.

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The Louvre Museum has 8.5 million visitors per year. This blog was viewed about 110,000 times in 2013. If it were an exhibit at the Louvre Museum, it would take about 5 days for that many people to see it.

Click here to see the complete report.