Using boto3 with Jython

Few days back I had a requirement that I had to use boto3 with Jython. boto3 is AWS EC2 python SDK that you can use to work with various Amazon Cloud API’s. Jython is the JVM implementation of Python. We were packaging our Jython scripts and boto3 and its dependencies inside a JAR. boto3 and Jython work great together when you use them in a normal way i.e. when boto3 can load its data model files from file system. This does not work when you package your script and its dependencies inside a JAR as the model files are then not available on the filesystem but are available on the classpath. In this blog, I will show you how we used boto3 to overcome this limitation. Continue reading “Using boto3 with Jython”

DevFest Austria 2013 Talk : Thinking Beyond RDBMS — Building Polyglot Persistence Java Applications

Couple of weeks back I gave a talk at DevFest Austria event on how to use different nosql datastores with OpenShift.  You can view the video here.

Building Single Page Web Applications with Backbone.js, JaxRS, MongoDB, and OpenShift

Backbone.js is a mature, popular, and lightweight javascript library which aims to bring structure to your client-side code. The goal of backbone.js is to bring MVC, a software architecture pattern, to client side development.Actually, backbone.js is an MV framework where view is also responsible for controller logic as well. In this blog, we will build a simple social bookmarking application.

In brief, MVC is a programming design pattern for separating the different concerns of your application. The M corresponds to Model and is used to define domain object. The V corresponds to View and is responsible for display logic. Finally, the C corresponds to Controller and is responsible for user interactions and interactions between Views and Models.

You can find source code of the application here https://github.com/shekhargulati/getbookmarks. Backbone.js is used by Linkedin, Foursquare, Wunderkit, Groupon, etc to build complex applications. You can view the full list here.

Read full blog at https://www.openshift.com/blogs/building-single-page-web-applications-with-backbonejs-jaxrs-mongodb-and-openshift

How To Build Location-Aware Web Applications using HTML5 and MongoDB

We all use location aware applications in our day-to-day life. Applications like Foursquare, Facebook places, etc help us share our location (or places we visit) with our friends and family. Applications such as Google Local help us find out which businesses are near our current location. So, if we need a good coffee shop recommendation, we can ask Google Local to find us all the coffee shops near our location. This not only helps the customer, but also helps businesses reach the right audience and sell their products more effectively. It is a win-win situation for both consumers and businesses.

To build such an application , you need the geolocation of the user. According to Wikipedia, “Geolocation is the identification of the real-world geographic location of an object”. Until now, there was no standard way to find the location of a user in a web application. We could use an open source library like Google Gears to get the geo location of a user but this library is not under development anymore and should only be used with older browsers, which do not support W3C GeoLocation API. The W3C GeoLocation API is a specification that provides standard scripted access to geographical information associated with the hosting device. Geo Location support is not officially part of HTML 5 specification but people use it interchangeably and most commonly you will hear about GeoLocation APIs with respect to HTML5. This API provides an abstraction layer on top of how geolocation information of a user is gathered. All modern browsers support the GeoLocation API. The below table is taken from http://caniuse.com/#feat=geolocation.

Read full blog at https://www.openshift.com/blogs/how-to-build-location-aware-web-applications-using-html5-and-mongodb

How To Run Apache Tomcat 8 on OpenShift

OpenShift PaaS gives developers the ability to try out new unsupported frameworks, programming languages, and servers. Today, while learning about WebSockets I learned that the current development version of Apache Tomcat i.e. version 8, has support for JSR356. JSR356 provides standard Java API for WebSockets. Although Apache Tomcat 7 provides support for WebSockets, it uses its own proprietary API. So, in order to test the JSR356 standard based WebSockets support in Apache Tomcat 8, I decided to give it a try. However, rather than installing Apache Tomcat 8 on my local development machine, I decided to install it on OpenShift.

Read full post at https://www.openshift.com/blogs/how-to-run-apache-tomcat-8-on-openshift

OpenShift Rails QuickStart

  1. Create a Ruby 1.9 application
    rhc app create -a railsdemo -t ruby-1.9
    
  2. After running the command railsdemo folder will get created in your directory. Run the command shown below. This will generate rails code in the railsdemo folder.It will ask you whether you want to override, say yes.
    rails new railsdemo
    
  3. Next run commands shown below
    cd railsdemo
    bundle install
    rails generate controller home index
    rm public/index.html
    
  4. Add the following route to config/routes.rb:
    root :to => "home#index"
    
  5. Generate Post entity with fields as shown below
    rails generate scaffold Post name:string title:string content:text
    
  6. Add Database support by uncommenting following lines from .openshift/action_hooks/deploy script
    pushd ${OPENSHIFT_REPO_DIR} > /dev/null
    bundle exec rake db:migrate RAILS_ENV="production"
    popd > /dev/null
    
  7. Add mysql cartridge
    rhc cartridge add -a railsdemo -c mysql-5.1
    
  8. Update config/database.yml
    production:
      adapter: mysql2
      encoding: utf8
      database: <%=ENV['OPENSHIFT_APP_NAME']%>
      pool: 5
      host: <%=ENV['OPENSHIFT_MYSQL_DB_HOST']%>
      port: <%=ENV['OPENSHIFT_MYSQL_DB_PORT']%>
      username: <%=ENV['OPENSHIFT_MYSQL_DB_USERNAME']%>
      password: <%=ENV['OPENSHIFT_MYSQL_DB_PASSWORD']%>
      socket: <%=ENV['OPENSHIFT_MYSQL_DB_SOCKET']%>
    
  9. Add gem in Gemfile
    gem 'mysql2'
    
  10. Add, Commit, and Push the Code
    git add .
    git commit -a -m "Initial setup"
    git push
    
  11. Finally you can view the application running at http://railsdemo-domainname.rhcloud.com and you can create new posts at http://railsdemo-domainname.rhcloud.com/posts

Deploy war on tomcat running on OpenShift

In this short blog, I will show you how you can deploy war file on Tomcat running on OpenShift. For Java, OpenShift supports three deployment options – JbossAS-7, JBossEAP-6, and JBossEWS-1.0 or Tomcat. In all of the three options it supports both binary deployment (war file) and source code deployment(maven project). I have talked about how you can do source code deployment on tomcat in my previous blog.

Step 1: Sign up for an OpenShift Account

If you don’t already have an OpenShift account, head on over to the website and signup with promo code ews. It is completely free and Red Hat gives every user three free Gears on which to run your applications. At the time of this writing, the combined resources allocated for each user is 1.5 GB of memory and 3 GB of disk space.

Step 2: Install the client tools on your machine

Note: If you would rather watch a screencast of this step, check out the following videos where I demo how to install the client tools.

Windows

Linux Ubuntu

Linux Fedora

OSX

The OpenShift client tools are written in a very popular programming language called Ruby. With OSX 10.6 or later and most Linux distributions, ruby is installed by default so installing the client tools is a snap. Simply issue the following command on your terminal application:

sudo gem install rhc

Step 3 : Setting up OpenShift

The rhc client tool makes it very easy to setup your openshift instance with ssh keys, git and your applications namespace. The namespace is a unique name per user which becomes part of your application url. For example, if your namespace is cix and application name is bookshop then url of the application will be https://bookshop-cix.rhcloud.com/. The command is shown below.

rhc setup -l openshift_login

Step 4 : Creating Tomcat Application

After installing the client tools and setting up OpenShift account, next step is to create the bookshop application. This is a very simple Spring JPA application which has only one entity called Book. So, a user can do CRUD operations on book entity. To create a book entity, execute the command shown below.

rhc app create -a bookshop -t jbossews-1.0

This will create an application container for us, called a gear, and setup all of the required SELinux policies and cgroup configuration. OpenShift will also setup a private git repository for you and propagate your DNS out world wide.

Step 5 : Download the WAR file

Because we are doing binary deployment in this blog, we have to remove the src folder and pom.xml file from git repository and add the war file.

git rm -rf src/ pom.xml
git commit -am &amp;quot;removing default files&amp;quot;

Next download the war file from http://bookshop-demo.googlecode.com/files/ROOT.war and copy the war file in webapps directory. The name ROOT for war makes sure that the application is accessible at the root context i.e. http://bookshop-cix.rhcloud.com/.

git add .
git commit -am &amp;quot;committing bookshop war file&amp;quot;

Step 6 : Pushing the code to OpenShift

Finally push the code to OpenShift using git.

git push

The git push command upload the binary to the application gear and runs the action hooks specified in .openshift/action_hooks folder.

Configuring JNDI DataSource with OpenShift Tomcat Cartridge

This short blog post will tell you how to configure JNDI datasource with OpenShift Tomcat cartridge. After configuring, we will also make it work with a sample Spring MVC application. Let’s get started.

Step 1 : Create Tomcat Application

After you have signed up for OpenShift and setup your account. Execute the rhc app create command as shown below to create tomcat application as shown below.

rhc app create -a tomcatjndidemo -t jbossews-1.0

Step 2 : Adding PostgreSQL Cartridge

Next we will add postgresql cartridge to our application by executing the command as shown below.

rhc cartridge add -a tomcatjndidemo -c postgresql-8.4

Step 3 : Defining Datasource in Tomcat Configuration Files

OpenShift gives you the flexibility to edit the tomcat configuration files. These files are location in .openshift/config folders inside tomcatjndidemo folder. If you look into this directory you will find that there are 5 files. For adding datasource we have to make changes in two files — context.xml and server.xml

In server.xml you have to define a resource under GlobalNamingResource as shown below.

<Resource name="jdbc/postgresqldb" auth="Container" type="javax.sql.DataSource"
	username="${env.OPENSHIFT_POSTGRESQL_DB_USERNAME}"        password="${env.OPENSHIFT_POSTGRESQL_DB_PASSWORD}"
	url="jdbc:postgresql://${env.OPENSHIFT_POSTGRESQL_DB_HOST}:${env.OPENSHIFT_POSTGRESQL_DB_PORT}/${env.OPENSHIFT_APP_NAME}"
	driverClassName="org.postgresql.Driver" initialSize="5" maxWait="5000"
	maxActive="120" maxIdle="5" validationQuery="select 1"
	poolPreparedStatements="true">
</Resource>

Next change that you have to make is in context.xml. You have to define resource link to the datasource as shown below.

 <ResourceLink name="jdbc/postgresqldb" global="jdbc/postgresqldb" type="javax.sql.DataSource"/>

Step 4 : Updating Tomcat Classpath with PostgreSQL JDBC Driver

Tomcat will require PostgreSQL JDBC driver jar to make connection with PostrgreSQL. You can’t copy the jars to tomcat lib directory as it is not writable. To do that, first ssh into the application instance and then download the jar in $OPENSHIFT_DATA_DIR as shown below.

cd $OPENSHIFT_DATA_DIR
wget http://repo1.maven.org/maven2/postgresql/postgresql/8.4-702.jdbc4/postgresql-8.4-702.jdbc4.jar

Next update the catalina.properties in your application .openshift/config folder to scan the OPENSHIFT_DATA_DIR as shown below. Please replace the openshift path with value of you OPENSHIFT_DATA_DIR environment variable.

common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,/var/lib/openshift/2e110c16da18478aa607f742d34b70fb/app-root/data/*.jar

Step 5 : Pushing Changes to OpenShift

Now you can push the changes to OpenShift as shown below.

git commit -am "made changes for tomcat jndi datasource"
git push

Step 6 : Testing the DataSource

The last step in this blog is to test the datasource configuration we added in step 3 and step 4. To do that pull the code from my github repository. The code is a simple Spring MVC application which do CRUD operations on Member entity. Please note that the code also contains the tomcat configuration changes. So, please update the catalina.properties with your $OPENSHIFT_DATA_DIR value.

git remote add jndi git://github.com/shekhargulati/tomcatjndidemo.git
git pull -s recursive -X theirs jndi master

Next do git push which will build the war file and deploy the application to tomcat. The application should be accessible at http://tomcatjndidemo-cix.rhcloud.com/

How to get Solr Up and Running On OpenShift

Full text search is a vital component in most enterprise or non-enterprise applications and Solr is one of the most popular choices. So, today I decided to spend sometime on getting Solr up and running on OpenShift. In this blog I am sharing all the steps required to get Solr running on OpenShift.

  1. Download the latest version of Solr. The current latest version is 3.5. You can get it from here http://www.apache.org/dyn/closer.cgi/lucene/solr/3.5.0
  2.  Install the OpenShift rhc ruby gem. You can follow steps https://www.redhat.com/openshift/community/kb/kb-e1000/installing-openshift-express-client-tools-on-non-rpm-based-systems
  3. Create a new jbossas-7 application using rhc gem. Type the command shown below.
    rhc-create-app -a solr -t jbossas-7 -d -l email
    
  4. Do a git remove the src and pom.xml files from the created solr maven project and commit the changes.
    git rm -rf src/ pom.xml
    git commit -a -m "removing default files"
    
  5. Copy the solr.war file which exists in apache-solr-3.5.0/example/webapps directory to deployment directory under solr maven project.
  6. Next solr needs a solr home directory. This directory contains a conf directory and lib directory. The default conf directory comes with solr installation and you can find at apache-solr-3.5.0/example/solr/conf. The lib directory should contains apache-solr-velocity-3.5.0.jar, commons-beanutils-1.7.0.jar,commons-collections-3.2.1.jar,velocity-1.6.4.jar,velocity-tools-2.0.jar. The solr home directory will also have index.
  7. On oepenshift you can put data in $OPENSHIFT_DATA_DIR. So create a folder solr.home under $OPENSHIFT_DATA_DIR directory.
  8. Push the conf and lib directory to $OPENSHIFT_DATA_DIR/solr.home directory using rsync.I zipped both conf and lib directory in one solr.zip file and extracted on remote machine.
    rsync -avz -e ssh solr.zip xxx@solr-india.rhcloud.com:$OPENSHIFT_DATA_DIR/solr.home
    
  9. Now you need to add solr.war, commit it and push it to openshift.
    git add .
    git commit -a -m "committing solr war"
    git push
    
  10. The above line will stop the jboss and deploy the war. But you will get exception because you didn’t specified solr home. To do that ssh into the openshift application instance and execute the command shown below.
    ctl_all stop
    export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=$OPENSHIFT_DATA_DIR/solr.home"
    ctl_all start
    

    Finally you will see up and running solr http://solr-india.rhcloud.com/solr/admin/

Say Hello to Jelastic

These days Platform as a Service (PaaS) is one of my interest areas and I like to play with different PaaS providers to see how easy or difficult it is to develop and deploy application on them. The best thing about most of the current new generation PaaS systems is that they don’t require you to change your code or learn new programming paradigm. Google App Engine is thing of past and is losing ground in PaaS race. For last six months I have spend some of my spare time on OpenShift and Cloud Foundry and one thing I can say is that I love both of the platforms. Today I decided to spend some time on Jelastic — seeing how easy or difficult is to deploy a simple Spring MongoDB application on it. According to Jelastic website

Jelastic is the next generation of Java hosting platforms which can run and scale ANY Java application with no code changes required

Jelastic provides a web ui using which you can create the deployment environment and upload your war file to it.To check the usability of the UI I decided that I will not refer to Jelastic documentation and will try to deploy the application based on my understanding. So in this blog I am sharing the steps I performed to deploy a simple Spring MongoDB application to Jelastic.

  1. To start I created a very simple simple moviestore application using Spring Roo. For those of you who are not aware of Spring Roo can refer to my article series at IBM Developerworks on Spring Roo.Once you have installed Spring Roo, fire the Roo shell and execute following commands. This will create a Spring MVC web application with MongoDB as backend.
    project --topLevelPackage com.shekhar.moviestore --projectName moviestore
    mongo setup --databaseName moviestore
    entity mongo --class ~.domain.Movie
    field string --fieldName title --notNull
    field string --fieldName description --notNull
    repository mongo --interface ~.repository.MovieRepository
    service --interface ~.service.MovieService
    web mvc setup
    web mvc all --package ~.web
    q
    
  2. You can test the application locally by first starting the MongoDB server and then starting the application using mvn tomcat:run.
  3. But the point is to test the application on Jelastic. So go to http://jelastic.com/ and sign up for free. You don’t need to pay anything. I choose North America hosting provider.
  4. Once you have registered at Jelastic login with your credentials at https://app.jelastic.servint.net/
  5. After you have logged in to Jelastic portal you will see a Create environment link on the left. In Jelastic you have to first create environment under which your application will run. Click on the environment link and choose MongoDB, Tomcat, Java 6 as the environment topology. This is shown in image below. I really liked the UI. It is sexy.
  6. When you press create it will take couple of minutes to create the environment. So please be patient.
  7.  You will receive an email from Jelastic with the MongoDB connection details. It will give you a url to access MongoDB from web UI and an admin username and password.In my case I received url http://mongodb-moviestore.jelastic.servint.net/. I am not going to share username and password.
  8. The MongoDB UI is a RockMongo MongoDB web client. Login into it using admin username and password and Rock 🙂
  9. Next we need to create a MongoDB database and user with which our application can connect. To create database first click on databases and then “Create new Database”. Enter the name of database as moviestore and press create button. Next click on newly created moviestore database and click more then authentication and then click on add user to create a new user. Create a user with username as moviestore and password as password and press Add user.
  10. Now that we have created a user we should update the database.properties and applicationContext-mongo.xml files which were created by Spring Roo. By default they were pointing to localhost. Update the files as shown below.
    database.properties

    #Updated at Tue Feb 28 12:26:32 IST 2012
    #Tue Feb 28 12:26:32 IST 2012
    mongo.host=mongodb-moviestore.jelastic.servint.net
    mongo.name=moviestore
    mongo.password=password
    mongo.port=27017
    mongo.username=moviestore
    

    applicationContext-mongo.xml

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <beans xmlns="http://www.springframework.org/schema/beans" xmlns:cloud="http://schema.cloudfoundry.org/spring" xmlns:context="http://www.springframework.org/schema/context" xmlns:mongo="http://www.springframework.org/schema/data/mongo" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd        http://www.springframework.org/schema/data/mongo        http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd        http://www.springframework.org/schema/beans        http://www.springframework.org/schema/beans/spring-beans-3.1.xsd        http://schema.cloudfoundry.org/spring http://schema.cloudfoundry.org/spring/cloudfoundry-spring-0.8.xsd">
    
        <mongo:db-factory dbname="${mongo.name}" host="${mongo.host}" id="mongoDbFactory" password="${mongo.password}" port="${mongo.port}" username="${mongo.username}"/>
    
        <mongo:repositories base-package="com.shekhar.moviestore"/>
    
        <!-- To translate any MongoExceptions thrown in @Repository annotated classes -->
        <context:annotation-config/>
    
        <bean class="org.springframework.data.mongodb.core.MongoTemplate" id="mongoTemplate">
            <constructor-arg ref="mongoDbFactory"/>
        </bean>
    
    </beans>
    
  11. Build the maven project by executing mvn clean install command.
  12. Then upload the war by clicking on upload link in the Jelastic web UI.This will take some time depending on your internet connection.
  13. After the war is uploaded you will see the war in deployment manager tab. Click deploy to moviestore environment to deploy to tomcat and select context as ROOT.
  14. Finally you will be able to view the application running at http://moviestore.jelastic.servint.net/

This was my first write up on Jelastic and I will continue experimenting with it and evaluating its capabilities. I will also spend time reading its documentation and see how it compare with other PaaS providers. Overall I was impressed with Jelastic and to me it looks like a good deployment option for Java applications.