Skip to content

Using Google App Engine with Grails

Using a Platform-as-a-Service (PaaS) provider has many advantages over rigging up your own server and maintaining your application yourself. First you don’t have to worry about the server infrastructure (OS, Security Patches, Networking etc.). But PaaS goes one step further and provides the hosting platform too. This means you should be able to just upload your application to the PaaS provider and your application is ready and accessible. Additionally there are even providers which offer their services free-of-charge in a limited setting (see http://code.google.com/intl/de-DE/appengine/kb/billing.html#freequota). Having a free hosting solution sounded great, so I gave GAE a shot.

First thing to check when trying something new with Grails: Is there already a plugin? Well, there is. And its pretty good too. You get up to speed in no time and don’t have to worry much about your dependency on GAE. The plugin handles the removal of the hibernate plugin (which you can’t use anymore, since GAE persistence is done using datanucleus), the creation of the needed configuration files and the deployment of your application.

In my situation I already had a pretty complete application which I “just” wanted to make available in the cloud. What I had to do was add annotations controlling the persistence behaviour to my domain classes (if you’re developing on a windows box, you need to check this as well: http://www.morkeleb.com/2009/07/27/grails-and-google-appengine-on-windows/). You’ll end up having something similar to this:

import javax.jdo.annotations.PersistenceCapable
import javax.jdo.annotations.IdentityType
import javax.jdo.annotations.Key
import javax.jdo.annotations.IdGeneratorStrategy
import javax.jdo.annotations.Persistent
import javax.jdo.annotations.PrimaryKey

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable = "true")
class Location {

  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  Key key

  @Persistent
  String name

  @Persistent
  String address

  @Persistent
  String city

  @Persistent
  String region
...
}

Now comes the tricky part. I mentioned that I already had a pretty complete application which, of course, was using a handful of plugins. It turns out you cannot use plugins which depend on hibernate. I had two plugins installed which depend on hibernate, searchable and commentable. Well, I didn’t use the searchable plugin extensively, I could have replaced it with some simple calls to findBy. The commentable plugin was not much of a loss either, I could have replaced it with some hours effort.

The next thing I stumbled upon was the restrictions on java classes which GAE allows. I was using the excellent groovy httpbuilder in my application which in turn is using HttpsURLConnection, which is a restricted class in GAE. At this point I didn’t continue with this experiment. Due to the restrictions GAE lays upon its users, it was not a viable solution for me.

The conclusion is of course to thoroughly read through the documentation which comes for GAE and especially for the JVM sandbox and the JRE white list, for all classes which you are allowed to use in your GAE application.

Other PaaS solutions include Amazons Elastic Beanstalk which makes a very good impression. But the pricing model is no alternative to GAE, since my application would be non-profit. VMWares CloudFoundry would be the most “native” PaaS solution for Grails apps, but the pricing is not available yet, plus you can only get on the platform by invitation. I will definitely keep an eye on both solutions.

Adding custom constraints per AST transformation

While developing a Grails plugin (https://github.com/drosowski/grails-concurrent-update) I needed to add a custom constraint to all domain classes which are annotated with a custom annotation. Looking for ways to accomplish this I stumbled upon AST transformations. AST transformations can be thought of as metaprogramming at compile time. There are already quite a few articles on this issue, so I won’t cover the basics here.

o The official groovy user guide covers this topic.
o or an article by Hamlet D’Arcy.

Also there are already some plugins where I recklessly copy&pasted some of the code (like the excellent multi-tenant plugin). But I could only find code to insert pre defined constraints like nullable, not custom constraints. So I needed

  1. a custom annotation, and
  2. the corresponding class which performs the AST transformation

The annotation was the easy part. You just have to define an interface in Grails’ src directory, looking something like this.

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
@GroovyASTTransformationClass("com.rosowski.grails.plugin.ConcurrentUpdateASTTransformation")
public @interface ConcurrentUpdate {
}

The next thing we need is the class which performs the AST transformation, as stated in the annotation of the interface. Please have a look at the sourcecode hosted at github for details, since I will only cover the building of the custom validator.

I decided to use AstBuilder, because it lets you build ASTNodes from String input, which is pretty convenient. So I declared my custom validator as a String, looking like this.

    private static final String checkVersion = "version(validator: { Long value, object ->\n" +
            "      if (value != null) {\n" +
            "        def storedVersion = object.getPersistentValue('version')\n" +
            "        if (storedVersion > value) {\n" +
            "          object.version = storedVersion\n" +
            "          object.isLocked = true\n" +
            "          return ['optimistic.locking.failure']\n" +
            "        }\n" +
            "        else {\n" +
            "          object.isLocked = false\n" +
            "          return true\n" +
            "        }\n" +
            "      }\n" +
            "      return true\n" +
            "    })";

Granted, it does look pretty clumsy and is hard to read. First I tried using a class which provided the constraint as a closure, but that didn’t work. Somehow the call getField on the classNode always returned null.

The corresponding code to transform the string into an ASTNode is done in the method getCheckVersion().

    private Expression getCheckVersion() {
        List<ASTNode> nodes = new AstBuilder().buildFromString(checkVersion);
        if (nodes != null && nodes.size() > 0) {
            BlockStatement block = (BlockStatement) nodes.get(0);
            List<Statement> statements = block.getStatements();
            if (statements != null && statements.size() > 0) {
                ReturnStatement returnStatement = (ReturnStatement) statements.get(0);
                return returnStatement.getExpression();
            }
        }
        return null;
    }

Next I can just add the resulting expression to the existing constraints closure found in the annotated class.

    private void addVersionConstraint(ClassNode classNode, Expression checkVersion) {
        FieldNode constraints = classNode.getField("constraints");
        ClosureExpression closure = (ClosureExpression) constraints.getInitialExpression();
        BlockStatement block = (BlockStatement) closure.getCode();
        block.addStatement(new ExpressionStatement(checkVersion));
        LOG.info("[OptimisticLockASTTransformation] Added constraint [version] to class [" + classNode.getName() + "]");
    }

And this way I have a custom constraint added to all classes annotated with @ConcurrentUpdate. Sweet, isn’t it?🙂

I would be glad to hear about your comments or improvements!

Upcoming Grails and Groovy content

Well, its been quite a while since my last post on this blog. I could say I didn’t have the time or the energy to blog, but to be honest I just didn’t have to say much. And according to my grandma its best to shut up if you don’t have anything important to say. My focus at work changed from using plain JEE and OFBiz to Grails. And with that I learned a lot which I would like to share. So hopefully the upcoming articles will be useful to somebody.

YUI – Howto build an editable dropdown box

I’ve been playing around with Yahoo’s UI lately. I think it has some pretty nice features (autocomplete, calendar etc.) plus it integrates well with Grails. For a personal project of mine, I had the requirement to display a list of suggestions to the user, but he/she should still be able to define their own values. I remembered an editable dropdown box which I saw on some desktop application.

<div id="gameTypeBox">
  <input type="text" id="gameType" name="gameType" value="${fieldValue(bean:game,field:'gameType')}" />
  <div id="gameTypeList">
  </div>
</div>
<script type="text/javascript">
  var myDataSource = new YAHOO.util.XHRDataSource('${createLink(controller:'game', action:'listGameTypes')}');
  myDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_JSON;
  myDataSource.responseSchema = {resultsList : "types"};

  var myAutoComp = new YAHOO.widget.AutoComplete('gameType','gameTypeList', myDataSource);
  myAutoComp.minQueryLength = 0;
  myAutoComp.queryDelay = 0;

  //show suggestions
  var showContainer = function(myAutoComp) {
    this.sendQuery('');
  };
  myAutoComp.textboxFocusEvent.subscribe(showContainer);
</script>

First of all, don’t get confused by the ${} constructs, they are Grails specific. As you can see it’s pretty straightforward. First you define your datasource. In this example its a XHR request returning a list of all gametypes in JSON format.The minQueryLength and queryDelay are both set to 0, since the suggestions should pop up as soon as the textfield is on focus. This is done in the next part, where we send an empty XHR query, so the corresponding method in the controller gets triggered and we get the list of gametypes. Next you subscribe the function to the textboxFocusEvent and you’re done. Of course you can still autocomplete the users text input, but this has to be done in the controller. In this example I first show all suggestions to the user. After that the list gets filtered by the users input.

Relocating a tomcat webapp to another server

Today I want to show you a little trick I used to relocate a tomcat webapplication to another server running with a new IP. The webapp is already running on the new server, but the hostname still points to the old server. I don’t want the users to have to wait until the DNS entry spread its way across the internet. So I decided to redirect all traffic from the old server to the new server. Ideally you have configured an apache webserver in front of your tomcat (for security, performance etc., see here). The article is a bit outdated, but it still has some valid points especially for the reasons on why you should put a webserver in front of your tomcat. Of course you would use mod_proxy_ajp nowadays for this task instead of mod_jk as recommended in the article.
Ok, back to the problem. In order to connect the webserver with tomcat they talk via ajp usually on port 8009. So what I did is just forward this port using ssh to the new server like this:

ssh -N -L 8009:192.168.1.2:8009 user@192.168.1.2

where 192.168.1.2 is the new servers IP address. This way the user doesn’t recognize he/she has been redirected. Sweet, isn’t it?
The apache on the new server is already configured to listen to HTTP requests and redirects them to localhost:8009 (where tomcat is listening). So when the hostname is switched to the new IP address the webserver running on the new server is forwarding the requests to tomcat. By using this trick we have a seamless transition during the server switch which is absolutely transparent for the user.

What other options can you think of?  I bet there are a hundred ways to do it with apache🙂

I love CollectionUtils

I just love the CollectionUtils. Today I was facing a problem, where I was iterating through a map and wanted to check if the key partially matches one element from a list of values. Still following? I think it gets much clearer when I go straight to the code. Let me just say this: CollectionUtils.exists  kicks ass!🙂

        final List<String> prefixes = UtilMisc.toList("first_", "second_", "third_");

        for (Map.Entry entry : context.entrySet()) {
            String value = null;
            String mapKey = null;
            String afterPrefix = null;
            if (entry.getKey() instanceof String) {
                final String key = (String) entry.getKey();
                if (CollectionUtils.exists(prefixes, new Predicate() {

                    public boolean evaluate(Object arg0) {
                        String element = (String) arg0;
                        if (key.startsWith(element)) {
                            return true;
                        }
                        return false;
                    }
                })) {
                    String[] strArr = key.split("_");
                    mapKey = strArr[0];
                    afterPrefix = strArr[1];
                    value = (String) entry.getValue();
                }
            }
        }

*EDIT: I should have mentioned that I’m talking about the Commons CollectionUtils. They can be found here.

Subversion over https – PITA

I’ve been using subversion for a long time. Recently I reconfigured the server to use https to have the subversion traffic encrypted. When you have a large codebase with many files and you’re following subversion best practices and branch your project for big changes you will have a problem. Over http/https the merging process will take forever. Well, this is of course exaggerated to make my point, but it is really slow. Compared to svn+ssh which I’m using now, https is a real performance killer.

At some point I read about turning off the path based checks for mod_authz_dav using “SVNPathAuthz Off”, but that helped little. I think the big overhead is the http protocol, which just isn’t meant for things like that.

My tip for everyone making extensive use of subversions branching and merging capabilities: Use svn+ssh over https for encrypted subversion traffic. It will speed things up significantly!

Follow

Get every new post delivered to your Inbox.