First thing to check when trying something new with Grails: Is there already a plugin? Well, there is. And its pretty good too. You get up to speed in no time and don’t have to worry much about your dependency on GAE. The plugin handles the removal of the hibernate plugin (which you can’t use anymore, since GAE persistence is done using datanucleus), the creation of the needed configuration files and the deployment of your application.

In my situation I already had a pretty complete application which I “just” wanted to make available in the cloud. What I had to do was add annotations controlling the persistence behaviour to my domain classes (if you’re developing on a windows box, you need to check this as well: http://www.morkeleb.com/2009/07/27/grails-and-google-appengine-on-windows/). You’ll end up having something similar to this:

import javax.jdo.annotations.PersistenceCapable
import javax.jdo.annotations.IdentityType
import javax.jdo.annotations.Key
import javax.jdo.annotations.IdGeneratorStrategy
import javax.jdo.annotations.Persistent
import javax.jdo.annotations.PrimaryKey

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable = "true")
class Location {

  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  Key key

  @Persistent
  String name

  @Persistent
  String address

  @Persistent
  String city

  @Persistent
  String region
...
}

Now comes the tricky part. I mentioned that I already had a pretty complete application which, of course, was using a handful of plugins. It turns out you cannot use plugins which depend on hibernate. I had two plugins installed which depend on hibernate, searchable and commentable. Well, I didn’t use the searchable plugin extensively, I could have replaced it with some simple calls to findBy. The commentable plugin was not much of a loss either, I could have replaced it with some hours effort.

The next thing I stumbled upon was the restrictions on java classes which GAE allows. I was using the excellent groovy httpbuilder in my application which in turn is using HttpsURLConnection, which is a restricted class in GAE. At this point I didn’t continue with this experiment. Due to the restrictions GAE lays upon its users, it was not a viable solution for me.

The conclusion is of course to thoroughly read through the documentation which comes for GAE and especially for the JVM sandbox and the JRE white list, for all classes which you are allowed to use in your GAE application.

Other PaaS solutions include Amazons Elastic Beanstalk which makes a very good impression. But the pricing model is no alternative to GAE, since my application would be non-profit. VMWares CloudFoundry would be the most “native” PaaS solution for Grails apps, but the pricing is not available yet, plus you can only get on the platform by invitation. I will definitely keep an eye on both solutions.

o The official groovy user guide covers this topic.
o or an article by Hamlet D’Arcy.

Also there are already some plugins where I recklessly copy&pasted some of the code (like the excellent multi-tenant plugin). But I could only find code to insert pre defined constraints like nullable, not custom constraints. So I needed

1. a custom annotation, and
2. the corresponding class which performs the AST transformation

The annotation was the easy part. You just have to define an interface in Grails’ src directory, looking something like this.

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
@GroovyASTTransformationClass("com.rosowski.grails.plugin.ConcurrentUpdateASTTransformation")
public @interface ConcurrentUpdate {
}

The next thing we need is the class which performs the AST transformation, as stated in the annotation of the interface. Please have a look at the sourcecode hosted at github for details, since I will only cover the building of the custom validator.

I decided to use AstBuilder, because it lets you build ASTNodes from String input, which is pretty convenient. So I declared my custom validator as a String, looking like this.

    private static final String checkVersion = "version(validator: { Long value, object ->\n" +
            "      if (value != null) {\n" +
            "        def storedVersion = object.getPersistentValue('version')\n" +
            "        if (storedVersion > value) {\n" +
            "          object.version = storedVersion\n" +
            "          object.isLocked = true\n" +
            "          return ['optimistic.locking.failure']\n" +
            "        }\n" +
            "        else {\n" +
            "          object.isLocked = false\n" +
            "          return true\n" +
            "        }\n" +
            "      }\n" +
            "      return true\n" +
            "    })";

The corresponding code to transform the string into an ASTNode is done in the method getCheckVersion().

    private Expression getCheckVersion() {
        List<ASTNode> nodes = new AstBuilder().buildFromString(checkVersion);
        if (nodes != null && nodes.size() > 0) {
            BlockStatement block = (BlockStatement) nodes.get(0);
            List<Statement> statements = block.getStatements();
            if (statements != null && statements.size() > 0) {
                ReturnStatement returnStatement = (ReturnStatement) statements.get(0);
                return returnStatement.getExpression();
            }
        }
        return null;
    }

Next I can just add the resulting expression to the existing constraints closure found in the annotated class.

    private void addVersionConstraint(ClassNode classNode, Expression checkVersion) {
        FieldNode constraints = classNode.getField("constraints");
        ClosureExpression closure = (ClosureExpression) constraints.getInitialExpression();
        BlockStatement block = (BlockStatement) closure.getCode();
        block.addStatement(new ExpressionStatement(checkVersion));
        LOG.info("[OptimisticLockASTTransformation] Added constraint [version] to class [" + classNode.getName() + "]");
    }

And this way I have a custom constraint added to all classes annotated with @ConcurrentUpdate. Sweet, isn’t it?

I would be glad to hear about your comments or improvements!

<div id="gameTypeBox">
  <input type="text" id="gameType" name="gameType" value="${fieldValue(bean:game,field:'gameType')}" />
  <div id="gameTypeList">
  </div>
</div>
<script type="text/javascript">
  var myDataSource = new YAHOO.util.XHRDataSource('${createLink(controller:'game', action:'listGameTypes')}');
  myDataSource.responseType = YAHOO.util.XHRDataSource.TYPE_JSON;
  myDataSource.responseSchema = {resultsList : "types"};

  var myAutoComp = new YAHOO.widget.AutoComplete('gameType','gameTypeList', myDataSource);
  myAutoComp.minQueryLength = 0;
  myAutoComp.queryDelay = 0;

  //show suggestions
  var showContainer = function(myAutoComp) {
    this.sendQuery('');
  };
  myAutoComp.textboxFocusEvent.subscribe(showContainer);
</script>

First of all, don’t get confused by the ${} constructs, they are Grails specific. As you can see it’s pretty straightforward. First you define your datasource. In this example its a XHR request returning a list of all gametypes in JSON format.The minQueryLength and queryDelay are both set to 0, since the suggestions should pop up as soon as the textfield is on focus. This is done in the next part, where we send an empty XHR query, so the corresponding method in the controller gets triggered and we get the list of gametypes. Next you subscribe the function to the textboxFocusEvent and you’re done. Of course you can still autocomplete the users text input, but this has to be done in the controller. In this example I first show all suggestions to the user. After that the list gets filtered by the users input.

ssh -N -L 8009:192.168.1.2:8009 user@192.168.1.2

where 192.168.1.2 is the new servers IP address. This way the user doesn’t recognize he/she has been redirected. Sweet, isn’t it?
The apache on the new server is already configured to listen to HTTP requests and redirects them to localhost:8009 (where tomcat is listening). So when the hostname is switched to the new IP address the webserver running on the new server is forwarding the requests to tomcat. By using this trick we have a seamless transition during the server switch which is absolutely transparent for the user.

What other options can you think of?  I bet there are a hundred ways to do it with apache

        final List<String> prefixes = UtilMisc.toList("first_", "second_", "third_");

        for (Map.Entry entry : context.entrySet()) {
            String value = null;
            String mapKey = null;
            String afterPrefix = null;
            if (entry.getKey() instanceof String) {
                final String key = (String) entry.getKey();
                if (CollectionUtils.exists(prefixes, new Predicate() {

                    public boolean evaluate(Object arg0) {
                        String element = (String) arg0;
                        if (key.startsWith(element)) {
                            return true;
                        }
                        return false;
                    }
                })) {
                    String[] strArr = key.split("_");
                    mapKey = strArr[0];
                    afterPrefix = strArr[1];
                    value = (String) entry.getValue();
                }
            }
        }

*EDIT: I should have mentioned that I’m talking about the Commons CollectionUtils. They can be found here.
]]> http://rosowski.wordpress.com/2008/12/18/i-love-collectionutils/feed/ 0 danielrosowski http://rosowski.wordpress.com/2008/11/19/subversion-over-https-pita/ http://rosowski.wordpress.com/2008/11/19/subversion-over-https-pita/#comments Wed, 19 Nov 2008 16:08:23 +0000 danielrosowski http://rosowski.wordpress.com/?p=29 ]]> I’ve been using subversion for a long time. Recently I reconfigured the server to use https to have the subversion traffic encrypted. When you have a large codebase with many files and you’re following subversion best practices and branch your project for big changes you will have a problem. Over http/https the merging process will take forever. Well, this is of course exaggerated to make my point, but it is really slow. Compared to svn+ssh which I’m using now, https is a real performance killer.

At some point I read about turning off the path based checks for mod_authz_dav using “SVNPathAuthz Off”, but that helped little. I think the big overhead is the http protocol, which just isn’t meant for things like that.

My tip for everyone making extensive use of subversions branching and merging capabilities: Use svn+ssh over https for encrypted subversion traffic. It will speed things up significantly!

I guess most people who are using Lucene came to the point where they want to score direct matches higher than wildcard matches. For example if we have the searchphrase “cup” and we have several documents in our index which contain the searchphrase like:

• doc 4711: name: cupholder
• doc 4712: name: cup

In most use-cases people want the direct match, in this case doc 4712 to appear as the first hit. Because cup is contained in both names, you get both documents as matches, but doc 4712 as the first match. So far so good. If you add more complexity to the documents perhaps by giving them a description, you might end up with a messed up scoring. This can have different causes (perhaps “cup” appears in the description of doc 4711 but not in 4712). Check out the Similarity class in Lucene and call searcher.explain() on your document to find out the reason for the score. What you want to do is to boost a direct match. Here is how I did it:

//where fieldName is 'name' or 'description'
Query tmpQuery = new TermQuery(new Term(fieldName, searchString));
//boost the direct match
tmpQuery.setBoost(2.0f);
query.add(tmpQuery, BooleanClause.Occur.SHOULD);

//search for other occurences
tmpQuery = new WildcardQuery(new Term(fieldName, "*" + searchString + "*"));
query.add(tmpQuery, BooleanClause.Occur.SHOULD);

I hope that helped somebody on his or her Lucene journey. Have a nice day.

David added some examples in the example application in OFBiz. Here is an excerpt from the services.xml showing the usage of the new execution engine:

    <service name="createExample" default-entity-name="Example" engine="entity-auto" invoke="create" auth="true">
        <description>Create a Example</description>
        <permission-service service-name="exampleGenericPermission" main-action="CREATE"/>
        <auto-attributes include="pk" mode="OUT" optional="false"/>
        <auto-attributes include="nonpk" mode="IN" optional="true"/>
        <override name="exampleTypeId" optional="false"/>
        <override name="statusId" optional="false"/>
        <override name="exampleName" optional="false"/>
    </service>

To get a grasp of what I’m talking about here, you should already have a general understanding of the basic Lucene concepts. There are some pretty good tutorials which cover the basics. Be sure to at least check out the introduction to the demo application which comes with Lucene. A very good tutorial about indexing has been written by Deng Peng Zhou.

As I started working with Lucene, I usually went with the standard search routine:

Hits result = searcher.search(query);

I’m running a webshop where products are assigned to categories. Soon there was the requirement to count the search hits in specific categories. An easy task with the Lucene HitCollector. Here’s my HitCollector

public class CountCollector extends TopFieldDocCollector {

    private Searcher searcher = null;
    private Map countMap = new HashMap();

    public CountCollector(Searcher searcher, IndexReader reader, Sort sorter, int maxSearchResults) throws IOException {
        super(reader, sorter, maxSearchResults);
        this.searcher = searcher;
    }

    public void collect(int doc, float score) {
        super.collect(doc, score);

        try {
            Document document = searcher.doc(doc);
            if (document != null) {
                    Field[] categoriesDoc = document.getFields("categories");
                    if (categoriesDoc != null && categoriesDoc.length > 0) {
                        for (int i = 0; i < categoriesDoc.length; i++) {
                            if (countMap.containsKey(categoriesDoc[i].stringValue())) {
                                countMap.put(categoriesDoc[i].stringValue(), new Long(countMap.get(categoriesDoc[i].stringValue()) + 1));
                            } else {
                                countMap.put(categoriesDoc[i].stringValue(), new Long("1"));
                            }
                        }
                   }
             }
        }
        catch (CorruptIndexException e) {
            System.err.println("ERROR: " + e.getMessage());
        }
        catch (IOException e) {
            System.err.println("ERROR: " + e.getMessage());
        }
    }
}

CountHitCollector collector = new CountHitCollector(searcher, indexReader, sorter, maxSearchResults);
searcher.search(finalQuery, collector);
ScoreDoc[] hits = collector.topDocs().scoreDocs;

Document doc = searcher.doc(hits[i].doc);

This post showed just one example for the usage of the HitCollector. There are many more cases for which it might come in handy. Tell me if this post was useful to you and perhaps for which case you used the HitCollector.