The boys at JBoss had recently released Hibernate Search 3.2.0 and it looked pretty sweet. The possibility of performance improvements around indexing was enough to make me upgrade.

Unfortunately, like most software frameworks, Hibernate Search introduced/forced some new dependencies on us. We had previously been using Seam 2.2.0 and Hibernate 3.3.0 (both JPA1), neither of which were compatible with Hibernate Search 3.2.0.

We’re using Maven, so I’ll quickly outline the dependency changes I had to make (all dependences are available via the JBoss Nexus Repository):

<dependency>
    <groupId>javax.persistence</groupId>
    <artifactId>persistence-api</artifactId>
    <!– bit of a hack to make sure we aren’t transitively pulling in the persistence api (should be referencing hibernate-jpa-2.0-api now) –->
    <version>explicitly-fail</version>
</dependency>

javax.persistence:persistence-api is the JPA1 API and is no longer relevant and should never be included as a dependency. The explicitly-fail version will trigger a build failure if it ever manages to sneak in.

<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.0-api</artifactId>
<version>1.0.0.Final</version>
</dependency>

org.hibernate.javax.persistence:hibernate-jpa-2.0-api is the new JPA2 API published by Hibernate. 

All previous hibernate dependencies had to be upgraded to 3.5.2.Final, and Hibernate Search bumped up to 3.2.0.Final.

If you’re using Seam, you’ll also need to upgrade it to 2.2.1.CR1.  Note that this is not yet a final release but is necessary in order to support JPA2.

Unfortunately, Seam 2.2.1.CR1 lacks a complete Hibernate Search integration.  I could no longer able properly inject a FullTextEntityManager (via @In).

Instead I was forced to obtain a FullTextSession programmatically:

private FullTextSession getFullTextSession()
{
    Session session = (Session) entityManager.getDelegate();
    return Search.getFullTextSession  (session.getSessionFactory().getCurrentSession());
}

Seam also enhanced it’s EntityManager so that it now returns a JDK proxy when getDelegate() is called. 

That would be fine and dandy, but unfortunately the proxy does not implement org.hibernate.classic.Session, as required by the FullTextSession in Hibernate Search. 

Session.getSessionFactory().getCurrentSession() will return an org.hibernate.classic.Session and does allow you to move forward. 

A bit annoying and hopefully something that will get corrected in the near future.  I’m hoping that either Hibernate Search will be upgraded to not require a classic session, or Seam will provide an EntityManager delegate implementing the classic session interface. Better yet, Seam should just provide a fully supported integration with Hibernate Search 3.2.0.

Honestly, that was about it as far as the upgrade went.  We were able to index using either the new MassIndexer API or the old suggested best practice from Hibernate Search 3.1. 

It’s interesting to note that on my test system, the MassIndexer API was slower than the previous indexing code. The documentation does cover many different tuning possibilities so it’s entirely possible the situation could be improved.

Also, the MassIndexer API runs in a series of transactions and unless your timeout is set sufficiently high (ie. higher than the default JBoss 5 minutes), you will see transaction timeouts.

I didn’t attempt to work around this and opt’d to stick with existing indexing code that already runs in a separate Seam @Asynchronous method (thus avoiding timeouts).

Somewhat less than ideal, but if you’re keen on running Hibernate Search 3.2.0 w/ Seam in JBoss 4.2.3, it is possible and not too much work.

The company I work for made the decision to OEM a product from a partner, rather than invest valuable development time to build something that has essentially been commoditized.

Development resources are at a premium and we consciously don’t want to spend any more time on this code base than we have to, we’re much better at adding value in other areas.

Not necessarily a bad or the wrong decision, we’ve just reached the point in time where we need to make functional changes to the OEM’d code base. 

Due Diligence

Obviously, before you entertain the outright purchase or license of a third party product/code base, there’s a few questions you ought to ask yourself:

- Is the technology stack compatible with our existing products?  (A member of the development staff should provide the answer)

- Does the product do significantly less/more than we will ever need it to? (The later is an important question, do you really want to maintain features that customers will never use? What’s the cost of re-implementing just the features the customer does need?)

- What’s the support model look like? (Who is fixing bugs, you or the source company? How are patches exchanged, or are they?)

Maintenance

During the course of our due diligence, we did send a couple developers on site to become familiar with not only the code base but in particular the build process around it.

This did pay dividends as we were able to setup Bamboo builds capable of automatically creating installation packages and running unit tests.

Significant win in my books.  Test coverage may not be the greatest, but if you can build a complete candidate release in 7 minutes, manual testing/verification remains a possibility.

That’s it for now. 

I’ll admit that I was a skeptic when the idea of adopting a 3rd party code base was first brought up.  Fortunately the changes we’ve been required to make thus far have been minor.  Technology stack is slightly different and anecdotally bug fixes seem to be taking ~3x longer then they would on our primary product, I expect that ratio to more or less even out as we familiarize ourselves with the patterns and structure in the new code base. 

The verdict’s still out on whether or not we’ll be able to add any new components of significant functionality while keeping things at all maintainable.

But of course that’s the challenge inherent with any software development activity!

The objective is to learn enough about the platform during these initial largely custom implementations, to be in a position to hopefully create and deliver a largely standalone solutions faster than we would using the typical Java technology stack.

The jury is still out, but what follows are some initial thoughts after working with the platform.  The team in question are a group of 3 senior developers working collectively on an application deployed in a single Salesforce Organization. 

Forgive my use or miss-use of Salesforce/DeveloperForce terminology.  It can get confusing at times

Version Control

Like any good development shop, we store all code and build artifacts in source control.  Subversion is our poison of choice.

Salesforce development is a bit different in that a lot of the configuration and development activities can occur in either the browser or their Eclipse-based IDE. 

Our strategy for getting code and configurations under source control was to use Eclipse and regularly sync with our development organizations (Salesforce jargon for a development space or area). 

Unbeknownst to us at the onset, there are certain configurations you can make using the browser that are not possible to make with the IDE.  It’s important that these configurations (user/queue creation, etc.) are documented externally (ie. in a spreadsheet).

Code and configurations are moved between developers using Subversion and the Eclipse synchronization capabilities (or the Salesforce migration tool).  By storing these artifacts in Subversion, we’re also able to do code reviews on them via Crucible.

Your mileage may vary with this approach, as we found the current Salesforce tooling to be quite poor at conflict resolution when it came to structural changes.  Moreover, we frequently had to fall back to manually making changes via the web browser. 

Very annoying. 

I think it’s safe to assume that despite all the redeeming qualities of the DeveloperForce platform, some significant gaps remain when it comes to the expectations of traditional software developers.  By traditional, I mean those of us that are used to using source control, doing code reviews, etc.  I guess Salesforce considers us a dying breed

Would I choose the platform to build a product I expected to maintain in the long-term and derive significant value from? No. 

Would I use it in the short-term as a RAD environment to build a prototype? Possibly, but not without first considering some of the more traditional alternatives (there are a number of schema-less data storage products available these days, combine that with a Grails, Seam, Rails or Django and you may very well be off to the races).

That’s enough for one night.  I plan on doing a follow-up once the team has more experience maintaining and migrating code after it’s been deployed to DeveloperForce.

Particularly interesting for me was his overview of the different mechanisms by which you can support multiple customer schemas securely and with decent performance.  The product I’m actively working on is likely headed in this direction, looks like Oracle VPD is Emmanuel’s preferred solution.  A quick search has turned up an add-on for Postgres called Veil which aims to provide row/column-level security similar to Oracle VPD.  Good to at least have a choice, but I imagine that when push comes to shove, Oracle will win out.

Shards has always looked interesting, will be nice when it finally hits GA and supports the JPA API.  Would make it slightly easier to incorporate and play around with.

Interesting thoughts on clustering Lucene/Hibernate Search.  We’re currently running it asynchronously (ie. @Asynchronous in Seam) on a single node but will likely need to look at pushing it to a second box and trying to get indexes in near real time without noticeable degradation to the front-end.

That was on top of the better part of a day that was spent by another developer digging into the code.

Development ain’t easy, and frameworks for all their glory strive to make the easy stuff easier and the difficult stuff… well it’s still difficult.

Take Seam for example, a couple simple @In annotations here and there, and all of a sudden you have an application up and running. 

But throw transactions and exceptions into the mix and the expected behaviour is up in the air.

Imagine the following scenario:

Component A calls Component B.update() which in turn calls EntityManager.persist(someEntity).

someEntity fails a database constraint and an EntityAlreadyExists exception is propagated from Component B.update(). 

Can Component A turn around and update someEntity and call Component B.update() again?

It depends.  In Seam there is a RollbackInterceptor behind the scenes that will rollback any transaction crossing an injection boundary (it’s slightly more complicated than that but we’ll leave that for another day). 

If Component A was a POJO Transactional Seam component, and Component B was a Seam EntityHome component, this wouldn’t work.  In this sitation, Component B throwing an exception would actually rollback the transaction before control was returned to Component A.  Component A could very well handle the exception but the underlying transaction would still be rolled back.

From the looks of the implementation, the EntityHome (Component B) is relying on the entity existing in its PersistenceContext when doing an update.  You can update() multiple times as long as the transaction remains open and long-running.

Throw an exception into the mix and that transaction is likely to get rolled back by the RollbackInterceptor and the persistence context reset.

From that point on, calling Component B.update() (equivalent to EntityHome.update()) is going to report a success but do nothing but flush an empty persistence context. 

The short term fix was to have Component B.update() re-merge in someEntity.  However, rather than perverting the framework, it likely makes sense to dig deeper into the implementations and merge Component A and Component B to prevent the RollbackInterceptor from firing and rolling back the transaction on an exception that is recoverable.

The things you learn!

Few key points:

• Design for Intermediates Users
• Use Tools that Help Beginners to Become Intermediates
• Less is More
• Design for the Probable, Provide for the Possible
• Eliminate Errors or Confirmation Dialogs

There’s an interesting closing comment discussing the need (or lack there of) for error or confirmation dialogs. 

“Cooper notes that while they [error dialogs] are used to signal that something went wrong with the code, the user tends to interpret them as “I’ve done something wrong.”  When users are told that they are wrong repeatedly, they start to hate your product”

I suspect there’s some truth behind the statement, however all things being equal, if your product is blowing up, there’s a good chance that users aren’t going to be happy campers anyways.

That being said, I agree that showing an error dialog with some cryptic exception message only decipherable by the originating author, is probably not doing anyone a favor.  

Using the Java IDE IntelliJ as an example, it’s quite rare for a user to get an actual error dialog.  Instead, there’s a little status icon in the corner that will blink red whenever there’s a problem.  Clicking it will provide additional details with the option of submitting it back as a bug report. 

It’s important to remember who the user is, and considering the article’s focus on differentiating between beginner, intermediate and expert users, this status icon approach is probably best suited for intermediate/expert users, and not beginners. 

The problem I’m trying to solve is simple, define a grammar for a basic search language (to potentially be extended over time), parse it into an AST and manipulate appropriately.

After some initial frustration, I did make headway on parts 1 + 2 of the plan. We’ll see how the rest plays out next week.

For the benefit of others, if you’re looking to get started with ANTLR, here’s a useful introduction blog post. A quick search on dzone would have saved some trial and error.

For what it’s worth, getting ANTLR hooked into Maven is pretty simple and there’s decent documentation available for that. The IntelliJ plugin for ANTLRWorks is quite slick and includes a nice debugger.

I’m going to have to pickup The Definitive ANTLR Reference if this implementation pans out.

Unitils is an open source library aimed at making unit testing easy and maintainable. Unitils builds further on existing libraries like DBUnit and EasyMock and integrates with JUnit and TestNG .

Unitils provides general asserion utilities, support for database testing, support for testing with mock objects and offers integration with Spring , Hibernate and the Java Persistence API (JPA). It has been designed to offer these services to unit tests in a very configurable and loosely coupled way. As a result, services can be added and extended very easily.

What particularly caught my attention was the support they’ve included for transactional test cases. By transactional, I mean test cases that will automatically cleanup after themselves. Simply include a @Transactional and you’re rocking.

In the past, we’ve rolled this capability ourselves with varying degrees of success. It’s easy to support rolling back of inserts but somewhat difficult to handle rolling back of deletes and other data mutations.

I’m just in the process of kicking off some new product development (actually, we’re one week into it but who’s counting) and general infrastructure (testing included) is something forefront on our minds.

For starters, we’ve got Cobertura and Selenium integrated into our build. General unit is covered with TestNG with Seam components integration tested using SeamTest/Embedded JBoss. Everything short of the Selenium tests are hitting freshly initialized in-memory databases.

It might be worth considering something like Unitils at this stage rather than building out our own tooling to accomplish much the same thing.

I’m currently working on a Seam-based prototype and Groovy is certainly applicable to aspects other than Hibernate but it was a good initial goal.

Required POM Changes

<build>
<plugins>
    <plugin>
        <groupId>org.codehaus.groovy.maven</groupId>
        <artifactId>gmaven-plugin</artifactId>
        <version>1.0-rc-2</version>
        <executions>
            <execution>
                <goals>
                    <goal>generateStubs</goal>
                    <goal>compile</goal>
                    <goal>generateTestStubs</goal>
                    <goal>testCompile</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>
</build>

The gmaven plugin is able to cross-compile Java and Groovy. The compilation phase will generate Java stubs corresponding to the groovy classes prior to compiling the actual Java classes. This allows for seamless dependencies to exist between Groovy and Java.

It’s important to note that your Groovy sources must (by default) be in a src/main/groovy folder.

<dependencies>
    <dependency>
        <groupId>org.codehaus.groovy.maven.runtime</groupId>
        <artifactId>gmaven-runtime-default</artifactId>
        <version>1.0-rc-2</version>
    </dependency>
</dependencies>

Results

Simple as that. Unit tests passed!

I’m able to take a class that looked like:

@Entity
public class Annotation
{
    private Long id;
    private String name;

    private Specimen specimen;
    private Patient patient;

    /**
     * Getter for property 'id'.
     *
     * @return Value for property 'id'.
     */
    @Id @GeneratedValue
    public Long getId()
    {
        return id;
    }

    /**
     * Setter for property 'id'.
     *
     * @param id Value to set for property 'id'.
     */
    public void setId(Long id)
    {
        this.id = id;
    }
...
}

and make it look like:

@Entity
public class Annotation
{
    @Id @GeneratedValue
    Long id;

    @Length(max=50) @NotNull
    String name;

    @ManyToOne @JoinColumn(name = "specimen_id")
    Specimen specimen;

    @ManyToOne @JoinColumn(name = "patient_id")
    Patient patient;
}

Much simpler and without a lot of unnecessary boilerplate.

Gotchas

Just a couple of things to be aware of. Simple stuff really if you actually read documentation and know what you’re doing

Firstly, Groovy sources have to live in src/main/groovy.

Secondly, don’t add private modifiers to your attributes if you want the generated Groovy stubs to include getters/setters. This is probably more obvious if you’re creating your Groovy classes from scratch. I forgot to remove them when I was converting from a Java POJO and had a minor WTF moment.

Next step will be to see how much Groovy could potentially be leveraged for other aspects of the system.

The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption.

The Memory Analyzer was developed to analyze productive heap dumps with hundreds of millions of objects. Once the heap dump is parsed, you can re-open it instantly, immediately get the retained size of single objects and quickly approximate the retained size of a set of objects. The reference chain to the Garbage Collection Roots then details why the object is not garbage collected.

Using these features, a report automatically extracts leak suspects. It includes details about the objects accumulated, the path to the GC Roots, plus general information like system properties.

The tool was actually contributed by SAP and is currently an incubation project over at Eclipse. It’s available as both an eclipse feature and a standalone eclipse RCP application.

Kudos to the team for taking the time to provide a standalone package!