Down the rabbits hole with neo4j (Part 1)

While standing on one foot, I would say that the neo4j project is an implementation for an embedded java graph database. Unlike the traditional relation databases, the graph database has no table structure for storing its data. Instead it uses a more dynamic (unstructured) network composition. Using mathematical terminology, a network of nodes is called a graph.

I’ll put my other foot down now.

Applications use an object data structure in order to describe and abstract the business domain they address. Stitching together the object oriented structure onto the table oriented data structure of a relational database is a major part of the time consumed in server side programming. This price has been, and still is willingly paid because of the absolute reliability of the traditional relational data structures.

Truth to be said, there are implementations that make much more sense when persisted using tables. However, I would dare to argue that when querying becomes complex and cumbersome, it just might be a sign that the persisting method isn’t appropriate.

The unstructured graph database leads the field when it comes to implementing an ad-hoc structure that is prone to runtime changes.

I decided to give it a try after watching Emil Efirem’s demonstration on the web and I’m sharing my experience as I played around with three aspects of use:

• Part 1: Converting plain java data objects into persisted neo nodes
• Part 2: Managing lookups
• Part 3: Experimenting With Transactions

Setting up a neo4j environment

The neo4j jars are available on: http://neo4j.org/download, under which there are two packages;

• The apoc package includes some extra utility jars, i.e. the indexing jar, the shell jar and some examples
• The kernel package includes the bare minimum, i.e. neo and jta jars

My example project’s source code is also available here, compatible for eclipse and intelliJ.

Part 1: Converting POJO into neo

In my mind’s eye I saw a plain old java data objects that would define my domain data structure. These objects would be passed to some kind of neo facilitating mechanism that would take care of the persistence for me.

Using annotations

Java annotations are a wonderful tool; I use Annotations to load metadata on class elements, when the metadata is not intended to be part of the primary business course of the class.

In this example, I have a type representing a person that has a couple of properties and lists of friends and foes. I annotated the fields with the hinting I needed to ease the conversion into neo.

Note that the conversion is handled at runtime so the annotation retention policy is set appropriately.

Here is the annotation declaration:

@Retention(RetentionPolicy.RUNTIME)
public @interface Persistance {
  Type type() default Type.Property;
  Peers relationType() default Peers.NA;
  public static enum Type {
    Property, Peer
  }
}

The annotated POJO, representing my person data is as follows:

public enum Peers implements RelationshipType {
  Friend, Foe, NA
}
public class Person {
  @Persistance(type = Persistance.Type.Property)
  public String name;

  @Persistance(type = Persistance.Type.Property)
  public String nickname;

  @Persistance(type = Persistance.Type.Peer, relationType = Peers.Friend)
  public List<Person> friends;

  @Persistance(type = Persistance.Type.Peer, relationType = Peers.Foe)
  public List<Person> foes;

And so on…

The conversion to the neo nodes

The neo world speaks in two basic terms, the node and the relation. Each can bear its own properties.

The nodes have no typing, which I think is a shame, but relationships do require typing. I went along with the recommendations and defined my relationship as an enum.

The snippet for the relationship type:

public enum Peers implements RelationshipType {
  Friend, Foe, NA
}

The persistence mechanism is a recursive breakdown of the POJO graph (the person and all this friends etc.) according to the hints I get on the annotations.

Within a neo transaction I call the converting method by passing the object I want to persist onto it:

    Transaction tx = neo.beginTx();
    Node node = null;
    try {
      node = objectToNode(object, null);
      tx.success();
      return node.getId();
    } catch (Exception e) {
      tx.failure();
      e.printStackTrace();
    } finally {
      tx.finish();
    }

The method objectToNode(..) is the recursive part and I load it with a stack map to prevent the recursive calls from looping forever on objects that were already processed.

    if (stack == null) {
      stack = new HashMap<Object, Node>();
    } else if (stack.containsKey(object)) {
      return stack.get(object);
    }
    Node node = neo.createNode();
    stack.put(object, node);
    Class cls = object.getClass();
    Field[] fields = cls.getDeclaredFields();
    for (Field field : fields) {
      // only persisting the fields that have my special annotation
      if (field.isAnnotationPresent(Persistance.class)) {
        processFieldData(node, object, field, stack);
      }
    }
    return node;

The method processFieldData(..) is where the annotations are processed. Properties are loaded on the nodes and hints for relationships invoke the recursive call that handles the linked objects as independent nodes.

      if (annotation instanceof Persistance) {
        Persistance p = (Persistance) annotation;
        log("processing annotation: " + p.type() + " "
            + p.relationType());
        if (p.type().equals(Persistance.Type.Property)) {
          // TODO: verify that the property is actually a primitive
          Object value = field.get(object);
          log("setting property: " + field.getName() + " " + value);
          // here is a neo setting of a property
          node.setProperty(field.getName(), value);
        } else if (p.type().equals(Persistance.Type.Peer)) {
          Object value = field.get(object);
          log("setting peer: " + field.getName() + " " + value);
          if (value instanceof List) {
            for (Object item : (List) value) {
              // here is another neo bit, where a new Node is
              // created by recursively calling on the converting
              // method with the child object
              // after the new node is handed back, i create the
              // relationship between the two
              Node otherNode = objectToNode(item, stack);
              node.createRelationshipTo(otherNode, p
                  .relationType());
            }
   

Download the complete java source code