Simple Xml-to-Jpa data service with Smooks, Play2 for java and Ebeans JPA engine

Simple Xml-to-Jpa data service with Smooks, Play2 for java and Ebeans JPA engine

When experimenting how to develop simple end-to-end xml-to-jpa service

• SOAP UI can be used to emulate client application. 
• Smooks Xml-to-java example can be used to define xml payload, transform rules and java domain classes. 
• Play2 template application can be customized to embed xml-to-java example with modifications for JPA persistence.

It is also possible to run Smooks example first from command line. Smooks xml-to-java example is here http://www.smooks.org/mediawiki/index.php?title=V1.5:xml-to-java

Why to use xml-transformations i.e. Smooks?

Applications interface definitions (XSD) and database definitions (Schema) evolve with time, and thus they should be separated. Application need transformation logic to convert information between company's internal structures (domain model) and external structures (public interfaces).

Why to use Ebean JPA engine?

Application's database structure (Schema) correlates with domain model and can be expressed using JPA entities or database dependent ddl (Database definition language). DDL files are typically set of SQL sentences with create table and create index definitions. JPA model is simple java class structure with specialized annotations. Play2 for Java contains Ebean, which uses JPA annotations, but saves you from using JPA container and creating your own DDL files as Database DDL's are automatically generated by Ebean and database is created on the fly.

Why not to use Jaxb?

XSD is interface definition of XML services, and can be used in Java apps thru Jaxb classes or by directly interpreting XML tree using SAX or DOM api's. With Smooks JAXB is not necessary when reading XML into object model. Smooks is using Xpath based selectors and Visitor logic with SAX event, and has efficient low-memory footprint engine which helps to translate xml tags to actions. So, why to use Jaxb if it is not needed?

XML services concept

Any client can send using http post xml messages containing "order" elements to play2 controller which is connected to "/order" url. 

On server side "order" elements are transformed to domain classes. Transformation rules are stored on smooks-config xml file. Ebean and H2 are persisting information.

Setting up environment

You need to install client and server parts

• Soap UI as Client http://sourceforge.net/projects/soapui/files/
• Play2 for java as Server http://www.playframework.com/download

As Play2 is full stack environment it already contains Netty IO and Ebean JPA.

• Netty is HTTP based non blocking app server http://netty.io/
• Ebean is JPA compatible object relational mapping engine http://www.avaje.org/

JBoss Smooks Transformation engine in turn is delivered thru Apache IVY as dependency http://www.smooks.org

Creating and testing sample Play2 app

Create your app

• Go to root of your development directory
• give command "play new xml-to-jpa" 

Start play shell

• go to project directory xml-to-jpa 
• give command "play" 

Test application

• start app with "~run" 
• go to "localhost:9000" 

To make code-compile-debug cycle smooth "~" is important addition to run, since it enables PHP alike dynamic development on java platform. Fancy, and works well.

More details of template play2 application at http://www.playframework.com/documentation/2.1.x/Home

Setting up IDE

Generate IDE configuration

• start shell "play" 
• generate configuration "eclipse with-source=true"

Create eclipse project

• Open Eclipse 
• Let Eclipse to create new workspace 
• Export / General / Java Project from existing sources 

If you prefer some other IDE please see http://www.playframework.com/documentation/2.1.x/IDE

Enabling H2 and Ebean

Enable Database support

• Open conf/application.conf 
• uncomment lines under "database configuration" to enable H2 support 
• uncomment lines under "ebean configuration" to enable JPA engine 

Change location of JPA model

• change "ebean.default" to point "example.models.*" as on Smooks example 

Extending app with JPA model

See how Smooks POJO domain model looks

• Open Order class source on github 
• https://github.com/smooks/smooks/blob/v1.5.1/smooks-examples/xml-to-java/src/main/java/example/model/Order.java

Create needed JPA classes

• Use example.model as package for classes 
• Create classes Header, CustomerOrder and OrderItem 
• note: we must use CustomerOrder instead of Order as Order is reserved word in SQL 

Add JPA and Ebean definitions

• Add play.db.ebean.Model as superclass 
• Define all classes as entities with @Entity annotation 
• add id fields and @Id annotations 
• add other needed fields from https://github.com/smooks/smooks/tree/v1.5.1/smooks-examples/xml-to-java/src/main/java/example/model
• Add back reference to OrderItem 
• Annotate classes with @ManyToOne, @OneToOne and @OneToMany JPA annotations 

Define find helper variable to main class of model

• Add static Finder as find variable to CustomerOrder 
• Note: Usage of Finder isn't mandatory, but it makes complex query operations single liners. 

Here's body of CustomerOrder class with JPA annotations & needed fields.

@Entitypublic class CustomerOrder extends Model {
@Id
public Long id;

/**
* Cascades saves, which means dependent object will be saved as main object is saved.
*/
@OneToOne (cascade = CascadeType.ALL, fetch=FetchType.EAGER)
public Header header;
/** Saves on collections are cascaded.  
* Fetches are eager, which might result some extra access to db.
*/
@OneToMany(cascade = CascadeType.ALL, mappedBy = "customerOrder", fetch=FetchType.EAGER)
public List<OrderItem> orderItems = new ArrayList<OrderItem> ();

/**
* find static member variable to help searches
*/
public static Finder<Long, CustomerOrder> find = new Finder<Long, CustomerOrder>(
Long.class, CustomerOrder.class);
}

If you wonder what exactly Model class and Finder do, please peek here http://www.playframework.com/documentation/2.1.1/JavaEbean

It’s important to note that if you want to have real JPA it can be plugged in, but it’s there not as per default http://www.playframework.com/documentation/2.1.1/JavaJPA

Review your entites

Document domain model

• Create class diagram with ObjectAid  http://www.objectaid.com/class-diagram
• Drag classes to canvas of diagram 

Domain model with completed JPA classes looks like this

Create database instance

Bootstrap database creation

• make sure application is compiled and running at port 9000 
• go to localhost:9000 
• Wait that play2 asks “Database 'default' needs evolution!” 
• Check generated database ddl and accept it 

CustomerOrder database ddl should look like

create table customer_order (
  id bigint not null,
  header_id bigint,
constraint pk_customer_order primary key (id));

You can access H2 database out of process to see empty database http://www.h2database.com/html/quickstart.html

Extending app with transformations

Add Smooks dependencies to SBT build

• Open file project/Build.scala 
• Add libs to appDependencies 
• "org.milyn" % "milyn-smooks-core" % "1.5.1", 
• "org.milyn" % "milyn-smooks-javabean" % "1.5.1", 
• add repository to settings method of play project 
• resolvers += "JBoss repository" at "http://repository.jboss.org/nexus/content/groups/public-jboss/" 
• update dependencies by compiling and starting project “play clean eclipse ~run“ 
• refresh project in eclipse with File / Refresh 

Prepare tranformation rules

• get smooks-config.xml from git to conf directory https://github.com/smooks/smooks/blob/v1.5.1/smooks-examples/xml-to-java/smooks-config.xml 
• Change class name "example.model.Order" to "example.model.CustomerOrder" at definition of “order” bean 
• Add to “orderItem” bean property “customerOrder” which references to “order” bean <jb:wiring property="customerOrder" beanIdRef="order" /> 

Add transformation logic

• See example at https://github.com/smooks/smooks/blob/v1.5.1/smooks-examples/xml-to-java/src/main/java/example/Main.java
• Get static runSmooks () method and copy it to app.controllers.Application 
• Add input parameter of type org.w3c.dom.document and replace input stream with it using http://docs.oracle.com/javase/7/docs/api/javax/xml/transform/dom/DOMSource.html
• Change html report creation to point logs directory 
• Result of refactoring can be seen here https://github.com/nikkijuk/smooks-play2-xml-to-jpa/blob/master/smooks-play2-xml-to-jpa/app/controllers/Application.java

Note: HTML report drains memory. HTML report is vital tool for checking your mappings / selectors, but should be used only with limited amount of data. Use it only for development and never with xml streams of megabytes.

Note: Instance of Smooks is thread safe and should be static as it's initialization per request basis is expensive. Here is is kept inside method scope to minimize changes.

Extending app with xml service

Define entry point

• Define http post route from /order to implementation method saveXml() at Application class in file conf/routes as "POST /order controllers.Application.saveXml()" 

Define controller implementation

• Create body for method Application.saveXML () 
• Add XML body parser 
• Get xml, transform it, save resulting object 
• As all of this is bit much with exception handling just peek code https://github.com/nikkijuk/smooks-play2-xml-to-jpa/blob/master/smooks-play2-xml-to-jpa/app/controllers/Application.java

For more background on flow of execution see discussion from previous post
http://nikkijuk.blogspot.de/2013/08/simple-java-xml-web-services.html

Web user interface concept

Play2 enables you to implement variation of MVC (model-view-controller) pattern.

As with xml service there's 

• Route to define endpoint for clients
• Controller for logic
• Domain classes for model  

But in addition to xml service we also have view template , which is used to render output to client.

Provide web user interface

Define view template

• Add simple template which renders order list 
• Define incoming parameter orders as list of CustomerOrder classes 
• Add iteration over list of orders 

Below is simple template -- if it doesn’t look all too familiar it’s because it’s written in Scala, more here http://www.playframework.com/documentation/2.1.1/JavaTemplates

@(orders: List[example.model.CustomerOrder])
@main("List of orders") {
<h1>List of orders</h1>
<ul>
@for(order <- orders){
      <li>@order.id @order.header.date @order.header.customerNumber @order.header.customerName </li>
}
</ul>
}

Define controller method

• Add method connecting route to template 
• Add logic to find all orders 
• Add logic to render template 

To make it readable I have kept method body on 2 lines.

public static Result list() {
  List<CustomerOrder> orders = CustomerOrder.find.findList();
  return ok(list.render(orders));
}

Define entry point

• Add get request to route "GET /orders controllers.Application.list()" 

See list of orders

• go to http://localhost:9000/orders
• You should see page with empty list

Prepare initial test data

Define test data

• Create yaml file to conf directory with name default-orders.yml 
• Fill file with one order having header and 2 line items 
• See example at https://github.com/nikkijuk/smooks-play2-xml-to-jpa/blob/master/smooks-play2-xml-to-jpa/conf/default-orders.yml

Load test data

• Add static constructor to app.controllers.Application class 
• Check that database is empty 
• Use Yaml.load method of SnakeYaml to load default-orders.yml 
• Result is at https://github.com/nikkijuk/smooks-play2-xml-to-jpa/blob/master/smooks-play2-xml-to-jpa/app/controllers/Application.java

static {
  try {
    if (Ebean.find(CustomerOrder.class).findRowCount() == 0) {
    // load yaml data
    Map<String, List<Object>> all = (Map<String, List<Object>>) Yaml
    .load("default-orders.yml");
    // Insert employees
    Ebean.save(all.get("orders"));
    Logger.info("Defaults added");
    }
  } catch (Exception e) {
    Logger.error("Defaults couldn't be added "+e.getMessage(), e);
  }
}

See list of orders

• go to http://localhost:9000/orders
• You should now see initial test data 

If you plan to extend test data changes are you need to check your Yaml is sane. let Yamllint to help http://yamllint.com/.

If you feel more academic interest then Yaml specification might help http://www.yaml.org/spec/1.2/spec.html

Testing with Soap UI & Browser

Setup SOAP ui

• Define post method of xml content to be sent to localhost:9000/order 

Get test message

• Get message from https://github.com/smooks/smooks/blob/v1.5.1/smooks-examples/xml-to-java/input-message.xml

Post it to service

• Send message to localhost:9000/order 

See list of orders

• go to http://localhost:9000/orders
• You should now see initial test data and your own added order

Simple java XML web services using Jaxb & Play2

Simple java XML services using Jaxb & Play2

Developing simple java XML services needs

• Data definition (XSD)
• Service interface (POST URL)
• Service body (Method implementing Post)
• Unmarshalling logic (XML -> object)

Data definition is used as an agreement between client and server. Client developer uses agreed xsd, defines xjb bindings for jaxb generation, and generate java classes which can be used to build request. Request is then sent to public interface of server on payload of http post request.

Posting single xml document

Server developer uses shared xsd, but can have different bindings and thus also different jaxb data model generated. Incoming post request is routed to controller, where xml is unmarshalled to jaxb model. Model can be from this on used as desired.

Data definition: Manual XSD-design 

My personal preference is to write XSD first and then generate java classes for client or server. In this way information structures are clean and client development is easy.

With JDK's basic tools one can easily get from XSD to java classes. Generation can be customized with configuration file. Your preferred IDE has most probably integrated generation to make this really simple.

http://docs.oracle.com/javase/6/docs/technotes/tools/share/xjc.html

Data definition: Russian Doll 

I selected Russian Doll XSD design. This is most simple way of designing XSD, but doesn't allow reusing of types as all types are embedded and local.

With Venetian Blind I needed to define XML root element using annotations on java code, which was to me undesired. If generation is one time operation this might not be problem, but to me it was.

http://www.xfront.com/GlobalVersusLocal.html

If you aim to develop self contained services whose interfaces are decoupled from database design and domain objects Russian Doll might be design choice for you.

I wanted to keep services separate and simple, but if you are doing complex SOA integrations you might prefer Salami Slices, Garden of Eden or Chameleon design patterns, i.e. reusing type definitions and overriding namespaces.

http://www.oracle.com/technetwork/java/design-patterns-142138.html

Java classes: JAXB generation

Jaxb generation is simple. In Eclipse just select File / New / Jaxb / Classes from schema and give schema file, package name and bindings file.

Customizing generation is part of basic process which is well supported and has bells and whistles more than simple use case needs.

http://docs.oracle.com/cd/E17802_01/webservices/webservices/docs/1.5/tutorial/doc/JAXBUsing4.html

Getting temporal datatypes "right" can be can be done by adjusting generation with external binding file. If you use Jodatime types conversion goes like below.

<!-- Transforms xml dates and dateTimes to Jodatime objects -->

<jaxb:bindings version="2.0" xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"

xmlns:xs="http://www.w3.org/2001/XMLSchema"

jaxb:extensionBindingPrefixes="xjc">

<jaxb:globalBindings mapSimpleTypeDef="false"

choiceContentProperty="true">

<jaxb:javaType xmlns="http://java.sun.com/xml/ns/jaxb"

name="org.joda.time.LocalDate" xmlType="xs:date"

parseMethod="org.joda.time.LocalDate.parse" printMethod="java.lang.String.valueOf" />

<jaxb:javaType xmlns="http://java.sun.com/xml/ns/jaxb"

name="org.joda.time.LocalTime" xmlType="xs:time"

parseMethod="org.joda.time.LocalTime.parse" printMethod="java.lang.String.valueOf" />

<jaxb:javaType xmlns="http://java.sun.com/xml/ns/jaxb"

name="org.joda.time.DateTime" xmlType="xs:dateTime" parseMethod="org.joda.time.DateTime.parse"

printMethod="java.lang.String.valueOf" />

</jaxb:globalBindings>

</jaxb:bindings>

Post interface: url

Here's play2 definition for route from external interface (url) to implementation (java method).

POST    /metrics/entity       controllers.MetricsEntityController.saveXml()

Play2 is full stack framework, and this is really all you need. No web.xml, no servlets, no..

Post method: signature and implementation

Here's play2 for java method, which is extracting org.w3c.dom.Document from http request.

@Of(Xml.class)

public static Result saveXml() {

Document xml = request().body().asXml();

...

Mime type on request needs to be 'text/xml' for asXml method to work. If you can't ensure right mime type is used TolerantXml allows relaxing checks here.

XML to Java implemenentation: Jaxb unmarshal

To unmarshall xml with jaxb you need to set up context and marshaller. Context is thread safe, so it can be initialized on static constructor.

// inits

static JAXBContext context;

static {

try {

// init jaxb

context = JAXBContext.newInstance(Result.class);

Logger.info("Jaxb initialized");

} catch (JAXBException jaxbe) {

Logger.error("Jaxb initialization failed", jaxbe);

throw new RuntimeException(jaxbe);

}

}

Unmarshalling is done on per request basis.

// results deserialization

Unmarshaller unmarshaller = context.createUnmarshaller();

computationResult = (Result) unmarshaller.unmarshal(xml);

At this point you have Jaxb object tree, which you can use to browse content of xml. What you do with it is up to you.

Conclusions

Java standard libraries with open source libraries and tools like jodatime allow straightforward java xml service development. Play2 is clean, simple and easy to use web development stack. Together Jaxb, Jodatime and Play2 provide all one needs to get from xml to java objects in plain http services.

http://www.playframework.com