Thursday, August 22, 2013

Simple Xml-to-Jpa data service with Smooks, Play2 for java and Ebeans JPA engine

Simple Xml-to-Jpa data service with Smooks, Play2 for java and Ebeans JPA engine

When experimenting how to develop simple end-to-end xml-to-jpa service
  • SOAP UI can be used to emulate client application. 
  • Smooks Xml-to-java example can be used to define xml payload, transform rules and java domain classes. 
  • Play2 template application can be customized to embed xml-to-java example with modifications for JPA persistence.
It is also possible to run Smooks example first from command line. Smooks xml-to-java example is here

Why to use xml-transformations i.e. Smooks?

Applications interface definitions (XSD) and database definitions (Schema) evolve with time, and thus they should be separated. Application need transformation logic to convert information between company's internal structures (domain model) and external structures (public interfaces).

Why to use Ebean JPA engine?

Application's database structure (Schema) correlates with domain model and can be expressed using JPA entities or database dependent ddl (Database definition language). DDL files are typically set of SQL sentences with create table and create index definitions. JPA model is simple java class structure with specialized annotations. Play2 for Java contains Ebean, which uses JPA annotations, but saves you from using JPA container and creating your own DDL files as Database DDL's are automatically generated by Ebean and database is created on the fly.

Why not to use Jaxb?

XSD is interface definition of XML services, and can be used in Java apps thru Jaxb classes or by directly interpreting XML tree using SAX or DOM api's. With Smooks JAXB is not necessary when reading XML into object model. Smooks is using Xpath based selectors and Visitor logic with SAX event, and has efficient low-memory footprint engine which helps to translate xml tags to actions. So, why to use Jaxb if it is not needed?

XML services concept

Any client can send using http post xml messages containing "order" elements to play2 controller which is connected to "/order" url. 

On server side "order" elements are transformed to domain classes. Transformation rules are stored on smooks-config xml file. Ebean and H2 are persisting information.

Setting up environment

You need to install client and server parts
As Play2 is full stack environment it already contains Netty IO and Ebean JPA.
JBoss Smooks Transformation engine in turn is delivered thru Apache IVY as dependency

Creating and testing sample Play2 app

Create your app
  • Go to root of your development directory
  • give command "play new xml-to-jpa" 

Start play shell
  • go to project directory xml-to-jpa 
  • give command "play" 
Test application
  • start app with "~run" 
  • go to "localhost:9000" 
To make code-compile-debug cycle smooth "~" is important addition to run, since it enables PHP alike dynamic development on java platform. Fancy, and works well.

More details of template play2 application at

Setting up IDE

Generate IDE configuration
  • start shell "play" 
  • generate configuration "eclipse with-source=true"
Create eclipse project
  • Open Eclipse 
  • Let Eclipse to create new workspace 
  • Export / General / Java Project from existing sources 
If you prefer some other IDE please see

Enabling H2 and Ebean

Enable Database support
  • Open conf/application.conf 
  • uncomment lines under "database configuration" to enable H2 support 
  • uncomment lines under "ebean configuration" to enable JPA engine 
Change location of JPA model
  • change "ebean.default" to point "example.models.*" as on Smooks example 

Extending app with JPA model

See how Smooks POJO domain model looks
Create needed JPA classes
  • Use example.model as package for classes 
  • Create classes Header, CustomerOrder and OrderItem 
  • note: we must use CustomerOrder instead of Order as Order is reserved word in SQL 
Add JPA and Ebean definitions
Define find helper variable to main class of model
  • Add static Finder as find variable to CustomerOrder 
  • Note: Usage of Finder isn't mandatory, but it makes complex query operations single liners. 
Here's body of CustomerOrder class with JPA annotations & needed fields.
@Entitypublic class CustomerOrder extends Model {
public Long id;

* Cascades saves, which means dependent object will be saved as main object is saved.
@OneToOne (cascade = CascadeType.ALL, fetch=FetchType.EAGER)
public Header header;
/** Saves on collections are cascaded.  
* Fetches are eager, which might result some extra access to db.
@OneToMany(cascade = CascadeType.ALL, mappedBy = "customerOrder", fetch=FetchType.EAGER)
public List<OrderItem> orderItems = new ArrayList<OrderItem> ();

* find static member variable to help searches
public static Finder<Long, CustomerOrder> find = new Finder<Long, CustomerOrder>(
Long.class, CustomerOrder.class);
If you wonder what exactly Model class and Finder do, please peek here

It’s important to note that if you want to have real JPA it can be plugged in, but it’s there not as per default

Review your entites

Document domain model
Domain model with completed JPA classes looks like this

Create database instance

Bootstrap database creation
  • make sure application is compiled and running at port 9000 
  • go to localhost:9000 
  • Wait that play2 asks “Database 'default' needs evolution!” 
  • Check generated database ddl and accept it 

CustomerOrder database ddl should look like
create table customer_order (
  id bigint not null,
  header_id bigint,
constraint pk_customer_order primary key (id));
You can access H2 database out of process to see empty database

Extending app with transformations

Add Smooks dependencies to SBT build
  • Open file project/Build.scala 
  • Add libs to appDependencies 
    • "org.milyn" % "milyn-smooks-core" % "1.5.1", 
    • "org.milyn" % "milyn-smooks-javabean" % "1.5.1", 
  • add repository to settings method of play project 
  • update dependencies by compiling and starting project “play clean eclipse ~run“ 
  • refresh project in eclipse with File / Refresh 

Prepare tranformation rules
Add transformation logic

Note: HTML report drains memory. HTML report is vital tool for checking your mappings / selectors, but should be used only with limited amount of data. Use it only for development and never with xml streams of megabytes.

Note: Instance of Smooks is thread safe and should be static as it's initialization per request basis is expensive. Here is is kept inside method scope to minimize changes.

Extending app with xml service

Define entry point
  • Define http post route from /order to implementation method saveXml() at Application class in file conf/routes as "POST /order controllers.Application.saveXml()" 
Define controller implementation
For more background on flow of execution see discussion from previous post

Web user interface concept

Play2 enables you to implement variation of MVC (model-view-controller) pattern.

As with xml service there's 
  • Route to define endpoint for clients
  • Controller for logic
  • Domain classes for model  
But in addition to xml service we also have view template , which is used to render output to client.

Provide web user interface

Define view template
  • Add simple template which renders order list 
  • Define incoming parameter orders as list of CustomerOrder classes 
  • Add iteration over list of orders 
Below is simple template -- if it doesn’t look all too familiar it’s because it’s written in Scala, more here
@(orders: List[example.model.CustomerOrder])
@main("List of orders") {
<h1>List of orders</h1>
@for(order <- orders){
      <li> @order.header.customerNumber @order.header.customerName </li>

Define controller method
  • Add method connecting route to template 
  • Add logic to find all orders 
  • Add logic to render template 
To make it readable I have kept method body on 2 lines.
public static Result list() {
  List<CustomerOrder> orders = CustomerOrder.find.findList();
  return ok(list.render(orders));
Define entry point
  • Add get request to route "GET /orders controllers.Application.list()" 

See list of orders

Prepare initial test data

Define test data

Load test data
static {
  try {
    if (Ebean.find(CustomerOrder.class).findRowCount() == 0) {
    // load yaml data
    Map<String, List<Object>> all = (Map<String, List<Object>>) Yaml
    // Insert employees"orders"));"Defaults added");
  } catch (Exception e) {
    Logger.error("Defaults couldn't be added "+e.getMessage(), e);

See list of orders
If you plan to extend test data changes are you need to check your Yaml is sane. let Yamllint to help

If you feel more academic interest then Yaml specification might help

Testing with Soap UI & Browser

Setup SOAP ui
  • Define post method of xml content to be sent to localhost:9000/order 
Get test message
Post it to service
  • Send message to localhost:9000/order 
See list of orders


Pseudozen said...

I'm getting this error:

play.api.Application$$anon$1: Execution exception[[RuntimeException: org.xml.sax
.SAXException: Failed to locate XSD resource '/META-INF/xsd/smooks/edi-1.1.xsd'
on classpath. Namespace: ''.]]

Any idea what's causing that?

Jukka Nikki said...

This message doesn't seem to have anything to do with my example app .. or do it? It looks to me like you wouldn't have right things included in classpath and that it would make sense to google a bit with error message or write well formulated question to more broad forum like