ReST Notes

This is a little lame. It's really a place to put notes until I get better organized. Much isn't about ReST at all, but more about Jersey. Also, peruse JSON Notes for additional and very relevant comments directly related to ReST.


Useful links not to forget...


ReST API design guidelines
  1. Resources (URIs). Use concrete names of resources rather than verbs, especially action verbs, like "createUser." Instead, something like GET user/1234, POST user (with body describing the new user), DELETE user/1234, etc.
  2. HTTP methods. GET, to retrieve information; HEAD, as GET, but only dealing with header section and status line; POST, create a resource; PUT, update a resource; DELETE, remove a resource; OPIONS, describe the communications options for a resource.
  3. Query parameters. Use for paging, filtering, sorting and searching.
  4. Status codes. Make proper use of HTTP status codes. Don't return 200 OK, for instance, after a POST call that created something (use 201 Created instead).

Design, part 2

Design for clients not for the data.

  1. root URL can be:
    • https://example.com/api/v1/...
    • https://api.example.com/v1/...
  2. query parameters are for retrieval of non-hierarchical data, including discovery, i.e.: URI + ?firstname={}&lastname={}, etc.

ReST verb primer

ReST HTTP status codes
200 OK. The request has successfully executed. Response depends upon the verb invoked.
201 Created. The request has successfully executed and a new resource has been created in the process. The response body is either empty or contains a representation containing URIs for the resource created. The Location header in the response should point to the URI as well.
202 Accepted. The request was valid and has been accepted but has not yet been processed. The response should include a URI to poll for status updates on the request. This allows asynchronous REST requests
204 No Content. The request was successfully processed but the server did not have any response. The client should not update its display
301 Moved Permanently. The requested resource is no longer located at the specified URL. The new Location should be returned in the response header. Only GET or HEAD requests should redirect to the new location. The client should update its bookmark if possible.
302 Found. The requested resource has temporarily been found somewhere else. The temporary Location should be returned in the response header. Only GET or HEAD requests should redirect to the new location. The client need not update its bookmark as the resource may return to this URL.
302 See Other. This response code is a way of responding to a valid request for a non-network addressable resource. There is a distinction between resources that can be found on the web and those that cannot, i.e.: 303 instead of 200. The redirected location will be reflected in the Location header of the response. This header will contain a reference to a document about the resource or perhaps some metadata about it.
304 Not Modified.
307 Temporary Redirect.
308 Permanent Redirect.
400 Bad Request. Client error or perceived as a client error that needs corrected.
401 Unauthorized. Lacking the necessary authorization permissions.
403 Forbidden. The server understands but forbids sending a response.
404 Not Found. The most famous error code Nothing to see here, try something else.
405 Method Not Allowed. The method used is not supported by the origin.
406 Not Acceptable.
410 Gone. Used instead of a 404 to state that a resource is intentionally missing.
411 Length Required.
412 Precondition Failed.
413 Entity Too Large. The payload is beyond what the server will send.
414 URI Too Long.
415 Unsupported Media Type.
417 Expectation Failed.
500 Internal Server Error.
501 Not Implemented.
502 Bad Gateway. The server, while acting as a gateway or proxy, received an invalid response from an inbound server it accessed while attempting to fulfill the request.
503 Service Unavailable.
504 Gateway Timeout.
505 HTTP Version Not Supported. Likely not secure enough.
508 Loop detected. Server quit the request to prevent an infinite loop.

Path parameters and objects

It's possible to do both path parameters AND an object thus:

@PUT
@Path( "/{id}" )
@Consumes( { "application/json", "application/xml" } )
@Produces( { "application/json", "application/xml" } )
public Response update( @PathParam( "id" ) Integer id, Account account )
{
    System.out.println( "id: " + id + "\naccount:" + account.toString() );
}

Understanding and converting entities in Jersey

What I mean is, if an argument to an entry point in your ReST service takes something from the HTTP command line, and it's complex like an entity, Jersey doesn't have a native understanding of it and you have to annotate the class that defines it. For example, ...

...
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Account implements Serializable
{
   private ObjectId oid;
   private String   name;
   private String   email;
   private String   password;
   ...

Not only the annotation, but the fact that it's serializable is important. And, Jersey can only serialize what it understands. (Look elsewhere in these notes for how to serialize HashMaps which Jersey doesn't understand.)

This is so that create() in the following service code will even get called by Jersey.

...

@Path( "/v1/account" )
public class AccountService
{
    ...

    private AccountManager accountManager;

    @POST
    @Consumes( { "application/json", "application/xml" } )
    @Produces( { "application/json", "application/xml" } )
    public Response create( Account account )
    {
        try
        {
            account = accountManager.create( account );
        ...

If you set a breakpoint on line 17 for instance and never get there when everything else (URL, method = POST, header contains "Content-Type: application/json;charset=utf-8", data is a well-formed JSON or XML, etc.), you may get HTTP Status Code 415 Media Type Unsupported. This is almost certainly the result of what you passed (to create() here) not being marked with @XmlRootElement.

Result

It's also so that Jersey will return a resulting object down the wire to your caller.

...

@Path( "/v1/account" )
public class AccountService
{
    ...

    private AccountManager accountManager;

    ...
    public Response create( Account account )
    ...

    @POST
    @Path( "/find" )
    @Produces( { "application/json", "application/xml" } )
    public Response read( Account account )
    {
        Account result;

        try
        {
            result = accountManager.readByEmailOrOid( account );
        }
        catch( AppException e )
        {
            return e.buildResponse();
        }
        catch( RuntimeException e )
        {
            return AppException.buildRuntimeResponse( e );
        }

        return Response.ok( result ).build();
    }
    ...

Don't forget handling of embedded objects

If you've got something embedded in your account object, such as an array of addresses, Jersey won't emit them when you pass back that object unless it knows enough. How that happens is via the @XmlElement annotation.

...
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlElement;

@XmlRootElement
public class Account implements Serializable
{
   private ObjectId oid;
   private String   name;
   private String   email;
   private String   password;

   @XmlElement
   private ArrayList< Address > addresses = new ArrayList< Address >();
   ...

Duplicate property names

Using @XmlRootElement and @XmlElement may not be enough in terms of annotations. There are problems that can arise that these don't handle.

For example, if you're embedding an object or array of objects in your entity that are themselves entities, you may run into the error below. Note that I didn't need this solution until I created the situation described, which is succinctly illustrated here:

@XmlRootElement
public class Account implements Serializable
{
   private ObjectId oid;
   private String   name;
   private String   email;
   private String   password;

   @XmlElement
   private ArrayList< Address > addresses = new ArrayList< Address >();
   ...
}

@XmlRootElement
public class Address implements Serializable
{
   private ObjectId oid;
   private String   street;
   private String   city;
   private String   state;
   ...
}
   Jul 4, 2012 7:53:14 AM com.sun.jersey.spi.container.ContainerResponse logException
   SEVERE: Mapped exception to response: 500 (Internal Server Error)
   javax.ws.rs.WebApplicationException: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: \
       2 counts of IllegalAnnotationExceptions
   Class has two properties of the same name "addresses"
	   this problem is related to the following location:
		   at public java.util.List com.hp.web.user.entity.Account.getAddresses()
		   at com.hp.web.user.entity.Account
	   this problem is related to the following location:
		   at java.util.List com.hp.web.user.entity.Account.addresses
		   at com.hp.web.user.entity.Account

It's apparently (and precisely) because I'm trying to use Address as an entity in its own right and also as an embedded field on Account.

Googling for a solution, you find the inelegant suggestion that you annotate the accessor (getter and/or setter) instead of the field. This is fine as far as it goes especially if it solves the problem. However, I find these Jersey annotations more self-documenting and easier to maintain if they are used with the member field near the top of the class code instead.

So, the solution I found while Googling is an annotation that says to Jersey, "look for accessor information on the field".

   @XmlAccessorType( XmlAccessType.FIELD )

I inserted this annotation on a line immediately preceding @XmlRootElement in the code in Account (not in Address).


XmlRootElement naming

If you're creating a data-transfer object (DTO) or some other such crutch for a "real" entity and trying to get arrays of them back through Jersey to the caller, you'll find that the name of what goes back will not be to your liking.

import java.io.Serializable;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlAccessorType;

@XmlAccessorType( XmlAccessType.FIELD )
@XmlRootElement()
public class AccountDto implements Serializable
{
    private ObjectId oid;
    ...

...will yield:

{
    "accountdto":
    {
        "oid":"13423abc218493...
    },
    {
        "oid":"723450bcdcba88...

Instead, you need to name it:

...
@XmlRootElement( name="account" )
public class AccountDto implements Serializable
{
    private String oid;
    ...

Null oid reported in JSON payload

Also, likely you'll not get the oid back as anything except null. Fix that by ensuring that...


Sample Jersey JAX-RS ReST suite with Spring and Hibernate

Here's the template of a ReST suite from the ReST service implementation all the way down to the POJO/bean. It includes also ample annotations from Spring and Hibernate, as well configuration files for same. And, it includes the SQL load script for the database table in question.

Note that I advise against using Spring at all. It's too heavy and wasteful for a lean and mean restlet.

This stuff is from work I've been doing and private details have been excised. Some mistakes may have been made and breakages created. It's only for giving an idea. Anyway, these are my private notes.

Files

  1. AccountService.java —ReST entry points
  2. AccountManager.java —business logic
  3. BaseDao.java —Hibernate-Spring glue
  4. AccountDao.java —data-access object
  5. Account.java —bean
  6. account-table.sql —SQL script
  7. spring-configuration.xml —Spring configuration
  8. database.properties —Hibernate/Spring configuation
  9. web.xml —web descriptor
AccountService.java:

These methods are the ReST entry points for a ReST web service. They call down to the business logic manager. In the code below, accountManager is @Autowired. In order for this to succeed, it must not be static (circa line 40 below). Autowired annotation is not supported on static fields. No compilation error will inform you of this; you'll see something like (logging statement from the Eclipse console):

14:33:36,072  WARN AutowiredAnnotationBeanPostProcessor:337 - Autowired annotation is not supported \
     on static fields: private static com.etretatlogiciels.web.user.manager.AccountManager \
     com.etretatlogiciels.web.user.service.AccountService.accountManager

...and HTTP will probably return 500 Internal Server Error.

package com.etretatlogiciels.web.user.service;

import java.net.URI;
import java.util.ArrayList;
import java.util.List;

import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

import com.etretatlogiciels.web.user.entity.Account;
import com.etretatlogiciels.web.user.manager.AccountManager;

/**
 * Handles CRUD requests surrounding the abstract Account with special handling for
 * reading (listing) all user accounts.
 */
@Path( "/v1/account" )
@Component( "accountService" )
@Scope( "request" )
public class AccountService
{
    private static Logger log = Logger.getLogger( AccountService.class );

    @Autowired
    private AccountManager accountManager;

    @POST
    @Consumes( { "application/json", "application/xml" } )
    @Produces( { "application/json", "application/xml" } )
    public Response create( Account account )
    {
        try
        {
            account = accountManager.create( account );
        }
        catch( Exception e )
        {
            log.error( "Error creating account " + account.getUsername() + " (" + account.getFullname() + ") as " + account.getId(), e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        URI createdUri = URI.create( "/usermgr/v1/account/" + account.getId() );
        return Response.created( createdUri ).entity( account ).build();
    }

    @GET
    @Produces( { "application/json", "application/xml" } )
    @Transactional( readOnly = true, propagation = Propagation.REQUIRED )
    public List< Account > read()
    {
        List< Account > accounts = new ArrayList< Account >();

        try
        {
            accounts = accountManager.findAll();
        }
        catch( Exception e )
        {
            log.error( "Error reading all accounts", e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        if( accounts.isEmpty() )
            throw new WebApplicationException( Response.Status.NOT_FOUND );

        return accounts;
    }

    @GET
    @Path( "/{id}" )
    @Produces( { "application/json", "application/xml" } )
    public Response read( @PathParam( "id" ) Integer id )
    {
        Account account;

        try
        {
            account = accountManager.findById( id );
        }
        catch( Exception e )
        {
            log.error( "Error reading account " + id, e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        if( account == null )
            throw new WebApplicationException( Response.Status.NOT_FOUND );

        return Response.ok( account ).build();
    }

    @GET
    @Path( "/{property}/{propertyvalue}" )
    @Produces( { "application/json", "application/xml" } )
    public List< Account > readByPropertyAndValue( @PathParam( "property" ) String property,
                                         @PathParam( "propertyvalue" ) String propertyvalue )
    {
        List< Account > accounts = new ArrayList< Account >();

        try
        {
            accounts = accountManager.findByPropertyAndValue( property, propertyvalue );
        }
        catch( Exception e )
        {
            log.error( "Error reading account " + propertyvalue, e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        if( accounts == null )
            throw new WebApplicationException( Response.Status.NOT_FOUND );

        return accounts;
    }

    @PUT
    @Path( "/{id}" )
    @Consumes( { "application/json", "application/xml" } )
    @Produces( { "application/json", "application/xml" } )
    public Response update( Account account )
    {
        try
        {
            accountManager.update( account );
        }
        catch( Exception e )
        {
            log.error( "Error updating account " + account.getId(), e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        return Response.noContent().build();
    }

    @DELETE
    @Path( "/{id}" )
    @Produces( { "application/json", "application/xml" } )
    public Response delete( @PathParam( "id" ) Integer id )
    {
        try
        {
            accountManager.delete( id );
        }
        catch( Exception e )
        {
            log.error( "Error deleting account " + id, e );
            throw new WebApplicationException( Response.Status.INTERNAL_SERVER_ERROR );
        }

        return Response.ok().build();
    }
}
AccountManager.java:

This class defines business logic, of which there is precious little here.

package com.etretatlogiciels.web.user.manager;

import java.util.List;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import com.etretatlogiciels.web.user.entity.Account;
import com.etretatlogiciels.web.user.entity.AccountDao;

@Transactional
@Service( "accountManager" )
public class AccountManager
{
    private static Logger log = Logger.getLogger( AccountManager.class );

    @Autowired( required = true )
    private AccountDao accountDao;

    public Account create( Account account )
    {
        log.info( "create new account " + account.getUsername() + " (" + account.getFullname() + ") as " + account.getId() );
        return accountDao.create( account );
    }

    public List< Account > findAll()
    {
        log.info( "list all supported accounts" );
        return accountDao.read();
    }

    public Account findById( Integer id )
    {
        log.info( "find account whose id is " + id );
        return accountDao.readById( id );
    }

    public List< Account > findByPropertyAndValue( String propertyName, String propertyValue )
    {
        log.info( "find account whose " + propertyName + " is " + propertyValue );
        return accountDao.readByProperty( propertyName, propertyValue );
    }

    public void update( Account account )
    {
        log.info( "update existing account" + account.getName() );
        accountDao.update( account );
    }

    public void delete( Integer id )
    {
        log.info( "delete existing account by id" + id );
        Account account = findById( id );
        accountDao.delete( account );
    }
}
BaseDao.java:

This is some of the more magical glue that make Spring and Hibernate work and play well together.

package com.etretatlogiciels.web.user.entity;

import org.hibernate.SessionFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.orm.hibernate3.HibernateTemplate;

public class BaseDao
{
    @Autowired
    public SessionFactory sessionFactory;

    @Autowired
    public HibernateTemplate hibernateTemplate;

    public BaseDao() { super(); }
}
AccountDao.java:

This class is the data-access object that sits atop the POJO (bean) mirroring the database table.

package com.etretatlogiciels.web.user.entity;

import java.util.List;

import org.apache.log4j.Logger;
import org.hibernate.Criteria;
import org.hibernate.criterion.MatchMode;
import org.hibernate.criterion.Restrictions;
import org.springframework.stereotype.Repository;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

@Transactional
@Repository( "accountDao" )
public class AccountDao extends BaseDao
{
    @SuppressWarnings( "unused" )
    private static Logger log = Logger.getLogger( AccountDao.class );

    @Transactional( readOnly = false, propagation = Propagation.MANDATORY )
    public Account create( Account account )
    {
        sessionFactory.getCurrentSession().save( account );
        return account;
    }

    @SuppressWarnings( "unchecked" )
    @Transactional( readOnly = true, propagation = Propagation.MANDATORY )
    public List< Account > read()
    {
        Criteria criteria = sessionFactory.getCurrentSession().createCriteria( Account.class );
        criteria.isReadOnly();
        return criteria.list();
    }

    @Transactional( readOnly = true, propagation = Propagation.MANDATORY )
    public Account readById( Integer id )
    {
        Criteria criteria = sessionFactory.getCurrentSession().createCriteria( Account.class );
        criteria.isReadOnly();
        criteria.add( Restrictions.idEq( id ) );
        return ( Account ) criteria.uniqueResult();
    }

    @SuppressWarnings( "unchecked" )
    @Transactional( readOnly = true, propagation = Propagation.MANDATORY )
    public List< Account > readByProperty( String propertyName, String propertyValue )
    {
        Criteria criteria = sessionFactory.getCurrentSession().createCriteria( Account.class );
        criteria.add( Restrictions.ilike( propertyName, propertyValue, MatchMode.EXACT ) );
        return criteria.list();
    }

    @Transactional( readOnly = false, propagation = Propagation.MANDATORY )
    public void update( Account account )
    {
        Account accountToModify = readById( account.getId() );
        accountToModify.setCode( account.getCode() );
        accountToModify.setName( account.getName() );
        sessionFactory.getCurrentSession().update( accountToModify );
    }

    @Transactional( readOnly = false, propagation = Propagation.MANDATORY )
    public void delete( Account account )
    {
        Account accountToModify = readById( account.getId() );
        sessionFactory.getCurrentSession().delete( accountToModify );
    }
}
Account.java:

This is the bean that describes what's in the database table. This simulates a reasonable user account record. Addresses, credit card information, etc. would be in yet other tables.

package com.etretatlogiciels.web.user.entity;

import java.io.Serializable;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.SequenceGenerator;
import javax.persistence.Table;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
@Entity
@Table( name = "account" )
public class Account implements Serializable
{
    private static final long serialVersionUID = -7095851069703408708L;

    private Integer id;
    private String  username;
    private String  firstname;
    private String  lastname;
    private String  fullname;
    private String  email;
    private String  telephone;
    private String  language;
    private Date    birthdate;
    private boolean loggedin;
    private Date    lastlogin;
    private Date    created;

    @Id
    @GeneratedValue( strategy=GenerationType.AUTO,  generator="account_seq_gen")
    @SequenceGenerator( name="account_seq_gen", sequenceName="account_sequence" )
    @Column( name = "id", unique = true, updatable = false )
    public Integer getId() { return id; }
    public void setId( Integer id ) { this.id = id; }

    @Column( username="username", nullable=false, length=128 )
    public String getUsername() { return username; }
    public void setUsername( String username ) { this.username = username; }

    @Column( firstname="firstname", nullable=true, length=64 )
    public String getFirstname() { return firstname; }
    public void setFirstname( String firstname ) { this.firstname = firstname; }

    @Column( lastname="lastname", nullable=true, length=64 )
    public String getLastname() { return lastname; }
    public void setLastname( String lastname ) { this.lastname = lastname; }

    @Column( fullname="fullname", nullable=true, length=256 )
    public String getFullname() { return fullname; }
    public void setFullname( String fullname ) { this.fullname = fullname; }

    @Column( email="email", nullable=false, length=128 )
    public String getEmail() { return email; }
    public void setEmail( String email ) { this.email = email; }

    @Column( telephone="telephone", nullable=true, length=32 )
    public String getTelephone() { return telephone; }
    public void setTelephone( String telephone ) { this.telephone = telephone; }

    @Column( language="language", nullable=true, length=2 )
    public String getLanguage() { return language; }
    public void setLanguage( String language ) { this.language = language; }

    @Column( name="birthdate", nullable=true )
    public Date getBirthdate() { return birthdate; }
    public void setBirthdate( Date birthdate ) { this.birthdate = birthdate; }

    @Column( name="loggedin", nullable=false )
    public boolean isLoggedin() { return loggedin; }
    public void setLoggedin( boolean loggedin ) { this.loggedin = loggedin; }

    @Column( name="lastlogin", nullable=true )
    public Date getLastlogin() { return lastlogin; }
    public void setLastlogin( Date lastlogin ) { this.lastlogin = lastlogin; }

    @Column( name="created", nullable=false )
    public Date getCreated() { return created; }
    public void setCreated( Date created ) { this.created = created; }

    /**
     * This just facilitates displaying an Account object in the Eclipse debugger.
     */
    public final String toString()
    {
        return "id = " + this.id + ", code = " + this.code + ", name = " + this.name;
    }
}
account-table.sql:

I much prefer MySQL, which doesn't demand all this sequence crap just to work perfectly. The harder case is PostgreSQL, so I show it instead.

DROP SEQUENCE IF EXISTS account_sequence CASCADE;

CREATE SEQUENCE account_sequence    --PostgreSQL doesn't support MySQL's AUTOINCREMENT
    INCREMENT 1
    MINVALUE 1
    MAXVALUE 9223372036854775807
    START 1
    CACHE 1;                        -- you'd think more, but it ends up affecting the sequence*

DROP TABLE IF EXISTS account;

CREATE TABLE account
(
    id        INTEGER      NOT NULL   PRIMARY KEY DEFAULT NEXTVAL( 'account_sequence' ),
    username  VARCHAR(128) NOT NULL,
    password  VARCHAR(256) NOT NULL,
    firstname VARCHAR(64)  NULL,
    lastname  VARCHAR(64)  NULL,
    fullname  VARCHAR(256) NULL,
    email     VARCHAR(128) NOT NULL,
    telephone VARCHAR(32)  NULL,
    language  VARCHAR(2)   NULL,
    birthdate DATE         NULL,
    loggedin  BOOLEAN      NULL       DEFAULT FALSE,
    lastlogin TIMESTAMP    NULL,
    created   TIMESTAMP    NULL,
);


-- Put in some test data (a few rows of account)...
START TRANSACTION;
INSERT INTO account (id,username,password,firstname,lastname,fullname,email,phone,birthdate,loggedin,lastlogin,created)
    VALUES ('zeke','h2342$!!s','Ezekiel','Mordecai','Ezekiel "Zeke" Mordecai','[email protected]',
            '339-555-1212','en','1984-08-11',FALSE,'2011-08-08 12:00:00','2011-08-08 12:00:00');
INSERT INTO account (id,username,password,firstname,lastname,fullname,email,phone,birthdate,loggedin,lastlogin,created)
    VALUES ('trumpet','password','Sergei','Elephant',u&'\0441\043B\043E\043D','[email protected]',
            '8412.555.1212','ru','1955-06-18',FALSE,'2011-08-08 12:00:00','2011-08-08 12:00:00');
INSERT INTO account (id,username,password,firstname,lastname,fullname,email,phone,birthdate,loggedin,lastlogin,created)
    VALUES ('ryota','pa$$xyz','Ryota','Natsume',u&'\68ee\9dd7\5916','[email protected]',
            '81 3 555-1212','jp','1970-05-08',FALSE,'2011-08-08 12:00:00','2011-08-08 12:00:00');
COMMIT;

-- Reset the sequencer or programmatically adding rows will not work until after a few attempts
-- fail since the sequencer will generate conflicting ids at first. MySQL doesn't need this
SELECT setval( 'auth_sequence', ( SELECT MAX( id ) FROM auth ) + 1 );

(* Setting the number of sequences to cache to other than 1 results in the next OID generated being that many beyond the previous one—not sure that's supposed to be, but it's my observation that it is.)

src/resources/spring-configuration.xml:

This is Spring hocus-pocus. It's not my objective to explain much of it.



    
    
    

    
    

    
        
            
                
          WEB-INF/classes/resources/database.properties
            
        
    

    
        
        
        
        
    

    
        

        
            
                ${hibernate.dialect.class}
                true
                thread
                org.hibernate.cache.NoCacheProvider
                true
                false
                false
            
        

        
            
                com.etretatlogiciels.web.user.entity.Account
            
        
    

    
        
    

    
        
    

src/resources/database.properties:

This is the variable part of what needs to be included in the Spring/Hibernate configuration because it's going to be different when running and testing in Eclipse versus Jenkins not to mention an real production deployment.

database.username=russ
database.password=test123

database.url=jdbc:postgresql://localhost/usermgrdb

jdbc.driver.class=org.postgresql.Driver
hibernate.dialect.class=org.hibernate.dialect.PostgreSQLDialect
WEB-INF/web.xml:

As this is a web server, there's a web.xml too. Mostly, it sets up a listener ready to initialize Spring. We're calling our servlet, "usermgr".



    usermgr

    
        org.springframework.web.context.ContextLoaderListener
    

    
        org.springframework.web.context.request.RequestContextListener
    

    
        contextConfigLocation
        /WEB-INF/classes/resources/spring-configuration.xml
    

    
        Jersey REST Service
        com.sun.jersey.spi.spring.container.servlet.SpringServlet
        
            com.sun.jersey.config.property.packages
            com.etretatlogiciels.web.user.service
        
        
            com.sun.jersey.spi.container.ResourceFilters
            com.etretatlogiciels.web.user.util.SharedSecurityFilter
        
        1
    

    
        Jersey REST Service
        /*
    

src/log4j.properties:
log4j.rootLogger=TRACE,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.SimpleLayout

### direct log messages to stdout
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
log4j.logger.org.hibernate=debug

### make Hibernate log its SQL
log4j.logger.org.hibernate.SQL=debug

### log JDBC bind parameters
log4j.logger.org.hibernate.type=debug

### log schema export/update
log4j.logger.org.hibernate.tool.hbm2ddl=debug

Libraries

For completeness, here is the list of third-party libraries consumed in this solution.

russ ~/dev/usermgr/lib $ tree
.
|-- aopalliance
|   |-- aopalliance.jar
|   `-- cglib-2.2.2.jar
|-- apache
|   |-- commons-collections-3.2.1.jar
|   |-- commons-configuration-1.7.jar
|   |-- commons-lang-2.6.jar
|   |-- httpclient-4.1.3.jar
|   |-- jakarta-oro-2.0.8.jar
|   |-- log4j-1.2.16.jar
|   `-- servlet-api.jar
|-- gson
|   `-- gson-1.7.1.jar
|-- hibernate
|   |-- antlr-2.7.6.jar
|   |-- c3p0-0.9.1.jar
|   |-- dom4j-1.6.1.jar
|   |-- hibernate3.jar
|   |-- hibernate-jpa-2.0-api-1.0.1.Final.jar
|   |-- javassist-3.12.0.GA.jar
|   `-- jta-1.1.jar
|-- jersey
|   |-- asm-3.1.jar
|   |-- jersey-client-1.6.jar
|   |-- jersey-core-1.6.jar
|   |-- jersey-json-1.6.jar
|   |-- jersey-server-1.6.jar
|   |-- jersey-spring-1.12-b01.jar
|   |-- oauth-server-1.10.jar
|   `-- oauth-signature-1.10.jar
|-- junit
|   `-- junit-4.10.jar
|-- postgres
|   `-- postgresql-9.1-901.jdbc4.jar
|-- slf4j
|   |-- slf4j-api-1.6.1.jar
|   `-- slf4j-nop-1.6.1.jar
`-- spring
    |-- org.springframework.aop-3.1.1.RELEASE.jar
    |-- org.springframework.asm-3.1.0.RC2.jar
    |-- org.springframework.beans-3.1.0.RC2.jar
    |-- org.springframework.context-3.1.0.RC2.jar
    |-- org.springframework.core-3.1.0.RC2.jar
    |-- org.springframework.expression-3.1.0.RC2.jar
    |-- org.springframework.jdbc-3.1.0.RC2.jar
    |-- org.springframework.orm-3.1.0.RC2.jar
    |-- org.springframework.transaction-3.1.0.RC2.jar
    `-- org.springframework.web-3.1.0.RC2.jar

Sample URIs to this service

To the path http://<hostname>:<port>/usermgr, add the URLs below:

Verb URL Operation Representation Sample payload
POST /account/ Create a new account http://<hostname>:<port>/usermgr/account/ {"username":"ryota", etc. }
GET /account/ Read all account http://<hostname>:<port>/usermgr/account/ (no payload)
GET /account/{id} Read one account http://<hostname>:<port>/usermgr/account/552 (no payload)
PUT /account/{id} Update a account http://<hostname>:<port>/usermgr/account/35 {"username":"trumpet", etc. }*
DELETE /account/{id} Delete an account http://<hostname>:<port>/usermgr/account/552 (no payload)

QueryParams

When coding a method that's too much like another, i.e.: same HTTP verb (GET, POST, etc.) same or similar parameter list, etc., you'll see this when you attempt to start your service:

Jul 2, 2012 5:20:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
SEVERE: The following errors and warnings have been detected with resource and/or provider classes:
  SEVERE: Producing media type conflict. The resource methods \
     public javax.ws.rs.core.Response com.etretatlogiciels.web.user.service.AddressService.readByType(org.bson.types.ObjectId,java.lang.String) \
     and public javax.ws.rs.core.Response com.etretatlogiciels.web.user.service.AddressService.read(org.bson.types.ObjectId) \
     can produce the same media type
Jul 2, 2012 5:20:57 PM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
com.sun.jersey.spi.inject.Errors$ErrorMessagesExceptionA
...

This is solved by remembering that QueryParams are optional: just overload the one method by adding the QueryParam and testing for it being null in the method. If null, it's the one (without) and if not, then the QueryParam is there.


How to serialize the lowly HashMap

In another note in here on embedded objects, we showed how Jersey can be induced to serialize arrays such that an entity get populated when your service is called. This doesn't happen for Maps; Jersey just doesn't do it. So, there's a work-around (or, something better than a mere work-around).

In stackoverflow, there is a stackoverflow discussion on how to do this. A lot of sample code, but also URLs were given in answer to my question. In the end, the major pieces in the solution were:

  1. The MapAdapter source code (see stackoverflow link).
  2. The annotations on my affected entity/dto code, beyond the ones already expected, were:
    @XmlAccessorType( XmlAccessType.FIELD )
    @XmlRootElement
    public class Person implements Serializable
    {
        private String firstname;
        private String lastname;
        private List< Address > addresses = new ArrayList< Address >();
    
        @XmlAnyElement
        @XmlJavaTypeAdapter( MapAdapter.class )
        private Map< String, String > data = new HashMap< String, String >();
        ...
    
  3. The four following libraries, excerpted from the latest EclipseLink download, were consumed. I also attached their source-code JARs thinking I might need to debug through, but this was not necessary.
    • org.eclipse.persistence.antlr_3.2.0.v201206041011.jar
    • org.eclipse.persistence.asm_3.3.1.v201206041142.jar
    • org.eclipse.persistence.core_2.4.0.v20120608-r11652.jar
    • org.eclipse.persistence.moxy_2.4.0.v20120608-r11652.jar
  4. No jaxb.properties file was needed in my entities or dto packages.
  5. No changes to web.xml were needed.

Jersey support for security via annotations

Some Jersey secret sauce.

package com.etretatlogiciels.web.user.util;

import java.util.Collections;
import java.util.List;

import org.apache.log4j.Logger;

import com.sun.jersey.api.model.AbstractMethod;
import com.sun.jersey.spi.container.ResourceFilter;
import com.sun.jersey.spi.container.ResourceFilterFactory;

/**
 * SharedSecurityFilter is used to wrap HTTP methods with security. If the method
 * or class has an annotation present it will call the appropriate security logic.
 *
 * Note that other types of security implementations can be added here, but for
 * now oAuth is the only supported type.
 */
public class SharedSecurityFilter implements ResourceFilterFactory
{
    private static Logger log = Logger.getLogger( SharedSecurityFilter.class );

    public SharedSecurityFilter()
    {
        log.info( "SecuredResourceFilterFactory is watching for @OAuthSecurity flags" );
    }

    @Override
    public List< ResourceFilter > create( AbstractMethod method )
    {
        if( method.isAnnotationPresent( OAuthSecurity.class ) )
            return Collections.< ResourceFilter > singletonList( new OAuthResourceFilter() );

        return null;
    }
}

Consider checking out the effect of using @WebFilter.


Quick-start for writing a ReST application/server

This is a complete template using all of the annotations discussed here. To get them all in one place, here are the imports used throughout this pseudocode.

import org.apache.log4j.Logger;
import org.bson.types.ObjectId;

import javax.ws.rs.POST;
import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.DELETE;
import javax.ws.rs.Path;
import javax.ws.rs.Consumes;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.PathParam;

import static javax.ws.rs.core.MediaType.APPLICATION_JSON;
import static javax.ws.rs.core.MediaType.APPLICATION_XML;

import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlRootElement;

import org.codehaus.jackson.annotate.JsonAutoDetect;
import org.codehaus.jackson.annotate.JsonAutoDetect.Visibility;
import org.codehaus.jackson.map.annotate.JsonSerialize;

import com.mongodb.DB;
import com.mongodb.DBCollection;
import org.mongodb.morphia.Datastore;
import org.mongodb.morphia.query.Query;

Service entry points

@Path( "/service" )
public class Service
{
    private static Logger log = Logger.getLogger( Service.class );

    private Manager manager = new Manager();

    @POST
    @Consumes( { APPLICATION_JSON, APPLICATION_XML } )
    @Produces( { APPLICATION_JSON, APPLICATION_XML } )
    public Response create( PojoDto proposed )
    {
        Pojo pojo = null;

        try
        {
            pojo = manager.create( proposed );
        }
        catch( Exception e )
        {
            return buildResponse();
        }
        catch( RuntimeException e )
        {
            return buildRuntimeResponse( e );
        }

        PojoDto dto = pojo.dto();

        URI createdUri = URI.create( pojo.getOid().toString() );
        return Response.created( createdUri ).entity( dto ).build();
    }

    @GET
    @Path( "/{oid}" )
    @Produces( { APPLICATION_JSON, APPLICATION_XML, TEXT_HTML } )
    public Response readByOid( @PathParam( "oid" )  ObjectId oid )
    {
        Pojo  pojo = null;

        try
        {
            pojo = manager.readByOid( oid );
        }
        catch( Exception e )
        {
            return buildResponse();
        }
        catch( RuntimeException e )
        {
            buildRuntimeResponse( e );
        }

        PojoDto dto = pojo.dto();

        return Response.ok( dto ).build();
    }

    @GET
    @Path( "/{identity}" )
    @Produces( { APPLICATION_JSON, APPLICATION_XML, TEXT_HTML } )
    public Response readByIdentity( @PathParam( "identity" )  String Identity )
    {
        List< Pojo >  pojos = null;

        try
        {
            pojos = manager.readByIdentity( identity );
        }
        catch( Exception e )
        {
            return buildResponse();
        }
        catch( RuntimeException e )
        {
            buildRuntimeResponse( e );
        }

        GenericEntity< List< T > > list = new GenericEntity< List< T > >( pojos ) { };

        return Response.ok( list ).build();
    }

    @PUT
    @Path( "/{oid}" )
    @Consumes( { APPLICATION_JSON, APPLICATION_XML } )
    @Produces( { APPLICATION_JSON, APPLICATION_XML } )
    public Response update( @PathParam( "oid" ) ObjectId oid, PojoDto proposed )
    {
        Pojo pojo = null;

        try
        {
            pojo = manager.update( oid, proposed );
        }
        catch( Exception e )
        {
            return buildResponse();
        }
        catch( RuntimeException e )
        {
            return buildRuntimeResponse( e );
        }

        PojoDto dto = pojo.dto();

        return Response.ok( dto ).build();
    }

    @DELETE
    @Path( "/{oid}" )
    @Produces( { APPLICATION_JSON, APPLICATION_XML } )
    public Response delete( @PathParam( "oid" ) ObjectId oid )
    {
        try
        {
            manager.delete( oid );
        }
        catch( AppException e )
        {
            return buildResponse();
        }
        catch( RuntimeException e )
        {
            return buildRuntimeResponse( e );
        }

        return Response.noContent().build();
    }
}

Business-level logic (manager)

public class Manager
{
    private static Logger log = Logger.getLogger( Manager.class );

    private Dao dao;

    public Manager( Dao dao )
    {
        this.dao = dao;
    }

    public Pojo create( PojoDto proposed ) throws Exception
    {
        return dao.create( proposed );
    }

    public Pojo readByOid( ObjectId oid ) throws Exception
    {
        return dao.readByOid( oid );
    }

    public Pojo readByIdentity( PojoDto dto ) throws Exception
    {
        return dao.readByIdentity( dto.getIdentity() );
    }

    public Pojo update( ObjectId oid, PojoDto proposed ) throws Exception
    {
        dao.update( oid, proposed );
        return proposed;
    }

    public Pojo delete( ObjectId oid ) throws Exception
    {
        dao.delete( oid );
    }
}

POJOs

@XmlAccessorType( XmlAccessType.FIELD )
public abstract class PojoCommonBase implements Serializable
{
    ObjectId oid;
    String   identity;
}
@XmlAccessorType( XmlAccessType.FIELD )
@XmlRootElement
@Entity( value=DatabaseProperties.COLLECTION, noClassnameStored=true )
public class Pojo extends PojoCommonBase implements Cloneable
{
}

DTOs

@XmlAccessorType( XmlAccessType.FIELD )
@XmlRootElement( name="pojo" )
@JsonSerialize( include=JsonSerialize.Inclusion.NON_NULL )
@JsonAutoDetect( fieldVisibility = Visibility.ANY, getterVisibility = Visibility.NONE, isGetterVisibility = Visibility.NONE, setterVisibility = Visibility.NONE )
public class PojoDto extends PojoCommonBase
{
}

DAOs

The example here happens to be MongoDB/Morphia-flavored. MongoDB is privately implemented and not shown here, but the MongoDB details are here only to suggest an implementation and are not germane to the ReST template.

public class Dao
{
    private static final Logger log = Logger.getLogger( Dao.class );

    private MongoDB      mongoDB;
    private DBCollection collection;
    private Datastore    datastore;

    private static class DaoHolder
    {
        public static final Dao INSTANCE = new Dao();
    }

    private Dao()
    {
        MongoDBManager manager = MongoDBManager.getInstance();

        mongoDB    = manager.getMongoByDatabaseName( DatabaseProperties.DATABASE );
        collection = mongoDB.getCollection( DatabaseProperties.DATABASE, DatabaseProperties.COLLECTION );
        datastore  = mongoDB.getDatastore( DatabaseProperties.DATABASE );
    }

    public static Dao getInstance()
    {
        log.trace( "Dao getInstance()" );
        return DaoHolder.INSTANCE;
    }

    public DBCollection  getCollection() { return this.collection; }
    public Datastore     getDatastore()  { return this.datastore; }
    public Class< Pojo > getClazz()      { return Pojo.class; }

    public Pojo create( Pojo pojo )
    {
        datastore.save( pojo );
        return pojo;
    }

    public Pojo readByOid( ObjectId oid )
    {
        Query< T > query = datastore.createQuery( Pojo.class )
                                         .filter( "_id", oid );
        return query.get();
    }

    public Pojo readByIdentity( String identity )
    {
        Query< Pojo > query = datastore.createQuery( Pojo.class );

        query.and( query.criteria( "identity" ).equal( identity );

        return query.get();
    }

    public void update( Pojo modify )
    {
        datastore.merge( modify );
    }

    public void delete( Pojo entity )
    {
        datastore.delete( entity );
    }
}

ReST reference implementation illustration


Importance of path and parameters

Imagine this request to upgrade Elmer Fudd to senior hunter:

PUT https://server/employees/665893
{
  "id" : "665893",
  "name: : "Elmer Fudd",
  "position" : "Senior Wabbit Hunter"
}

The employee id in the path identifies Fudd uniquely. It may be questioned as to why duplicate that id in the path when it's going to be in the request body anyway.

The answer is that based on a path, https://server/employees, Fudd is not uniquely identified as an existing resource. Failure to do that violates an important ReST principle as the path no longer points at one, discrete resource.


ReST-ify SQL data

A comprehensive, top-level article:
http://java.dzone.com/articles/restify-your-sql-data


Distribution: ReST or web service versus a JAR

As a web service...

By JAR...


Testing JAX-RS web service URIs without mocks

Test your JAX-RS 2.0 Web Service URIs… Without Mocks

Please note that there are mocking frameworks that are good for testing web services:


HTTP ETag (entity tag)

This is something in the HTTP header and is useful for web cache validation, that is, making caches more efficient. It may save bandwidth, but most of all, it can be used to avoid data corruption.

The use of an ETag is optional.

The practice of (ReST) APIs being used by multiple clients that modify the same data at the same time, leads to lost updates and stalled deletes that slowly ruin entries in the database. Using the concurrency control designed into the HTTP protocol by means of the ETag is a simple and effective way to protect resource integrity.

Without simply copying or reproducing it, it would be hard to beat Kenneth Lange's illustration of ETags (I hope it doesn't disappear), see Avoid Data Corruption in Your REST API with ETags. See also Wikipedia: HTTP ETag.

A typical example

Protect against accidental deletions and updates by requiring callers to fish an ETag out of an explicit GET plus id of the object prior to passing that ETag back in the header as If-Match: ETag when calling DELETE or PUT.


Logging the ReST implementation

Logging should be by design, for example, ...

If microservice (multiple servers running, load-balancing, etc.), logging should be centrally gathered via ELK (Filebeat → Logstash/Elasticsearch).


Tools and technologies for the ReST implementation
  1. Design (Development): Swagger
  2. Testing: Postman, RESTed, cURL, Wireshark
  3. Hosting: HTTP Caching, DNS, TLS
  4. Performance: Loader.io
  5. Observability: ELK, Zipkin
  6. Management: API Gateway