Wednesday, 4 May 2016

OpenIDM: Sequential Number Generator

The problem 

Working on a Proof of Concept recently the customer wanted numbers to be allocated to new users in sequence. They wanted the sequence to start at 300000000 and increase by 1 each time a new user was created. This was to 'guarantee' uniqueness of the number allocated to each user.

The solution

I decided to make use of the generic repository object capability of OpenIDM. This is a little known feature but something I've blogged about in the past (See: http://yaunap.blogspot.co.uk/2015/09/data-driven-preferences-list-in-openidm.html) But just storing a number wasn't enough...I needed some logic to increment the number and store the result, as well as an API in order to 'GetNextId' when a new user was created.
So there are several parts to this:
  1. Repository Object 
  2. Script to read stored number, increment it, store it and return it 
  3. Custom endpoint in OpenIDM to expose the script to clients 

Repository object 

Creating the repository object is straightforward. I called my object 'counter'. It will have a single field/value called 'lastid' seeded with 300000000 (note limitation below):
curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin"  --header "Content-Type: application/json" --data '{"lastid":"300000000" }' --request POST http://localhost:8080/openidm/repo/counter?_action=create

Script

The script is also fairly simple:
(function getNextId () {
  //read last number from custom repo object, add 1, store it back, return it!
  var  lastid = (openidm.read('repo/counter/lastid'));
  var nextid = Number(lastid['lastid']) + 1;
  lastid['lastid'] = nextid;
  openidm.update('repo/counter/lastid',null,lastid);
  return {nextid:nextid.toString()};
}());
This needs to be saved to a new .js file in the 'script' directory under the OpenIDM deployment directory. e.g. openidm/script/getNextId.js

Custom Endpoint

The custom endpoint is a simple definition (creating custom endpoints is well documented in the Integrators Guide):
{
    "type" : "text/javascript",
    "file" : "script/getNextId.js"
} 

 This needs to be saved to a .json file in the 'conf' directory. The naming convention is important. It must start with 'endpoint-'. Anything after the '-' will be the name of API you can access over HTTP. Therefore a file called:
openidm/conf/endpoint-getnextid.json
will be addressable thus:
curl -H "X-OpenIDM-Username: openidm-admin" -H "X-OpenIDM-Password: openidm-admin" 'http://localhost:8080/openidm/endpoint/getnextid'


Limitation: 

The endpoint only handles javascript numbers with values less than 2147483648 If you want larger numbers (i.e. starting at 3000000000) then the logic will need to support some way of handling the larger number.

Counter Reset:

To reset the counter, use:
curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin"  --header "Content-Type: application/json" --data '{"lastid":"300000000" }' --header "If-Match: *" --request PUT http://localhost:8080/openidm/repo/counter/lastid

4 comments:

  1. Hey,
    i've tried to implement your guidiance in my openidm, but i get an error that i dont realy understand..

    {"code":500,"reason":"Internal Server Error","message":"TypeError: Cannot read property \"lastid\" from null (/home/testuser/Downloads/openidm/script/getNextId.js#4) in /home/testuser/Downloads/openidm/script/getNextId.js at line number 4 at column number 0","detail":{"fileName":"/home/testuser/Downloads/openidm/script/getNextId.js","columnNumber":0,"lineNumber":4}}

    ..i dont know what's wrong at this position, i copied your script and dont do any changes on it. Did i forget something? I would be very nice if you could help me at this topic.

    best regards
    Marcel

    ReplyDelete
    Replies
    1. Were you able to successfully create the Repository Object?
      You might find the default permissions stop this.
      One way around this is to create another script and endpoint just for initialising the object.
      Script in script/initNextId.js:
      (function initNextId () {
      openidm.create('repo/counter', 'lastid', {"lastid":"300000000" });
      return {nextid:"300000000"};
      } ());

      endpoint in conf/endpoint-initnextid.json:
      {
      "type" : "text/javascript",
      "file" : "script/initNextId.js"
      }

      Then to create the repo object, run:
      curl -H "X-OpenIDM-Username: openidm-admin" -H "X-OpenIDM-Password: openidm-admin" 'http://localhost:8080/openidm/endpoint/initnextid'

      After that calling:
      curl -H "X-OpenIDM-Username: openidm-admin" -H "X-OpenIDM-Password: openidm-admin" 'http://localhost:8080/openidm/endpoint/getnextid'
      should work fine.

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Thank you so much for your proof of concept!
    I've made it thread safe and I've improved it to have multiple counters referenced with an argument in the request:
    (function(){
    var tryCount = 0;
    while (true){
    try {
    var val = request["resourcePath"];
    var lastid = openidm.read('repo/counter/uuid');
    var nextid = 0;
    if (lastid == null){
    lastid = openidm.create('repo/counter/','uuid', {'description':"This object is a counter for the uuid"});
    }
    nextid = (lastid[val]!=null) ? Number(lastid[val]) + 1 : 0;
    lastid[val] = nextid;
    openidm.update('repo/counter/uuid',lastid['_rev'],lastid);
    logger.info("Success finding value");
    break;
    } catch (e) {
    if (tryCount++ > 10){
    throw "Error while generating uuid for " + val;
    }
    logger.info("Error while generating unique id" + e.message);
    // wait random time to avoid conflict
    var now = new Date().getTime();
    var time = Math.random()*1000;
    while(new Date().getTime() < now + time){ /* do nothing */ }
    }
    }
    return {nextid:nextid.toString()};

    })();

    ReplyDelete