Merge pull request #208 from gtorodelvalle/feature/support-importing-…

…packages

Support the importing of NPM packages in attribute-function-interpolator

CHANGES_NEXT_RELEASE

@@ -0,0 +1 @@
+- [FEATURE] Support the importing of NPM packages in attribute-function-interpolator interpolators

README.md

@@ -244,6 +244,7 @@ An example simulation configuration file is shown next to give you a glimpse of
 The simulation configuration file accepts the following JSON properties or entries:
 
 * **exports**: The FIWARE Device Simulation provides a templating mechanism to avoid repeating text into simulation configuration files as well as to facilitate the edition of these files. More information about this templating mechanism just after the description of the rest of the properties which may be used in a simulation configuration file.
+* **require**: An array of names and/or paths of NPM packages to be required before running the simulation. This property is related to the `attribute-function-interpolator` detailed below. It makes it possible to `require()` these NPM packages directly in the code associated to these `attribute-function-interpolator`.
 * **domain**: Includes information about the service and subservice (i.e., service path) to use in the requests. It is mandatory in case any `entities` are included in the simulation configuration (see below).
     * **service**: The service to use in the requests.
     * **subservice**: The subservice (i.e., service path) to use in the requests.
@@ -334,6 +335,10 @@ The simulation configuration file accepts the following JSON properties or entri
                 * A valid attribute value using the `text-rotation-interpolator` is: `"text-rotation-interpolator({\"units\": \"seconds\", \"text\": [[0,\"PENDING\"],[15,\"REQUESTED\"],[30,[[50,\"COMPLETED\"],[50,\"ERROR\"]]],[45,\"REMOVED\"]]})"`. For example, according to this text rotation interpolation specification, if the current time seconds is between 0 and 15 it will return the value `PENDING`, if it is between 15 and 30 it will return the value `REQUESTED`, if it is between 30 and 45 it will return the value `COMPLETED` with a probability of 50% and `ERROR` with a probability of 50%.
             8. **`attribute-function-interpolator`**: It returns the result of the evaluation of some Javascript code. This code may include references to any entity's attributes values stored in the Context Broker. This interpolator accepts a string (properly escaped) with the Javascript code to evaluate. In this Javascript code, references to entity's attribute values may be included using the notation: `${{<entity-id>:#:<entity-type>}{<attribute-name>}}`, substituting the `<entity-id>`, `<entity-type>` and `<attribute-name}` by their concrete values. Take into consideration that the type specification of the entity (i.e., `:#:<entity-type>`, inluding the `:#:` separator) is optional and can be omited, in which case the entity type will not be considered when retrieving the entity and the corresponding attribute value from the Context Broker.
                 * A valid attribute value using the `attribute-function-interpolator` is: `"attribute-function-interpolator(${{Entity:001}{active:001}} + Math.pow(${{Entity:002}{active:001}},2))"`.
+                * An advanced feature incorporated to the `attribute-function-interpolator` is the possibility to `require` packages directly in the Javascript code to be evaluated. Obviously, all the capabilities related to referencing entity attributes are supported too in this case. To use it, please follow the next steps:
+                    1. Include a `require` property in your simulation configuration file setting its value to an array including the names and/or paths of the NPM packages you will be using in any of your `attribute-function-interpolator` interpolators. These packages will be required before proceding with the simulation and made available to your `attribute-function-interpolator` code which uses them. For example: `"require": ["postfix-calculate"]`.
+                    2. The result of the evaluation of your code should be assigned to the `module.exports` property (this is due to the fact that this functionality leans on the [`eval` NPM package](https://www.npmjs.com/package/eval) which imposes this restriction).
+                * A valid attribute value using this advanced mode of the `attribute-function-interpolator` is: `"attribute-function-interpolator(var postfixCalculate = require('postfix-calculate'); module.exports = postfixCalculate('${{Entity:001}{active:001}} 1 +');)"`, where the result of the evaluation (this is, the value assigned to `module.exports`) will be the result of adding 1 to the value of the `active:001` attribute of the `Entity:001` entity, according to the [`postfix-calculate` NPM](https://www.npmjs.com/package/postfix-calculate) functionality.
         * **metadata**: Array of metadata information to be associated to the attribute on the update. Each metadata array entry is an object including 3 properties:
             * **name**: The metadata name.
             * **type**: The metadata type.

bin/fiwareDeviceSimulatorCLI

@@ -107,6 +107,10 @@ function executeCommand() {
     logops.debug('update-request event:', ev);
   });
 
+  progressEmitter.on('info', function(ev) {
+    logops.info('info event:', ev.message);
+  });
+
   progressEmitter.on('error', function(ev) {
     logops.error('error event:', ev);
   });

bin/fiwareDeviceSimulatorTranspilerCLI

@@ -71,7 +71,7 @@ function executeCommand() {
     return logops.error('The output file path (\'' + outputConfigurationFilePath + '\') already exists');
   }
 
-  fiwareDeviceSimulatorTranspiler.compose(require(inputConfigurationFilePath), function(err, newConfigurationObj) {
+  fiwareDeviceSimulatorTranspiler.transpile(require(inputConfigurationFilePath), function(err, newConfigurationObj) {
     if (err) {
       return logops.error('Error when transpiling the simulation configuration file (\'' +
         inputConfigurationFilePath + '\'): ' + err);

lib/errors/fdsErrors.js

@@ -45,6 +45,17 @@ function NGSIVersionNotSupported(message) {
 }
 NGSIVersionNotSupported.prototype = Object.create(Error.prototype);
 
+/**
+ * Package not imported error
+ * @param {String} message Human-readable description of the error
+ */
+function PackageNotImported(message) {
+  Error.call(this, message);
+  this.name = 'PackageNotImported';
+  this.message = message;
+}
+PackageNotImported.prototype = Object.create(Error.prototype);
+
 /**
  * Protocol not supported error
  * @param {String} message Human-readable description of the error
@@ -92,6 +103,7 @@ ValueResolutionError.prototype = Object.create(Error.prototype);
 module.exports = {
   InvalidInterpolationSpec: InvalidInterpolationSpec,
   NGSIVersionNotSupported: NGSIVersionNotSupported,
+  PackageNotImported: PackageNotImported,
   ProtocolNotSupported: ProtocolNotSupported,
   SimulationConfigurationNotValid: SimulationConfigurationNotValid,
   TokenNotAvailable: TokenNotAvailable,

lib/fiwareDeviceSimulator.js

@@ -28,6 +28,7 @@ var async = require('async');
 var EventEmitter = require('events').EventEmitter;
 var lolex = require('lolex');
 var mqtt = require('mqtt');
+var npmInstall = require('npm-install-package');
 var scheduler = require('node-schedule');
 var request = require('request');
 var time = require('time');
@@ -317,6 +318,19 @@ function emitScheduled(schedule, elementType, element, attributes) {
   eventEmitter.emit('update-scheduled', event2Emit);
 }
 
+/**
+ * Emits an "info" event
+ * @param  {Object} info The information to emit
+ */
+function emitInfo(info) {
+  eventEmitter.emit(
+    'info',
+    {
+      message: info
+    }
+  );
+}
+
 /**
  * Emits an "error" event
  * @param  {Object} err The error to emit
@@ -1155,6 +1169,53 @@ function onTokenResponse(err, response, body) {
   /* jshint camelcase: true */
 }
 
+/**
+ * Transpiles a simulation configuration into another simulation configuration resolving the possible import()
+ * directives
+ * @param  {Object}   configuration The input simulation configuration
+ * @param  {Function} callback      The callback
+ */
+function transpile(configuration, callback) {
+  emitInfo('Starting the simulation configuration transpiling...');
+  fiwareDeviceSimulatorTranspiler.transpile(configuration, function(err, newConfiguration) {
+    if (err) {
+      return process.nextTick(callback.bind(null, err));
+    }
+    emitInfo('Simulation configuration transpiling successfully completed!');
+    return process.nextTick(callback.bind(null, err, newConfiguration));
+  });
+}
+
+/**
+ * Imports the packages included in the require property of the simulation configuration
+ * @param  {Object}   configuration The simulation configuration
+ * @param  {Function} callback      The callback
+ */
+function importPackages(configuration, callback) {
+  if (configuration.require) {
+    emitInfo('Requiring the following NPM packages: ' + configuration.require);
+    npmInstall(
+      configuration.require,
+      {
+        cache: true,
+        silent: true
+      },
+      function(err) {
+        var error;
+        if (err) {
+          error = new fdsErrors.PackageNotImported('Some of the packages included in the \'require\' list (' +
+            configuration.require + ') could not be imported');
+        }
+        emitInfo('NPM packages (' + configuration.require + ') successfully required and available!');
+        delete configuration.require;
+        return process.nextTick(callback.bind(null, error, configuration));
+      }
+    );
+  } else {
+    process.nextTick(callback.bind(null, null, configuration));
+  }
+}
+
 /**
  * Starts a simulation according to certain simulation configuration
  * @param  {Object}       config   A JSON simulation configuration Object
@@ -1200,44 +1261,43 @@ function start(config, theFromDate, theToDate, interval, margin, theDelay) {
   realFromDate = null;
   isEnded = false;
   cancelAllJobs();
-  fiwareDeviceSimulatorTranspiler.compose(config, function(err, newConfig) {
+  async.waterfall([
+    // Needed to be able to return the eventEmitter and start emitting events
+    function(callback) {
+      return process.nextTick(callback);
+    },
+    async.apply(transpile, config),
+    fiwareDeviceSimulatorValidator.validateConfiguration,
+    importPackages
+  ], function(err, newConfig) {
     if (err) {
       process.nextTick(function notifySimulationConfigurationError() {
         emitError(err);
         end();
       });
     } else {
-      fiwareDeviceSimulatorValidator.validateConfiguration(newConfig, function(err) {
-        if (err) {
-          process.nextTick(function notifySimulationConfigurationError() {
-            emitError(err);
-            end();
-          });
+      if (theFromDate) {
+        clock = lolex.install(theFromDate.getTime());
+      }
+      configuration = newConfig;
+      maximumNotRespondedUpdateRequests = margin;
+      delay = theDelay;
+      progressInfoInterval = interval;
+      fromDate = theFromDate;
+      toDate = theToDate;
+      if (configuration.authentication) {
+        if (configuration.authentication.retry) {
+          process.nextTick(
+            async.retry.bind(null, configuration.authentication.retry, requestToken, onTokenResponse));
+          nextTick();
         } else {
-          if (theFromDate) {
-            clock = lolex.install(theFromDate.getTime());
-          }
-          configuration = newConfig;
-          maximumNotRespondedUpdateRequests = margin;
-          delay = theDelay;
-          progressInfoInterval = interval;
-          fromDate = theFromDate;
-          toDate = theToDate;
-          if (configuration.authentication) {
-            if (configuration.authentication.retry) {
-              process.nextTick(
-                async.retry.bind(null, configuration.authentication.retry, requestToken, onTokenResponse));
-              nextTick();
-            } else {
-              process.nextTick(requestToken.bind(null, onTokenResponse));
-              nextTick();
-            }
-          } else {
-            process.nextTick(scheduleJobs);
-            nextTick();
-          }
+          process.nextTick(requestToken.bind(null, onTokenResponse));
+          nextTick();
         }
-      });
+      } else {
+        process.nextTick(scheduleJobs);
+        nextTick();
+      }
     }
   });
   return eventEmitter;

lib/interpolators/attributeFunctionInterpolator.js

@@ -26,6 +26,7 @@
 var ROOT_PATH = require('app-root-path');
 var async = require('async');
 var deasync = require('deasync');
+var _eval = require('eval');
 var request = require('request');
 var fdsErrors = require(ROOT_PATH + '/lib/errors/fdsErrors');
 
@@ -223,17 +224,21 @@ module.exports = function(interpolationSpec, theDomainConf, theContextBrokerConf
         });
       });
 
-      /* jshint evil: true */
       var evaluatedValue;
       try {
-        evaluatedValue = eval(evalStr);
+        if (evalStr.indexOf('module.exports') !== -1) {
+          evaluatedValue = _eval(evalStr, true);
+        } else {
+          /* jshint evil: true */
+          evaluatedValue = eval(evalStr);
+          /* jshint evil: false */
+        }
       } catch (exception) {
         return callback(
           new fdsErrors.ValueResolutionError('Error when evaluating the Javascript code ' +
             'for an attribute-function-interpolator resolution with spec: \'' + interpolationSpec + '\''));
       }
       callback(null, evaluatedValue);
-      /* jshint evil: false */
     });
   }
 

lib/transpilers/fiwareDeviceSimulatorTranspiler.js

@@ -214,7 +214,7 @@ function replacer(match) {
  * @param  {Object}   configuration The original configuration
  * @param  {Function} callback      The callback
  */
-function compose(configuration, callback) {
+function transpile(configuration, callback) {
   var configurationStr,
       newConfigurationStr,
       newConfigurationObj;
@@ -238,5 +238,5 @@ function compose(configuration, callback) {
 }
 
 module.exports = {
-  compose: compose
+  transpile: transpile
 };

lib/validators/fiwareDeviceSimulatorValidator.js

@@ -49,6 +49,29 @@ function isEntities(simulationConfiguration) {
   return false;
 }
 
+/**
+ * Checks if the require information is valid
+ * @param  {Object}   simulationConfiguration The simulation configuration object
+ * @param  {Function} callback                The callback
+ */
+function validateRequire(simulationConfiguration, callback) {
+  if (simulationConfiguration.require) {
+    if (Array.isArray(simulationConfiguration.require)) {
+      for (var ii = 0; ii < simulationConfiguration.require.length; ii++) {
+        if (typeof simulationConfiguration.require[ii] !== 'string') {
+          return callback(new fdsErrors.SimulationConfigurationNotValid('The \'require\' configuration information ' +
+            'is not an array of NPM packages names'));
+        }
+      }
+      return setImmediate(callback);
+    } else {
+      return callback(new fdsErrors.SimulationConfigurationNotValid('The \'require\' configuration information ' +
+        'is not an array of NPM packages names'));
+    }
+  }
+  return setImmediate(callback);
+}
+
 /**
  * Checks if the domain information is included
  * @param  {Object}   simulationConfiguration The simulation configuration object
@@ -613,7 +636,7 @@ function validateAttribute(attributeType, parentType, parentIndex, attribute, at
     async.apply(validateSchedule, attribute.schedule, parentType, parentIndex)
   ], function(err) {
     if (err) {
-      callback(err);
+      return callback(err);
     } else {
       if (attribute.metadata) {
         if (!Array.isArray(attribute.metadata)) {
@@ -778,13 +801,16 @@ function validateEntitiesConfiguration(simulationConfiguration, callback) {
 function validateConfiguration(theSimulationConfiguration, callback) {
   simulationConfiguration = theSimulationConfiguration;
   async.series([
+    async.apply(validateRequire, simulationConfiguration),
     async.apply(validateDomain, simulationConfiguration),
     async.apply(validateContextBrokerConfiguration, simulationConfiguration),
     async.apply(validateAuthenticationConfiguration, simulationConfiguration),
     async.apply(validateIoTAConfiguration, simulationConfiguration),
     async.apply(validateEntitiesConfiguration, simulationConfiguration),
     async.apply(validateDevicesConfiguration, simulationConfiguration)
-  ], callback);
+  ], function(err) {
+    return setImmediate(callback.bind(null, err, simulationConfiguration));
+  });
 }
 
 module.exports = {

package.json

@@ -49,12 +49,14 @@
     "async": "^2.0.1",
     "commander": "^2.9.0",
     "deasync": "^0.1.8",
+    "eval": "^0.1.1",
     "humanize-duration": "^3.9.1",
     "linear-interpolator": "^1.0.2",
     "logops": "^1.0.1",
     "lolex": "^1.5.1",
     "mqtt": "^1.14.0",
     "node-schedule": "^1.2.0",
+    "npm-install-package": "^1.1.0",
     "request": "^2.74.0",
     "time": "^0.11.4",
     "turf-along": "^3.0.12",

test/unit/fiwareDeviceSimulator_test.js

@@ -5444,7 +5444,7 @@ describe('fiwareDeviceSimulator tests', function() {
 
   describe('authorization', function() {
     beforeEach(function(done) {
-      fiwareDeviceSimulatorTranspiler.compose(simulationConfiguration, function(err, newSimulationConfiguration) {
+      fiwareDeviceSimulatorTranspiler.transpile(simulationConfiguration, function(err, newSimulationConfiguration) {
         if (err) {
           return done(err);
         }
@@ -5518,7 +5518,7 @@ describe('fiwareDeviceSimulator tests', function() {
     function simulationTestSuite(type, options){
       beforeEach(function(done) {
         simulationConfiguration = require(ROOT_PATH + '/test/unit/configurations/simulation-configuration.json');
-        fiwareDeviceSimulatorTranspiler.compose(simulationConfiguration, function(err, newSimulationConfiguration) {
+        fiwareDeviceSimulatorTranspiler.transpile(simulationConfiguration, function(err, newSimulationConfiguration) {
           if (err) {
             return done(err);
           }