Skip to content

Commit

Permalink
Merge pull request #1 from DiUS/iot-tunnel
Browse files Browse the repository at this point in the history
Ideally [this issue in the SDK](aws/aws-iot-device-sdk-js#357) would have been resolved before the merge, but on the balance of things I'd rather get this feature onto main before the holiday break, even if it means living with the Horrible Workaround(tm) for a while.
  • Loading branch information
jmattsson authored Dec 21, 2020
2 parents 8ec4360 + 65c549c commit 57659fd
Show file tree
Hide file tree
Showing 6 changed files with 255 additions and 2 deletions.
76 changes: 76 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,17 @@ Without delving into the details, here are some examples how the different facet
1. Launch chariotd with `--fleetprov` to enable initial device provisioning via AWS fleet provisioning: `chariotd --clientid=myclient --cacert=/path/to/AmazonRootCA1.pem --certstore=/var/chariotd/certs --fleetprov=/var/chariotd/fp.json`
1. If there is no device certificate available in the certstore, a fleet provisioning attempt will be started, and if successful the new certificate is saved to the certstore.
### Using secure tunneling
1. Ensure the [`localproxy`](https://github.com/aws-samples/aws-iot-securetunneling-localproxy) has been installed.
1. Ensure the Amazon root CA certificate is available in an OpenSSL [compatible](https://github.com/aws-samples/aws-iot-securetunneling-localproxy#certificate-setup) location.
1. Launch chariotd with `--tunnelmappings` to listen for tunnel requests: `chairotd --clientid=myclient --cacert=/path/to/AmazonRootCA1.pem --certstore=/var/chariotd/certs --tunnelmappings=MyThing:SSH=22,HTTPS=443 --tunnelproxy=/path/to/bin/localproxy`
1. If necessary, explicitly point chariotd at the directory where the Amazon root CA certificate can be found using e.g. `--tunnelcadir=/var/chariotd/tunnelca`.
1. Open a tunnel session using e.g. the [AWS Console](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling-tutorial.html#open-tunnel) or the AWS CLI tool (`aws iotsecuretunneling open-tunnel ...`).
1. Start a local proxy in [source mode](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling-tutorial.html#start-local-proxy), e.g. `localproxy -r ap-southeast-2 -s SSH=2222,HTTPS=4443 -t AQGAAXgU4...`.
1. Check that the tunnel session shows both endpoints connected, e.g. using the AWS Console or the CLI (`aws iotsecuretunneling describe-tunnel --tunnel-id=...`).
1. Connect through the proxy, e.g. `ssh -p 2222 user@localhost`.
## Running chariotd
Expand All @@ -69,6 +80,9 @@ Past this the features are pick-and-choose.
- `--messages` - Instructs chariotd to watch for MQTT publish requests. The specifics are available in the [Message publishing](#message-publishing) section.
- `--commands` - Makes chariotd listen for command requests. Available commands which may be sent are listed in the [Command requests](#command-requests) section.
- `--keepalive` - This can be used to override the keepalive interval, which by default is set to 1200. Devices communicating through a [NAT](https://en.wikipedia.org/wiki/Network_address_translation) device may need to use a lower keepalive interval to avoid the session state expiring in the NAT device.
- `--tunnelmappings` - Enables listening for AWS IoT Secure Tunneling requests. Details available in the [Secure Tunneling](#secure-tunneling) section.
- `--tunnelproxy` - Specifies the path to the secure tunneling proxy client to use.
- `--tunnelcadir` - Specifies the directory where the tunnel proxy client can find the Amazon root CA certificate in an OpenSSL compatible format.
## Restarting chariotd
Expand Down Expand Up @@ -315,6 +329,35 @@ Causes chariotd to run a fleet provisioning attempt. The main use case for this
If chariotd was started without `--fleetprov` this command is not available, for obvious reasons.
#### open-tunnel
Parameters: JSON object
```
{
"thing": "YourThingName",
"region": "xx-xyzzy-n",
"services": [ "Service1", ... ],
"token": "AQGAAXiZyfA6..."
}
```
Requests chariotd to launch a new proxy client instance using the provided settings.
If chariotd was started without a `--tunnelmappings` option for this thing, or if the tunnel request's services list does not match the tunnel session, the open-tunnel command will fail.
#### close-tunnel
Parameters: JSON object
```
{
"thing": "YourThingName"
}
```
Causes chariotd to shut down the active tunnel for the specified thing, provided it had launched the proxy client already. Otherwise a no-op.
## Fleet provisioning
Expand Down Expand Up @@ -368,3 +411,36 @@ Example JSON outfile:
}
```
## Secure Tunneling
For those wishing to use the AWS IoT Secure Tunneling feature, chariotd provides easy integration when using the [AWS IoT Secure Tunneling local proxy](https://github.com/aws-samples/aws-iot-securetunneling-localproxy). A different proxy client may be used as long as it provides the same interface in terms of environment variables and command line options. If the tunnel proxy client isn't called `localproxy` or is not in the path it must be explicitly specified using `--tunnelproxy=/path/to/proxyclient`.
When opening a tunnel session for a thing, the access token is sent to the device automatically on a reserved MQTT topic. When chariotd receives a message on such a topic it will spawn a proxy client, providing the access token (and other details). Note that AWS IoT sends the access token only *once*. If the device is not online at the time, the token may be lost. If this is a concern, you could provide an additional way of providing it to the device, and issue an explicit open-tunnel command to chariotd. See the [open-tunnel description](#open-tunnel) for details. One idea would be to provide the tunnel details via the device shadow, and implement a service which issues the open-tunnel/close-tunnel commands.
If a new tunnel request is received for a thing, chariotd will terminate the previous proxy instance, if known. Proxy clients are run "backgrounded" so they may outlive the chariotd instance. If chariotd has been restarted and receives a new tunnel request for which there is an old proxy client running, chariotd will be unable to terminate the old proxy client instance. Likewise it will not be able to action [close-tunnel commands](#close-tunnel) for it.
### Tunnel port mappings
To enable secure tunneling for a device, one or more service names must be mapped to a port number. The trivial case is a single service, e.g. "SSH=22". Multiple port mappings are specified separated by a comma, e.g. "SSH=22,HTTPS=443". The service names are arbitrary, but must match what is used in the tunnel setup.
Worth noting is that it is perfectly valid to provide more port mappings to chariotd than what ends up used in a tunnel session. When invoking the proxy client, chariotd will only pass the services actually requested in the tunnel session.
### Local proxy CA requirements
The standard local proxy client expects the Amazon root CA certificate to be available in a manner OpenSSL understands. On many systems it may already be available in a default location, in which case no special attention is required.
If that is not the case, then the CA certificate must be provided in a directory which has been set up to be OpenSSL compatible. Please refer to the [localproxy documentation](https://github.com/aws-samples/aws-iot-securetunneling-localproxy#certificate-setup) on how to prepare such a directory, and then pass `--tunnelcadir=/path/to/ca-dir` to chariotd on startup.
### Using a different local proxy client
If using a proxy client other than the AWS reference implementation, the following interface must be compatible. If necessary, use a wrapper shell script to translate as needed.
Environment variables:
- `AWSIOT_TUNNEL_REGION` - The region string, e.g. 'ap-southeast-2'.
- `AWSIOT_TUNNEL_ACCESS_TOKEN` - The destination access token.
Command line options:
- `-d SVC1=PORT2,SVC2=PORT2...` - One or more mappings between service name and port.
- `-c CADIR` - Root CA certificate directory. Only used if `--tunnelcadir` is passed to chariotd.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "chariotd",
"version": "1.0.0",
"version": "0.0.0",
"description": "Daemon for bringing AWS IoT Core services into Linux user-space.",
"main": "src/main.js",
"bin": {
Expand Down
9 changes: 9 additions & 0 deletions src/cmdline_opts.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,17 @@ const cmdline = Getopt.create([
'watch for message to publish to mqtt in DIR' ],
[ '', 'commands=DIR',
'watch for command requests in DIR' ],

[ '', 'tunnelmappings=THING:PORTMAPPINGS+',
'accept tunnel requests for THING, using PORTMAPPINGS e.g. SSH=22,HTTP=80' ],
[ '', 'tunnelproxy=PATH',
'use PATH as tunnel proxy binary (default "localproxy")' ],
[ '', 'tunnelcadir=DIR',
'tell tunnel proxy to look for OpenSSL-compatibly named CA cert in DIR' ],

[ '', 'keepalive=SECONDS',
'send keep-alive ping every SECONDS (default 1200)' ],

[ 'h', 'help', 'this help' ]
]).bindHelp().parseSystem();

Expand Down
1 change: 0 additions & 1 deletion src/dirwatch.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ function prune(dir, max) {

function DirWatch(basedir, cb, opts) {
this._basedir = basedir;
this._cb = cb;
this._max_failed = ((opts || {}).max_failed != null) ?
opts.max_failed : MAX_PRUNE_KEEP_DEFAULT;

Expand Down
47 changes: 47 additions & 0 deletions src/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ const CertStore = require('./certstore.js');
const Shadow = require('./shadow.js');
const DirWatch = require('./dirwatch.js');
const FleetProvisioning = require('./fleet_provisioning.js');
const { applyHorribleReservedTopicWorkaround, SecureTunnel } = require('./secure_tunnel.js');
const services = require('./services.js');
const { options } = require('./cmdline_opts.js');
const shadowMerge = require('./shadow_merge.js');
Expand Down Expand Up @@ -90,6 +91,24 @@ function handleCommand(dir, fname) {
else
console.warn(`Ignoring impossible request to reprovision - no fleet provisioning details provided on startup.`);
break;
case 'open-tunnel': {
const obj = JSON.parse(fs.readFileSync(`${dir}/${fname}`));
if (tunnels[obj.thing] != null)
tunnels[obj.thing].launch(obj.token, obj.services, obj.region);
else
console.warn(
`Ignored open-tunnel request for '${obj.thing}': not configured`);
break;
}
case 'close-tunnel': {
const obj = JSON.parse(fs.readFileSync(`${dir}/${fname}`));
if (tunnels[obj.thing] != null)
tunnels[obj.thing].terminate();
else
console.warn(
`Ignored close-tunnel request for '${obj.thing}': not configured`);
break;
}
default:
console.warn(`Ignored unknown command '${fname}'.`);
break;
Expand Down Expand Up @@ -150,6 +169,22 @@ for (const arg of (options.defaultshadow || [])) {
console.info(`Default shadow content generator available for ${thing}.`);
}

// Note tunnel configs, if any
const tunnels = {};
const tunnel_topics = {};
for (const arg of (options.tunnelmappings || [])) {
const [ thing, list ] = splitArg(arg);
const tunnel = new SecureTunnel(
thing, list, options.tunnelproxy, options.tunnelcadir);
tunnels[thing] = tunnel;
tunnel_topics[tunnel.topic()] = tunnel;
console.info(`Secure tunnel config loaded for ${thing}.`);
}
// TODO: Remove this workaround as soon as the aws-iot-device-sdk is fixed!
if (Object.keys(tunnels).length > 0)
applyHorribleReservedTopicWorkaround();


const shadows = {};

const cmdwatch = (options.commands != null) ?
Expand Down Expand Up @@ -212,6 +247,13 @@ function connect() {
else // thing is already registered, we may do updates & fetches
go();
}
for (const thing in tunnels) {
comms.subscribe(tunnels[thing].topic(), err => {
if (err != null)
console.warn(
`Failed to subscribe to tunnel topic for '${thing}': ${err}`);
});
}
});
comms.on('status', (thing, stat, token, resp) => {
comms_attempts = 0;
Expand All @@ -226,6 +268,11 @@ function connect() {
++comms_attempts;
checkCommsAttempts();
});
comms.on('message', (topic, payload) => {
const tunnel = tunnel_topics[topic];
if (tunnel != null)
tunnel.handleMessage(payload);
});
comms.on('error', err => {
console.error('AWS IoT Core connection reported error:', err);
checkCommsAttempts();
Expand Down
122 changes: 122 additions & 0 deletions src/secure_tunnel.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
/* Copyright(C) 2019-2020 DiUS Computing Pty Ltd */
'use strict';
const child_process = require('child_process');

function SecureTunnel(thing, portmappings, localproxy, caDir) {
this._thing = thing;
this._localproxy = localproxy || 'localproxy';
this._caDir = caDir;
this._mappings = {};
try {
portmappings.split(',').forEach(portmap => {
const [ key, port ] = portmap.split('=');
this._mappings[key] = +port;
if (isNaN(this._mappings[key]))
throw new Error(`invalid port number: ${port}`);
});
}
catch(e) {
console.error(
`Error parsing '${thing}' portmappings "${portmappings}": ${e}`);
process.exit(1);
}
}


SecureTunnel.prototype.thing = function() {
return this._thing;
}


SecureTunnel.prototype.topic = function() {
return `$aws/things/${this._thing}/tunnels/notify`;
}


SecureTunnel.prototype.handleMessage = function(buf) {
try {
const tunnelConfig = JSON.parse(buf);
if (tunnelConfig.clientMode != 'destination')
throw new Error(`unsupported mode '${tunnelConfig.clientMode}'`);

this.launch(
tunnelConfig.clientAccessToken,
tunnelConfig.services,
tunnelConfig.region);
}
catch(e) {
console.error(`Unable to open tunnel for ${this._thing}: ${e}`);
}
}


SecureTunnel.prototype.launch = function(accessToken, serviceList, region) {
if (this._proc != null) {
console.warn(`Stopping previous tunnel proxy for '${this._thing}'.`);
this._proc.kill();
}

const enabled = [];
for (const svc of serviceList) {
if (this._mappings[svc] != null)
enabled.push(`${svc}=${this._mappings[svc]}`);
else
console.warn(
`Requested tunnel service '${svc}' for '${this._thing}' not mapped!`);
}
if (enabled.length == 0)
throw new Error(`requested services not available!`);

const args = [ '-d', enabled.join(','), ];
if (this._caDir != null) {
args.push('-c');
args.push(this._caDir);
}
const env = {
AWSIOT_TUNNEL_ACCESS_TOKEN: accessToken,
AWSIOT_TUNNEL_REGION: region,
};

this._proc = child_process.spawn(
this._localproxy, args, { env, stdio: 'inherit', detached: true });
const pid = this._proc.pid;
console.info(`Starting tunnel proxy for '${this._thing}' with services "${serviceList.join(', ')}", pid ${pid}.`);
this._proc.on('exit', (code, sig) => {
const msg = `Tunnel proxy for '${this._thing}', pid ${pid} exited`;
if (code != null)
console.info(`${msg} with code ${code}.`);
else
console.info(`${msg} due to ${sig}.`);
// Clean up, but only if this._proc has already been replaced
if (this._proc != null && this._proc.pid == pid)
this._proc = null;
});

this._proc.unref(); // don't wait for proxy exit before we can exit
}


SecureTunnel.prototype.terminate = function() {
if (this._proc != null) {
console.info(`Terminating tunnel proxy for '${this._thing}', pid ${this._proc.pid} on request...`);
this._proc.kill();
}
}


function applyHorribleReservedTopicWorkaround() {
const substring = String.prototype.substring;
String.prototype.substring = function(x, y) {
var res = substring.call(this, x, y);
if (x == 0 && y == 12 && res == '$aws/things/') {
res = 'OneUglyHack!';
}
return res;
}
}


module.exports = {
SecureTunnel,
applyHorribleReservedTopicWorkaround,
};

0 comments on commit 57659fd

Please sign in to comment.