diff --git a/README.rst b/README.rst index 120b63c..2490b1e 100644 --- a/README.rst +++ b/README.rst @@ -1,12 +1,12 @@ ============= Cayenne Agent ============= -The Cayenne agent is a full featured client for the `Cayenne IoT project builder `_. It sends system information as well as sensor and actuator date and responds to actuator messages initiated from the Cayenne dashboard and mobile apps. The Cayenne agent currently supports Rasbian on the Raspberry Pi but it can be extended to support additional Linux flavors and other platforms. +The Cayenne agent is a full featured client for the `Cayenne IoT project builder `_. It sends system information as well as sensor and actuator data and responds to actuator messages initiated from the Cayenne dashboard and mobile apps. The Cayenne agent currently supports Rasbian on the Raspberry Pi but it can be extended to support additional Linux flavors and other platforms. ************ Requirements ************ -* `Python 3 `_. +* `Python 3.3 or newer `_. * pip3 - Python 3 package manager. This should already be available in Python 3.4+ and above. If it isn't in can be installed using the system package manager. Via `apt-get` this would be: :: @@ -127,9 +127,9 @@ To verify that the sensor/actuator works correctly you can test it with the foll * Create a new sensor using ``myDevices.sensors.SensorsClient.AddSensor`` using the appropriate device name and any args required by your device. * Get the sensor values using ``myDevices.sensors.SensorsClient.SensorsInfo`` and make sure the sensor data is correct. * If the new device is an actuator set the actuator value using ``myDevices.sensors.SensorsClient.SensorCommand``. -* Delete the sensor using ``myDevices.sensors.SensorsClient.DeleteSensor``. +* Delete the sensor using ``myDevices.sensors.SensorsClient.RemoveSensor``. -An example demonstrating these functions is available in ``myDevices.test.client_test.py``. +An example demonstrating these functions is available in ``myDevices.test.sensors_test.py``. *Note:* For security reasons the Cayenne agent is designed to be able to run from an account without root privileges. If any of your sensor/actuator code requires root access consider running just that portion of your code via a separate process that can be launched using sudo. For example, the ``myDevices.devices.digital.ds2408`` module uses this method to write data. @@ -164,19 +164,19 @@ System info Information about the device, including CPU, RAM, etc., is currently retrieved via a few different modules. To support a different board you may need to update the agent code for the following items, if applicable: General System Info - General system info, including CPU, RAM, memory, etc. is retrieved via ``myDevices.os.systeminfo.py`` and ``myDevices.os.cpu.py``. These are mostly implemented using cross platform libraries so they may already provide support for your board. If not, they should be modified or overridden to provide the appropriate system info. If your board does not support all the data values currently implemented you can just provide default values where necessary, though this may affect the data display in the Cayenne dashboard. + General system info, including CPU, RAM, memory, etc. is retrieved via ``myDevices.system.systeminfo.py`` and ``myDevices.system.cpu.py``. These are mostly implemented using cross platform libraries so they may already provide support for your board. If not, they should be modified or overridden to provide the appropriate system info. If your board does not support all the data values currently implemented you can just provide default values where necessary, though this may affect the data display in the Cayenne dashboard. Hardware Info - Hardware info, including make, model, etc. is retrieved via ``myDevices.os.hardware.py``. This should be modified or overridden to provide the appropriate hardware info for your board. + Hardware info, including make, model, etc. is retrieved via ``myDevices.system.hardware.py``. This should be modified or overridden to provide the appropriate hardware info for your board. Pin Mapping - The mapping of the on-board pins is provided in ``myDevices.utils.version.py`` with the ``MAPPING`` list. This list provides the available GPIO pin numbers as well as the voltage ("V33", "V50"), ground ("GND") and do-not-connect ("DNC") pins. This should be updated with the mapping for your board. However, the Cayenne dashboard is currently built to display the Raspberry Pi GPIO layout so if your board's pin layout is significantly different it may not display correctly in the GPIO tab. + The mapping of the on-board pins is provided in ``myDevices.devices.digital.gpio.py`` with the ``MAPPING`` list. This list provides the available GPIO pin numbers as well as the voltage ("V33", "V50"), ground ("GND") and do-not-connect ("DNC") pins. This should be updated with the mapping for your board. However, the Cayenne dashboard is currently built to display the Raspberry Pi GPIO layout so if your board's pin layout is significantly different it may not display correctly in the GPIO tab. Settings -------- -Currently the Raspberry Pi agent has settings for enabling/disabling the device tree, SPI, I²C, serial and camera. These are set via the ``myDevices.os.raspiconfig`` module which runs a separate Bash script at ``/etc/myDevices/scripts/config.sh``. If any of these settings are available on your board and you would like to support them you can override or replace ``myDevices.os.raspiconfig.py``. Otherwise the settings functionality can be ignored. +Currently the Raspberry Pi agent has settings for enabling/disabling the device tree, SPI, I²C, serial and camera. These are set via the ``myDevices.system.raspiconfig`` module which runs a separate Bash script at ``/etc/myDevices/scripts/config.sh``. If any of these settings are available on your board and you would like to support them you can override or replace ``myDevices.system.raspiconfig.py``. Otherwise the settings functionality can be ignored. -*Note:* For security reasons the Cayenne agent is designed to be able to run from an account without root privileges. If any of your I/O, system info or settings code requires root access consider running it via a separate process that can be launched using sudo. For example, the ``myDevices.os.raspiconfig`` module uses this method to update config settings. +*Note:* For security reasons the Cayenne agent is designed to be able to run from an account without root privileges. If any of your I/O, system info or settings code requires root access consider running it via a separate process that can be launched using sudo. For example, the ``myDevices.system.raspiconfig`` module uses this method to update config settings. ************ Contributing diff --git a/myDevices/__init__.py b/myDevices/__init__.py index 122f144..295c145 100644 --- a/myDevices/__init__.py +++ b/myDevices/__init__.py @@ -1,7 +1,4 @@ -from time import sleep - -try: - import ipgetter -except: - pass - +""" +This package contains the Cayenne agent, which is a full featured client for the Cayenne IoT project builder: https://cayenne.mydevices.com. It sends system information as well as sensor and actuator data and responds to actuator messages initiated from the Cayenne dashboard and mobile apps. +""" +__version__ = '2.0.0' diff --git a/myDevices/__main__.py b/myDevices/__main__.py index f06a68e..14e5788 100644 --- a/myDevices/__main__.py +++ b/myDevices/__main__.py @@ -1,40 +1,48 @@ -from myDevices.utils.config import Config +""" +This module is the main entry point for the Cayenne agent. It processes any command line parameters and launches the client. +""" from os import path, getpid, remove +from sys import __excepthook__, argv, maxsize +from threading import Thread +from myDevices.utils.config import Config from myDevices.cloud.client import CloudServerClient from myDevices.utils.logger import exception, setDebug, info, debug, error, logToFile, setInfo -from sys import excepthook, __excepthook__, argv, maxsize -from threading import Thread from signal import signal, SIGUSR1, SIGINT from resource import getrlimit, setrlimit, RLIMIT_AS -from myDevices.os.services import ProcessInfo -from myDevices.os.daemon import Daemon +from myDevices.system.services import ProcessInfo +from myDevices.utils.daemon import Daemon -def setMemoryLimit(rsrc, megs = 200): +def setMemoryLimit(rsrc, megs=200): + """Set the memory usage limit for the agent process""" size = megs * 1048576 soft, hard = getrlimit(rsrc) - setrlimit(rsrc, (size, hard)) #limit to one kilobyte - soft, hard = getrlimit(rsrc) - info ('Limit changed to :'+ str( soft)) + setrlimit(rsrc, (size, hard)) + try: #Only set memory limit on 32-bit systems if maxsize <= 2**32: setMemoryLimit(RLIMIT_AS) -except Exception as e: - error('Cannot set limit to memory: ' + str(e)) +except Exception as ex: + print('Cannot set limit to memory: ' + str(ex)) client = None pidfile = '/var/run/myDevices/cayenne.pid' def signal_handler(signal, frame): - if client: + """Handle program interrupt so the agent can exit cleanly""" + if client and client.connected: if signal == SIGINT: info('Program interrupt received, client exiting') client.Destroy() remove(pidfile) else: client.Restart() + elif signal == SIGINT: + remove(pidfile) + raise SystemExit signal(SIGUSR1, signal_handler) signal(SIGINT, signal_handler) + def exceptionHook(exc_type, exc_value, exc_traceback): """Make sure any uncaught exceptions are logged""" debug('Daemon::exceptionHook ') @@ -46,7 +54,7 @@ def exceptionHook(exc_type, exc_value, exc_traceback): def threadExceptionHook(): """Make sure any child threads hook exceptions. This should be called before any threads are created.""" - debug('Daemon::threadExceptionHook ') + debug('Daemon::threadExceptionHook') init_original = Thread.__init__ def init(self, *args, **kwargs): init_original(self, *args, **kwargs) @@ -79,51 +87,50 @@ def displayHelp(): exit() def writePidToFile(pidfile): + """Write the process ID to a file to prevent multiple agents from running at the same time""" if path.isfile(pidfile): info(pidfile + " already exists, exiting") with open(pidfile, 'r') as file: pid = int(file.read()) if ProcessInfo.IsRunning(pid) and pid != getpid(): - Daemon.Exit() - return + raise SystemExit pid = str(getpid()) with open(pidfile, 'w') as file: file.write(pid) + def main(argv): + """Main entry point for starting the agent client""" global pidfile configfile = None - scriptfile = None logfile = None - isDebug = False i = 1 setInfo() while i < len(argv): if argv[i] in ["-c", "-C", "--config-file"]: configfile = argv[i+1] - i+=1 + i += 1 elif argv[i] in ["-l", "-L", "--log-file"]: logfile = argv[i+1] - i+=1 + i += 1 elif argv[i] in ["-h", "-H", "--help"]: displayHelp() elif argv[i] in ["-d", "--debug"]: setDebug() elif argv[i] in ["-P", "--pidfile"]: pidfile = argv[i+1] - i+=1 - i+=1 + i += 1 + i += 1 if configfile == None: configfile = '/etc/myDevices/Network.ini' writePidToFile(pidfile) logToFile(logfile) - # SET HOST AND PORT config = Config(configfile) - HOST = config.get('CONFIG','ServerAddress', 'cloud.mydevices.com') - PORT = config.getInt('CONFIG','ServerPort', 8181) + HOST = config.get('CONFIG', 'ServerAddress', 'mqtt.mydevices.com') + PORT = config.getInt('CONFIG', 'ServerPort', 8883) CayenneApiHost = config.get('CONFIG', 'CayenneApi', 'https://api.mydevices.com') - # CREATE SOCKET - global client + global client client = CloudServerClient(HOST, PORT, CayenneApiHost) + client.Start() if __name__ == "__main__": try: diff --git a/myDevices/cloud/apiclient.py b/myDevices/cloud/apiclient.py index 82393ab..3a8eb18 100644 --- a/myDevices/cloud/apiclient.py +++ b/myDevices/cloud/apiclient.py @@ -2,6 +2,10 @@ from concurrent.futures import ThreadPoolExecutor import json from myDevices.utils.logger import error, exception +from myDevices.system.hardware import Hardware +from myDevices.system.systeminfo import SystemInfo +from myDevices.cloud import cayennemqtt +from myDevices.devices.digital.gpio import NativeGPIO class CayenneApiClient: def __init__(self, host): @@ -36,31 +40,55 @@ def sendRequest(self, method, uri, body=None): return None return response exception("No data received") + + def getMessageBody(self, inviteCode): + body = {'id': inviteCode} + hardware = Hardware() + if hardware.Serial and hardware.isRaspberryPi(): + body['type'] = 'rpi' + body['hardware_id'] = hardware.Serial + else: + hardware_id = hardware.getMac() + if hardware_id: + body['type'] = 'mac' + body['hardware_id'] = hardware_id + try: + system_data = [] + cayennemqtt.DataChannel.add(system_data, cayennemqtt.SYS_HARDWARE_MAKE, value=hardware.getManufacturer(), type='string', unit='utf8') + cayennemqtt.DataChannel.add(system_data, cayennemqtt.SYS_HARDWARE_MODEL, value=hardware.getModel(), type='string', unit='utf8') + system_info = SystemInfo() + capacity_data = system_info.getMemoryInfo((cayennemqtt.CAPACITY,)) + capacity_data += system_info.getDiskInfo((cayennemqtt.CAPACITY,)) + for item in capacity_data: + system_data.append(item) + body['properties'] = {} + body['properties']['pinmap'] = NativeGPIO().MAPPING + if system_data: + body['properties']['sysinfo'] = system_data + except: + exception('Error getting system info') + return json.dumps(body) def authenticate(self, inviteCode): - body = json.dumps({'id': inviteCode}) + body = self.getMessageBody(inviteCode) url = '/things/key/authenticate' return self.sendRequest('POST', url, body) def activate(self, inviteCode): - body = json.dumps({'id': inviteCode}) + body = self.getMessageBody(inviteCode) url = '/things/key/activate' return self.sendRequest('POST', url, body) - def getId(self, content): + def getCredentials(self, content): if content is None: return None body = content.decode("utf-8") if body is None or body is "": return None - return json.loads(body)['id'] + return json.loads(body) def loginDevice(self, inviteCode): - response = self.authenticate(inviteCode) + response = self.activate(inviteCode) if response and response.status_code == 200: - return self.getId(response.content) - if not response or response.status_code == 412: - response = self.activate(inviteCode) - if response and response.status_code == 200: - return self.getId(response.content) + return self.getCredentials(response.content) return None diff --git a/myDevices/cloud/cayennemqtt.py b/myDevices/cloud/cayennemqtt.py new file mode 100644 index 0000000..3f99e98 --- /dev/null +++ b/myDevices/cloud/cayennemqtt.py @@ -0,0 +1,246 @@ +import time +from json import loads, decoder +from ssl import PROTOCOL_TLSv1_2 +import paho.mqtt.client as mqtt +from myDevices.utils.logger import debug, error, exception, info, logJson, warn + +# Topics +DATA_TOPIC = 'data/json' +COMMAND_TOPIC = 'cmd' +COMMAND_JSON_TOPIC = 'cmd.json' +COMMAND_RESPONSE_TOPIC = 'response' + +# Data Channels +SYS_HARDWARE_MAKE = 'sys:hw:make' +SYS_HARDWARE_MODEL = 'sys:hw:model' +SYS_OS_NAME = 'sys:os:name' +SYS_OS_VERSION = 'sys:os:version' +SYS_NET = 'sys:net' +SYS_STORAGE = 'sys:storage' +SYS_RAM = 'sys:ram' +SYS_CPU = 'sys:cpu' +SYS_I2C = 'sys:i2c' +SYS_SPI = 'sys:spi' +SYS_UART = 'sys:uart' +SYS_ONEWIRE = 'sys:1wire' +SYS_DEVICETREE = 'sys:devicetree' +SYS_GPIO = 'sys:gpio' +SYS_POWER_RESET = 'sys:pwr:reset' +SYS_POWER_HALT = 'sys:pwr:halt' +AGENT_VERSION = 'agent:version' +AGENT_DEVICES = 'agent:devices' +AGENT_MANAGE = 'agent:manage' +DEV_SENSOR = 'dev' + +# Channel Suffixes +IP = 'ip' +SPEEDTEST = 'speedtest' +SSID = 'ssid' +USAGE = 'usage' +CAPACITY = 'capacity' +LOAD = 'load' +TEMPERATURE = 'temp' +VALUE = 'value' +FUNCTION = 'function' + + +class DataChannel: + @staticmethod + def add(data_list, prefix, channel=None, suffix=None, value=None, type=None, unit=None, name=None): + """Create data channel dict and append it to a list""" + data_channel = prefix + if channel is not None: + data_channel += ':' + str(channel) + if suffix is not None: + data_channel += ';' + str(suffix) + data = {} + data['channel'] = data_channel + data['value'] = value + if type is not None: + data['type'] = type + if unit is not None: + data['unit'] = unit + if name is not None: + data['name'] = name + data_list.append(data) + + +class CayenneMQTTClient: + """Cayenne MQTT Client class. + + This is the main client class for connecting to Cayenne and sending and recFUeiving data. + + Standard usage: + * Set on_message callback, if you are receiving data. + * Connect to Cayenne using the begin() function. + * Call loop() at intervals (or loop_forever() once) to perform message processing. + * Send data to Cayenne using write functions: virtualWrite(), celsiusWrite(), etc. + * Receive and process data from Cayenne in the on_message callback. + + The on_message callback can be used by creating a function and assigning it to CayenneMQTTClient.on_message member. + The callback function should have the following signature: on_message(topic, message) + If it exists this callback is used as the default message handler. + """ + client = None + root_topic = "" + connected = False + on_message = None + + def begin(self, username, password, clientid, hostname='mqtt.mydevices.com', port=8883): + """Initializes the client and connects to Cayenne. + + username is the Cayenne username. + password is the Cayenne password. + clientid is the Cayennne client ID for the device. + hostname is the MQTT broker hostname. + port is the MQTT broker port. + """ + self.root_topic = 'v1/{}/things/{}'.format(username, clientid) + self.client = mqtt.Client(client_id=clientid, clean_session=True, userdata=self) + self.client.on_connect = self.connect_callback + self.client.on_disconnect = self.disconnect_callback + self.client.on_message = self.message_callback + self.client.username_pw_set(username, password) + if port != 1883: + self.client.tls_set(ca_certs='/etc/ssl/certs/ca-certificates.crt', tls_version=PROTOCOL_TLSv1_2) + self.client.connect(hostname, port, 60) + info('Connecting to {}:{}'.format(hostname, port)) + + def connect_callback(self, client, userdata, flags, rc): + """The callback for when the client connects to the server. + + client is the client instance for this callback. + userdata is the private user data as set in Client() or userdata_set(). + flags are the response flags sent by the broker. + rc is the connection result. + """ + if rc != 0: + # MQTT broker error codes + broker_errors = { + 1 : 'unacceptable protocol version', + 2 : 'identifier rejected', + 3 : 'server unavailable', + 4 : 'bad user name or password', + 5 : 'not authorized', + } + raise Exception("Connection failed, " + broker_errors.get(rc, "result code " + str(rc))) + else: + info("Connected with result code "+str(rc)) + self.connected = True + # Subscribing in on_connect() means that if we lose the connection and + # reconnect then subscriptions will be renewed. + client.subscribe(self.get_topic_string(COMMAND_TOPIC, True)) + client.subscribe(self.get_topic_string(COMMAND_JSON_TOPIC, False)) + + def disconnect_callback(self, client, userdata, rc): + """The callback for when the client disconnects from the server. + + client is the client instance for this callback. + userdata is the private user data as set in Client() or userdata_set(). + rc is the connection result. + """ + info("Disconnected with result code "+str(rc)) + self.connected = False + reconnected = False + while not reconnected: + try: + self.client.reconnect() + reconnected = True + except: + print("Reconnect failed, retrying") + time.sleep(5) + + def message_callback(self, client, userdata, msg): + """The callback for when a message is received from the server. + + client is the client instance for this callback. + userdata is the private user data as set in Client() or userdata_set(). + msg is the received message. + """ + try: + message = {} + if msg.topic[-len(COMMAND_JSON_TOPIC):] == COMMAND_JSON_TOPIC: + payload = loads(msg.payload.decode()) + message['payload'] = payload['value'] + message['cmdId'] = payload['cmdId'] + channel = payload['channel'].split('/')[-1].split(';') + else: + payload = msg.payload.decode().split(',') + if len(payload) > 1: + message['cmdId'] = payload[0] + message['payload'] = payload[1] + else: + message['payload'] = payload[0] + channel = msg.topic.split('/')[-1].split(';') + message['channel'] = channel[0] + if len(channel) > 1: + message['suffix'] = channel[1] + debug('message_callback: {}'.format(message)) + if self.on_message: + self.on_message(message) + except: + exception('Error processing message: {} {}'.format(msg.topic, str(msg.payload))) + + def get_topic_string(self, topic, append_wildcard=False): + """Return a topic string. + + topic: the topic substring + append_wildcard: if True append the single level topics wildcard (+)""" + if append_wildcard: + return '{}/{}/+'.format(self.root_topic, topic) + else: + return '{}/{}'.format(self.root_topic, topic) + + def disconnect(self): + """Disconnect from Cayenne. + """ + self.client.disconnect() + + def loop(self, timeout=1.0): + """Process Cayenne messages. + + This should be called regularly to ensure Cayenne messages are sent and received. + + timeout: The time in seconds to wait for incoming/outgoing network + traffic before timing out and returning. + """ + self.client.loop(timeout) + + def loop_start(self): + """This is part of the threaded client interface. Call this once to + start a new thread to process network traffic. This provides an + alternative to repeatedly calling loop() yourself. + """ + self.client.loop_start() + + def loop_stop(self): + """This is part of the threaded client interface. Call this once to + stop the network thread previously created with loop_start(). This call + will block until the network thread finishes. + """ + self.client.loop_stop() + + def publish_packet(self, topic, packet, qos=0, retain=False): + """Publish a packet. + + topic: topic substring. + packet: JSON packet to publish. + qos: quality of service level to use. + retain: if True, the message will be set as the "last known good"/retained message for the topic. + """ + debug('Publish to {}'.format(self.get_topic_string(topic))) + self.client.publish(self.get_topic_string(topic), packet, qos, retain) + + def publish_response(self, msg_id, error_message=None): + """Send a command response to Cayenne. + + This should be sent when a command message has been received. + msg_id is the ID of the message received. + error_message is the error message to send. This should be set to None if there is no error. + """ + topic = self.get_topic_string(COMMAND_RESPONSE_TOPIC) + if error_message: + payload = "error,%s=%s" % (msg_id, error_message) + else: + payload = "ok,%s" % (msg_id) + self.client.publish(topic, payload) diff --git a/myDevices/cloud/client.py b/myDevices/cloud/client.py index ffb5e21..0d5f973 100644 --- a/myDevices/cloud/client.py +++ b/myDevices/cloud/client.py @@ -1,1152 +1,573 @@ -from socket import SOCK_STREAM, socket, AF_INET, gethostname, SHUT_RDWR -from ssl import CERT_REQUIRED, wrap_socket +""" +This module contains the main agent client for connecting to the Cayenne server. The client connects +to server, retrives system info as well as sensor and actuator info and sends that data to the server. +It also responds messages from the server, to set actuator values, change system config settings, etc. +""" + from json import dumps, loads -from socket import error as socket_error -from threading import Thread, RLock +from threading import Thread, Event from time import strftime, localtime, tzset, time, sleep from queue import Queue, Empty -from enum import Enum, unique -from ctypes import CDLL, CFUNCTYPE, create_string_buffer, c_char_p, c_bool, c_int, c_void_p +from myDevices import __version__ from myDevices.utils.config import Config -import myDevices.ipgetter from myDevices.utils.logger import exception, info, warn, error, debug, logJson -from myDevices.os import services from myDevices.sensors import sensors -from myDevices.os.hardware import Hardware -from myDevices.wifi import WifiManager -from myDevices.cloud.scheduler import SchedulerEngine +from myDevices.system.hardware import Hardware +# from myDevices.cloud.scheduler import SchedulerEngine from myDevices.cloud.download_speed import DownloadSpeed from myDevices.cloud.updater import Updater -from myDevices.os.raspiconfig import RaspiConfig -from myDevices.os.daemon import Daemon -from myDevices.os.threadpool import ThreadPool -from myDevices.utils.history import History -from select import select -from hashlib import sha256 -from resource import getrusage, RUSAGE_SELF +from myDevices.system.systemconfig import SystemConfig +from myDevices.utils.daemon import Daemon +from myDevices.utils.threadpool import ThreadPool +# from myDevices.utils.history import History +from myDevices.utils.subprocess import executeCommand +# from hashlib import sha256 from myDevices.cloud.apiclient import CayenneApiClient +import myDevices.cloud.cayennemqtt as cayennemqtt -@unique -class PacketTypes(Enum): - PT_ACK= 0 - PT_START_COLLECTION= 1 - PT_STOP_COLLECTION= 2 - PT_UTILIZATION= 3 - PT_SYSTEM_INFO= 4 - PT_PROCESS_LIST= 5 - PT_DRIVE_LIST= 6 - PT_DEFRAG_ANALYSIS= 7 - PT_STARTUP_APPLICATIONS= 8 - PT_DEFRAG_DRIVE= 9 - PT_DEFRAG_COMPLETED= 10 - PT_START_RDS= 11 - PT_STOP_RDS= 12 - PT_START_SCAN= 13 - PT_CHECK_SCAN= 14 - PT_STOP_SCAN= 15 - PT_START_FIX= 16 - PT_CHECK_FIX= 17 - PT_STOP_FIX= 18 - PT_SCAN_RESPONSE= 19 - PT_FIX_RESPONSE= 20 - PT_END_SCAN= 21 - PT_DEFRAG_CHECK= 22 - PT_DEVICE_INFO_MOBILE= 23 - PT_LOCK_DESKTOP= 24 - PT_RESTART_COMPUTER= 25 - PT_SHUTDOWN_COMPUTER= 26 - PT_KILL_PROCESS= 27 - PT_CMD_NOTIFY= 28 - PT_PRINTER_INFO= 29 - PT_PRINT_TEST_PAGE= 30 - PT_ENABLE_FIREWALL= 31 - PT_WINDOWS_UPDATES= 32 - PT_LAUNCH_TASKMGR= 33 - PT_MALWARE_SCAN= 34 - PT_CANCEL_MALWARE_SCAN= 35 - PT_START_RDS_LOCAL_INIT= 36 - PT_MALWARE_GET_ITEMS= 37 - PT_MALWARE_DELETE_ITEMS= 38 - PT_MALWARE_RESTORE_ITEMS= 39 - PT_REQUEST_SCHEDULES= 40 - PT_UPDATE_SCHEDULES= 41 - PT_MALWARE_THREAT_DETECTED= 42 - PT_TOGGLE_MALWARE_SCANNER= 43 - PT_MALWARE_SCANNER_STATE= 44 - PT_AGENT_MESSAGE= 45 - PT_DRIVE_ANALYSIS= 46 - PT_FILE_TRANSFER= 47 - PT_PRINT_JOBS= 48 - PT_DISPLAY_WEB_PAGE= 49 - PT_PRODUCT_INFO= 50 - PT_UNINSTALL_AGENT= 51 - PT_ASK_RDS_CONTROL= 52 - PT_ANS_RDS_CONTROL= 53 - PT_REMOTE_CONTROL_ENDED= 54 - PT_STOP_RD_INVITE= 55 - PT_CLOSE_SALSA_CONNECTION= 56 - PT_INITIALIZED= 57 - PT_LOCATION = 58 - PT_SUPPORTED_SENSORS = 59 - PT_MACHINE_SENSORS = 60 - PT_ADD_SENSOR = 61 - PT_REMOVE_SENSOR = 62 - PT_UPDATE_SENSOR = 63 - PT_DEVICE_COMMAND = 64 - PT_DEVICE_COMMAND_RESPONSE = 65 - PT_ADD_SCHEDULE = 66 - PT_REMOVE_SCHEDULE = 67 - PT_GET_SCHEDULES = 68 - PT_NOTIFICATION = 69 - PT_DATA_CHANGED = 70 - PT_HISTORY_DATA = 71 - PT_HISTORY_DATA_RESPONSE = 72 - PT_DISCOVERY = 73 - PT_AGENT_CONFIGURATION = 74 - - -NETWORK_SETTINGS='/etc/myDevices/Network.ini' -APP_SETTINGS='/etc/myDevices/AppSettings.ini' -GENERAL_SLEEP_THREAD=0.20#was 0.05 - - - -# from contextlib import contextmanager -# -# @contextmanager -#import sys -# def stdout_redirector(stream): -# old_stdout = sys.stdout -# sys.stdout = stream -# try: -# yield -# finally: -# sys.stdout = old_stdout - #with open(filepathdebug, "w+") as f: #replace filepath & filename - # with stdout_redirector(f): - #tracker.print_diff() - #countDebug=countDebug+1 - #print(str(datetime.now()) + ' Count: ' + str(countDebug)) - -# debugCount=0 -# try: -# import resource -# import gc -# gc.enable() -# from objgraph import * -# import random -# from datetime import datetime -# -# #from pympler.tracker import SummaryTracker -# #tracker = SummaryTracker() -# except Exception as e: -# error('failed to load debug modules: ' + str(e)) -debugCount=0 -def Debug(): - try: - global debugCount - debugCount = debugCount + 1 - resUsage=getrusage(RUSAGE_SELF) - size=resUsage.ru_maxrss - info("Memory usage : " + str(debugCount) + " size: " + str(size)) - info("Resouce usage info: " + str(resUsage)) - # Memory leaks display currently commented out - # show_growth() - # obj=get_leaking_objects() - # warn('Leaking objects size='+str(len(obj))) - # filepathdebug='/var/log/myDebug'+str(debugCount) - # with open(filepathdebug, "w+") as f: #replace filepath & filename - # f.write('Debug resouce iteration: ' + str(debugCount) + " size: " + str(size)) - # f.write('Leaking objects size='+str(len(obj)) + '\n') - # f.write('Leaking objects size='+str(typestats()) + '\n') - # f.write('Leaking objects'+str(obj) + '\n') - except Exception as e: - error('failed to track memory: ' + str(e)) +NETWORK_SETTINGS = '/etc/myDevices/Network.ini' +APP_SETTINGS = '/etc/myDevices/AppSettings.ini' +GENERAL_SLEEP_THREAD = 0.20 def GetTime(): + """Return string with the current time""" tzset() - cur=time() - val=strftime("%Y-%m-%dT%T", localtime(cur)) - timezone=strftime("%z", localtime(cur)) - hourtime=int(timezone[1:3]) - timezone=timezone[:1] + str(int(timezone[1:3]))+':'+ timezone[3:7] + cur = time() + val = strftime("%Y-%m-%dT%T", localtime(cur)) + timezone = strftime("%z", localtime(cur)) + hourtime = int(timezone[1:3]) + timezone = timezone[:1] + str(int(timezone[1:3]))+':'+ timezone[3:7] if hourtime == 0: - timezone='' + timezone = '' return val + timezone -class OSInfo(Thread): + +class OSInfo(): + """Class for getting information about the OS""" + def __init__(self): - #debug("OS Info init") + """Initialize variables with OS information""" try: - sleep(GENERAL_SLEEP_THREAD) - f = open('/etc/os-release','r') - for line in f: - splitLine = line.split('=') - if len(splitLine) < 2: - continue - key = splitLine[0].strip() - value = splitLine[1].strip().replace('"', '') - if key=='PRETTY_NAME': - self.PRETTY_NAME = value - continue - if key=='NAME': - self.NAME = value - continue - if key=='VERSION_ID': - self.VERSION_ID = value - continue - if key=='VERSION': - self.VERSION = value - continue - if key=='ID_LIKE': - self.ID_LIKE = value - continue - if key=='ID': - self.ID = value - continue - if key=='ANSI_COLOR': - self.ANSI_COLOR = value - continue - if key=='HOME_URL': - self.HOME_URL = value - continue - f.close() - except: - exception ("OSInfo Unexpected error") -#READER THREAD -class ReaderThread(Thread): - def __init__(self, name, client): - debug('ReaderThread init') - Thread.__init__(self, name=name) - self.cloudClient = client - self.Continue = True - def run(self): - debug('ReaderThread run') - debug('ReaderThread continue?:' + str(self.Continue) ) - while self.Continue: - try: - sleep(GENERAL_SLEEP_THREAD) - if self.cloudClient.connected == False: - continue - #debug('ReaderThread - Reading message') - #self.cloudClient.mutex.acquire() - bReturned = self.cloudClient.ReadMessage() - # if bReturned: - # #debug('ReaderThread process message') - # t1 = Thread(target=self.cloudClient.ProcessMessage) - # t1.start() - except: - exception ("ReaderThread Unexpected error") - return - def stop(self): - debug('ReaderThread stop') - self.Continue = False + with open('/etc/os-release', 'r') as os_file: + for line in os_file: + splitLine = line.split('=') + if len(splitLine) < 2: + continue + key = splitLine[0].strip() + value = splitLine[1].strip().replace('"', '') + keys = ('VERSION_ID', 'ID') + if key in keys: + setattr(self, key, value) + except: + exception("OSInfo Unexpected error") + + class ProcessorThread(Thread): + """Class for processing messages from the server on a thread""" + def __init__(self, name, client): + """Initialize processor thread""" debug('ProcessorThread init') Thread.__init__(self, name=name) self.cloudClient = client self.Continue = True + def run(self): - debug('ProcessorThread run') - debug('ProcessorThread continue?:' + str(self.Continue) ) + """Process messages from the server until the thread is stopped""" + debug('ProcessorThread run, continue: ' + str(self.Continue)) while self.Continue: try: - sleep(GENERAL_SLEEP_THREAD) + if self.cloudClient.exiting.wait(GENERAL_SLEEP_THREAD): + return self.cloudClient.ProcessMessage() except: - exception ("ProcessorThread Unexpected error") + exception("ProcessorThread Unexpected error") return + def stop(self): + """Stop processing messages from the server""" debug('ProcessorThread stop') self.Continue = False -#WRITER THREAD + + class WriterThread(Thread): + """Class for sending messages to the server on a thread""" + def __init__(self, name, client): + """Initialize writer thread""" debug('WriterThread init') Thread.__init__(self, name=name) self.cloudClient = client self.Continue = True + def run(self): + """Send messages to the server until the thread is stopped""" debug('WriterThread run') while self.Continue: - sleep(GENERAL_SLEEP_THREAD) try: - if self.cloudClient.connected == False: - continue - message = self.cloudClient.DequeuePacket() - if not message: + if self.cloudClient.exiting.wait(GENERAL_SLEEP_THREAD): + return + if self.cloudClient.mqttClient.connected == False: + info('WriterThread mqttClient not connected') continue - self.cloudClient.SendMessage(message) - del message - message = None + topic, message = self.cloudClient.DequeuePacket() + if message: + # debug('WriterThread, topic: {} {}'.format(topic, message)) + if not isinstance(message, str): + message = dumps(message) + self.cloudClient.mqttClient.publish_packet(topic, message) + message = None + self.cloudClient.writeQueue.task_done() except: - exception ("WriterThread Unexpected error") + exception("WriterThread Unexpected error") return + def stop(self): + """Stop sending messages to the server""" debug('WriterThread stop') self.Continue = False -#Run function at timed intervals + class TimerThread(Thread): + """Class to run a function on a thread at timed intervals""" + def __init__(self, function, interval, initial_delay=0): + """Set function to run at intervals and start thread""" Thread.__init__(self) self.setDaemon(True) self.function = function self.interval = interval self.initial_delay = initial_delay self.start() + def run(self): + """Run function at intervals""" sleep(self.initial_delay) while True: try: self.function() - sleep(self.interval + GENERAL_SLEEP_THREAD) + sleep(self.interval) except: - exception("TimerThread Unexpected error") - + exception("TimerThread Unexpected error") + + class CloudServerClient: + """Class to connect to the server and send and receive data""" + def __init__(self, host, port, cayenneApiHost): + """Initialize the client configuration""" self.HOST = host self.PORT = port self.CayenneApiHost = cayenneApiHost - self.onMessageReceived = None - self.onMessageSent = None - self.initialized = False - self.machineName = gethostname() self.config = Config(APP_SETTINGS) - inviteCode = self.config.get('Agent', 'InviteCode', fallback=None) - if not inviteCode: - error('No invite code found in {}'.format(APP_SETTINGS)) - print('Please input an invite code. This can be retrieved from the Cayenne dashboard by adding a new Raspberry Pi device.\n' - 'The invite code will be part of the script name shown there: rpi_[invitecode].sh.') - inviteCode = input('Invite code: ') - if inviteCode: - self.config.set('Agent', 'InviteCode', inviteCode) - else: - print('No invite code set, exiting.') - quit() - self.installDate=None - try: - self.installDate = self.config.get('Agent', 'InstallDate', fallback=None) - except: - pass - if not self.installDate: - self.installDate = int(time()) - self.config.set('Agent', 'InstallDate', self.installDate) self.networkConfig = Config(NETWORK_SETTINGS) - #self.defaultRDServer = self.networkConfig.get('CONFIG','RemoteDesktopServerAddress') - self.schedulerEngine = SchedulerEngine(self, 'client_scheduler') - self.Initialize() - self.CheckSubscription() - self.FirstRun() - self.updater = Updater(self.config, self.OnUpdateConfig) - self.updater.start() - self.initialized = True + self.username = self.config.get('Agent', 'Username', None) + self.password = self.config.get('Agent', 'Password', None) + self.clientId = self.config.get('Agent', 'ClientID', None) + self.connected = False + self.exiting = Event() def __del__(self): + """Delete the client""" self.Destroy() - def OnUpdateConfig(self): - pass - # info('Requesting PT_AGENT_CONFIGURATION ') - # data = {} - # data['MachineName'] = self.MachineId - # data['Timestamp'] = int(time()) - # data['Platform'] = 1 # raspberrypi platform id is 1 - # data['PacketType'] = PacketTypes.PT_AGENT_CONFIGURATION.value - # self.EnqueuePacket(data) - def Initialize(self): - #debug('CloudServerClient init') + + def Start(self): + """Connect to server and start background threads""" try: - self.mutex = RLock() + self.installDate=None + try: + self.installDate = self.config.get('Agent', 'InstallDate', fallback=None) + except: + pass + if not self.installDate: + self.installDate = int(time()) + self.config.set('Agent', 'InstallDate', self.installDate) + if not self.username and not self.password and not self.clientId: + self.CheckSubscription() + if not self.Connect(): + error('Error starting agent') + return + # self.schedulerEngine = SchedulerEngine(self, 'client_scheduler') + self.sensorsClient = sensors.SensorsClient() self.readQueue = Queue() self.writeQueue = Queue() - self.pingRate = 10 - self.pingTimeout = 35 - self.waitPing = 0 - self.lastPing = time()-self.pingRate - 1 - self.PublicIP = myDevices.ipgetter.myip() self.hardware = Hardware() self.oSInfo = OSInfo() - self.downloadSpeed = DownloadSpeed(self.config) - self.MachineId = None - self.connected = False - self.exiting = False - self.Start self.count = 10000 self.buff = bytearray(self.count) - #start thread only after init of other fields - self.sensorsClient = sensors.SensorsClient() - self.sensorsClient.SetDataChanged(self.OnDataChanged, self.BuildPT_SYSTEM_INFO) - self.processManager=services.ProcessManager() - self.serviceManager=services.ServiceManager() - self.wifiManager = WifiManager.WifiManager() - self.writerThread = WriterThread('writer',self) + self.downloadSpeed = DownloadSpeed(self.config) + self.downloadSpeed.getDownloadSpeed() + self.sensorsClient.SetDataChanged(self.OnDataChanged) + self.writerThread = WriterThread('writer', self) self.writerThread.start() - self.readerThread = ReaderThread('reader',self) - self.readerThread.start() self.processorThread = ProcessorThread('processor', self) self.processorThread.start() - #TimerThread(self.RequestSchedules, 600, 10) - TimerThread(self.CheckConnectionAndPing, self.pingRate) - self.sentHistoryData = {} - self.historySendFails = 0 - self.historyThread = Thread(target=self.SendHistoryData) - self.historyThread.setDaemon(True) - self.historyThread.start() + TimerThread(self.SendSystemInfo, 300) + TimerThread(self.SendSystemState, 30, 5) + self.updater = Updater(self.config) + self.updater.start() + # self.sentHistoryData = {} + # self.historySendFails = 0 + # self.historyThread = Thread(target=self.SendHistoryData) + # self.historyThread.setDaemon(True) + # self.historyThread.start() except Exception as e: exception('Initialize error: ' + str(e)) + def Destroy(self): + """Destroy client and stop client threads""" info('Shutting down client') - self.exiting = True - self.sensorsClient.StopMonitoring() + self.exiting.set() + if hasattr(self, 'sensorsClient'): + self.sensorsClient.StopMonitoring() if hasattr(self, 'schedulerEngine'): self.schedulerEngine.stop() if hasattr(self, 'updater'): self.updater.stop() if hasattr(self, 'writerThread'): self.writerThread.stop() - if hasattr(self, 'readerThread'): - self.readerThread.stop() if hasattr(self, 'processorThread'): self.processorThread.stop() ThreadPool.Shutdown() - self.Stop() + self.Disconnect() info('Client shut down') - # def Test(self): - # message = {} - # message['PacketType'] = PacketTypes.PT_DEVICE_COMMAND.value - # message['Type'] = '' - # message['Service'] = 'config' - # message['Id']=1021 - # parameters = {} - # parameters['id'] = 16 - # parameters['arguments'] = 'Asia/Tokyo' - # message['Parameters'] = parameters - # self.ExecuteMessage(message) - # message = {} - # message['PacketType'] = PacketTypes.PT_DEVICE_COMMAND.value - # message['Type'] = '' - # message['Service'] = 'config' - # message['Id']=1021 - # parameters = {} - # parameters['id'] = 15 - # parameters['arguments'] = '' - # message['Parameters'] = parameters - # self.ExecuteMessage(message) - # message = {} - # message['PacketType'] = PacketTypes.PT_DEVICE_COMMAND.value - # message['Type'] = '' - # message['Service'] = 'config' - # message['Id']=1021 - # parameters = {} - # parameters['id'] = 0 - # parameters['arguments'] = 'test' - # message['Parameters'] = parameters - # self.ExecuteMessage(message) - def FirstRun(self): - # self.BuildPT_LOCATION() - self.BuildPT_SYSTEM_INFO() - # data = {} - # data['MachineName'] = self.MachineId - # data['Timestamp'] = int(time()) - # data['PacketType'] = PacketTypes.PT_UTILIZATION.value - # self.processManager.RefreshProcessManager() - # data['VisibleMemory'] = 1000000 - # data['AvailableMemory'] = 100000 - # data['AverageProcessorUsage'] = 20 - # data['PeakProcessorUsage'] = 98 - # data['AverageMemoryUsage'] = 30 - # data['PeakMemoryUsage'] = 99 - # data['PercentProcessorTime'] = 80 - # self.EnqueuePacket(data) - # data['PacketType'] = PacketTypes.PT_PROCESS_LIST.value - # self.EnqueuePacket(data) - # data['PacketType'] = PacketTypes.PT_DRIVE_LIST.value - # self.EnqueuePacket(data) - # data['PacketType'] = PacketTypes.PT_PRINTER_INFO.value - # self.EnqueuePacket(data) - self.RequestSchedules() - # self.BuildPT_LOCATION() - self.OnUpdateConfig() - def BuildPT_LOCATION(self): - data = {} - data['position'] = {} - data['position']['latitude'] = '30.022112' - data['position']['longitude'] = '45.022112' - data['position']['accuracy'] = '20' - data['position']['method'] = 'Windows location provider' - data['provider'] = 'other' - data['time'] = int(time()) - data['PacketType'] = PacketTypes.PT_LOCATION.value - data['MachineName'] = self.MachineId - self.EnqueuePacket(data) - def BuildPT_UTILIZATION(self): - #debug('BuildPT_UTILIZATION') - data = {} - data['MachineName'] = self.MachineId - data['Timestamp'] = int(time()) - data['PacketType'] = PacketTypes.PT_UTILIZATION.value - self.processManager.RefreshProcessManager() - data['VisibleMemory'] = self.processManager.VisibleMemory - data['AvailableMemory'] = self.processManager.AvailableMemory - data['AverageProcessorUsage'] = self.processManager.AverageProcessorUsage - data['PeakProcessorUsage'] = self.processManager.PeakProcessorUsage - data['AverageMemoryUsage'] = self.processManager.AverageMemoryUsage - data['PeakMemoryUsage'] = self.processManager.AverageMemoryUsage - data['PercentProcessorTime'] = self.processManager.PercentProcessorTime - self.EnqueuePacket(data) - def OnDataChanged(self, raspberryValue): - data = {} - data['MachineName'] = self.MachineId - data['PacketType'] = PacketTypes.PT_DATA_CHANGED.value - data['Timestamp'] = int(time()) - data['RaspberryInfo'] = raspberryValue + + def OnDataChanged(self, data): + """Enqueue a packet containing changed system data to send to the server""" + info('Send changed data: {}'.format([{item['channel']:item['value']} for item in data])) self.EnqueuePacket(data) - del data - del raspberryValue - def BuildPT_SYSTEM_INFO(self): + + def SendSystemInfo(self): + """Enqueue a packet containing system info to send to the server""" try: - data = {} - data['MachineName'] = self.MachineId - data['PacketType'] = PacketTypes.PT_SYSTEM_INFO.value - data['Timestamp'] = int(time()) - data['IpAddress'] = self.PublicIP - data['GatewayMACAddress'] = self.hardware.getMac() - raspberryValue = {} - raspberryValue['NetworkSpeed'] = str(self.downloadSpeed.getDownloadSpeed()) - raspberryValue['AntiVirus'] = 'None' - raspberryValue['Firewall'] = 'iptables' - raspberryValue['FirewallEnabled'] = 'true' - raspberryValue['ComputerMake'] = self.hardware.getManufacturer() - raspberryValue['ComputerModel'] = self.hardware.getModel() - raspberryValue['OsName'] = self.oSInfo.ID - raspberryValue['OsBuild'] = self.oSInfo.ID_LIKE if hasattr(self.oSInfo, 'ID_LIKE') else self.oSInfo.ID - raspberryValue['OsArchitecture'] = self.hardware.Revision - raspberryValue['OsVersion'] = self.oSInfo.VERSION_ID - raspberryValue['ComputerName'] = self.machineName - raspberryValue['AgentVersion'] = self.config.get('Agent', 'Version', fallback='1.0.1.0') - raspberryValue['InstallDate'] = self.installDate - raspberryValue['GatewayMACAddress'] = self.hardware.getMac() - with self.sensorsClient.sensorMutex: - raspberryValue['SystemInfo'] = self.sensorsClient.currentSystemInfo - raspberryValue['SensorsInfo'] = self.sensorsClient.currentSensorsInfo - raspberryValue['BusInfo'] = self.sensorsClient.currentBusInfo - raspberryValue['OsSettings'] = RaspiConfig.getConfig() - raspberryValue['NetworkId'] = WifiManager.Network.GetNetworkId() - raspberryValue['WifiStatus'] = self.wifiManager.GetStatus() - try: - history = History() - history.SaveAverages(raspberryValue) - except: - exception('History error') - data['RaspberryInfo'] = raspberryValue + data = [] + cayennemqtt.DataChannel.add(data, cayennemqtt.SYS_OS_NAME, value=self.oSInfo.ID) + cayennemqtt.DataChannel.add(data, cayennemqtt.SYS_OS_VERSION, value=self.oSInfo.VERSION_ID) + cayennemqtt.DataChannel.add(data, cayennemqtt.AGENT_VERSION, value=self.config.get('Agent', 'Version', __version__)) + cayennemqtt.DataChannel.add(data, cayennemqtt.SYS_POWER_RESET, value=0) + cayennemqtt.DataChannel.add(data, cayennemqtt.SYS_POWER_HALT, value=0) + config = SystemConfig.getConfig() + if config: + channel_map = {'I2C': cayennemqtt.SYS_I2C, 'SPI': cayennemqtt.SYS_SPI, 'Serial': cayennemqtt.SYS_UART, + 'OneWire': cayennemqtt.SYS_ONEWIRE, 'DeviceTree': cayennemqtt.SYS_DEVICETREE} + for key, channel in channel_map.items(): + try: + cayennemqtt.DataChannel.add(data, channel, value=config[key]) + except: + pass + info('Send system info: {}'.format([{item['channel']:item['value']} for item in data])) + self.EnqueuePacket(data) + except Exception: + exception('SendSystemInfo unexpected error') + + def SendSystemState(self): + """Enqueue a packet containing system information to send to the server""" + try: + data = [] + download_speed = self.downloadSpeed.getDownloadSpeed() + if download_speed: + cayennemqtt.DataChannel.add(data, cayennemqtt.SYS_NET, suffix=cayennemqtt.SPEEDTEST, value=download_speed, type='bw', unit='mbps') + data += self.sensorsClient.systemData + info('Send system state: {} items'.format(len(data))) self.EnqueuePacket(data) - logJson('PT_SYSTEM_INFO: ' + dumps(data), 'PT_SYSTEM_INFO') - del raspberryValue - del data - data=None except Exception as e: exception('ThreadSystemInfo unexpected error: ' + str(e)) - Debug() - def BuildPT_STARTUP_APPLICATIONS(self): - ThreadPool.Submit(self.ThreadServiceManager) - def ThreadServiceManager(self): - self.serviceManager.Run() - sleep(GENERAL_SLEEP_THREAD) - data = {} - data['MachineName'] = self.MachineId - data['PacketType'] = PacketTypes.PT_STARTUP_APPLICATIONS.value - data['ProcessList'] = self.serviceManager.GetServiceList() - self.EnqueuePacket(data) - def BuildPT_PROCESS_LIST(self): - ThreadPool.Submit(self.ThreadProcessManager) - def ThreadProcessManager(self): - self.processManager.Run() - sleep(GENERAL_SLEEP_THREAD) - data = {} - data['MachineName'] = self.MachineId - data['PacketType'] = PacketTypes.PT_PROCESS_LIST.value - data['ProcessList'] = self.processManager.GetProcessList() - self.EnqueuePacket(data) - def ProcessPT_KILL_PROCESS(self, message): - #debug('ProcessPT_KILL_PROCESS') - pid = message['Pid'] - retVal = self.processManager.KillProcess(int(pid)) - data = {} - data['MachineName'] = self.MachineId - data['PacketType'] = PacketTypes.PT_AGENT_MESSAGE.value - data['Type'] = 'Info' - if retVal: - data['Message'] = 'Process Killed!' - else: - data['Message'] = 'Process not Killed!' - self.EnqueuePacket(data) + def CheckSubscription(self): - inviteCode = self.config.get('Agent','InviteCode') + """Check that an invite code is valid""" + inviteCode = self.config.get('Agent', 'InviteCode', fallback=None) + if not inviteCode: + error('No invite code found in {}'.format(APP_SETTINGS)) + print('Please input an invite code. This can be retrieved from the Cayenne dashboard by adding a new Raspberry Pi device.\n' + 'The invite code will be part of the script name shown there: rpi_[invitecode].sh.') + inviteCode = input('Invite code: ') + if inviteCode: + self.config.set('Agent', 'InviteCode', inviteCode) + else: + print('No invite code set, exiting.') + raise SystemExit + inviteCode = self.config.get('Agent', 'InviteCode') cayenneApiClient = CayenneApiClient(self.CayenneApiHost) - authId = cayenneApiClient.loginDevice(inviteCode) - if authId == None: + credentials = cayenneApiClient.loginDevice(inviteCode) + if credentials == None: error('Registration failed for invite code {}, closing the process'.format(inviteCode)) - Daemon.Exit() + raise SystemExit else: - info('Registration succeeded for invite code {}, auth id = {}'.format(inviteCode, authId)) + info('Registration succeeded for invite code {}, credentials = {}'.format(inviteCode, credentials)) self.config.set('Agent', 'Initialized', 'true') - self.MachineId = authId - @property - def Start(self): - #debug('Start') - if self.connected: - ret = False - error('Start already connected') - else: - info('Connecting to: {}:{}'.format(self.HOST, self.PORT)) - count = 0 - with self.mutex: - count+=1 - while self.connected == False and count < 30: - try: - self.sock = None - self.wrappedSocket = None - ##debug('Start wrap_socket') - self.sock = socket(AF_INET, SOCK_STREAM) - #self.wrappedSocket = wrap_socket(self.sock, ca_certs="/etc/myDevices/ca.crt", cert_reqs=CERT_REQUIRED) - self.wrappedSocket = wrap_socket(self.sock) - self.wrappedSocket.connect((self.HOST, self.PORT)) - info('myDevices cloud connected') - self.connected = True - except socket_error as serr: - Daemon.OnFailure('cloud', serr.errno) - error ('Start failed: ' + str(self.HOST) + ':' + str(self.PORT) + ' Error:' + str(serr)) - self.connected = False - sleep(30-count) + try: + self.username = credentials['mqtt']['username'] + self.password = credentials['mqtt']['password'] + self.clientId = credentials['mqtt']['clientId'] + self.config.set('Agent', 'Username', self.username) + self.config.set('Agent', 'Password', self.password) + self.config.set('Agent', 'ClientID', self.clientId) + except: + exception('Invalid credentials, closing the process') + raise SystemExit + + def Connect(self): + """Connect to the server""" + self.connected = False + count = 0 + while self.connected == False and count < 30 and not self.exiting.is_set(): + try: + self.mqttClient = cayennemqtt.CayenneMQTTClient() + self.mqttClient.on_message = self.OnMessage + self.mqttClient.begin(self.username, self.password, self.clientId, self.HOST, self.PORT) + self.mqttClient.loop_start() + self.connected = True + except OSError as oserror: + Daemon.OnFailure('cloud', oserror.errno) + error('Connect failed: ' + str(self.HOST) + ':' + str(self.PORT) + ' Error:' + str(oserror)) + if self.exiting.wait(30): + # If we are exiting return immediately + return self.connected + count += 1 return self.connected - def Stop(self): - #debug('Stop started') + + def Disconnect(self): + """Disconnect from the server""" Daemon.Reset('cloud') - ret = True - if self.connected == False: - ret = False - error('Stop not connected') - else: - with self.mutex: - try: - self.wrappedSocket.shutdown(SHUT_RDWR) - self.wrappedSocket.close() - info('myDevices cloud disconnected') - except socket_error as serr: - debug(str(serr)) - error ('myDevices cloud disconnected error:' + str(serr)) - ret = False - self.connected = False - #debug('Stop finished') - return ret + try: + if hasattr(self, 'mqttClient'): + self.mqttClient.loop_stop() + info('myDevices cloud disconnected') + except: + exception('Error stopping client') + def Restart(self): - if not self.exiting: + """Restart the server connection""" + if not self.exiting.is_set(): debug('Restarting cycle...') sleep(1) - self.Stop() - self.Start + self.Disconnect() + self.Connect() - def SendMessage(self,message): - logJson(message, 'SendMessage') - ret = True - if self.connected == False: - error('SendMessage fail') - ret = False - else: - try: - data = bytes(message, 'UTF-8') - max_size=16383 - if len(data) > max_size: - start = 0 - current=max_size - end = len(data) - self.wrappedSocket.send(data[start:current]) - while current < end: - start = current - current = start + max_size if start + max_size < end else end - self.wrappedSocket.send(data[start:current]) - else: - self.wrappedSocket.send(data) - if self.onMessageSent: - self.onMessageSent(message) - message = None - except socket_error as serr: - error ('SendMessage:' + str(serr)) - ret = False - Daemon.OnFailure('cloud', serr.errno) - sleep(1) - except IOError as ioerr: - debug('IOError: ' + str(ioerr)) - self.Restart() - #Daemon.OnFailure('cloud', ioerr.errno) - except socket_error as serr: - Daemon.OnFailure('cloud', serr.errno) - except: - exception('SendMessage error') - return ret def CheckJson(self, message): + """Check if a JSON message is valid""" try: test = loads(message) except ValueError: return False return True - def ReadMessage(self): - ret = True - if self.connected == False: - ret = False - else: - try: - self.count=4096 - timeout_in_seconds=10 - ready = select([self.wrappedSocket], [], [], timeout_in_seconds) - if ready[0]: - message = self.wrappedSocket.recv(self.count).decode() - buffering = len(message) == 4096 - while buffering and message: - if self.CheckJson(message): - buffering = False - else: - more = self.wrappedSocket.recv(self.count).decode() - if not more: - buffering = False - else: - message += more - try: - if message: - messageObject = loads(message) - del message - self.readQueue.put(messageObject) - else: - error('ReadMessage received empty message string') - except: - exception('ReadMessage error: ' + str(message)) - return False - Daemon.Reset('cloud') - except IOError as ioerr: - debug('IOError: ' + str(ioerr)) - self.Restart() - #Daemon.OnFailure('cloud', ioerr.errno) - except socket_error as serr: - Daemon.OnFailure('cloud', serr.errno) - except: - exception('ReadMessage error') - ret = False - sleep(1) - Daemon.OnFailure('cloud') - return ret + + def OnMessage(self, message): + """Add message from the server to the queue""" + info('OnMessage: {}'.format(message)) + self.readQueue.put(message) + def RunAction(self, action): - #execute action in machine + """Run a specified action""" debug('RunAction') - if 'MachineName' in action and self.MachineId != action['MachineName']: - debug('Scheduler action is not assigned for this machine: ' + str(action)) - return self.ExecuteMessage(action) - def SendNotification(self, notify, subject, body): - info('SendNotification: ' + str(notify) + ' ' + str(subject) + ' ' + str(body)) - try: - data = {} - data['PacketType'] = PacketTypes.PT_NOTIFICATION.value - data['MachineName'] = self.MachineId - data['Subject'] = subject - data['Body'] = body - data['Notify'] = notify - self.EnqueuePacket(data) - except: - debug('') - return False - return True + def ProcessMessage(self): + """Process a message from the server""" try: messageObject = self.readQueue.get(False) if not messageObject: - return False + return False except Empty: return False - with self.mutex: - retVal = self.CheckPT_ACK(messageObject) - if retVal: + self.ExecuteMessage(messageObject) + + def ExecuteMessage(self, message): + """Execute an action described in a message object""" + if not message: return - self.ExecuteMessage(messageObject) - def CheckPT_ACK(self, messageObject): + channel = message['channel'] + info('ExecuteMessage: {}'.format(message)) + if channel in (cayennemqtt.SYS_POWER_RESET, cayennemqtt.SYS_POWER_HALT): + self.ProcessPowerCommand(message) + elif channel.startswith(cayennemqtt.DEV_SENSOR): + self.ProcessSensorCommand(message) + elif channel.startswith(cayennemqtt.SYS_GPIO): + self.ProcessGpioCommand(message) + elif channel == cayennemqtt.AGENT_DEVICES: + self.ProcessDeviceCommand(message) + elif channel in (cayennemqtt.SYS_I2C, cayennemqtt.SYS_SPI, cayennemqtt.SYS_UART, cayennemqtt.SYS_ONEWIRE): + self.ProcessConfigCommand(message) + elif channel == cayennemqtt.AGENT_MANAGE: + self.ProcessAgentCommand(message) + else: + info('Unknown message') + + def ProcessPowerCommand(self, message): + """Process command to reboot/shutdown the system""" + error_message = None try: - packetType = int(messageObject['PacketType']) - if packetType == PacketTypes.PT_ACK.value: - self.lastPing = time() - return True - except: - debug('') - error('CheckPT_ACK failure: ' + str(messageObject)) - return False - def ExecuteMessage(self, messageObject): - if not messageObject: - return - info("ExecuteMessage: " + str(messageObject['PacketType']) ) - packetType = int(messageObject['PacketType']) - if packetType == PacketTypes.PT_UTILIZATION.value: - self.BuildPT_UTILIZATION() - info(PacketTypes.PT_UTILIZATION) - return - if packetType == PacketTypes.PT_SYSTEM_INFO.value: - self.BuildPT_SYSTEM_INFO() - info(PacketTypes.PT_SYSTEM_INFO) - return - if packetType == PacketTypes.PT_UNINSTALL_AGENT.value: - command = "sudo /etc/myDevices/uninstall/uninstall.sh" - services.ServiceManager.ExecuteCommand(command) - return - if packetType == PacketTypes.PT_STARTUP_APPLICATIONS.value: - self.BuildPT_STARTUP_APPLICATIONS() - info(PacketTypes.PT_STARTUP_APPLICATIONS) - return - if packetType == PacketTypes.PT_PROCESS_LIST.value: - self.BuildPT_PROCESS_LIST() - info(PacketTypes.PT_PROCESS_LIST) - return - if packetType == PacketTypes.PT_KILL_PROCESS.value: - self.ProcessPT_KILL_PROCESS(messageObject) - info(PacketTypes.PT_KILL_PROCESS) - return - if packetType == PacketTypes.PT_INITIALIZED.value: - #self.libMYOPX.SetSubscription(messageObject) - info(PacketTypes.PT_INITIALIZED) - return - if packetType == PacketTypes.PT_PRODUCT_INFO.value: - self.config.set('Subscription', 'ProductCode', messageObject['ProductCode']); - info(PacketTypes.PT_PRODUCT_INFO) - return - if packetType == PacketTypes.PT_START_RDS_LOCAL_INIT.value: - error('PT_START_RDS_LOCAL_INIT not implemented') - info(PacketTypes.PT_START_RDS_LOCAL_INIT) - return - if packetType == PacketTypes.PT_RESTART_COMPUTER.value: - info(PacketTypes.PT_RESTART_COMPUTER) - data={} - data['PacketType'] = PacketTypes.PT_AGENT_MESSAGE.value - data['MachineName'] = self.MachineId - data['Message'] = 'Computer Restarted!' - self.EnqueuePacket(data) - command = "sudo shutdown -r now" - services.ServiceManager.ExecuteCommand(command) - return - if packetType == PacketTypes.PT_SHUTDOWN_COMPUTER.value: - info(PacketTypes.PT_SHUTDOWN_COMPUTER) - data={} - data['PacketType'] = PacketTypes.PT_AGENT_MESSAGE.value - data['MachineName'] = self.MachineId - data['Message'] = 'Computer Powered Off!' + self.EnqueueCommandResponse(message, error_message) + commands = {cayennemqtt.SYS_POWER_RESET: 'sudo shutdown -r now', cayennemqtt.SYS_POWER_HALT: 'sudo shutdown -h now'} + if int(message['payload']) == 1: + debug('Processing power command') + data = [] + cayennemqtt.DataChannel.add(data, message['channel'], value=1) + self.EnqueuePacket(data) + self.writeQueue.join() + output, result = executeCommand(commands[message['channel']]) + debug('ProcessPowerCommand: {}, result: {}, output: {}'.format(message, result, output)) + if result != 0: + error_message = 'Error executing shutdown command' + except Exception as ex: + error_message = '{}: {}'.format(type(ex).__name__, ex) + if error_message: + error(error_message) + data = [] + cayennemqtt.DataChannel.add(data, message['channel'], value=0) self.EnqueuePacket(data) - command = "sudo shutdown -h now" - services.ServiceManager.ExecuteCommand(command) - return - if packetType == PacketTypes.PT_SUPPORTED_SENSORS.value: - self.sensorsClient.SupportedSensorsUpdate(messageObject) - info(PacketTypes.PT_SUPPORTED_SENSORS) - return - if packetType == PacketTypes.PT_MACHINE_SENSORS.value: - self.sensorsClient.OnDbSensors(messageObject) - info(PacketTypes.PT_MACHINE_SENSORS) - return - if packetType == PacketTypes.PT_AGENT_CONFIGURATION.value: - info('PT_AGENT_CONFIGURATION: ' + str(messageObject.Data)) - self.config.setCloudConfig(messageObject.Data) - return - if packetType == PacketTypes.PT_ADD_SENSOR.value: - try: - info(PacketTypes.PT_ADD_SENSOR) - parameters = None - deviceName = None - deviceClass = None - description = None - #for backward compatibility check the DisplayName and overwrite it over the other variables - displayName = None - if 'DisplayName' in messageObject: - displayName = messageObject['DisplayName'] - if 'Parameters' in messageObject: - parameters = messageObject['Parameters'] + def ProcessAgentCommand(self, message): + """Process command to manage the agent state""" + error = None + try: + if message['suffix'] == 'uninstall': + output, result = executeCommand('sudo -n /etc/myDevices/uninstall/uninstall.sh', disablePipe=True) + debug('ProcessAgentCommand: {}, result: {}, output: {}'.format(message, result, output)) + if result != 0: + error = 'Error uninstalling agent' + # elif message['suffix'] == 'config': + # for key, value in message['payload'].items(): + # if value is None: + # info('Remove config item: {}'.format(key)) + # self.config.remove('Agent', key) + # else: + # info('Set config item: {} {}'.format(key, value)) + # self.config.set('Agent', key, value) + else: + error = 'Unknown agent command: {}'.format(message['suffix']) + except Exception as ex: + error = '{}: {}'.format(type(ex).__name__, ex) + self.EnqueueCommandResponse(message, error) - if 'DeviceName' in messageObject: - deviceName = messageObject['DeviceName'] - else: - deviceName = displayName + def ProcessConfigCommand(self, message): + """Process system configuration command""" + error = None + try: + value = 1 - int(message['payload']) #Invert the value since the config script uses 0 for enable and 1 for disable + command_id = {cayennemqtt.SYS_I2C: 11, cayennemqtt.SYS_SPI: 12, cayennemqtt.SYS_UART: 13, cayennemqtt.SYS_ONEWIRE: 19} + result, output = SystemConfig.ExecuteConfigCommand(command_id[message['channel']], value) + debug('ProcessConfigCommand: {}, result: {}, output: {}'.format(message, result, output)) + if result != 0: + error = 'Error executing config command' + except Exception as ex: + error = '{}: {}'.format(type(ex).__name__, ex) + self.EnqueueCommandResponse(message, error) + + def ProcessGpioCommand(self, message): + """Process GPIO command""" + error = None + try: + channel = int(message['channel'].replace(cayennemqtt.SYS_GPIO + ':', '')) + result = self.sensorsClient.GpioCommand(message.get('suffix', 'value'), channel, message['payload']) + debug('ProcessGpioCommand result: {}'.format(result)) + if result == 'failure': + error = 'GPIO command failed' + except Exception as ex: + error = '{}: {}'.format(type(ex).__name__, ex) + self.EnqueueCommandResponse(message, error) - if 'Description' in messageObject: - description = messageObject['Description'] - else: - description = deviceName - - if 'Class' in messageObject: - deviceClass = messageObject['Class'] - - retValue = True - retValue = self.sensorsClient.AddSensor(deviceName, description, deviceClass, parameters) - except Exception as ex: - exception ("PT_ADD_SENSOR Unexpected error"+ str(ex)) - retValue = False - data = {} - if 'Id' in messageObject: - data['Id'] = messageObject['Id'] - #0 - None, 1 - Pending, 2-Success, 3 - Not responding, 4 - Failure - if retValue: - data['State'] = 2 - else: - data['State'] = 4 - data['PacketType'] = PacketTypes.PT_UPDATE_SENSOR.value - data['MachineName'] = self.MachineId - self.EnqueuePacket(data) - return - if packetType == PacketTypes.PT_REMOVE_SENSOR.value: - try: - info(PacketTypes.PT_REMOVE_SENSOR) - retValue = False - if 'Name' in messageObject: - Name = messageObject['Name'] - retValue = self.sensorsClient.RemoveSensor(Name) - data = {} - data['Name'] = Name - data['PacketType'] = PacketTypes.PT_REMOVE_SENSOR.value - data['MachineName'] = self.MachineId - data['Response'] = retValue - self.EnqueuePacket(data) - except Exception as ex: - exception ("PT_REMOVE_SENSOR Unexpected error"+ str(ex)) - retValue = False - return - if packetType == PacketTypes.PT_DEVICE_COMMAND.value: - info(PacketTypes.PT_DEVICE_COMMAND) - self.ProcessDeviceCommand(messageObject) - return - if packetType == PacketTypes.PT_ADD_SCHEDULE.value: - info(PacketTypes.PT_ADD_SCHEDULE.value) - retVal = self.schedulerEngine.AddScheduledItem(messageObject, True) - if 'Update' in messageObject: - messageObject['Update'] = messageObject['Update'] - messageObject['PacketType'] = PacketTypes.PT_ADD_SCHEDULE.value - messageObject['MachineName'] = self.MachineId - messageObject['Status'] = str(retVal) - self.EnqueuePacket(messageObject) - return - if packetType == PacketTypes.PT_REMOVE_SCHEDULE.value: - info(PacketTypes.PT_REMOVE_SCHEDULE) - retVal = self.schedulerEngine.RemoveScheduledItem(messageObject) - messageObject['PacketType'] = PacketTypes.PT_REMOVE_SCHEDULE.value - messageObject['MachineName'] = self.MachineId - messageObject['Status'] = str(retVal) - self.EnqueuePacket(messageObject) - return - if packetType == PacketTypes.PT_GET_SCHEDULES.value: - info(PacketTypes.PT_GET_SCHEDULES) - schedulesJson = self.schedulerEngine.GetSchedules() - data['Schedules'] = schedulesJson - data['PacketType'] = PacketTypes.PT_GET_SCHEDULES.value - data['MachineName'] = self.MachineId - self.EnqueuePacket(data) - return - if packetType == PacketTypes.PT_UPDATE_SCHEDULES.value: - info(PacketTypes.PT_UPDATE_SCHEDULES) - retVal = self.schedulerEngine.UpdateSchedules(messageObject) - return - if packetType == PacketTypes.PT_HISTORY_DATA_RESPONSE.value: - info(PacketTypes.PT_HISTORY_DATA_RESPONSE) - try: - id = messageObject['Id'] - history = History() - if messageObject['Status']: - history.Sent(True, self.sentHistoryData[id]['HistoryData']) - self.historySendFails = 0 - else: - history.Sent(False, self.sentHistoryData[id]['HistoryData']) - self.historySendFails += 1 - del self.sentHistoryData[id] - except: - exception('Processing history response packet failed') - return - info("Skipping not required packet: " + str(packetType)) - def ProcessDeviceCommand(self, messageObject): - # t1 = Thread(target=self.ThreadDeviceCommand) - # t1.start() - # def ThreadDeviceCommand(self): - commandType = messageObject['Type'] - commandService = messageObject['Service'] - parameters = messageObject['Parameters'] - info('PT_DEVICE_COMMAND: ' + dumps(messageObject)) - debug('ProcessDeviceCommand: ' + commandType + ' ' + commandService + ' ' + str(parameters)) - id = messageObject['Id'] - sensorId = None - if 'SensorId' in messageObject: - sensorId = messageObject['SensorId'] - data = {} - retValue = '' - if commandService == 'wifi': - if commandType == 'status': - retValue = self.wifiManager.GetStatus() - if commandType == 'scan': - retValue = self.wifiManager.GetWirelessNetworks() - if commandType == 'setup': - try: - ssid = parameters["ssid"] - password = parameters["password"] - interface = parameters["interface"] - retValue = self.wifiManager.Setup(ssid, password, interface) - except: - retValue = False - if commandService == 'services': - serviceName = parameters['ServiceName'] - if commandType == 'status': - retValue = self.serviceManager.Status(serviceName) - if commandType == 'start': - retValue = self.serviceManager.Start(serviceName) - if commandType == 'stop': - retValue = self.serviceManager.Stop(serviceName) - if commandService == 'sensor': - debug('SENSOR_COMMAND processing: ' + str(parameters)) - method = None + def ProcessSensorCommand(self, message): + """Process sensor command""" + error = None + try: + sensor_info = message['channel'].replace(cayennemqtt.DEV_SENSOR + ':', '').split(':') + sensor = sensor_info[0] channel = None - value = None - driverClass = None - sensorType = None - sensorName = None - if 'SensorName' in parameters: - sensorName = parameters["SensorName"] - if 'DriverClass' in parameters: - driverClass = parameters["DriverClass"] - if commandType == 'enable': - sensor = None - enable = None - if 'Sensor' in parameters: - sensor = parameters["Sensor"] - if 'Enable' in parameters: - enable = parameters["Enable"] - retValue = self.sensorsClient.EnableSensor(sensor, enable) + if len(sensor_info) > 1: + channel = sensor_info[1] + result = self.sensorsClient.SensorCommand(message.get('suffix', 'value'), sensor, channel, message['payload']) + debug('ProcessSensorCommand result: {}'.format(result)) + if result is False: + error = 'Sensor command failed' + except Exception as ex: + error = '{}: {}'.format(type(ex).__name__, ex) + self.EnqueueCommandResponse(message, error) + + def ProcessDeviceCommand(self, message): + """Process a device command to add/edit/remove a sensor""" + error = None + try: + payload = message['payload'] + info('ProcessDeviceCommand payload: {}'.format(payload)) + if message['suffix'] == 'add': + result = self.sensorsClient.AddSensor(payload['sensorId'], payload['description'], payload['class'], payload['args']) + elif message['suffix'] == 'edit': + result = self.sensorsClient.EditSensor(payload['sensorId'], payload['description'], payload['class'], payload['args']) + elif message['suffix'] == 'delete': + result = self.sensorsClient.RemoveSensor(payload['sensorId']) else: - if commandType == 'edit': - description = sensorName - device = None - if "Description" in parameters: - description=parameters["Description"] - if "Args" in parameters: - args=parameters["Args"] - retValue = self.sensorsClient.EditSensor(sensorName, description, driverClass, args) - else: - if 'Channel' in parameters: - channel = parameters["Channel"] - if 'Method' in parameters: - method = parameters["Method"] - if 'Value' in parameters: - value = parameters["Value"] - if 'SensorType' in parameters: - sensorType = parameters["SensorType"] - #(self, commandType, sensorName, sensorType, driverClass, method, channel, value): - retValue = self.sensorsClient.SensorCommand(commandType, sensorName, sensorType, driverClass, method, channel, value) - if commandService == 'gpio': - method = parameters["Method"] - channel = parameters["Channel"] - value = parameters["Value"] - debug('ProcessDeviceCommand: ' + commandService + ' ' + method + ' ' + str(channel) + ' ' + str(value)) - retValue = str(self.sensorsClient.GpioCommand(commandType, method, channel, value)) - debug('ProcessDeviceCommand gpio returned value: ' + retValue) - if commandService == 'config': - try: - config_id = parameters["id"] - arguments = parameters["arguments"] - (retValue, output) = RaspiConfig.Config(config_id, arguments) - data["Output"] = output - retValue = str(retValue) - except: - exception ("Exception on config") - data['Response'] = retValue - data['Id'] = id - data['PacketType'] = PacketTypes.PT_DEVICE_COMMAND_RESPONSE.value - data['MachineName'] = self.MachineId - info('PT_DEVICE_COMMAND_RESPONSE: ' + dumps(data)) - if sensorId: - data['SensorId'] = sensorId - self.EnqueuePacket(data) - #if commandService == 'processes': #Kill command is handled with PT_KILL_PROCESS - def EnqueuePacket(self,message): - message['PacketTime'] = GetTime() - #datetime.now().strftime("%Y-%m-%dT%H:%M:%S%z") - json_data = dumps(message)+ '\n' - message = None - #debug(json_data) - self.writeQueue.put(json_data) + error = 'Unknown device command: {}'.format(message['suffix']) + debug('ProcessDeviceCommand result: {}'.format(result)) + if result is False: + error = 'Device command failed' + except Exception as ex: + error = '{}: {}'.format(type(ex).__name__, ex) + self.EnqueueCommandResponse(message, error) + + def EnqueueCommandResponse(self, message, error): + """Send response after processing a command message""" + debug('EnqueueCommandResponse error: {}'.format(error)) + if error: + response = 'error,{}={}'.format(message['cmdId'], error) + else: + response = 'ok,{}'.format(message['cmdId']) + info(response) + self.EnqueuePacket(response, cayennemqtt.COMMAND_RESPONSE_TOPIC) + + def EnqueuePacket(self, message, topic=cayennemqtt.DATA_TOPIC): + """Enqueue a message packet to send to the server""" + packet = (topic, message) + self.writeQueue.put(packet) + def DequeuePacket(self): - packet = None + """Dequeue a message packet to send to the server""" + packet = (None, None) try: - packet = self.writeQueue.get() + packet = self.writeQueue.get(False) except Empty: - packet = None + pass return packet - def CheckConnectionAndPing(self): - ticksStart = time() - with self.mutex: - try: - if(ticksStart - self.lastPing > self.pingTimeout): - #debug('CheckConnectionAndPing EXPIRED - trying to reconnect') - self.Stop() - self.Start - self.lastPing = time() - self.pingRate - 1 - warn('Restarting cloud connection -> CheckConnectionAndPing EXPIRED: ' + str(self.lastPing)) - if (ticksStart - self.waitPing >= self.pingRate): - #debug("CheckConnectionAndPing sending ACK packet") - self.SendAckPacket() - except: - debug('') - error('CheckConnectionAndPing error') - def SendAckPacket(self): - data = {} - debug('Last ping: ' + str(self.lastPing) + ' Wait ping: ' + str(self.waitPing)) - data['MachineName'] = self.MachineId - data['IPAddress'] = self.PublicIP - data['PacketType'] = PacketTypes.PT_ACK.value - self.EnqueuePacket(data) - self.waitPing = time() - def RequestSchedules(self): - data = {} - data['MachineName'] = self.MachineId - data['Stored'] = "dynamodb" - data['PacketType'] = PacketTypes.PT_REQUEST_SCHEDULES.value - self.EnqueuePacket(data) - def SendHistoryData(self): - try: - info('SendHistoryData start') - history = History() - history.Reset() - while True: - try: - #If there is no acknowledgment after a minute we assume failure - sendFailed = [key for key, item in self.sentHistoryData.items() if (item['Timestamp'] + 60) < time()] - info('SendHistoryData previously SendFailed items: ' + str(sendFailed)) - for id in sendFailed: - self.historySendFails += len(sendFailed) - history.Sent(False, self.sentHistoryData[id]['HistoryData']) - del self.sentHistoryData[id] - historyData = history.GetHistoricalData() - if historyData: - data = {} - info('SendHistoryData historyData: ' + str(historyData)) - data['MachineName'] = self.MachineId - data['Timestamp'] = int(time()) - data['PacketType'] = PacketTypes.PT_HISTORY_DATA.value - id = sha256(dumps(historyData).encode('utf8')).hexdigest() - data['Id'] = id - data['HistoryData'] = historyData - info('Sending history data, id = {}'.format(id)) - debug('SendHistoryData historyData: ' + str(data)) - self.EnqueuePacket(data) - #this will keep acumulating - self.sentHistoryData[id] = data - except Exception as ex: - exception('SendHistoryData error' + str(ex)) - delay = 60 - if self.historySendFails > 2: - delay = 120 - if self.historySendFails > 4: - #Wait an hour if we keep getting send failures. - delay = 3600 - self.historySendFails = 0 - sleep(delay) - except Exception as ex: - exception('SendHistoryData general exception: ' + str(ex)) + + # def SendHistoryData(self): + # """Enqueue a packet containing historical data to send to the server""" + # try: + # info('SendHistoryData start') + # history = History() + # history.Reset() + # while True: + # try: + # #If there is no acknowledgment after a minute we assume failure + # sendFailed = [key for key, item in self.sentHistoryData.items() if (item['Timestamp'] + 60) < time()] + # info('SendHistoryData previously SendFailed items: ' + str(sendFailed)) + # for id in sendFailed: + # self.historySendFails += len(sendFailed) + # history.Sent(False, self.sentHistoryData[id]['HistoryData']) + # del self.sentHistoryData[id] + # historyData = history.GetHistoricalData() + # if historyData: + # data = {} + # info('SendHistoryData historyData: ' + str(historyData)) + # data['MachineName'] = self.MachineId + # data['Timestamp'] = int(time()) + # data['PacketType'] = PacketTypes.PT_HISTORY_DATA.value + # id = sha256(dumps(historyData).encode('utf8')).hexdigest() + # data['Id'] = id + # data['HistoryData'] = historyData + # info('Sending history data, id = {}'.format(id)) + # debug('SendHistoryData historyData: ' + str(data)) + # self.EnqueuePacket(data) + # #this will keep accumulating + # self.sentHistoryData[id] = data + # except Exception as ex: + # exception('SendHistoryData error' + str(ex)) + # delay = 60 + # if self.historySendFails > 2: + # delay = 120 + # if self.historySendFails > 4: + # #Wait an hour if we keep getting send failures. + # delay = 3600 + # self.historySendFails = 0 + # sleep(delay) + # except Exception as ex: + # exception('SendHistoryData general exception: ' + str(ex)) \ No newline at end of file diff --git a/myDevices/cloud/download_speed.py b/myDevices/cloud/download_speed.py index 96b4f5f..7ab5eba 100644 --- a/myDevices/cloud/download_speed.py +++ b/myDevices/cloud/download_speed.py @@ -1,13 +1,15 @@ +""" +This module provides a class for testing download speed +""" from datetime import datetime, timedelta from os import path, remove from urllib import request, error from myDevices.utils.logger import exception, info, warn, error, debug -from threading import Thread from time import sleep from random import randint from socket import error as socket_error -from myDevices.os.daemon import Daemon -from myDevices.os.threadpool import ThreadPool +from myDevices.utils.daemon import Daemon +from myDevices.utils.threadpool import ThreadPool defaultUrl = "https://updates.mydevices.com/test/10MB.zip" download_path = "/etc/myDevices/test" @@ -16,69 +18,73 @@ defaultDownloadRate = 24*60*60 class DownloadSpeed(): + """Class for checking download speed""" + def __init__(self, config): + """Initialize variables and start download speed test""" self.downloadSpeed = None - self.uploadSpeed = None self.testTime = None self.isRunning = False self.Start() self.config = config #add a random delay to the start of download - self.delay = randint(0,100) + self.delay = randint(0, 100) + def Start(self): - #thread = Thread(target = self.Test) - #thread.start() + """Start download speed thread""" ThreadPool.Submit(self.Test) + def Test(self): + """Run speed test""" if self.isRunning: return False self.isRunning = True sleep(1) self.TestDownload() sleep(1) - self.TestUpload() self.testTime = datetime.now() self.isRunning = False return True + def TestDownload(self): + """Test download speed by retrieving a file""" try: - a = datetime.now() - info('Excuting regular download test for network speed') - url = self.config.cloudConfig.DownloadSpeedTestUrl if 'DownloadSpeedTestUrl' in self.config.cloudConfig else defaultUrl + info('Executing regular download test for network speed') + url = self.config.get('Agent', 'DownloadSpeedTestUrl', defaultUrl) debug(url + ' ' + download_path) + a = datetime.now() request.urlretrieve(url, download_path) - request.urlcleanup() b = datetime.now() + request.urlcleanup() c = b - a if path.exists(download_path): size = path.getsize(download_path)/mb - self.downloadSpeed = size/c.total_seconds() + self.downloadSpeed = size/c.total_seconds() * 8 #Convert to megabits per second remove(download_path) return True except socket_error as serr: - error ('TestDownload:' + str(serr)) - ret = False - Daemon.OnFailure('cloud', serr.errno) - return + error ('TestDownload:' + str(serr)) + ret = False + Daemon.OnFailure('cloud', serr.errno) + return except: exception('TestDownload Failed') return False - def TestUpload(self): - debug('Network Speed TestUpload - Not implemented') - self.uploadSpeed = 0 + def IsDownloadTime(self): + """Return true if it is time to run a new download speed test""" if self.testTime is None: return True - downloadRate = int(self.config.cloudConfig.DownloadSpeedTestRate) if 'DownloadSpeedTestRate' in self.config.cloudConfig else defaultDownloadRate - if self.testTime +timedelta(seconds=downloadRate+self.delay ) < datetime.now(): + downloadRate = self.config.getInt('Agent', 'DownloadSpeedTestRate', defaultDownloadRate) + if self.testTime + timedelta(seconds=downloadRate+self.delay ) < datetime.now(): return True return False + def getDownloadSpeed(self): + """Start a new download speed test if necessary and return the download speed""" if self.IsDownloadTime(): self.Start() return self.downloadSpeed - def getUploadSpeed(self): - return self.uploadSpeed def Test(): from myDevices.utils.config import Config diff --git a/myDevices/cloud/scheduler.py b/myDevices/cloud/scheduler.py index 5c96128..f29a944 100644 --- a/myDevices/cloud/scheduler.py +++ b/myDevices/cloud/scheduler.py @@ -193,7 +193,7 @@ def ProcessAction(self, scheduleItem): subject = scheduleItem.title #build an array of device names #if this fails to be sent, save it in the DB and resubmit it - runStatus = self.client.SendNotification(scheduleItem.notify, subject, body) + runStatus = False #self.client.SendNotification(scheduleItem.notify, subject, body) sleep(1) if runStatus == False: error('Notification ' + str(scheduleItem.notify) + ' was not sent') diff --git a/myDevices/cloud/updater.py b/myDevices/cloud/updater.py index 932ac0c..19f9bd0 100644 --- a/myDevices/cloud/updater.py +++ b/myDevices/cloud/updater.py @@ -1,7 +1,6 @@ from myDevices.utils.logger import exception, info, warn, error, debug, setDebug from time import time, sleep from sched import scheduler -from myDevices.os import services from distutils.version import LooseVersion, StrictVersion from os import mkdir, path from threading import Thread @@ -9,16 +8,17 @@ from datetime import datetime, timedelta import random from myDevices.utils.config import Config +from myDevices.utils.subprocess import executeCommand -SETUP_NAME = 'myDevicesSetup_raspberrypi.sh' +SETUP_NAME = 'update_raspberrypi.sh' INSTALL_PATH = '/etc/myDevices/' UPDATE_PATH = INSTALL_PATH + 'updates/' -UPDATE_CFG = UPDATE_PATH + 'update' +UPDATE_CFG = UPDATE_PATH + 'updatecfg' SETUP_PATH = UPDATE_PATH + SETUP_NAME -TIME_TO_CHECK = 60 + random.randint(60, 300) #seconds - at least 2 minutes or +TIME_TO_CHECK = 60 + random.randint(60, 300) #seconds - at least 2 minutes TIME_TO_SLEEP = 60 -UPDATE_URL = 'https://updates.mydevices.com/raspberry/update' +UPDATE_URL = 'https://updates.mydevices.com/raspberry/updatecfg' SETUP_URL = 'https://updates.mydevices.com/raspberry/' try: @@ -38,29 +38,21 @@ def __init__(self, config, onUpdateConfig = None): self.setDaemon(True) self.appSettings = config self.onUpdateConfig = onUpdateConfig - self.env = self.appSettings.get('Agent','Environment', fallback='live') + self.env = self.appSettings.get('Agent', 'Environment', fallback='live') global SETUP_URL global UPDATE_URL - global TIME_TO_CHECK - - if self.env == "live": + if self.env == 'live': SETUP_URL = SETUP_URL + SETUP_NAME else: - SETUP_URL = SETUP_URL + self.env + "_" + SETUP_NAME + SETUP_URL = SETUP_URL + self.env + '_' + SETUP_NAME UPDATE_URL = UPDATE_URL + self.env - - if 'UpdateUrl' in config.cloudConfig: - UPDATE_URL = config.cloudConfig.UpdateUrl - if 'UpdateCheckRate' in config.cloudConfig: - interval = int(config.cloudConfig.UpdateCheckRate) - TIME_TO_CHECK = interval + random.randint(0, interval*10) - if 'SetupUrl' in config.cloudConfig: - SETUP_URL = config.cloudConfig.SetupUrl + # UPDATE_URL = self.appSettings.get('Agent', 'UpdateUrl', UPDATE_URL) + # SETUP_URL = self.appSettings.get('Agent', 'SetupUrl', SETUP_URL) self.scheduler = scheduler(time, sleep) self.Continue = True self.currentVersion = '' self.newVersion = '' - self.downloadUrl = '' + # self.downloadUrl = '' self.UpdateCleanup() self.startTime = datetime.now() - timedelta(days=1) @@ -98,29 +90,26 @@ def CheckUpdate(self): elapsedTime=now-self.startTime if elapsedTime.total_seconds() < TIME_TO_CHECK: return - self.startTime = datetime.now() if path.exists(UPDATE_PATH) == True: error('myDevices updater another update in progress') return sleep(1) # Run the update as root - retCode = services.ServiceManager.ExecuteCommand("sudo python3 -m myDevices.cloud.doupdatecheck") + executeCommand('sudo python3 -m myDevices.cloud.doupdatecheck') def DoUpdateCheck(self): mkdir(UPDATE_PATH) sleep(1) try: - self.currentVersion = self.appSettings.get('Agent', 'Version', fallback='1.0.1.0') + self.currentVersion = self.appSettings.get('Agent', 'Version', fallback=None) except: - error('Updater Current Version not found') - + error('Error getting agent version') sleep(1) if not self.currentVersion: - error('Current version not available. Cannot update agent.') + info('Current version not available. Cannot update agent.') self.UpdateCleanup() return - retValue = self.RetrieveUpdate() sleep(1) if retValue is False: @@ -129,7 +118,6 @@ def DoUpdateCheck(self): return sleep(1) retValue = self.CheckVersion(self.currentVersion, self.newVersion) - if retValue is True: info('Update needed, current version: {}, update version: {}'.format(self.currentVersion, self.newVersion)) retValue = self.ExecuteUpdate() @@ -143,23 +131,27 @@ def DoUpdateCheck(self): self.UpdateCleanup() def SetupUpdater(self): + # global TIME_TO_CHECK + # TIME_TO_CHECK = self.appSettings.get('Agent', 'UpdateCheckRate', TIME_TO_CHECK) self.scheduler.enter(TIME_TO_CHECK, 1, self.CheckUpdate, ()) def RetrieveUpdate(self): try: info('Checking update version') - debug( UPDATE_URL + ' ' + UPDATE_CFG ) + debug('Retrieve update config: {} {}'.format(UPDATE_URL, UPDATE_CFG)) retValue = self.DownloadFile(UPDATE_URL, UPDATE_CFG) if retValue is False: - error('failed to download update file') + error('Failed to download update file') return retValue updateConfig = Config(UPDATE_CFG) try: self.newVersion = updateConfig.get('UPDATES','Version') - self.downloadUrl = updateConfig.get('UPDATES','Url') except: - error('Updater missing: update version or Url') - + error('Updater missing version') + # try: + # self.downloadUrl = updateConfig.get('UPDATES','Url') + # except: + # error('Updater missing url') info('Updater retrieve update success') return True except: @@ -168,7 +160,7 @@ def RetrieveUpdate(self): def DownloadFile(self, url, localPath): try: - info( url + ' ' + localPath) + info(url + ' ' + localPath) with urlopen(url) as response: with open(localPath, 'wb') as output: output.write(response.read()) @@ -179,16 +171,16 @@ def DownloadFile(self, url, localPath): return False def ExecuteUpdate(self): - debug('') + debug('Execute update: {} {}'.format(SETUP_URL, SETUP_PATH)) retValue = self.DownloadFile(SETUP_URL, SETUP_PATH) if retValue is False: return retValue command = "chmod +x " + SETUP_PATH - (output, returncode) = services.ServiceManager.ExecuteCommand(command) + (output, returncode) = executeCommand(command) del output command = "nohup " + SETUP_PATH + ' -update >/var/log/myDevices/myDevices.update 2>/var/log/myDevices/myDevices.update.err' debug('execute command started: {}'.format(command)) - (output, returncode) = services.ServiceManager.ExecuteCommand(command) + (output, returncode) = executeCommand(command) del output debug('Updater execute command finished') diff --git a/myDevices/devices/bus.py b/myDevices/devices/bus.py index 5281514..348aba5 100644 --- a/myDevices/devices/bus.py +++ b/myDevices/devices/bus.py @@ -17,32 +17,71 @@ import subprocess from myDevices.utils.logger import debug, info -from myDevices.utils.version import OS_VERSION, OS_RASPBIAN_JESSIE, OS_RASPBIAN_WHEEZY - -BUSLIST = { - "I2C": { - "enabled": False, - "gpio": {0:"SDA", 1:"SCL", 2:"SDA", 3:"SCL"}, - "modules": ["i2c-bcm2708", "i2c-dev"] - }, - - "SPI": { - "enabled": False, - "gpio": {7:"CE1", 8:"CE0", 9:"MISO", 10:"MOSI", 11:"SCLK"}, - "modules": ["spi-bcm2708" if OS_VERSION == OS_RASPBIAN_WHEEZY else "spi-bcm2835"] - }, - - "UART": { - "enabled": False, - "gpio": {14:"TX", 15:"RX"} - }, - - "ONEWIRE": { - "enabled": False, - "gpio": {4:"DATA"}, - "modules": ["w1-gpio"], - "wait": 2} -} +from myDevices.system.version import OS_VERSION, OS_JESSIE, OS_WHEEZY +from myDevices.system.hardware import Hardware + +hardware = Hardware() +if hardware.isTinkerBoard(): + # Tinker Board only supports I2C and SPI for now. These are enabled by default and + # don't need to load any modules. + BUSLIST = { + "I2C": { + "enabled": True, + }, + + "SPI": { + "enabled": True, + } + } +elif hardware.isBeagleBone(): + BUSLIST = { + "I2C": { + "enabled": True, + }, + + "SPI": { + "enabled": False, + "gpio": {17:"SPI0_CS", 18:"SPI0_D1", 21:"SPI0_D1", 22:"SPI0_SCLK"}, + "configure_pin_command": "sudo config-pin P9.{} spi" + } + } +else: + # Raspberry Pi + BUSLIST = { + "I2C": { + "enabled": False, + "gpio": {0:"SDA", 1:"SCL", 2:"SDA", 3:"SCL"}, + "modules": ["i2c-bcm2708", "i2c-dev"] + }, + + "SPI": { + "enabled": False, + "gpio": {7:"CE1", 8:"CE0", 9:"MISO", 10:"MOSI", 11:"SCLK"}, + "modules": ["spi-bcm2708" if OS_VERSION == OS_WHEEZY else "spi-bcm2835"] + }, + + "UART": { + "enabled": False, + "gpio": {14:"TX", 15:"RX"} + }, + + "ONEWIRE": { + "enabled": False, + "gpio": {4:"DATA"}, + "modules": ["w1-gpio"], + "wait": 2} + } + +def enableBus(bus): + loadModules(bus) + configurePins(bus) + BUSLIST[bus]["enabled"] = True + +def configurePins(bus): + if "configure_pin_command" in BUSLIST[bus]: + for pin in BUSLIST[bus]["gpio"].keys(): + command = BUSLIST[bus]["configure_pin_command"].format(pin) + subprocess.call(command.split(' ')) def loadModule(module): subprocess.call(["sudo", "modprobe", module]) @@ -59,8 +98,6 @@ def loadModules(bus): info("Sleeping %ds to let %s modules load" % (BUSLIST[bus]["wait"], bus)) time.sleep(BUSLIST[bus]["wait"]) - BUSLIST[bus]["enabled"] = True - def unloadModules(bus): info("Unloading %s modules" % bus) for module in BUSLIST[bus]["modules"]: @@ -95,7 +132,7 @@ def checkAllBus(): class Bus(): def __init__(self, busName, device, flag=os.O_RDWR): - loadModules(busName) + enableBus(busName) self.busName = busName self.device = device self.flag = flag diff --git a/myDevices/devices/digital/__init__.py b/myDevices/devices/digital/__init__.py index 6d0d4fd..fdd8a63 100644 --- a/myDevices/devices/digital/__init__.py +++ b/myDevices/devices/digital/__init__.py @@ -47,13 +47,13 @@ def __getFunction__(self, channel): def __setFunction__(self, channel, func): raise NotImplementedError - def __digitalRead__(self, chanel): + def __digitalRead__(self, channel): raise NotImplementedError def __portRead__(self): raise NotImplementedError - def __digitalWrite__(self, chanel, value): + def __digitalWrite__(self, channel, value): raise NotImplementedError def __portWrite__(self, value): @@ -80,7 +80,6 @@ def setFunction(self, channel, value): self.__setFunction__(channel, value) return self.__getFunction__(channel) - #@request("POST", "%(channel)d/function/%(value)s") def setFunctionString(self, channel, value): value = value.lower() @@ -94,7 +93,6 @@ def setFunctionString(self, channel, value): raise ValueError("Bad Function") return self.getFunctionString(channel) - #@request("GET", "%(channel)d/value") @response("%d") def digitalRead(self, channel): diff --git a/myDevices/devices/digital/ds2408.py b/myDevices/devices/digital/ds2408.py index e539dab..0a53b72 100644 --- a/myDevices/devices/digital/ds2408.py +++ b/myDevices/devices/digital/ds2408.py @@ -78,7 +78,7 @@ def readByte(self): def writeByte(self, value): try: info('DS2408 writeByte {} {} {}'.format(self.slave, value, bytearray([value]))) - command = 'sudo python3 -m myDevices.devices.writevalue /sys/bus/w1/devices/{}/output {}'.format(self.slave, value) + command = 'sudo python3 -m myDevices.devices.writevalue -f /sys/bus/w1/devices/{}/output -b {}'.format(self.slave, value) subprocess.call(command.split()) except Exception as ex: error('DS2408 writeByte error: {}'.format(ex)) diff --git a/myDevices/devices/digital/gpio.py b/myDevices/devices/digital/gpio.py index 3a68dfc..35ce58d 100644 --- a/myDevices/devices/digital/gpio.py +++ b/myDevices/devices/digital/gpio.py @@ -16,66 +16,77 @@ import mmap from time import sleep from myDevices.utils.types import M_JSON -from myDevices.utils.logger import debug, info, error +from myDevices.utils.logger import debug, info, error, exception +from myDevices.utils.singleton import Singleton from myDevices.devices.digital import GPIOPort -from myDevices.decorators.rest import request, response +from myDevices.decorators.rest import response +from myDevices.system.hardware import BOARD_REVISION, Hardware +from myDevices.system.systemconfig import SystemConfig +from myDevices.utils.subprocess import executeCommand +try: + import ASUS.GPIO as gpio_library +except: + gpio_library = None + FSEL_OFFSET = 0 # 0x0000 PINLEVEL_OFFSET = 13 # 0x0034 / 4 BLOCK_SIZE = (4*1024) -GPIO_COUNT = 54 -FUNCTIONS = ["IN", "OUT", "ALT5", "ALT4", "ALT0", "ALT1", "ALT2", "ALT3", "PWM"] +class NativeGPIO(Singleton, GPIOPort): + IN = 0 + OUT = 1 -class NativeGPIO(GPIOPort): - GPIO_COUNT = 54 + ASUS_GPIO = 44 - IN = 0 - OUT = 1 - ALT5 = 2 - ALT4 = 3 - ALT0 = 4 - ALT1 = 5 - ALT2 = 6 - ALT3 = 7 - PWM = 8 + LOW = 0 + HIGH = 1 - LOW = 0 - HIGH = 1 + PUD_OFF = 0 + PUD_DOWN = 1 + PUD_UP = 2 - PUD_OFF = 0 - PUD_DOWN = 1 - PUD_UP = 2 + RATIO = 1 + ANGLE = 2 - RATIO = 1 - ANGLE = 2 - instance = None + MAPPING = [] def __init__(self): - if not NativeGPIO.instance: - GPIOPort.__init__(self, 54) - self.export = range(54) - self.post_value = True - self.post_function = True - self.gpio_setup = [] - self.gpio_reset = [] - self.valueFile = [0 for i in range(54)] - self.functionFile = [0 for i in range(54)] - for i in range(54): - # Export the pins here to prevent a delay when accessing the values for the - # first time while waiting for the file group to be set - self.__checkFilesystemExport__(i) + self.setPinMapping() + GPIOPort.__init__(self, max(self.pins) + 1) + self.post_value = True + self.post_function = True + self.gpio_setup = [] + self.gpio_reset = [] + self.gpio_map = None + self.pinFunctionSet = set() + self.valueFile = {pin:None for pin in self.pins} + self.functionFile = {pin:None for pin in self.pins} + for pin in self.pins: + # Export the pins here to prevent a delay when accessing the values for the + # first time while waiting for the file group to be set + self.__checkFilesystemExport__(pin) + if gpio_library: + gpio_library.setmode(gpio_library.ASUS) + else: try: with open('/dev/gpiomem', 'rb') as gpiomem: self.gpio_map = mmap.mmap(gpiomem.fileno(), BLOCK_SIZE, prot=mmap.PROT_READ) + except FileNotFoundError: + pass except OSError as err: error(err) - NativeGPIO.instance = self def __del__(self): - if self.gpio_map: + if hasattr(self, 'gpio_map'): self.gpio_map.close() + for value in self.valueFile.values(): + if value: + value.close() + for value in self.functionFile.values(): + if value: + value.close() class SetupException(BaseException): pass @@ -139,7 +150,7 @@ def close(self): self.__digitalWrite__(gpio, g["value"]) def checkDigitalChannelExported(self, channel): - if not channel in self.export: + if not channel in self.pins: raise Exception("Channel %d is not allowed" % channel) def checkPostingFunctionAllowed(self): @@ -149,6 +160,12 @@ def checkPostingFunctionAllowed(self): def checkPostingValueAllowed(self): if not self.post_value: raise ValueError("POSTing value to native GPIO not allowed") + + def __getFunctionFilePath__(self, channel): + return "/sys/class/gpio/gpio%s/direction" % channel + + def __getValueFilePath__(self, channel): + return "/sys/class/gpio/gpio%s/value" % channel def __checkFilesystemExport__(self, channel): #debug("checkExport for channel %d" % channel) @@ -157,36 +174,47 @@ def __checkFilesystemExport__(self, channel): try: with open("/sys/class/gpio/export", "a") as f: f.write("%s" % channel) + except PermissionError: + command = 'sudo python3 -m myDevices.devices.writevalue -f /sys/class/gpio/export -t {}'.format(channel) + executeCommand(command) except Exception as ex: - #error('Failed on __checkFilesystemExport__: ' + str(channel) + ' ' + str(ex)) + error('Failed on __checkFilesystemExport__: ' + str(channel) + ' ' + str(ex)) return False return True def __checkFilesystemFunction__(self, channel): - if self.functionFile[channel] == 0: + if not self.functionFile[channel]: #debug("function file %d not open" %channel) valRet = self.__checkFilesystemExport__(channel) if not valRet: return + mode = 'w+' + if (gpio_library or Hardware().isBeagleBone()) and os.geteuid() != 0: + #On devices with root permissions on gpio files open the file in read mode from non-root process + mode = 'r' for i in range(10): try: - self.functionFile[channel] = open("/sys/class/gpio/gpio%s/direction" % channel, "w+") + self.functionFile[channel] = open(self.__getFunctionFilePath__(channel), mode) break except PermissionError: # Try again since the file group might not have been set to the gpio group # since there is a delay when the gpio channel is first exported sleep(0.01) - + def __checkFilesystemValue__(self, channel): - if self.valueFile[channel] == 0: + if not self.valueFile[channel]: #debug("value file %d not open" %channel) valRet = self.__checkFilesystemExport__(channel) if not valRet: return + mode = 'w+' + if (gpio_library or Hardware().isBeagleBone()) and os.geteuid() != 0: + #On devices with root permissions on gpio files open the file in read mode from non-root process + mode = 'r' for i in range(10): try: - self.valueFile[channel] = open("/sys/class/gpio/gpio%s/value" % channel, "w+") + self.valueFile[channel] = open(self.__getValueFilePath__(channel), mode) break except PermissionError: # Try again since the file group might not have been set to the gpio group @@ -194,30 +222,30 @@ def __checkFilesystemValue__(self, channel): sleep(0.01) def __digitalRead__(self, channel): - self.__checkFilesystemValue__(channel) - #self.checkDigitalChannelExported(channel) try: - r = self.valueFile[channel].read(1) + self.__checkFilesystemValue__(channel) + value = self.valueFile[channel].read(1) self.valueFile[channel].seek(0) - if r[0] == '1': + if value[0] == '1': return self.HIGH else: return self.LOW except: - #error('Error' + str(ex)) return -1 def __digitalWrite__(self, channel, value): self.__checkFilesystemValue__(channel) - #self.checkDigitalChannelExported(channel) - #self.checkPostingValueAllowed() try: - if (value == 1): - self.valueFile[channel].write('1') + if value == 1: + value = '1' else: - self.valueFile[channel].write('0') - self.valueFile[channel].seek(0) - pass + value = '0' + try: + self.valueFile[channel].write(value) + self.valueFile[channel].seek(0) + except: + command = 'sudo python3 -m myDevices.devices.writevalue -f {} -t {}'.format(self.__getValueFilePath__(channel), value) + executeCommand(command) except: pass @@ -235,14 +263,28 @@ def __getFunction__(self, channel): self.__checkFilesystemFunction__(channel) self.checkDigitalChannelExported(channel) try: - r = self.functionFile[channel].read() + # If we haven't already set the channel function on an ASUS device, we use the GPIO + # library to get the function. Otherwise we just fall through and read the file itself + # since we can assume the pin is a GPIO pin and reading the function file is quicker + # than launching a separate process. + if gpio_library and channel not in self.pinFunctionSet: + if os.geteuid() == 0: + value = gpio_library.gpio_function(channel) + else: + value, err = executeCommand('sudo python3 -m myDevices.devices.readvalue -c {}'.format(channel)) + return int(value.splitlines()[0]) + # If this is not a GPIO function return it, otherwise check the function file to see + # if it is an IN or OUT pin since the ASUS library doesn't provide that info. + if value != self.ASUS_GPIO: + return value + function = self.functionFile[channel].read() self.functionFile[channel].seek(0) - if (r.startswith("out")): + if function.startswith("out"): return self.OUT else: return self.IN except Exception as ex: - #error('Failed on __getFunction__: '+ str(channel) + ' ' + str(ex)) + error('Failed on __getFunction__: '+ str(channel) + ' ' + str(ex)) return -1 def __setFunction__(self, channel, value): @@ -250,24 +292,30 @@ def __setFunction__(self, channel, value): self.checkDigitalChannelExported(channel) self.checkPostingFunctionAllowed() try: - if (value == self.IN): - self.functionFile[channel].write("in") + if value == self.IN: + value = 'in' else: - self.functionFile[channel].write("out") - self.functionFile[channel].seek(0) + value = 'out' + try: + self.functionFile[channel].write(value) + self.functionFile[channel].seek(0) + except: + command = 'sudo python3 -m myDevices.devices.writevalue -f {} -t {}'.format(self.__getFunctionFilePath__(channel), value) + executeCommand(command) + self.pinFunctionSet.add(channel) except Exception as ex: - error('Failed on __setFunction__: ' + str(channel) + ' ' + str(ex)) + exception('Failed on __setFunction__: ' + str(channel) + ' ' + str(ex)) pass def __portRead__(self): value = 0 - for i in self.export: + for i in self.pins: value |= self.__digitalRead__(i) << i return value def __portWrite__(self, value): - if len(self.export) < 54: - for i in self.export: + if len(self.pins) <= value.bit_length(): + for i in self.pins: if self.getFunction(i) == self.OUT: self.__digitalWrite__(i, (value >> i) & 1) else: @@ -276,15 +324,21 @@ def __portWrite__(self, value): #@request("GET", "*") @response(contentType=M_JSON) def wildcard(self, compact=False): + if gpio_library and os.geteuid() != 0: + #If not root on an ASUS device get the pin states as root + value, err = executeCommand('sudo python3 -m myDevices.devices.readvalue --pins') + value = value.splitlines()[0] + import json + return json.loads(value) if compact: f = "f" v = "v" else: f = "function" v = "value" - values = {} - for i in self.export: + self.system_config = SystemConfig.getConfig() + for i in self.pins + self.overlay_pins: if compact: func = self.getFunction(i) else: @@ -297,13 +351,295 @@ def getFunction(self, channel): def getFunctionString(self, channel): f = self.getFunction(channel) - try: - function_string = FUNCTIONS[f] - except: - function_string = 'UNKNOWN' + function_string = 'UNKNOWN' + functions = {0:'IN', 1:'OUT', 2:'ALT5', 3:'ALT4', 4:'ALT0', 5:'ALT1', 6:'ALT2', 7:'ALT3', 8:'PWM', + 40:'SERIAL', 41:'SPI', 42:'I2C', 43:'PWM', 44:'GPIO', 45:'TS_XXXX', 46:'RESERVED', 47:'I2S'} + if f >= 0: + try: + function_string = functions[f] + except: + pass + try: + # On Raspberry Pis using the spi_bcm2835 driver SPI chip select is done via software rather than hardware + # so the pin function is OUT instead of ALT0. Here we override that (and the I2C to be safe) so the GPIO map + # in the UI will display the appropriate pin info. + if channel in self.spi_pins and self.system_config['SPI'] == 1: + function_string = functions[4] + if channel in self.i2c_pins and self.system_config['I2C'] == 1: + function_string = functions[4] + except: + pass + try: + # If 1-Wire is enabled specify the pin function as a device tree overlay. + if channel in self.overlay_pins: + function_string = 'OVERLAY' + except: + pass return function_string - def input(self, channel): - value = self.__digitalRead__(channel) - return value - + def setPinMapping(self): + hardware = Hardware() + if hardware.isTinkerBoard(): + self.MAPPING = [{'name': 'GPIO', + 'map': [ + {'power': 'V33'}, + {'power': 'V50'}, + {'gpio': 252}, + {'power': 'V50'}, + {'gpio': 253}, + {'power': 'GND'}, + {'gpio': 17}, + {'gpio': 161}, + {'power': 'GND'}, + {'gpio': 160}, + {'gpio': 164}, + {'gpio': 184}, + {'gpio': 166}, + {'power': 'GND'}, + {'gpio': 167}, + {'gpio': 162}, + {'power': 'V33'}, + {'gpio': 163}, + {'gpio': 257}, + {'power': 'GND'}, + {'gpio': 256}, + {'gpio': 171}, + {'gpio': 254}, + {'gpio': 255}, + {'power': 'GND'}, + {'gpio': 251}, + {'dnc': True}, + {'dnc': True}, + {'gpio': 165}, + {'power': 'GND'}, + {'gpio': 168}, + {'gpio': 239}, + {'gpio': 238}, + {'power': 'GND'}, + {'gpio': 185}, + {'gpio': 223}, + {'gpio': 224}, + {'gpio': 187}, + {'power': 'GND'}, + {'gpio': 188} + ]}] + elif hardware.isBeagleBone(): + self.MAPPING = [{'name': 'P9', + 'map': [ + {'power': 'GND'}, + {'power': 'GND'}, + {'power': 'V33'}, + {'power': 'V33'}, + {'power': 'V50'}, + {'power': 'V50'}, + {'power': 'V50'}, + {'power': 'V50'}, + {'power': 'PWR'}, + {'power': 'RST'}, + {'gpio': 30}, + {'gpio': 60}, + {'gpio': 31}, + {'gpio': 50}, + {'gpio': 48}, + {'gpio': 51}, + {'gpio': 5}, + {'gpio': 4}, + {'alt0': {'channel': 'sys:i2c:2', 'name': 'SCL'}}, + {'alt0': {'channel': 'sys:i2c:2', 'name': 'SDA'}}, + {'gpio': 3}, + {'gpio': 2}, + {'gpio': 49}, + {'gpio': 15}, + {'gpio': 117}, + {'gpio': 14}, + {'gpio': 115}, + {'gpio': 113, 'alt0': {'channel': 'sys:spi:1', 'name': 'CS0'}}, + {'gpio': 111, 'alt0': {'channel': 'sys:spi:1', 'name': 'D0'}}, + {'gpio': 112, 'alt0': {'channel': 'sys:spi:1', 'name': 'D1'}}, + {'gpio': 110, 'alt0': {'channel': 'sys:spi:1', 'name': 'SCLK'}}, + {'power': 'VDD_ADC'}, + {'analog': 4}, + {'power': 'GNDA_ADC'}, + {'analog': 6}, + {'analog': 5}, + {'analog': 2}, + {'analog': 3}, + {'analog': 0}, + {'analog': 1}, + {'gpio': 20}, + {'gpio': 7}, + {'power': 'GND'}, + {'power': 'GND'}, + {'power': 'GND'}, + {'power': 'GND'}]}, + {'name': 'P8', + 'map': [ + {'power': 'GND'}, + {'power': 'GND'}, + {'gpio': 38}, + {'gpio': 39}, + {'gpio': 34}, + {'gpio': 35}, + {'gpio': 66}, + {'gpio': 67}, + {'gpio': 69}, + {'gpio': 68}, + {'gpio': 45}, + {'gpio': 44}, + {'gpio': 23}, + {'gpio': 26}, + {'gpio': 47}, + {'gpio': 46}, + {'gpio': 27}, + {'gpio': 65}, + {'gpio': 22}, + {'gpio': 63}, + {'gpio': 62}, + {'gpio': 37}, + {'gpio': 36}, + {'gpio': 33}, + {'gpio': 32}, + {'gpio': 61}, + {'gpio': 86}, + {'gpio': 88}, + {'gpio': 87}, + {'gpio': 89}, + {'gpio': 10}, + {'gpio': 11}, + {'gpio': 9}, + {'gpio': 81}, + {'gpio': 8}, + {'gpio': 80}, + {'gpio': 78}, + {'gpio': 79}, + {'gpio': 76}, + {'gpio': 77}, + {'gpio': 74}, + {'gpio': 75}, + {'gpio': 72}, + {'gpio': 73}, + {'gpio': 70}, + {'gpio': 71} + ]}] + else: + if BOARD_REVISION == 1: + self.MAPPING = [{'name': 'P1', + 'map': [ + {'power': 'V33'}, + {'power': 'V50'}, + {'gpio': 0, 'alt0': {'channel': 'sys:i2c', 'name': 'SDA'}}, + {'power': 'V50'}, + {'gpio': 1, 'alt0': {'channel': 'sys:i2c', 'name': 'SCL'}}, + {'power': 'GND'}, + {'gpio': 4, 'overlay': {'channel': 'sys:1wire', 'name': 'DATA'}}, + {'gpio': 14, 'alt0': {'channel': 'sys:uart', 'name': 'TX'}}, + {'power': 'GND'}, + {'gpio': 15, 'alt0': {'channel': 'sys:uart', 'name': 'RX'}}, + {'gpio': 17}, + {'gpio': 18}, + {'gpio': 21}, + {'power': 'GND'}, + {'gpio': 22}, + {'gpio': 23}, + {'power': 'V33'}, + {'gpio': 24}, + {'gpio': 10, 'alt0': {'channel': 'sys:spi', 'name': 'MOSI'}}, + {'power': 'GND'}, + {'gpio': 9, 'alt0': {'channel': 'sys:spi', 'name': 'MISO'}}, + {'gpio': 25}, + {'gpio': 11, 'alt0': {'channel': 'sys:spi', 'name': 'SCLK'}}, + {'gpio': 8, 'alt0': {'channel': 'sys:spi', 'name': 'CE0'}}, + {'power': 'GND'}, + {'gpio': 7, 'alt0': {'channel': 'sys:spi', 'name': 'CE1'}} + ]}] + elif BOARD_REVISION == 2: + self.MAPPING = [{'name': 'P1', + 'map': [ + {'power': 'V33'}, + {'power': 'V50'}, + {'gpio': 2, 'alt0': {'channel': 'sys:i2c', 'name': 'SDA'}}, + {'power': 'V50'}, + {'gpio': 3, 'alt0': {'channel': 'sys:i2c', 'name': 'SCL'}}, + {'power': 'GND'}, + {'gpio': 4, 'overlay': {'channel': 'sys:1wire', 'name': 'DATA'}}, + {'gpio': 14, 'alt0': {'channel': 'sys:uart', 'name': 'TX'}}, + {'power': 'GND'}, + {'gpio': 15, 'alt0': {'channel': 'sys:uart', 'name': 'RX'}}, + {'gpio': 17}, + {'gpio': 18}, + {'gpio': 27}, + {'power': 'GND'}, + {'gpio': 22}, + {'gpio': 23}, + {'power': 'V33'}, + {'gpio': 24}, + {'gpio': 10, 'alt0': {'channel': 'sys:spi', 'name': 'MOSI'}}, + {'power': 'GND'}, + {'gpio': 9, 'alt0': {'channel': 'sys:spi', 'name': 'MISO'}}, + {'gpio': 25}, + {'gpio': 11, 'alt0': {'channel': 'sys:spi', 'name': 'SCLK'}}, + {'gpio': 8, 'alt0': {'channel': 'sys:spi', 'name': 'CE0'}}, + {'power': 'GND'}, + {'gpio': 7, 'alt0': {'channel': 'sys:spi', 'name': 'CE1'}} + ]}] + elif BOARD_REVISION == 3: + self.MAPPING = [{'name': 'P1', + 'map': [ + {'power': 'V33'}, + {'power': 'V50'}, + {'gpio': 2, 'alt0': {'channel': 'sys:i2c', 'name': 'SDA'}}, + {'power': 'V50'}, + {'gpio': 3, 'alt0': {'channel': 'sys:i2c', 'name': 'SCL'}}, + {'power': 'GND'}, + {'gpio': 4, 'overlay': {'channel': 'sys:1wire', 'name': 'DATA'}}, + {'gpio': 14, 'alt0': {'channel': 'sys:uart', 'name': 'TX'}}, + {'power': 'GND'}, + {'gpio': 15, 'alt0': {'channel': 'sys:uart', 'name': 'RX'}}, + {'gpio': 17}, + {'gpio': 18}, + {'gpio': 27}, + {'power': 'GND'}, + {'gpio': 22}, + {'gpio': 23}, + {'power': 'V33'}, + {'gpio': 24}, + {'gpio': 10, 'alt0': {'channel': 'sys:spi', 'name': 'MOSI'}}, + {'power': 'GND'}, + {'gpio': 9, 'alt0': {'channel': 'sys:spi', 'name': 'MISO'}}, + {'gpio': 25}, + {'gpio': 11, 'alt0': {'channel': 'sys:spi', 'name': 'SCLK'}}, + {'gpio': 8, 'alt0': {'channel': 'sys:spi', 'name': 'CE0'}}, + {'power': 'GND'}, + {'gpio': 7, 'alt0': {'channel': 'sys:spi', 'name': 'CE1'}}, + {'dnc': True}, + {'dnc': True}, + {'gpio': 5}, + {'power': 'GND'}, + {'gpio': 6}, + {'gpio': 12}, + {'gpio': 13}, + {'power': 'GND'}, + {'gpio': 19}, + {'gpio': 16}, + {'gpio': 26}, + {'gpio': 20}, + {'power': 'GND'}, + {'gpio': 21} + ]}] + if isinstance(self.MAPPING, list): + self.pins = [] + self.overlay_pins = [] + self.spi_pins = [] + self.i2c_pins = [] + self.system_config = SystemConfig.getConfig() + for header in self.MAPPING: + self.pins.extend([pin['gpio'] for pin in header['map'] if 'gpio' in pin]) + try: + if Hardware().isRaspberryPi(): + if self.system_config['OneWire'] == 1: + self.overlay_pins.extend([pin['gpio'] for pin in header['map'] if 'overlay' in pin and pin['overlay']['channel'] == 'sys:1wire']) + self.pins = [pin for pin in self.pins if pin not in self.overlay_pins] + self.spi_pins.extend([pin['gpio'] for pin in header['map'] if 'alt0' in pin and pin['alt0']['channel'] == 'sys:spi']) + self.i2c_pins.extend([pin['gpio'] for pin in header['map'] if 'alt0' in pin and pin['alt0']['channel'] == 'sys:i2c']) + except: + pass diff --git a/myDevices/devices/digital/helper.py b/myDevices/devices/digital/helper.py index f45bcc1..4c5460a 100644 --- a/myDevices/devices/digital/helper.py +++ b/myDevices/devices/digital/helper.py @@ -42,7 +42,7 @@ def setGPIOInstance(self): if self.gpioname != "GPIO": self.gpio = instance.deviceInstance(self.gpioname) else: - self.gpio = GPIO.instance + self.gpio = GPIO() if self.gpio: self.gpio.setFunction(self.channel, GPIO.IN) diff --git a/myDevices/devices/i2c.py b/myDevices/devices/i2c.py index 4be8d1f..eea281f 100644 --- a/myDevices/devices/i2c.py +++ b/myDevices/devices/i2c.py @@ -14,7 +14,7 @@ import fcntl -from myDevices.utils.version import BOARD_REVISION +from myDevices.system.hardware import BOARD_REVISION, Hardware from myDevices.devices.bus import Bus # /dev/i2c-X ioctl commands. The ioctl's parameter is always an @@ -45,22 +45,30 @@ SLAVES = [None for i in range(128)] class I2C(Bus): - def __init__(self, slave): + def __init__(self, slave, allow_duplicates=False): global SLAVES - if SLAVES[slave] != None: + if SLAVES[slave] != None and not allow_duplicates: raise Exception("SLAVE_ADDRESS_USED") self.channel = 0 - if BOARD_REVISION > 1: + hardware = Hardware() + if BOARD_REVISION > 1 or hardware.isTinkerBoard(): self.channel = 1 + elif hardware.isBeagleBone(): + self.channel = 2 Bus.__init__(self, "I2C", "/dev/i2c-%d" % self.channel) self.slave = slave if fcntl.ioctl(self.fd, I2C_SLAVE, self.slave): raise Exception("Error binding I2C slave 0x%02X" % self.slave) - SLAVES[self.slave] = self + # Since we now allow duplicates, e.g. BMP180_TEMPERATURE & BMP180_PRESSURE, we might need to + # change the SLAVES list to store a reference count or base class name as a way of making sure + # only the same base sensor type is duplicated. That doesn't seem to be an issue currently though + # so for now we just ignore the SLAVES check. + if not allow_duplicates: + SLAVES[self.slave] = self def __str__(self): return "I2C(slave=0x%02X)" % self.slave diff --git a/myDevices/devices/manager.py b/myDevices/devices/manager.py index d4ca9de..19155f3 100644 --- a/myDevices/devices/manager.py +++ b/myDevices/devices/manager.py @@ -2,6 +2,7 @@ import os.path import json as JSON from time import sleep, time +from threading import RLock from myDevices.utils import logger from myDevices.utils import types from myDevices.utils.config import Config @@ -12,6 +13,9 @@ PACKAGES = [serial, digital, analog, sensor, shield] DYNAMIC_DEVICES = {} DEVICES_JSON_FILE = "/etc/myDevices/devices.json" + +mutex = RLock() + def deviceDetector(): logger.debug('deviceDetector') try: @@ -29,8 +33,6 @@ def deviceDetector(): except Exception as e: logger.error("Device detector: %s" % e) - sleep(5) - def findDeviceClass(name): for package in PACKAGES: if hasattr(package, name): @@ -48,34 +50,36 @@ def findDeviceClass(name): return None def saveDevice(name, install_date): - logger.debug('saveDevice: ' + str(name)) - if name not in DEVICES: - return - #never save to json devices that are manually added - if DEVICES[name]['origin'] == 'manual': - return - DYNAMIC_DEVICES[name] = DEVICES[name] - DEVICES[name]['install_date'] = install_date - json_devices = getJSON(DYNAMIC_DEVICES) - with open(DEVICES_JSON_FILE, 'w') as outfile: - outfile.write(json_devices) + with mutex: + logger.debug('saveDevice: ' + str(name)) + if name not in DEVICES: + return + #never save to json devices that are manually added + if DEVICES[name]['origin'] == 'manual': + return + DYNAMIC_DEVICES[name] = DEVICES[name] + DEVICES[name]['install_date'] = install_date + json_devices = getJSON(DYNAMIC_DEVICES) + with open(DEVICES_JSON_FILE, 'w') as outfile: + outfile.write(json_devices) def removeDevice(name): - if name in DEVICES: - if name in DYNAMIC_DEVICES: - if hasattr(DEVICES[name]["device"], 'close'): - DEVICES[name]["device"].close() - del DEVICES[name] - del DYNAMIC_DEVICES[name] - json_devices = getJSON(DYNAMIC_DEVICES) - with open(DEVICES_JSON_FILE, 'w') as outfile: - outfile.write(json_devices) - logger.debug("Deleted device %s" % name) - return (200, None, None) - logger.error("Cannot delete %s, found but not added via REST" % name) - return (403, None, None) - logger.error("Cannot delete %s, not found" % name) - return (404, None, None) + with mutex: + if name in DEVICES: + if name in DYNAMIC_DEVICES: + if hasattr(DEVICES[name]["device"], 'close'): + DEVICES[name]["device"].close() + del DEVICES[name] + del DYNAMIC_DEVICES[name] + json_devices = getJSON(DYNAMIC_DEVICES) + with open(DEVICES_JSON_FILE, 'w') as outfile: + outfile.write(json_devices) + logger.debug("Deleted device %s" % name) + return (200, None, None) + logger.error("Cannot delete %s, found but not added via REST" % name) + return (403, None, None) + logger.error("Cannot delete %s, not found" % name) + return (404, None, None) def addDeviceJSON(json): @@ -106,62 +110,63 @@ def addDeviceJSON(json): return (500, "ERROR", "text/plain") def updateDevice(name, json): - if not name in DEVICES: - return (404, None, None) - - if "name" in json: -# forbid name changed -# if json["name"] != name: -# return (403, "FORBIDDEN", "text/plain") + with mutex: + if not name in DEVICES: + return (404, None, None) - if json["name"] != name and json["name"] in DEVICES: - return (403, "ALREADY_EXISTS", "text/plain") + if "name" in json: + # forbid name changed + # if json["name"] != name: + # return (403, "FORBIDDEN", "text/plain") - logger.info("Edit %s" % name) - (c, d, t) = removeDevice(name) - if c == 200: - (c, d, t) = addDeviceJSON(json) - - return (c, d, t) + if json["name"] != name and json["name"] in DEVICES: + return (403, "ALREADY_EXISTS", "text/plain") + logger.info("Edit %s" % name) + (c, d, t) = removeDevice(name) + if c == 200: + (c, d, t) = addDeviceJSON(json) + + return (c, d, t) def addDevice(name, device, description, args, origin): - if name in DEVICES: - logger.error("Device <%s> already exists" % name) - return -1 - logger.debug('addDevice: ' + str(name) + ' ' + str(device)) -# if '/' in device: -# deviceClass = device.split('/')[0] -# else: -# deviceClass = device - try: - constructor = findDeviceClass(device) - except Exception as ex: - logger.debug('findDeviceClass failure:' + str(ex)) - return 0 - logger.debug('constructor class found ' + str(constructor)) - if constructor == None: - raise Exception("Device driver not found for %s" % device) + with mutex: + if name in DEVICES: + logger.error("Device <%s> already exists" % name) + return -1 + logger.debug('addDevice: ' + str(name) + ' ' + str(device)) + # if '/' in device: + # deviceClass = device.split('/')[0] + # else: + # deviceClass = device + try: + constructor = findDeviceClass(device) + except Exception as ex: + logger.debug('findDeviceClass failure:' + str(ex)) + return 0 + logger.debug('constructor class found ' + str(constructor)) + if constructor == None: + raise Exception("Device driver not found for %s" % device) - instance = None - try: - if len(args) > 0: - instance = constructor(**args) - else: - instance = constructor() - logger.debug('Adding instance ' + str(instance)) - addDeviceInstance(name, device, description, instance, args, origin) - return 1 - except Exception as e: - logger.error("Error while adding device %s(%s) : %s" % (name, device, e)) - # addDeviceInstance(name, device, description, None, args, origin) - removeDevice(name) - return 0 + instance = None + try: + if len(args) > 0: + instance = constructor(**args) + else: + instance = constructor() + logger.debug('Adding instance ' + str(instance)) + addDeviceInstance(name, device, description, instance, args, origin) + return 1 + except Exception as e: + logger.error("Error while adding device %s(%s) : %s" % (name, device, e)) + # addDeviceInstance(name, device, description, None, args, origin) + removeDevice(name) + return 0 def addDeviceConf(devices, origin): for (name, params) in devices: values = params.split(" ") - driver = values[0]; + driver = values[0] description = name args = {} i = 1 @@ -223,13 +228,14 @@ def addDeviceInstance(name, device, description, instance, args, origin): } def closeDevices(): - devices = [k for k in DEVICES.keys()] - for name in devices: - device = DEVICES[name]["device"] - logger.debug("Closing device %s - %s" % (name, device)) - del DEVICES[name] - if hasattr(device, 'close'): - device.close() + with mutex: + devices = [k for k in DEVICES.keys()] + for name in devices: + device = DEVICES[name]["device"] + logger.debug("Closing device %s - %s" % (name, device)) + del DEVICES[name] + if hasattr(device, 'close'): + device.close() def getJSON(devices_list): return types.jsonDumps(getDeviceList(devices_list)) diff --git a/myDevices/devices/onewire.py b/myDevices/devices/onewire.py index a03a4f8..40f1246 100644 --- a/myDevices/devices/onewire.py +++ b/myDevices/devices/onewire.py @@ -74,7 +74,7 @@ def deviceList(self): devices.append(line) else: devices = lines - return devices; + return devices def read(self): data = "" @@ -90,15 +90,18 @@ def detectOneWireDevices(): devices = [] debug('detectOneWireDevices') slaveList = "/sys/bus/w1/devices/w1_bus_master1/w1_master_slaves" - with open(slaveList) as f: - lines = f.read().split("\n") - for line in lines: - debug(line) - if (len(line) > 0) and ('-' in line): - (family, addr) = line.split("-") - if family in FAMILIES: - device = {'name': addr, 'description': FAMILIES[family], 'device': FAMILIES[family], 'args': {'slave': line}} - debug(str(device)) - devices.append(device) + try: + with open(slaveList) as f: + lines = f.read().split("\n") + for line in lines: + debug(line) + if (len(line) > 0) and ('-' in line): + (family, addr) = line.split("-") + if family in FAMILIES: + device = {'name': addr, 'description': FAMILIES[family], 'device': FAMILIES[family], 'args': {'slave': line}} + debug(str(device)) + devices.append(device) + except FileNotFoundError as err: + debug('Error detecting 1-wire devices: {}'.format(err)) return devices diff --git a/myDevices/devices/readvalue.py b/myDevices/devices/readvalue.py new file mode 100644 index 0000000..9634640 --- /dev/null +++ b/myDevices/devices/readvalue.py @@ -0,0 +1,25 @@ +""" +This script reads GPIO info. It is intended to be launched via sudo in order to +read data from files that require root access so the main agent code can run from +a non-root process and only elevate to root when necessary. +""" +import sys +# from myDevices.utils.logger import setInfo, info, error, logToFile +from myDevices.devices.digital.gpio import NativeGPIO + +if __name__ == '__main__': + # Read data using script so it can be called via sudo, sends the data to the main process by writing to stdout + # setInfo() + # logToFile() + i = 1 + while i < len(sys.argv): + if sys.argv[i] in ['-p', '--pins']: + import json + gpio = NativeGPIO() + print(json.dumps(gpio.wildcard())) + if sys.argv[i] in ['-c', '--channel']: + gpio = NativeGPIO() + i += 1 + channel = int(sys.argv[i]) + print(gpio.getFunction(channel)) + i += 1 diff --git a/myDevices/devices/sensor/__init__.py b/myDevices/devices/sensor/__init__.py index 2a453e9..9d3e4c2 100644 --- a/myDevices/devices/sensor/__init__.py +++ b/myDevices/devices/sensor/__init__.py @@ -187,10 +187,10 @@ def getHumidityPercent(self): return self.__getHumidity__() * 100 DRIVERS = {} -DRIVERS["bmp085"] = ["BMP085", "BMP180"] +DRIVERS["bmp085"] = ["BMP085", "BMP180", "BMP180_TEMPERATURE", "BMP180_PRESSURE"] DRIVERS["onewiretemp"] = ["DS1822", "DS1825", "DS18B20", "DS18S20", "DS28EA00"] DRIVERS["tmpXXX"] = ["TMP36", "TMP75", "TMP102", "TMP275"] DRIVERS["tslXXXX"] = ["TSL2561", "TSL2561CS", "TSL2561T", "TSL4531", "TSL45311", "TSL45313", "TSL45315", "TSL45317"] -DRIVERS["vcnl4000"] = ["VCNL4000"] +DRIVERS["vcnl4000"] = ["VCNL4000", "VCNL4000_LUMINOSITY", "VCNL4000_DISTANCE"] DRIVERS["hytXXX"] = ["HYT221"] diff --git a/myDevices/devices/sensor/bmp085.py b/myDevices/devices/sensor/bmp085.py index e1083b2..a87c2bf 100644 --- a/myDevices/devices/sensor/bmp085.py +++ b/myDevices/devices/sensor/bmp085.py @@ -18,9 +18,12 @@ from myDevices.devices.sensor import Temperature, Pressure class BMP085(I2C, Temperature, Pressure): - def __init__(self, altitude=0, external=None): - I2C.__init__(self, 0x77) - Pressure.__init__(self, altitude, external) + def __init__(self, altitude=0, external=None, temperature=True, pressure=True): + self.temperature = temperature + self.pressure = pressure + I2C.__init__(self, 0x77, True) + if self.pressure: + Pressure.__init__(self, altitude, external) self.ac1 = self.readSignedInteger(0xAA) self.ac2 = self.readSignedInteger(0xAC) @@ -38,7 +41,12 @@ def __str__(self): return "BMP085" def __family__(self): - return [Temperature.__family__(self), Pressure.__family__(self)] + family = [] + if self.temperature: + family.append(Temperature.__family__(self)) + if self.pressure: + family.append(Pressure.__family__(self)) + return family def readUnsignedInteger(self, address): d = self.readRegisters(address, 2) @@ -100,9 +108,22 @@ def __getPascal__(self): return int(p) class BMP180(BMP085): - def __init__(self, altitude=0, external=None): - BMP085.__init__(self, altitude, external) + def __init__(self, altitude=0, external=None, temperature=True, pressure=True): + BMP085.__init__(self, altitude, external, temperature, pressure) def __str__(self): return "BMP180" +class BMP180_TEMPERATURE(BMP180): + def __init__(self, altitude=0, external=None): + BMP180.__init__(self, altitude, external, temperature=True, pressure=False) + + def __str__(self): + return "BMP180_TEMPERATURE" + +class BMP180_PRESSURE(BMP180): + def __init__(self, altitude=0, external=None): + BMP180.__init__(self, altitude, external, temperature=False, pressure=True) + + def __str__(self): + return "BMP180_PRESSURE" \ No newline at end of file diff --git a/myDevices/devices/sensor/vcnl4000.py b/myDevices/devices/sensor/vcnl4000.py index cbbb378..c1b3e37 100644 --- a/myDevices/devices/sensor/vcnl4000.py +++ b/myDevices/devices/sensor/vcnl4000.py @@ -23,7 +23,7 @@ from myDevices.devices.i2c import I2C from myDevices.devices.sensor import Luminosity, Distance from myDevices.utils.types import toint -from myDevices.utils.logger import debug +from myDevices.utils.logger import debug class VCNL4000(I2C, Luminosity, Distance): REG_COMMAND = 0x80 @@ -52,8 +52,10 @@ class VCNL4000(I2C, Luminosity, Distance): MASK_PROX_READY = 0b00100000 MASK_AMB_READY = 0b01000000 - def __init__(self, slave=0b0010011, current=20, frequency=781, prox_threshold=15, prox_cycles=10, cal_cycles= 5): - I2C.__init__(self, toint(slave)) + def __init__(self, slave=0b0010011, current=20, frequency=781, prox_threshold=15, prox_cycles=10, cal_cycles=5, luminosity=True, distance=True): + self.luminosity = luminosity + self.distance = distance + I2C.__init__(self, toint(slave), True) self.setCurrent(toint(current)) self.setFrequency(toint(frequency)) self.prox_threshold = toint(prox_threshold) @@ -68,7 +70,12 @@ def __str__(self): return "VCNL4000(slave=0x%02X)" % self.slave def __family__(self): - return [Luminosity.__family__(self), Distance.__family__(self)] + family = [] + if self.luminosity: + family.append(Luminosity.__family__(self)) + if self.distance: + family.append(Distance.__family__(self)) + return family def __setProximityTiming__(self): self.writeRegister(self.REG_PROX_ADJUST, self.VAL_MOD_TIMING_DEF) @@ -85,12 +92,10 @@ def calibrate(self): self.offset = self.__measureOffset__() debug ("VCNL4000: offset = %d" % (self.offset)) return self.offset - def setCurrent(self, current): self.current = current self.__setCurrent__() - def getCurrent(self): return self.__getCurrent__() @@ -141,9 +146,20 @@ def __getCurrent__(self): return bits_current * 10 def __getLux__(self): - self.writeRegister(self.REG_COMMAND, self.VAL_START_AMB) - while not (self.readRegister(self.REG_COMMAND) & self.MASK_AMB_READY): - time.sleep(0.001) + count = 0 + while count < 5: + # This try catch loop is a hacky fix for issues when making an initial ambient light reading + # using a VCNL4010 sensor. It should be removed when true VCNL4010 support is added. + try: + self.writeRegister(self.REG_COMMAND, self.VAL_START_AMB) + while not (self.readRegister(self.REG_COMMAND) & self.MASK_AMB_READY): + time.sleep(0.001) + count = 10 + except OSError as ex: + count = count + 1 + time.sleep(0.05) + if count >= 5: + raise ex light_bytes = self.readRegisters(self.REG_AMB_RESULT_HIGH, 2) light_word = light_bytes[0] << 8 | light_bytes[1] return self.__calculateLux__(light_word) @@ -206,6 +222,19 @@ def __readProximityCounts__(self): while not (self.readRegister(self.REG_COMMAND) & self.MASK_PROX_READY): time.sleep(0.001) proximity_bytes = self.readRegisters(self.REG_PROX_RESULT_HIGH, 2) - debug ("VCNL4000: prox raw value = %d" % (proximity_bytes[0] << 8 | proximity_bytes[1])) + debug("VCNL4000: prox raw value = %d" % (proximity_bytes[0] << 8 | proximity_bytes[1])) return (proximity_bytes[0] << 8 | proximity_bytes[1]) - \ No newline at end of file + +class VCNL4000_LUMINOSITY(VCNL4000): + def __init__(self): + VCNL4000.__init__(self, luminosity=True, distance=False) + + def __str__(self): + return "VCNL4000_LUMINOSITY" + +class VCNL4000_DISTANCE(VCNL4000): + def __init__(self): + VCNL4000.__init__(self, luminosity=False, distance=True) + + def __str__(self): + return "VCNL4000_DISTANCE" \ No newline at end of file diff --git a/myDevices/devices/shield/piface.py b/myDevices/devices/shield/piface.py index 81bc23e..722fed5 100644 --- a/myDevices/devices/shield/piface.py +++ b/myDevices/devices/shield/piface.py @@ -17,72 +17,15 @@ from myDevices.decorators.rest import request, response -class PiFaceDigital(): +class PiFaceDigital(MCP23S17): def __init__(self, board=0): - mcp = MCP23S17(0, 0x20+toint(board)) - mcp.writeRegister(mcp.getAddress(mcp.IODIR, 0), 0x00) # Port A as output - mcp.writeRegister(mcp.getAddress(mcp.IODIR, 8), 0xFF) # Port B as input - mcp.writeRegister(mcp.getAddress(mcp.GPPU, 0), 0x00) # Port A PU OFF - mcp.writeRegister(mcp.getAddress(mcp.GPPU, 8), 0xFF) # Port B PU ON - self.mcp = mcp self.board = toint(board) + MCP23S17.__init__(self, 0, 0x20 + self.board) + self.writeRegister(self.getAddress(self.IODIR, 0), 0x00) # Port A as output + self.writeRegister(self.getAddress(self.IODIR, 8), 0xFF) # Port B as input + self.writeRegister(self.getAddress(self.GPPU, 0), 0x00) # Port A PU OFF + self.writeRegister(self.getAddress(self.GPPU, 8), 0xFF) # Port B PU ON def __str__(self): return "PiFaceDigital(%d)" % self.board - def __family__(self): - return "GPIOPort" - - def checkChannel(self, channel): -# if not channel in range(8): - if not channel in range(16): - raise ValueError("Channel %d invalid" % channel) - - #@request("GET", "%(channel)d/value") - @response("%d") - def digitalRead(self, channel): - self.checkChannel(channel) -# return not self.mcp.digitalRead(channel+8) - return self.mcp.digitalRead(channel) - - #@request("POST", "%(channel)d/value/%(value)d") - @response("%d") - def digitalWrite(self, channel, value): - self.checkChannel(channel) - return self.mcp.digitalWrite(channel, value) - -# #@request("GET", "digital/output/%(channel)d") -# @response("%d") -# def digitalReadOutput(self, channel): -# self.checkChannel(channel) -# return self.mcp.digitalRead(channel) - -# #@request("GET", "digital/*") -# @response(contentType=M_JSON) -# def readAll(self): -# inputs = {} -# outputs = {} -# for i in range(8): -# inputs[i] = self.digitalRead(i) -# outputs[i] = self.digitalReadOutput(i) -# return {"input": inputs, "output": outputs} - - #@request("GET", "*") - @response(contentType=M_JSON) - def readAll(self): -# inputs = {} -# outputs = {} -# for i in range(8): -# inputs[i] = self.digitalRead(i) -# outputs[i] = self.digitalReadOutput(i) -# return {"input": inputs, "output": outputs} - - values = {} - for i in range(16): - values[i] = {"function": self.mcp.getFunctionString(i), "value": int(self.mcp.digitalRead(i))} - return values - - #@request("GET", "count") - @response("%d") - def digitalCount(self): - return 16 diff --git a/myDevices/devices/spi.py b/myDevices/devices/spi.py index bab451e..2cc5c78 100644 --- a/myDevices/devices/spi.py +++ b/myDevices/devices/spi.py @@ -17,8 +17,9 @@ import ctypes import struct -from myDevices.utils.version import PYTHON_MAJOR +from sys import version_info from myDevices.devices.bus import Bus +from myDevices.system.hardware import Hardware # from spi/spidev.h _IOC_NRBITS = 8 @@ -84,11 +85,17 @@ def SPI_IOC_MESSAGE(count): SPI_IOC_WR_MAX_SPEED_HZ = _IOW(SPI_IOC_MAGIC, 4, 4) class SPI(Bus): - def __init__(self, chip=0, mode=0, bits=8, speed=0): - Bus.__init__(self, "SPI", "/dev/spidev0.%d" % chip) + def __init__(self, chip=0, mode=0, bits=8, speed=0, init=True): + bus = 0 + hardware = Hardware() + if hardware.isTinkerBoard(): + bus = 2 + elif hardware.isBeagleBone(): + bus = 1 + Bus.__init__(self, "SPI", "/dev/spidev%d.%d" % (bus, chip)) self.chip = chip - val8 = array.array('B', [0]) + val8 = array.array('B', [0]) val8[0] = mode if fcntl.ioctl(self.fd, SPI_IOC_WR_MODE, val8): raise Exception("Cannot write SPI Mode") @@ -114,13 +121,13 @@ def __init__(self, chip=0, mode=0, bits=8, speed=0): raise Exception("Cannot read SPI Max speed") self.speed = struct.unpack('I', val32)[0] assert((self.speed == speed) or (speed == 0)) - + def __str__(self): return "SPI(chip=%d, mode=%d, speed=%dHz)" % (self.chip, self.mode, self.speed) def xfer(self, txbuff=None): length = len(txbuff) - if PYTHON_MAJOR >= 3: + if version_info.major >= 3: _txbuff = bytes(txbuff) _txptr = ctypes.create_string_buffer(_txbuff) else: @@ -142,4 +149,3 @@ def xfer(self, txbuff=None): fcntl.ioctl(self.fd, SPI_IOC_MESSAGE(len(data)), data) _rxbuff = ctypes.string_at(_rxptr, length) return bytearray(_rxbuff) - \ No newline at end of file diff --git a/myDevices/devices/writevalue.py b/myDevices/devices/writevalue.py index 8becd63..783cc25 100644 --- a/myDevices/devices/writevalue.py +++ b/myDevices/devices/writevalue.py @@ -1,14 +1,34 @@ +""" +This script writes a value to a file. It is intended to be launched via sudo +in order to write to files that require root access so the main agent code can +run from a non-root process and only elevate to root when necessary. +""" import sys -from myDevices.utils.logger import setInfo, info, error, logToFile +# from myDevices.utils.logger import setInfo, info, error, logToFile if __name__ == '__main__': # Write value to file in script so it can be called via sudo - setInfo() - logToFile() + # setInfo() + # logToFile() + filepath = None + mode = 'w' + value = None + i = 1 + while i < len(sys.argv): + if sys.argv[i] in ["-f", "-F", "--file"]: + filepath = sys.argv[i + 1] + i += 1 + elif sys.argv[i] in ["-t", "-T", "--text"]: + value = sys.argv[i + 1] + i += 1 + elif sys.argv[i] in ["-b", "-B", "--bytearray"]: + value = bytearray([int(sys.argv[i + 1])]) + mode = 'wb' + i += 1 + i += 1 try: - info('Write value {} to {}'.format(sys.argv[2], sys.argv[1])) - with open(sys.argv[1], "wb") as out_file: - out_file.write(bytearray([int(sys.argv[2])])) + # info('Write value {} to {}'.format(value, filepath)) + with open(filepath, mode) as out_file: + out_file.write(value) except Exception as ex: - error('Error writing value {}'.format(ex)) - + print('Error writing value {}'.format(ex), file=sys.stderr) diff --git a/myDevices/os/daemon.py b/myDevices/os/daemon.py deleted file mode 100644 index 603bbe4..0000000 --- a/myDevices/os/daemon.py +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env python -from sys import exit -from datetime import datetime -from os.path import getmtime -from myDevices.utils.logger import exception, info, warn, error, debug -from myDevices.os.services import ServiceManager - -#defining reset timeout in seconds -RESET_TIMEOUT=30 -FAILURE_COUNT=1000 -PYTHON_BIN='/usr/bin/python3' -failureCount={} -startFailure={} -errorList= (-3, -2, 12, 9, 24) -class Daemon: - def OnFailure(component, error=0): - #-3=Temporary failure in name resolution - info('Daemon failure handling ' + str(error)) - if error in errorList: - Daemon.Restart() - if component not in failureCount: - Daemon.Reset(component) - failureCount[component]+=1 - now = datetime.now() - if startFailure[component]==0: - startFailure[component]=now - elapsedTime=now-startFailure[component] - if (elapsedTime.total_seconds() >= RESET_TIMEOUT) or ( failureCount[component] > FAILURE_COUNT): - warn('Daemon::OnFailure myDevices is going to restart after ' +str(component) + ' failed: ' + str(elapsedTime.total_seconds()) + ' seconds and ' + str(failureCount) + ' times') - Daemon.Restart() - def Reset(component): - startFailure[component]=0 - failureCount[component]=0 - def Restart(): - try: - info('Daemon Restarting myDevices' ) - (output, returncode) = ServiceManager.ExecuteCommand('sudo service myDevices restart') - debug(str(output) + ' ' + str(returncode)) - del output - except: - exception ("Daemon::Restart Unexpected error") - Daemon.Exit() - def Exit(): - info('Critical failure. Closing myDevices process...') - exit('Daemon::Exit Closing agent. Critical failure.') - - - - diff --git a/myDevices/os/hardware.py b/myDevices/os/hardware.py deleted file mode 100644 index 0db63d7..0000000 --- a/myDevices/os/hardware.py +++ /dev/null @@ -1,64 +0,0 @@ -from uuid import getnode -from myDevices.utils.logger import exception, info, warn, error, debug - -class Hardware: - def __init__(self): - self.Revision = "0" - try: - with open('/proc/cpuinfo','r') as f: - for line in f: - splitLine = line.split(':') - if len(splitLine) < 2: - continue - key = splitLine[0].strip() - value = splitLine[1].strip() - if key=='Revision': - self.Revision = value - except: - exception ("Error reading cpuinfo") - self.model = 'Unknown' - if self.Revision == 'Beta': - self.model = 'Model B (Beta)' - if self.Revision in ('000d', '000e', '000f', '0002', '0003', '0004', '0005', '0006'): - self.model = 'Model B' - if self.Revision in ('0007', '0008', '0009'): - self.model = 'Model A' - if self.Revision in ('0010', '0013', '900032'): - self.model = 'Model B +' - if self.Revision in ('0011', '0014'): - self.model = 'Compute Module' - if self.Revision in ('0012', '0015'): - self.model = 'Model A+' - if self.Revision in ('a01041', 'a21041', 'a22042'): - self.model = 'Pi 2 Model B' - if self.Revision in ('900092', '900093'): - self.model = 'Zero' - if self.Revision in ('9000c1',): - self.model = 'Zero W' - if self.Revision in ('a02082', 'a22082'): - self.model = 'Pi 3 Model B' - self.manufacturer = 'Element14/Premier Farnell' - if self.Revision in ('a01041', '900092', 'a02082', '0012', '0011', '0010', '000e', '0008', '0004'): - self.manufacturer = 'Sony, UK' - if self.Revision in ('0014', '0015', 'a21041', 'a22082'): - self.manufacturer = 'Embest, China' - if self.Revision in ('0005', '0009', '000f'): - self.manufacturer = 'Qisda' - if self.Revision in ('0006', '0007', '000d'): - self.manufacturer = 'Egoman' - - def getManufacturer(self): - return self.manufacturer - - def getModel(self): - return self.model - - def getMac(self, format=2): - if format < 2: - format = 2 - if format > 4: - format = 4 - mac_num = hex(getnode()).replace('0x', '').upper() - mac = '-'.join(mac_num[i : i + format] for i in range(0, 11, format)) - #debug("Mac:" + mac) - return mac diff --git a/myDevices/os/raspiconfig.py b/myDevices/os/raspiconfig.py deleted file mode 100644 index cdddf05..0000000 --- a/myDevices/os/raspiconfig.py +++ /dev/null @@ -1,67 +0,0 @@ -from myDevices.utils.logger import exception, info, warn, error, debug -from myDevices.os.services import ServiceManager -from time import sleep -from myDevices.os.threadpool import ThreadPool - -CUSTOM_CONFIG_SCRIPT = "/etc/myDevices/scripts/config.sh" - -class RaspiConfig: - def Config(config_id, parameters): - if config_id == 0: - return RaspiConfig.ExpandRootfs() - else: - return RaspiConfig.ExecuteConfigCommand(config_id, parameters) - def ExpandRootfs(): - command = "sudo raspi-config --expand-rootfs" - debug('ExpandRootfs command:' + command) - (output, returnCode) = ServiceManager.ExecuteCommand(command) - debug('ExpandRootfs command:' + command + " retCode: " + returnCode) - output = 'reboot required' - return (returnCode, output) - def ExecuteConfigCommand(config_id, parameters): - debug('RaspiConfig::config') - command = "sudo " + CUSTOM_CONFIG_SCRIPT + " " + str(config_id) + " " + str(parameters) - (output, returnCode) = ServiceManager.ExecuteCommand(command) - debug('ExecuteConfigCommand '+ str(config_id) + ' args: ' + str(parameters) + ' retCode: ' + str(returnCode) + ' output: ' + output ) - if "reboot required" in output: - ThreadPool.Submit(RaspiConfig.RestartService) - #del output - return (returnCode, output) - def RestartService(): - sleep(5) - command = "sudo shutdown -r now" - (output, returnCode) = ServiceManager.ExecuteCommand(command) - def getConfig(): - configItem = {} - try: - (returnCode, output) = RaspiConfig.ExecuteConfigCommand(17, '') - if output: - print('output: ' + output) - values = output.strip().split(' ') - configItem['Camera'] = {} - for i in values: - val1 = i.split('=') - configItem['Camera'][val1[0]] = int(val1[1]) - del output - except: - exception('Camera config') - - try: - (returnCode, output) = RaspiConfig.ExecuteConfigCommand(10, '') - if output: - print('output: ' + output) - configItem['DeviceTree'] = int(output.strip()) - del output - (returnCode, output) = RaspiConfig.ExecuteConfigCommand(18, '') - if output: - print('output: ' + output) - configItem['Serial'] = int(output.strip()) - del output - (returnCode, output) = RaspiConfig.ExecuteConfigCommand(20, '') - if output: - print('output: ' + output) - configItem['OneWire'] = int(output.strip()) - del output - except: - exception('Camera config') - return configItem \ No newline at end of file diff --git a/myDevices/os/systeminfo.py b/myDevices/os/systeminfo.py deleted file mode 100644 index ac6c793..0000000 --- a/myDevices/os/systeminfo.py +++ /dev/null @@ -1,181 +0,0 @@ -import psutil -import netifaces -from ctypes import CDLL, c_char_p -from myDevices.utils.logger import exception, info, warn, error, debug, logJson -from os import path, getpid -from json import loads, dumps -from myDevices.os.cpu import CpuInfo - - -class SystemInfo(): - """Class to get system CPU, memory, uptime, storage and network info""" - - def getSystemInformation(self): - """Get a dict containing CPU, memory, uptime, storage and network info""" - system_info = {} - try: - cpu_info = CpuInfo() - system_info['Cpu'] = cpu_info.get_cpu_info() - system_info['CpuLoad'] = cpu_info.get_cpu_load() - system_info['Memory'] = self.getMemoryInfo() - system_info['Uptime'] = self.getUptime() - system_info['Storage'] = self.getDiskInfo() - system_info['Network'] = self.getNetworkInfo() - except: - exception('Error retrieving system info') - finally: - return system_info - - def getMemoryInfo(self): - """Get a dict containing the memory info - - Returned dict example:: - - { - 'used': 377036800, - 'total': 903979008, - 'buffers': 129654784, - 'cached': 135168000, - 'processes': 112214016, - 'free': 526942208, - 'swap': { - 'used': 0, - 'free': 104853504, - 'total': 104853504 - } - } - """ - memory = {} - try: - vmem = psutil.virtual_memory() - memory['total'] = vmem.total - memory['free'] = vmem.available - memory['used'] = memory['total'] - memory['free'] - memory['buffers'] = vmem.buffers - memory['cached'] = vmem.cached - memory['processes'] = memory['used'] - swap = psutil.swap_memory() - memory['swap'] = {} - memory['swap']['total'] = swap.total - memory['swap']['free'] = swap.free - memory['swap']['used'] = swap.used - except Exception as e: - exception('Error getting memory info') - return memory - - def getUptime(self): - """Get system uptime as a dict - - Returned dict example:: - - { - 'uptime': 90844.69, - 'idle': 391082.64 - } - """ - info = {} - uptime = 0.0 - idle = 0.0 - try: - with open('/proc/uptime', 'r') as f_stat: - lines = [line.split(' ') for content in f_stat.readlines() for line in content.split('\n') if line != ''] - uptime = float(lines[0][0]) - idle = float(lines[0][1]) - except Exception as e: - exception('Error getting uptime') - info['uptime'] = uptime - return info - - def getDiskInfo(self): - """Get system uptime as a dict - - Returned dict example:: - - { - 'list': [{ - 'filesystem': 'ext4', - 'size': 13646516224, - 'use': 0.346063, - 'mount': '/', - 'device': '/dev/root', - 'available': 8923963392, - 'used': 4005748736 - }, { - "device": "/dev/mmcblk0p5", - "filesystem": "vfat", - "mount": "/boot", - }] - } - """ - disk_list = [] - try: - for partition in psutil.disk_partitions(True): - disk = {} - disk['filesystem'] = partition.fstype - disk['mount'] = partition.mountpoint - disk['device'] = partition.device - try: - usage = psutil.disk_usage(partition.mountpoint) - if usage.total: - disk['size'] = usage.total - disk['used'] = usage.used - disk['available'] = usage.free - disk['use'] = round((usage.total - usage.free) / usage.total, 6) - except: - pass - disk_list.append(disk) - except: - exception('Error getting disk info') - info = {} - info['list'] = disk_list - return info - - def getNetworkInfo(self): - """Get network information as a dict - - Returned dict example:: - - { - "eth0": { - "ip": { - "address": "192.168.0.25", - }, - "ipv6": [{ - "address": "2001:db8:3c4d::1a2f:1a2b", - }], - "mac": "aa:bb:cc:dd:ee:ff", - }, - "wlan0": { - "ipv6": [{ - "address": "2001:db8:3c4d::1a2f:1a2b", - }], - "mac": "aa:bb:cc:dd:ee:ff" - } - } - """ - network_info = {} - try: - for interface in netifaces.interfaces(): - addresses = netifaces.ifaddresses(interface) - interface_info = {} - try: - addr = addresses[netifaces.AF_INET][0]['addr'] - interface_info['ip'] = {} - interface_info['ip']['address'] = addr - except: - pass - try: - interface_info['ipv6'] = [{'address': addr['addr'].split('%')[0]} for addr in addresses[netifaces.AF_INET6]] - except: - pass - try: - interface_info['mac'] = addresses[netifaces.AF_LINK][0]['addr'] - except: - pass - if interface_info: - network_info[interface] = interface_info - except: - exception('Error getting network info') - info = {} - info['list'] = network_info - return info diff --git a/myDevices/os/threadpool.py b/myDevices/os/threadpool.py deleted file mode 100644 index 96879e9..0000000 --- a/myDevices/os/threadpool.py +++ /dev/null @@ -1,12 +0,0 @@ -from concurrent.futures import ThreadPoolExecutor -from myDevices.utils.singleton import Singleton -import inspect - -executor = ThreadPoolExecutor(max_workers=4) -class ThreadPool(Singleton): - def Submit(something): - future = executor.submit(something) - def SubmitParam(*arg): - executor.submit(*arg) - def Shutdown(): - executor.shutdown() \ No newline at end of file diff --git a/myDevices/sensors/sensors.py b/myDevices/sensors/sensors.py index d0603b9..2dd63d0 100644 --- a/myDevices/sensors/sensors.py +++ b/myDevices/sensors/sensors.py @@ -1,45 +1,41 @@ +""" +This module provides a class for interfacing with sensors and actuators. It can add, edit and remove +sensors and actuators as well as monitor their states and execute commands. +""" from myDevices.utils.logger import exception, info, warn, error, debug, logJson from time import sleep, time -from json import loads,dumps -from threading import Thread, RLock -from myDevices.os import services -from copy import deepcopy +from json import loads, dumps +from threading import RLock, Event +from myDevices.system import services from datetime import datetime, timedelta from os import path, getpid -from urllib import parse -import http.client as httplib -from myDevices.os.daemon import Daemon +from myDevices.utils.daemon import Daemon from myDevices.cloud.dbmanager import DbManager -from myDevices.os.threadpool import ThreadPool -from hashlib import sha1 -from http.client import CannotSendRequest -import urllib.request as req -from myDevices.utils.version import MAPPING +from myDevices.utils.threadpool import ThreadPool from myDevices.devices.bus import checkAllBus, BUSLIST from myDevices.devices.digital.gpio import NativeGPIO as GPIO from myDevices.devices import manager from myDevices.devices import instance from myDevices.utils.types import M_JSON -from myDevices.os.systeminfo import SystemInfo +from myDevices.system.systeminfo import SystemInfo +from myDevices.cloud import cayennemqtt -REFRESH_FREQUENCY = 0.10 #seconds -SENSOR_INFO_SLEEP = 0.05 +REFRESH_FREQUENCY = 5 #seconds +# SENSOR_INFO_SLEEP = 0.05 class SensorsClient(): + """Class for interfacing with sensors and actuators""" + def __init__(self): + """Initialize the bus and sensor info and start monitoring sensor states""" self.sensorMutex = RLock() - self.systemMutex = RLock() - self.continueMonitoring = False + self.exiting = Event() self.onDataChanged = None self.onSystemInfo = None - self.currentBusInfo = self.previousBusInfo = None - self.currentSensorsInfo = self.previousSensorsInfo = None - self.currentSystemInfo = self.previousSystemInfo = None + self.systemData = [] + self.currentSystemState = [] self.disabledSensors = {} - self.sensorsRefreshCount = 0 - self.retrievingSystemInfo = False self.disabledSensorTable = "disabled_sensors" - self.systemInfoRefreshList = [] checkAllBus() self.gpio = GPIO() manager.addDeviceInstance("GPIO", "GPIO", "GPIO", self.gpio, [], "system") @@ -49,164 +45,84 @@ def __init__(self): for row in results: self.disabledSensors[row[0]] = 1 self.StartMonitoring() - def SetDataChanged(self, onDataChanged = None, onSystemInfo = None): - #one parameter only that contains data in a dictionary for changed items only + + def SetDataChanged(self, onDataChanged=None, onSystemInfo=None): + """Set callbacks to call when data has changed + + Args: + onDataChanged: Function to call when sensor data changes + onSystemInfo: Function to call when system info changes + """ self.onDataChanged = onDataChanged self.onSystemInfo = onSystemInfo + def StartMonitoring(self): - self.continueMonitoring = True - #thread = Thread(target = self.Monitor) - #thread.start() + """Start thread monitoring sensor data""" ThreadPool.Submit(self.Monitor) + def StopMonitoring(self): - self.continueMonitoring = False + """Stop thread monitoring sensor data""" + self.exiting.set() + def Monitor(self): - nextTime = datetime.now() - nextTimeSystemInfo = datetime.now() + """Monitor bus/sensor states and system info and report changed data via callbacks""" debug('Monitoring sensors and os resources started') - while self.continueMonitoring: + while not self.exiting.is_set(): try: - if datetime.now() > nextTime: - self.raspberryValue = None - self.raspberryValue = {} - refreshTime = int(time()) - if datetime.now() > nextTimeSystemInfo: - with self.systemMutex: - if not self.retrievingSystemInfo: - ThreadPool.Submit(self.MonitorSystemInformation()) - nextTimeSystemInfo = datetime.now() + timedelta(seconds=5) + if not self.exiting.wait(REFRESH_FREQUENCY): + self.currentSystemState = [] + self.MonitorSystemInformation() self.MonitorSensors() self.MonitorBus() - if self.onDataChanged != None: - self.onDataChanged(self.raspberryValue) - bResult=self.RemoveRefresh(refreshTime) - if bResult == True and self.onSystemInfo != None: - self.onSystemInfo() - self.sensorsRefreshCount += 1 - nextTime = datetime.now() + timedelta(seconds=REFRESH_FREQUENCY) - sleep(REFRESH_FREQUENCY) + if self.currentSystemState != self.systemData: + changedSystemData = self.currentSystemState + if self.systemData: + changedSystemData = [x for x in self.currentSystemState if x not in self.systemData] + if self.onDataChanged and changedSystemData: + self.onDataChanged(changedSystemData) + self.systemData = self.currentSystemState except: - exception("Monitoring sensors and os resources failed: " + str() ) - debug('Monitoring sensors and os resources Finished') + exception('Monitoring sensors and os resources failed') + debug('Monitoring sensors and os resources finished') + def MonitorSensors(self): - if self.continueMonitoring == False: + """Check sensor states for changes""" + if self.exiting.is_set(): return - debug(str(time()) + ' Get sensors info ' + str(self.sensorsRefreshCount)) - self.SensorsInfo() - debug(str(time()) + ' Got sensors info ' + str(self.sensorsRefreshCount)) - with self.sensorMutex: - if self.SHA_Calc(self.currentSensorsInfo) != self.SHA_Calc(self.previousSensorsInfo): - #do merge - mergedSensors = self.ChangedSensorsList() - if self.previousSensorsInfo: - del self.previousSensorsInfo - self.previousSensorsInfo = None - if mergedSensors == None: - self.previousSensorsInfo = self.currentSensorsInfo - return - self.raspberryValue['SensorsInfo'] = mergedSensors - self.previousSensorsInfo = self.currentSensorsInfo - debug(str(time()) + ' Merge sensors info ' + str(self.sensorsRefreshCount)) - def ChangedSensorsList(self): - if self.continueMonitoring == False: - return None - if self.previousSensorsInfo == None: - return None - if self.currentSensorsInfo == None: - return None - changedSensors = [] - previousSensorsDictionary = dict((i['sensor'], i) for i in self.previousSensorsInfo) - for item in self.currentSensorsInfo: - oldItem = None - if item['sensor'] in previousSensorsDictionary: - oldItem = previousSensorsDictionary[item['sensor']] - if self.SHA_Calc(item) != self.SHA_Calc(oldItem): - changedSensors.append(item) - else: - changedSensors.append(item) - return changedSensors + self.currentSystemState += self.SensorsInfo() + def MonitorBus(self): - if self.continueMonitoring == False: + """Check bus states for changes""" + if self.exiting.is_set(): return - debug(str(time()) + ' Get bus info ' + str(self.sensorsRefreshCount)) - self.BusInfo() - debug(str(time()) + ' Got bus info ' + str(self.sensorsRefreshCount)) - if self.SHA_Calc(self.currentBusInfo) != self.SHA_Calc(self.previousBusInfo): - self.raspberryValue['BusInfo'] = self.currentBusInfo - if self.previousBusInfo: - del self.previousBusInfo - self.previousBusInfo = None - self.previousBusInfo = self.currentBusInfo + self.currentSystemState += self.BusInfo() + def MonitorSystemInformation(self): - if self.continueMonitoring == False: + """Check system info for changes""" + if self.exiting.is_set(): return - debug(str(time()) + ' Get system info ' + str(self.sensorsRefreshCount)) - self.SystemInformation() - debug(str(time()) + ' Got system info ' + str(self.sensorsRefreshCount)) - firstSHA = self.SHA_Calc(self.currentSystemInfo) - secondSHA = self.SHA_Calc(self.previousSystemInfo) - if firstSHA != secondSHA: - if self.previousSystemInfo: - del self.previousSystemInfo - self.previousSystemInfo = None - self.previousSystemInfo = self.currentSystemInfo - self.raspberryValue['SystemInfo'] = self.currentSystemInfo + self.currentSystemState += self.SystemInformation() + def SystemInformation(self): - with self.systemMutex: - self.retrievingSystemInfo = True + """Return dict containing current system info, including CPU, RAM, storage and network info""" + newSystemInfo = [] try: systemInfo = SystemInfo() newSystemInfo = systemInfo.getSystemInformation() - if newSystemInfo: - self.currentSystemInfo = newSystemInfo - except Exception as ex: - exception('SystemInformation failed: '+str(ex)) - with self.systemMutex: - self.retrievingSystemInfo = False - return self.currentSystemInfo - def SHA_Calc(self, object): - if object == None: - return '' - try: - strVal = dumps(object) - except: - exception('SHA_Calc failed for:' + str(object)) - return '' - return self.SHA_Calc_str(strVal) - def SHA_Calc_str(self, stringVal): - m = sha1() - m.update(stringVal.encode('utf8')) - sDigest = str(m.hexdigest()) - return sDigest - def AppendToDeviceList(self, device_list, source, device_type): - device = source.copy() - del device['origin'] - device['name'] = parse.unquote(device['name']) - device['type'] = device_type - if len(source['type']) > 1: - device['hash'] = self.SHA_Calc_str(device['name']+device['type']) - else: - device['hash'] = self.SHA_Calc_str(device['name']+device['device']) - if device['hash'] in self.disabledSensors: - device['enabled'] = 0 - else: - device['enabled'] = 1 - device_list.append(device) - def GetDevices(self): - manager.deviceDetector() - device_list = manager.getDeviceList() - devices = [] - for dev in device_list: - try: - if len(dev['type']) == 0: - self.AppendToDeviceList(devices, dev, '') - else: - for device_type in dev['type']: - self.AppendToDeviceList(devices, dev, device_type) - except: - exception ("Failed to get device: {}".format(dev)) - return devices + except Exception: + exception('SystemInformation failed') + return newSystemInfo + def CallDeviceFunction(self, func, *args): + """Call a function for a sensor/actuator device and format the result value type + + Args: + func: Function to call + args: Parameters to pass to the function + + Returns: + True for success, False otherwise. + """ result = func(*args) if result != None: if hasattr(func, "contentType"): @@ -218,98 +134,86 @@ def CallDeviceFunction(self, func, *args): else: response = result return response + def BusInfo(self): - json = {} - for (bus, value) in BUSLIST.items(): - json[bus] = int(value["enabled"]) - gpios = {} - for gpio in range(GPIO.GPIO_COUNT): - gpios[gpio] = {} - gpios[gpio]['function'] = self.gpio.getFunctionString(gpio) - gpios[gpio]['value'] = int(self.gpio.input(gpio)) - json['GPIO'] = gpios - json['GpioMap'] = MAPPING - self.currentBusInfo = json - return self.currentBusInfo + """Return a dict with current bus info""" + bus_info = [] + gpio_state = self.gpio.wildcard() + for key, value in gpio_state.items(): + cayennemqtt.DataChannel.add(bus_info, cayennemqtt.SYS_GPIO, key, cayennemqtt.VALUE, value['value']) + cayennemqtt.DataChannel.add(bus_info, cayennemqtt.SYS_GPIO, key, cayennemqtt.FUNCTION, value['function']) + return bus_info + def SensorsInfo(self): - devices = self.GetDevices() - debug(str(time()) + ' Got devices info ' + str(self.sensorsRefreshCount)) + """Return a list with current sensor states for all enabled sensors""" + manager.deviceDetector() + devices = manager.getDeviceList() + sensors_info = [] if devices is None: - return {} - for value in devices: - sensor = instance.deviceInstance(value['name']) - if 'enabled' not in value or value['enabled'] == 1: - sleep(SENSOR_INFO_SLEEP) - try: - if value['type'] == 'Temperature': - value['Celsius'] = self.CallDeviceFunction(sensor.getCelsius) - value['Fahrenheit'] = self.CallDeviceFunction(sensor.getFahrenheit) - value['Kelvin'] =self.CallDeviceFunction(sensor.getKelvin) - if value['type'] == 'Pressure': - value['Pascal'] = self.CallDeviceFunction(sensor.getPascal) - if value['type'] == 'Luminosity': - value['Lux'] = self.CallDeviceFunction(sensor.getLux) - if value['type'] == 'Distance': - value['Centimeter'] = self.CallDeviceFunction(sensor.getCentimeter) - value['Inch'] = self.CallDeviceFunction(sensor.getInch) - if value['type'] in ('ADC', 'DAC'): - value['channelCount'] = self.CallDeviceFunction(sensor.analogCount) - value['maxInteger'] = self.CallDeviceFunction(sensor.analogMaximum) - value['resolution'] = self.CallDeviceFunction(sensor.analogResolution) - value['allInteger'] = self.CallDeviceFunction(sensor.analogReadAll) - value['allVolt'] = self.CallDeviceFunction(sensor.analogReadAllVolt) - value['allFloat'] = self.CallDeviceFunction(sensor.analogReadAllFloat) - if value['type'] == 'DAC': - value['vref'] = self.CallDeviceFunction(sensor.analogReference) - if value['type'] == 'PWM': - value['channelCount'] = self.CallDeviceFunction(sensor.pwmCount) - value['maxInteger'] = self.CallDeviceFunction(sensor.pwmMaximum) - value['resolution'] = self.CallDeviceFunction(sensor.pwmResolution) - value['all'] = self.CallDeviceFunction(sensor.pwmWildcard) - if value['type'] == 'Humidity': - value['float']=self.CallDeviceFunction(sensor.getHumidity) - value['percent']=self.CallDeviceFunction(sensor.getHumidityPercent) - if value['type'] == 'PiFaceDigital': - value['all'] = self.CallDeviceFunction(sensor.readAll) - if value['type'] in ('DigitalSensor', 'DigitalActuator'): - value['value'] = self.CallDeviceFunction(sensor.read) - if value['type'] == 'GPIOPort': - value['channelCount'] = self.CallDeviceFunction(sensor.digitalCount) - value['all'] = self.CallDeviceFunction(sensor.wildcard) - if value['type'] == 'AnalogSensor': - value['integer'] = self.CallDeviceFunction(sensor.read) - value['float'] = self.CallDeviceFunction(sensor.readFloat) - value['volt'] = self.CallDeviceFunction(sensor.readVolt) - if value['type'] == 'ServoMotor': - value['angle'] = self.CallDeviceFunction(sensor.readAngle) - if value['type'] == 'AnalogActuator': - value['float'] = self.CallDeviceFunction(sensor.readFloat) - except: - exception ("Sensor values failed: "+ value['type'] + " " + value['name']) - try: - if 'hash' in value: - value['sensor'] = value['hash'] - del value['hash'] - except KeyError: - pass - with self.sensorMutex: - if self.currentSensorsInfo: - del self.currentSensorsInfo - self.currentSensorsInfo = None - self.currentSensorsInfo = devices - devices = None - if self.sensorsRefreshCount == 0: - info('System sensors info at start '+str(self.currentSensorsInfo)) - debug(('New sensors info retrieved: {}').format(self.sensorsRefreshCount)) - logJson('Sensors Info updated: ' + str(self.currentSensorsInfo)) - return self.currentSensorsInfo + return sensors_info + for device in devices: + sensor = instance.deviceInstance(device['name']) + if 'enabled' not in device or device['enabled'] == 1: + sensor_types = {'Temperature': {'function': 'getCelsius', 'data_args': {'type': 'temp', 'unit': 'c'}}, + 'Humidity': {'function': 'getHumidityPercent', 'data_args': {'type': 'rel_hum', 'unit': 'p'}}, + 'Pressure': {'function': 'getPascal', 'data_args': {'type': 'bp', 'unit': 'pa'}}, + 'Luminosity': {'function': 'getLux', 'data_args': {'type': 'lum', 'unit': 'lux'}}, + 'Distance': {'function': 'getCentimeter', 'data_args': {'type': 'prox', 'unit': 'cm'}}, + 'ServoMotor': {'function': 'readAngle', 'data_args': {'type': 'analog_actuator'}}, + 'DigitalSensor': {'function': 'read', 'data_args': {'type': 'digital_sensor', 'unit': 'd'}}, + 'DigitalActuator': {'function': 'read', 'data_args': {'type': 'digital_actuator', 'unit': 'd'}}, + 'AnalogSensor': {'function': 'readFloat', 'data_args': {'type': 'analog_sensor'}}, + 'AnalogActuator': {'function': 'readFloat', 'data_args': {'type': 'analog_actuator'}}} + extension_types = {'ADC': {'function': 'analogReadAllFloat'}, + 'DAC': {'function': 'analogReadAllFloat'}, + 'PWM': {'function': 'pwmWildcard'}, + 'GPIOPort': {'function': 'wildcard'}} + for device_type in device['type']: + try: + display_name = device['description'] + except: + display_name = None + if device_type in sensor_types: + try: + sensor_type = sensor_types[device_type] + func = getattr(sensor, sensor_type['function']) + if len(device['type']) > 1: + channel = '{}:{}'.format(device['name'], device_type.lower()) + else: + channel = device['name'] + cayennemqtt.DataChannel.add(sensors_info, cayennemqtt.DEV_SENSOR, channel, value=self.CallDeviceFunction(func), name=display_name, **sensor_type['data_args']) + except: + exception('Failed to get sensor data: {} {}'.format(device_type, device['name'])) + else: + try: + extension_type = extension_types[device_type] + func = getattr(sensor, extension_type['function']) + values = self.CallDeviceFunction(func) + for pin, value in values.items(): + cayennemqtt.DataChannel.add(sensors_info, cayennemqtt.DEV_SENSOR, device['name'] + ':' + str(pin), cayennemqtt.VALUE, value, name=display_name) + except: + exception('Failed to get extension data: {} {}'.format(device_type, device['name'])) + logJson('Sensors info: {}'.format(sensors_info)) + return sensors_info + def AddSensor(self, name, description, device, args): + """Add a new sensor/actuator + + Args: + name: Name of sensor to add + description: Sensor description + device: Sensor device class + args: Sensor specific args + + Returns: + True for success, False otherwise. + """ info('AddSensor: {}, {}, {}, {}'.format(name, description, device, args)) bVal = False try: sensorAdd = {} if name: - sensorAdd['name'] = req.pathname2url(name) + sensorAdd['name'] = name if device: sensorAdd['device'] = device if args: @@ -321,16 +225,27 @@ def AddSensor(self, name, description, device, args): info('Add device returned: {}'.format(retValue)) if retValue[0] == 200: bVal = True - self.AddRefresh() except: bVal = False return bVal + def EditSensor(self, name, description, device, args): + """Edit an existing sensor/actuator + + Args: + name: Name of sensor to edit + description: New sensor description + device: New sensor device class + args: New sensor specific args + + Returns: + True for success, False otherwise. + """ info('EditSensor: {}, {}, {}, {}'.format(name, description, device, args)) bVal = False try: - sensorEdit= {} - name = req.pathname2url(name) + sensorEdit = {} + name = name sensorEdit['name'] = name sensorEdit['device'] = device sensorEdit['description'] = description @@ -338,48 +253,45 @@ def EditSensor(self, name, description, device, args): with self.sensorMutex: retValue = manager.updateDevice(name, sensorEdit) info('Edit device returned: {}'.format(retValue)) - try: - hashKey = self.SHA_Calc_str(name+device) - with self.sensorMutex: - if self.currentSensorsInfo: - currentSensorsDictionary = dict((i['sensor'], i) for i in self.currentSensorsInfo) - sensorData = currentSensorsDictionary[hashKey] - sensor = sensorData[hashKey] - raspberryValue = {} - sensor['args'] = args - sensor['description'] = description - raspberryValue['SensorsInfo'] = [] - raspberryValue['SensorsInfo'].append(sensor) - if self.onDataChanged != None: - self.onDataChanged(raspberryValue) - except: - pass if retValue[0] == 200: bVal = True - self.AddRefresh() except: - exception ("Edit sensor failed") + exception("Edit sensor failed") bVal = False return bVal - def DeleteSensor(self, name): + + def RemoveSensor(self, name): + """Remove an existing sensor/actuator + + Args: + name: Name of sensor to remove + + Returns: + True for success, False otherwise. + """ bVal = False try: - sensorRemove = req.pathname2url(name) + sensorRemove = name with self.sensorMutex: retValue = manager.removeDevice(sensorRemove) info('Remove device returned: {}'.format(retValue)) if retValue[0] == 200: bVal = True - self.AddRefresh() except: - exception ("Remove sensor failed") + exception("Remove sensor failed") bVal = False return bVal - def RemoveSensor(self, name): - debug('') - return self.DeleteSensor(name) + def EnableSensor(self, sensor, enable): - #sensor is the hash composed from name and device class/type + """Enable a sensor/actuator + + Args: + sensor: Hash composed from name and device class/type + enable: 1 to enable, 0 to disable + + Returns: + True for success, False otherwise. + """ info('Enable sensor: ' + str(sensor) + ' ' + str(enable)) try: if sensor is None: @@ -403,108 +315,70 @@ def EnableSensor(self, sensor, enable): return False self.AddRefresh() return True - def AddRefresh(self): - self.systemInfoRefreshList.append(int(time())) - def RemoveRefresh(self, newRefresh): - bReturn = False - for i in self.systemInfoRefreshList: - if i < newRefresh: - self.systemInfoRefreshList.remove(i) - bReturn = True - return bReturn - def GpioCommand(self, commandType, method, channel, value): - debug('') - info('GpioCommand ' + commandType + ' method ' + method + ' Channel: ' + str(channel) + ' Value: ' + str(value)) - if commandType == 'function': - if method == 'POST': - debug('setFunction:' + str(channel) + ' ' + str(value)) - return str(self.gpio.setFunctionString(channel, value)) - if method == 'GET': - debug('getFunction:' + str(channel) + ' ' + str(value)) - return str(self.gpio.getFunctionString(channel)) - if commandType == 'value': - if method == 'POST': - debug('digitalWrite:' + str(channel) + ' ' + str(value)) - retVal = str(self.gpio.digitalWrite(channel, value)) - return retVal - if method == 'GET': - debug('digitalRead:' + str(channel)) - return str(self.gpio.digitalRead(channel)) - if commandType == 'integer': - if method == 'POST': - debug('portWrite:' + str(value)) - return str(self.gpio.portWrite(value)) - if method == 'GET': - debug('portRead:' ) - return str(self.gpio.portRead()) - debug.log('GpioCommand not set') + + def GpioCommand(self, command, channel, value): + """Execute onboard GPIO command + + Args: + command: Type of command to execute + channel: GPIO pin + value: Value to use for writing data + + Returns: + String containing command specific return value on success, or 'failure' on failure + """ + info('GpioCommand {}, channel {}, value {}'.format(command, channel, value)) + if command == 'function': + if value.lower() in ('in', 'input'): + return str(self.gpio.setFunctionString(channel, 'in')) + elif value.lower() in ('out', 'output'): + return str(self.gpio.setFunctionString(channel, 'out')) + elif command in ('value', ''): + return self.gpio.digitalWrite(channel, int(value)) + debug('GPIO command failed') return 'failure' - def SensorCommand(self, commandType, sensorName, sensorType, driverClass, method, channel, value): - retVal = False - info('SensorCommand: {} SensorName {} SensorType {} DriverClass {} Method {} Channel {} Value {}'.format(commandType, sensorName, sensorType, driverClass, method, channel, value) ) + + def SensorCommand(self, command, sensorId, channel, value): + """Execute sensor/actuator command + + Args: + command: Type of command to execute + sensorId: Sensor id + channel: Pin/channel on device, None if there is no channel + value: Value to use for setting the sensor state + + Returns: + Command specific return value on success, False on failure + """ + result = False + info('SensorCommand: {}, sensor {}, channel {}, value {}'.format(command, sensorId, channel, value)) try: - self.AddRefresh() - debug('') - actuators=('GPIOPort', 'ServoMotor', 'AnalogActuator', 'LoadSensor', 'PiFaceDigital', 'DistanceSensor', 'Thermistor', 'Photoresistor', 'LightDimmer', 'LightSwitch', 'DigitalSensor', 'DigitalActuator', 'MotorSwitch', 'RelaySwitch', 'ValveSwitch', 'MotionSensor') - gpioExtensions=('GPIOPort', 'PiFaceDigital') - if driverClass is None: - hashKey = self.SHA_Calc_str(sensorName+sensorType) - else: - hashKey = self.SHA_Calc_str(sensorName+driverClass) + commands = {'integer': {'function': 'write', 'value_type': int}, + 'value': {'function': 'write', 'value_type': int}, + 'function': {'function': 'setFunctionString', 'value_type': str}, + 'angle': {'function': 'writeAngle', 'value_type': float}, + 'float': {'function': 'writeFloat', 'value_type': float}, + 'volt': {'function': 'writeVolt', 'value_type': float}} with self.sensorMutex: - if hashKey in self.disabledSensors: - return retVal - sensor = instance.deviceInstance(sensorName) + if sensorId in self.disabledSensors: + info('Sensor disabled') + return result + sensor = instance.deviceInstance(sensorId) if not sensor: info('Sensor not found') - return retVal - if (sensorType in actuators) or (driverClass in actuators): - if sensorType in gpioExtensions or driverClass in gpioExtensions: - if commandType == 'integer' or commandType == 'value': - retVal = str(self.CallDeviceFunction(sensor.write, int(channel), int(value))) - return retVal + return result + if command in commands: + info('Sensor found: {}'.format(instance.DEVICES[sensorId])) + func = getattr(sensor, commands[command]['function']) + value = commands[command]['value_type'](value) + if channel: + result = self.CallDeviceFunction(func, int(channel), value) else: - if commandType == 'integer': - retVal = str(self.CallDeviceFunction(sensor.write, int(value))) - return retVal - if commandType == 'function': - retVal = str(self.CallDeviceFunction(sensor.setFunctionString, channel, value)) - return retVal - if commandType == 'angle': - retVal = self.CallDeviceFunction(sensor.writeAngle, value) - return retVal - if commandType == 'float': - retVal = self.CallDeviceFunction(sensor.writeFloat, float(value)) - return retVal - if commandType == 'integer': - retVal = float(self.CallDeviceFunction(sensor.write, int(channel), int(value))) - return retVal - if commandType == 'float': - retVal = float(self.CallDeviceFunction(sensor.writeFloat, int(channel), float(value))) - return retVal - if commandType == 'volt': - retVal = float(self.CallDeviceFunction(sensor.writeVolt, int(channel), float(value))) - return retVal - if commandType == 'angle': - retVal = float(self.CallDeviceFunction(sensor.writeAngle, int(channel), float(value))) - return retVal - warn('Command not implemented: ' + commandType) - return retVal - except Exception as ex: - exception('SensorCommand failed with: ' +str(ex)) - pass - finally: - #looks like this breaks actuators refresh by updating and not sending data changed - #then refresh never comes for the specific sensor - #ThreadPool.SubmitParam(self.Monitor, hashKey) - return retVal - # def SensorCommandTest(self): - # # self.SensorCommand('integer', 'ledswitch', 'DigitalActuator', '', '', 1) - # # sleep(5) - # # self.SensorCommand('integer', 'motorswitch', 'DigitalActuator', '', '', 1) - # sleep(5) - # self.SensorCommand('integer', 'motorswitch', 'DigitalActuator', '', '', 0) - # sleep(5) - # self.SensorCommand('integer', 'ledswitch', 'DigitalActuator', '', '', 0) - -#SensorCommandTest() + result = self.CallDeviceFunction(func, value) + return result + warn('Command not implemented: {}'.format(command)) + return result + except Exception: + exception('SensorCommand failed') + return result + diff --git a/myDevices/os/__init__.py b/myDevices/system/__init__.py similarity index 100% rename from myDevices/os/__init__.py rename to myDevices/system/__init__.py diff --git a/myDevices/os/cpu.py b/myDevices/system/cpu.py similarity index 92% rename from myDevices/os/cpu.py rename to myDevices/system/cpu.py index 0b9f36a..1df45fa 100644 --- a/myDevices/os/cpu.py +++ b/myDevices/system/cpu.py @@ -7,7 +7,8 @@ class CpuInfo(object): """Class for retrieving CPU info""" - def get_cpu_info(self): + @staticmethod + def get_cpu_info(): """Return CPU temperature, load average and usage info as a dict""" info = {} info['temperature'] = self.get_cpu_temp() @@ -15,7 +16,8 @@ def get_cpu_info(self): info["usage"] = self.get_cpu_usage() return info - def get_cpu_usage(self): + @staticmethod + def get_cpu_usage(): """Return dict with overall CPU usage""" usage = {} try: @@ -25,8 +27,9 @@ def get_cpu_usage(self): except: exception('Error getting CPU usage info') return usage - - def get_cpu_load(self, interval = 1): + + @staticmethod + def get_cpu_load(interval = 1): """Return CPU load :param interval: time interval in seconds to wait when calculating CPU usage @@ -39,7 +42,8 @@ def get_cpu_load(self, interval = 1): exception('Error getting CPU load info') return cpu_load - def get_cpu_temp(self): + @staticmethod + def get_cpu_temp(): """Get CPU temperature""" info = {} thermal_dirs = glob('/sys/class/thermal/thermal_zone*') @@ -62,7 +66,8 @@ def get_cpu_temp(self): exception('Error getting CPU temperature') return temp - def get_load_avg(self): + @staticmethod + def get_load_avg(): """Get CPU average load for the last one, five, and 10 minute periods""" info = {} file = "/proc/loadavg" diff --git a/myDevices/system/hardware.py b/myDevices/system/hardware.py new file mode 100644 index 0000000..703e3f3 --- /dev/null +++ b/myDevices/system/hardware.py @@ -0,0 +1,148 @@ +""" +This module provides constants for the board revision info and pin mapping as well as +a class for getting hardware info, including manufacturer, model and MAC address. +""" +import re +import sys +from myDevices.utils.logger import exception, info, warn, error, debug + +BOARD_REVISION = 0 +CPU_REVISION = "0" +CPU_HARDWARE = "" + +try: + with open("/proc/cpuinfo") as f: + cpuinfo = f.read() + rc = re.compile("Revision\s*:\s(.*)\n") + result = rc.search(cpuinfo) + if result: + CPU_REVISION = result.group(1) + if CPU_REVISION.startswith("1000"): + CPU_REVISION = CPU_REVISION[-4:] + if CPU_REVISION != "0000": + cpurev = int(CPU_REVISION, 16) + if cpurev < 0x04: + BOARD_REVISION = 1 + elif cpurev < 0x10: + BOARD_REVISION = 2 + else: + BOARD_REVISION = 3 + rc = re.compile("Hardware\s*:\s(.*)\n") + result = rc.search(cpuinfo) + CPU_HARDWARE = result.group(1) +except: + exception("Error reading cpuinfo") + + +class Hardware: + """Class for getting hardware info, including manufacturer, model and MAC address.""" + + def __init__(self): + """Initialize board revision and model info""" + self.Revision = '0' + self.Serial = None + try: + with open('/proc/cpuinfo','r') as f: + for line in f: + splitLine = line.split(':') + if len(splitLine) < 2: + continue + key = splitLine[0].strip() + value = splitLine[1].strip() + if key == 'Revision': + self.Revision = value + if key == 'Serial' and value != len(value) * '0': + self.Serial = value + except: + exception ("Error reading cpuinfo") + self.model = 'Unknown' + if self.Revision == 'Beta': + self.model = 'Raspberry Pi Model B (Beta)' + if self.Revision in ('000d', '000e', '000f', '0002', '0003', '0004', '0005', '0006'): + self.model = 'Raspberry Pi Model B' + if self.Revision in ('0007', '0008', '0009'): + self.model = 'Raspberry Pi Model A' + if self.Revision in ('0010', '0013', '900032'): + self.model = 'Raspberry Pi Model B +' + if self.Revision in ('0011', '0014'): + self.model = 'Raspberry Pi Compute Module' + if self.Revision in ('0012', '0015'): + self.model = 'Raspberry Pi Model A+' + if self.Revision in ('a01040', 'a01041', 'a21041', 'a22042'): + self.model = 'Raspberry Pi 2 Model B' + if self.Revision in ('900092', '900093', '920093'): + self.model = 'Raspberry Pi Zero' + if self.Revision in ('9000c1',): + self.model = 'Raspberry Pi Zero W' + if self.Revision in ('a02082', 'a22082', 'a32082'): + self.model = 'Raspberry Pi 3 Model B' + if self.Revision in ('a020d3'): + self.model = 'Raspberry Pi 3 Model B+' + if self.Revision in ('a020a0'): + self.model = 'Raspberry Pi Compute Module 3' + if 'Rockchip' in CPU_HARDWARE: + self.model = 'Tinker Board' + self.manufacturer = 'Element14/Premier Farnell' + if self.Revision in ('a01041', '900092', 'a02082', '0012', '0011', '0010', '000e', '0008', '0004', 'a020d3', 'a01040', 'a020a0'): + self.manufacturer = 'Sony, UK' + if self.Revision in ('a32082'): + self.manufacturer = 'Sony, Japan' + if self.Revision in ('0014', '0015', 'a21041', 'a22082', '920093'): + self.manufacturer = 'Embest, China' + if self.Revision in ('0005', '0009', '000f'): + self.manufacturer = 'Qisda' + if self.Revision in ('0006', '0007', '000d'): + self.manufacturer = 'Egoman' + if self.Revision == '0000': + if 'Rockchip' in CPU_HARDWARE: + self.manufacturer = 'ASUS' + else: + try: + with open('/proc/device-tree/model', 'r') as model_file: + for line in model_file: + if 'BeagleBone' in line: + index = line.index('BeagleBone') + self.manufacturer = line[:index - 1].strip(' \n\t\0') + self.model = line[index:].strip(' \n\t\0') + break + except: + exception ("Error reading model") + + + def getManufacturer(self): + """Return manufacturer name as string""" + return self.manufacturer + + def getModel(self): + """Return model name as string""" + return self.model + + def getMac(self): + """Return MAC address as a string or None if no MAC address is found""" + # Import netifaces here to prevent error importing this module in setup.py + import netifaces + interfaces = ['eth0', 'wlan0'] + try: + interfaces.append(netifaces.gateways()['default'][netifaces.AF_INET][1]) + except: + pass + for interface in interfaces: + try: + return netifaces.ifaddresses(interface)[netifaces.AF_LINK][0]['addr'] + except ValueError: + pass + except: + exception('Error getting MAC address') + return None + + def isRaspberryPi(self): + """Return True if device is a Raspberry Pi""" + return 'Raspberry Pi' in self.model + + def isTinkerBoard(self): + """Return True if device is a Tinker Board""" + return 'Tinker Board' == self.model + + def isBeagleBone(self): + """Return True if device is a BeagleBone""" + return 'BeagleBone' in self.model diff --git a/myDevices/ipgetter/__init__.py b/myDevices/system/ipgetter.py similarity index 94% rename from myDevices/ipgetter/__init__.py rename to myDevices/system/ipgetter.py index 5f8f95c..519389e 100644 --- a/myDevices/ipgetter/__init__.py +++ b/myDevices/system/ipgetter.py @@ -19,8 +19,8 @@ >>> ipgetter.IPgetter().test() Number of servers: 47 - IP's : - 8.8.8.8 = 47 ocurrencies + IPs: + 8.8.8.8 = 47 occurrences Copyright 2014 phoemur@gmail.com @@ -49,7 +49,6 @@ def myip(): class IPgetter(object): - ''' This class is designed to fetch your external IP address from the internet. It is used mostly when behind a NAT. @@ -108,7 +107,6 @@ def get_externalip(self): ''' This function gets your IP from a random server ''' - myip = '' for i in range(7): myip = self.fetch(choice(self.server_list)) @@ -131,7 +129,7 @@ def fetch(self, server): url = opener.open(server, timeout=2) content = url.read() - # Didn't want to import chardet. Prefered to stick to stdlib + # Didn't want to import chardet. Preferred to stick to stdlib if PY3K: try: content = content.decode('UTF-8') @@ -155,7 +153,6 @@ def test(self): on the list when retrieving your IP. All results should be the same. ''' - resultdict = {} for server in self.server_list: resultdict.update(**{server: self.fetch(server)}) @@ -163,9 +160,9 @@ def test(self): ips = sorted(resultdict.values()) ips_set = set(ips) print('\nNumber of servers: {}'.format(len(self.server_list))) - print("IP's :") - for ip, ocorrencia in zip(ips_set, map(lambda x: ips.count(x), ips_set)): - print('{0} = {1} ocurrenc{2}'.format(ip if len(ip) > 0 else 'broken server', ocorrencia, 'y' if ocorrencia == 1 else 'ies')) + print("IPs:") + for ip, occurrence in zip(ips_set, map(lambda x: ips.count(x), ips_set)): + print(' {0} = {1} occurrences'.format(ip if len(ip) > 0 else 'broken server', occurrence)) print('\n') print(resultdict) diff --git a/myDevices/os/services.py b/myDevices/system/services.py similarity index 63% rename from myDevices/os/services.py rename to myDevices/system/services.py index 1f185d9..0d7ed29 100644 --- a/myDevices/os/services.py +++ b/myDevices/system/services.py @@ -1,44 +1,27 @@ +""" +This module provides classes for retrieving process and service info, as well as managing processes and services. +""" from subprocess import Popen, PIPE from enum import Enum, unique -from myDevices.utils.logger import exception, info, warn, error, debug -from psutil import Process, process_iter, virtual_memory, cpu_percent from threading import RLock +from psutil import Process, process_iter, virtual_memory, cpu_percent +from myDevices.utils.logger import exception, info, warn, error, debug +from myDevices.utils.subprocess import executeCommand + class ProcessInfo: + """Class for getting process info and killing processes""" + def __init__(self): + """Initialize process information""" self.Name = None - self.Pid = None - self.Username = None + self.Pid = None + self.Username = None self.Cmdline = None - # self.Cwd = None - # self.Process = None - # self.CpuPercent = None - # self.Exe = None - # self.Status = None - # self.CreateTime = None - # self.MemoryPercent = None - # self.NumThreads = None - # self.NumFds = None - def Suspend(self): - debug('ProcessManager::Suspend Name:' + self.Name + ' PID:' + str(self.Pid)) - try: - process = Process(self.Pid) - process.suspend() - except Exception as ex: - error('ProcessInfo::Suspend failed Name:' + self.Name + ' PID:' + str(self.Pid) + ' Exception:' + str(ex)) - return False - return True - def Resume(self): - debug('ProcessManager::Resume Name:' + self.Name + ' PID:' + str(self.Pid)) - try: - process = Process(self.Pid) - process.resume() - except Exception as ex: - error('ProcessInfo::Resume failed Name:' + self.Name + ' PID:' + str(self.Pid) + ' Exception:' + str(ex)) - return False - return True + def Terminate(self): - print('ProcessManager::Terminate Name:' + self.Name + ' PID:' + str(self.Pid)) + """Terminate the process""" + info('ProcessManager::Terminate Name:' + self.Name + ' PID:' + str(self.Pid)) try: process = Process(self.Pid) process.terminate() @@ -46,16 +29,10 @@ def Terminate(self): error('ProcessInfo::Terminate failed Name:' + self.Name + ' PID:' + str(self.Pid) + ' Exception:' + str(ex)) return False return True - def Wait(self, waitTimeout): - debug('ProcessManager::Wait Name:' + self.Name + ' PID:' + str(self.Pid)) - try: - process = Process(self.Pid) - process.wait(timeout = waitTimeout) - except Exception as ex: - error('ProcessInfo::Wait failed Name:' + self.Name + ' PID:' + str(self.Pid) + ' Exception:' + str(ex)) - return False - return True + + @staticmethod def IsRunning(pid): + """Return True if process with specified pid is running""" try: process = Process(pid) except Exception as ex: @@ -64,7 +41,10 @@ def IsRunning(pid): return True class ProcessManager: + """Class for retrieving running processes and processor usage info""" + def __init__(self): + """Initialize process and processor info""" debug('') self.mapProcesses = {} self.VisibleMemory = 0 @@ -77,7 +57,9 @@ def __init__(self): self.totalMemoryCount = 0 self.totalProcessorCount = 0 self.mutex = RLock() + def Run(self): + """Get running process info""" debug('') try: running_processes = [] @@ -91,18 +73,8 @@ def Run(self): processInfo.Name = p.name() if callable(p.name) else p.name processInfo.Username = p.username() if callable(p.username) else p.username processInfo.Cmdline = p.cmdline() if callable(p.cmdline) else p.cmdline - # processInfo.Cwd = p.getcwd() - # processInfo.Process = p - # processInfo.CpuPercent = p.get_cpu_percent(interval=0.1) - # processInfo.Exe = p.exe - # processInfo.Status = p.status - # processInfo.CreateTime = p.create_time - # processInfo.MemoryPercent = p.get_memory_percent() - # processInfo.NumThreads = p.get_num_threads() - # processInfo.NumFds = p.get_num_fds() self.mapProcesses[p.pid] = processInfo - except Exception as e: - # debug('ProcessManager::Run Exception {} on pid {}'.format(e, pid)) + except Exception: pass remove = [key for key in self.mapProcesses.keys() if key not in running_processes] for key in remove: @@ -111,7 +83,9 @@ def Run(self): except: exception('ProcessManager::Run failed') debug('ProcessManager::Run retrieved {} processes'.format(len(self.mapProcesses))) + def GetProcessList(self): + """Return list of running processes""" process_list = [] with self.mutex: for key, value in self.mapProcesses.items(): @@ -125,7 +99,9 @@ def GetProcessList(self): process['pid'] = value.Pid process_list.append(process) return process_list + def KillProcess(self, pid): + """Kill the process specified by pid""" retVal = False with self.mutex: process = self.mapProcesses.get(pid) @@ -138,7 +114,9 @@ def KillProcess(self, pid): debug('KillProcess: {}'.format(e)) pass return retVal + def RefreshProcessManager(self): + """Refresh processor usage and memory info""" try: if self.VisibleMemory: del self.VisibleMemory @@ -164,23 +142,28 @@ def RefreshProcessManager(self): self.PeakMemoryUsage = self.AvailableMemory except: exception('ProcessManager::RefreshProcessManager failed') - + @unique class ServiceState(Enum): - Unkown = 0 + Unknown = 0 Running = 1 NotRunning = 2 NotAvailable = 3 - + class ServiceManager: + """Class for retrieving service info and managing services""" + def __init__(self): + """Initialize service info""" self.Init = True self.mapServices = {} self.mutex = RLock() + def Run(self): + """Get info about services""" debug('ServiceManager::Run') with self.mutex: - (output, returnCode) = ServiceManager.ExecuteCommand("service --status-all") + (output, returnCode) = executeCommand("service --status-all") servicesList = output.split("\n") service_names = [] for line in servicesList: @@ -201,68 +184,42 @@ def Run(self): del self.mapServices[key] debug('ServiceManager::Run retrieved ' + str(len(self.mapServices)) + ' services') del output + def GetServiceList(self): + """Return list of services""" service_list = [] with self.mutex: for key, value in self.mapServices.items(): process = {} - process['ProcessName'] = str(key) + process['ProcessName'] = str(key) process['ProcessDescription'] = str(value) - process['CompanyName'] = str(key) + process['CompanyName'] = str(key) service_list.append(process) return service_list + def Start(self, serviceName): + """Start the named service""" debug('ServiceManager::Start') command = "sudo service " + serviceName + " start" - (output, returnCode) = ServiceManager.ExecuteCommand(command) + (output, returnCode) = executeCommand(command) debug('ServiceManager::Start command:' + command + " output: " + output) del output return returnCode + def Stop(self, serviceName): + """Stop the named service""" debug('ServiceManager::Stop') command = "sudo service " + serviceName + " stop" - (output, returnCode) = ServiceManager.ExecuteCommand(command) + (output, returnCode) = executeCommand(command) debug('ServiceManager::Stop command:' + command + " output: " + output) del output return returnCode + def Status(self, serviceName): + """Get the status of the named service""" debug('ServiceManager::Status') command = "service " + serviceName + " status" - (output, returnCode) = ServiceManager.ExecuteCommand(command) + (output, returnCode) = executeCommand(command) debug('ServiceManager::Stop command:' + command + " output: " + output) del output return returnCode - def SetMemoryLimits(): - try: - from resource import getrlimit, setrlimit, RLIMIT_AS - soft, hard = getrlimit(RLIMIT_AS) - setrlimit(RLIMIT_AS, (hard, hard)) - soft, hard = getrlimit(RLIMIT_AS) - except: - pass - def ExecuteCommand(command, increaseMemoryLimit=False): - debug('ServiceManager::ExecuteCommand: ' + command) - output = "" - returncode = 1 - try: - setLimit = None - if increaseMemoryLimit: - setLimit = ServiceManager.SetMemoryLimits - process = Popen(command, stdout=PIPE, shell=True, preexec_fn=setLimit) - processOutput = process.communicate() - returncode = process.wait() - returncode = process.returncode - debug('ServiceManager::ExecuteCommand: ' + str(processOutput)) - if processOutput and processOutput[0]: - output = str(processOutput[0].decode('utf-8')) - processOutput = None - except OSError as oserror: - warn('ServiceManager::ExecuteCommand handled: ' + command + ' Exception:' + str(traceback.format_exc())) - from myDevices.os.daemon import Daemon - Daemon.OnFailure('services', oserror.errno) - except: - exception('ServiceManager::ExecuteCommand failed: ' + command) - debug('ServiceManager::ExecuteCommand: ' + command + ' ' + str(output)) - retOut = str(output) - del output - return (retOut, returncode) \ No newline at end of file diff --git a/myDevices/system/systemconfig.py b/myDevices/system/systemconfig.py new file mode 100644 index 0000000..8c59d03 --- /dev/null +++ b/myDevices/system/systemconfig.py @@ -0,0 +1,67 @@ +""" +This module provides a class for modifying system configuration settings. +""" +from time import sleep +from myDevices.utils.logger import exception, info, warn, error, debug +from myDevices.utils.subprocess import executeCommand +from myDevices.utils.threadpool import ThreadPool +from myDevices.system.hardware import Hardware + +CUSTOM_CONFIG_SCRIPT = "/etc/myDevices/scripts/config.sh" + +class SystemConfig: + """Class for modifying configuration settings""" + + @staticmethod + def ExpandRootfs(): + """Expand the filesystem""" + command = "sudo raspi-config --expand-rootfs" + debug('ExpandRootfs command:' + command) + (output, returnCode) = executeCommand(command) + debug('ExpandRootfs command:' + command + " retCode: " + returnCode) + output = 'reboot required' + return (returnCode, output) + + @staticmethod + def ExecuteConfigCommand(config_id, parameters=''): + """Execute specified command to modify configuration + + Args: + config_id: Id of command to run + parameters: Parameters to use when executing command + """ + if not Hardware().isRaspberryPi(): + return (1, 'Not supported') + debug('SystemConfig::ExecuteConfigCommand') + if config_id == 0: + return SystemConfig.ExpandRootfs() + command = "sudo " + CUSTOM_CONFIG_SCRIPT + " " + str(config_id) + " " + str(parameters) + (output, returnCode) = executeCommand(command) + debug('ExecuteConfigCommand '+ str(config_id) + ' args: ' + str(parameters) + ' retCode: ' + str(returnCode) + ' output: ' + output ) + if "reboot required" in output: + ThreadPool.Submit(SystemConfig.RestartDevice) + return (returnCode, output) + + @staticmethod + def RestartDevice(): + """Reboot the device""" + sleep(5) + command = "sudo shutdown -r now" + (output, returnCode) = executeCommand(command) + + @staticmethod + def getConfig(): + """Return dict containing configuration settings""" + config = {} + if not Hardware().isRaspberryPi(): + return config + commands = {10: 'DeviceTree', 18: 'Serial', 20: 'OneWire', 21: 'I2C', 22: 'SPI'} + for command, name in commands.items(): + try: + (returnCode, output) = SystemConfig.ExecuteConfigCommand(command) + if output: + config[name] = 1 - int(output.strip()) #Invert the value since the config script uses 0 for enable and 1 for disable + except: + exception('Get config') + debug('SystemConfig: {}'.format(config)) + return config diff --git a/myDevices/system/systeminfo.py b/myDevices/system/systeminfo.py new file mode 100644 index 0000000..caeed8a --- /dev/null +++ b/myDevices/system/systeminfo.py @@ -0,0 +1,157 @@ +""" +This module retrieves information about the system, including CPU, RAM, disk and network data. +""" + +import psutil +import netifaces +from myDevices.utils.logger import exception +from myDevices.system.cpu import CpuInfo +from myDevices.cloud import cayennemqtt +from myDevices.wifi import WifiManager + +class SystemInfo(): + """Class to get system CPU, memory, uptime, storage and network info""" + + def getSystemInformation(self): + """Get a dict containing CPU, memory, uptime, storage and network info""" + system_info = [] + try: + system_info += self.getCpuInfo() + system_info += self.getMemoryInfo((cayennemqtt.USAGE,)) + system_info += self.getDiskInfo((cayennemqtt.USAGE,)) + system_info += self.getNetworkInfo() + except: + exception('Error retrieving system info') + return system_info + + def getCpuInfo(self): + """Get CPU information as a list formatted for Cayenne MQTT + + Returned list example:: + + [{ + 'channel': 'sys:cpu;load', + 'value': 12.8, + 'type': 'cpuload', + 'unit': 'p' + }, { + 'channel': 'sys:cpu;temp', + 'value': 50.843, + 'type': 'temp', + 'unit': 'c' + }] + """ + cpu_info = [] + try: + cayennemqtt.DataChannel.add(cpu_info, cayennemqtt.SYS_CPU, suffix=cayennemqtt.LOAD, value=psutil.cpu_percent(1), type='cpuload', unit='p') + cayennemqtt.DataChannel.add(cpu_info, cayennemqtt.SYS_CPU, suffix=cayennemqtt.TEMPERATURE, value=CpuInfo.get_cpu_temp(), type='temp', unit='c') + except: + exception('Error getting CPU info') + return cpu_info + + def getMemoryInfo(self, types): + """Get memory information as a list formatted for Cayenne MQTT. + + Args: + types: Iterable containing types of memory info to retrieve matching cayennemqtt suffixes, e.g. cayennemqtt.USAGE + + Returned list example:: + + [{ + 'channel': 'sys:ram;capacity', + 'value': 968208384, + 'type': 'memory', + 'type': 'b' + }, { + 'channel': 'sys:ram;usage', + 'value': 296620032, + 'type': 'memory', + 'type': 'b' + }] + """ + memory_info = [] + try: + vmem = psutil.virtual_memory() + if not types or cayennemqtt.USAGE in types: + cayennemqtt.DataChannel.add(memory_info, cayennemqtt.SYS_RAM, suffix=cayennemqtt.USAGE, value=vmem.total - vmem.available, type='memory', unit='b') + if not types or cayennemqtt.CAPACITY in types: + cayennemqtt.DataChannel.add(memory_info, cayennemqtt.SYS_RAM, suffix=cayennemqtt.CAPACITY, value=vmem.total, type='memory', unit='b') + except: + exception('Error getting memory info') + return memory_info + + def getDiskInfo(self, types): + """Get disk information as a list formatted for Cayenne MQTT + + Args: + types: Iterable containing types of disk info to retrieve matching cayennemqtt suffixes, e.g. cayennemqtt.USAGE + + Returned list example:: + + [{ + 'channel': 'sys:storage:/;capacity', + 'value': 13646516224, + 'type': 'memory', + 'type': 'b' + }, { + 'channel': 'sys:storage:/;usage', + 'value': 6353821696, + 'type': 'memory', + 'type': 'b' + }, { + 'channel': 'sys:storage:/mnt/cdrom;capacity', + 'value': 479383552, + 'type': 'memory', + 'type': 'b' + }, { + 'channel': 'sys:storage:/mnt/cdrom;usage', + 'value': 0, + 'type': 'memory', + 'type': 'b' + }] + """ + storage_info = [] + try: + for partition in psutil.disk_partitions(True): + try: + if partition.mountpoint == '/': + usage = psutil.disk_usage(partition.mountpoint) + if usage.total: + if not types or cayennemqtt.USAGE in types: + cayennemqtt.DataChannel.add(storage_info, cayennemqtt.SYS_STORAGE, partition.mountpoint, cayennemqtt.USAGE, usage.used, type='memory', unit='b') + if not types or cayennemqtt.CAPACITY in types: + cayennemqtt.DataChannel.add(storage_info, cayennemqtt.SYS_STORAGE, partition.mountpoint, cayennemqtt.CAPACITY, usage.total, type='memory', unit='b') + except: + pass + except: + exception('Error getting disk info') + return storage_info + + def getNetworkInfo(self): + """Get network information as a list formatted for Cayenne MQTT + + Returned list example:: + + [{ + 'channel': 'sys:net;ip', + 'value': '192.168.0.2' + }, { + 'channel': 'sys:net;ssid', + 'value': 'myWifi' + }] + """ + network_info = [] + try: + wifi_manager = WifiManager.WifiManager() + wifi_status = wifi_manager.GetStatus() + default_interface = netifaces.gateways()['default'][netifaces.AF_INET][1] + try: + cayennemqtt.DataChannel.add(network_info, cayennemqtt.SYS_NET, suffix=cayennemqtt.SSID, value=wifi_status[default_interface]['ssid']) + except: + pass + addresses = netifaces.ifaddresses(default_interface) + addr = addresses[netifaces.AF_INET][0]['addr'] + cayennemqtt.DataChannel.add(network_info, cayennemqtt.SYS_NET, suffix=cayennemqtt.IP, value=addr) + except: + exception('Error getting network info') + return network_info diff --git a/myDevices/system/version.py b/myDevices/system/version.py new file mode 100644 index 0000000..8f895c8 --- /dev/null +++ b/myDevices/system/version.py @@ -0,0 +1,19 @@ +""" +This module contains code to retrieve the OS version. +""" +OS_VERSION = 0 +OS_WHEEZY = 1 +OS_JESSIE = 2 +OS_STRETCH = 3 + +try: + with open("/etc/apt/sources.list") as f: + sources = f.read() + if "wheezy" in sources: + OS_VERSION = OS_WHEEZY + elif "jessie" in sources: + OS_VERSION = OS_JESSIE + elif "stretch" in sources: + OS_VERSION = OS_STRETCH +except: + pass diff --git a/myDevices/test/apiclient_test.py b/myDevices/test/apiclient_test.py new file mode 100644 index 0000000..cc95080 --- /dev/null +++ b/myDevices/test/apiclient_test.py @@ -0,0 +1,21 @@ +import unittest +from myDevices.utils.logger import exception, setDebug, info, debug, error, logToFile, setInfo +from myDevices.cloud.apiclient import CayenneApiClient +from json import loads + + +class ApiClientTest(unittest.TestCase): + def testMessageBody(self): + cayenneApiClient = CayenneApiClient('https://api.mydevices.com') + message = loads(cayenneApiClient.getMessageBody('invite_code')) + self.assertIn('id', message) + self.assertIn('type', message) + self.assertIn('hardware_id', message) + self.assertIn('properties', message) + self.assertIn('sysinfo', message['properties']) + self.assertIn('pinmap', message['properties']) + + +if __name__ == '__main__': + setInfo() + unittest.main() diff --git a/myDevices/test/cayennemqtt_test.py b/myDevices/test/cayennemqtt_test.py new file mode 100644 index 0000000..7b7bbfd --- /dev/null +++ b/myDevices/test/cayennemqtt_test.py @@ -0,0 +1,71 @@ +import unittest +import warnings +import myDevices.cloud.cayennemqtt as cayennemqtt +import paho.mqtt.client as mqtt +from time import sleep +from json import dumps, loads + +TEST_USERNAME = "user" +TEST_PASSWORD = "password" +TEST_CLIENT_ID = "id" +TEST_HOST = "localhost" +TEST_PORT = 1883 + +class CayenneMQTTTest(unittest.TestCase): + def setUp(self): + # print('setUp') + self.mqttClient = cayennemqtt.CayenneMQTTClient() + self.mqttClient.on_message = self.OnMessage + self.mqttClient.begin(TEST_USERNAME, TEST_PASSWORD, TEST_CLIENT_ID, TEST_HOST, TEST_PORT) + self.mqttClient.loop_start() + self.testClient = mqtt.Client("testID") + self.testClient.on_message = self.OnTestMessage + # self.testClient.on_log = self.OnTestLog + self.testClient.username_pw_set("testClient", "testClientPass") + self.testClient.connect(TEST_HOST, TEST_PORT, 60) + (result, messageID) = self.testClient.subscribe(self.mqttClient.get_topic_string(cayennemqtt.DATA_TOPIC)) + self.assertEqual(result, mqtt.MQTT_ERR_SUCCESS) + self.testClient.loop_start() + + def tearDown(self): + # print('tearDown') + self.mqttClient.loop_stop() + self.mqttClient.disconnect() + self.testClient.loop_stop() + self.testClient.disconnect() + + def OnMessage(self, message): + self.receivedMessage = message + # print('OnMessage: {}'.format(self.receivedMessage)) + + def OnTestMessage(self, client, userdata, message): + self.receivedTopic = message.topic + self.receivedMessage = message.payload.decode() + # print('OnTestMessage: {} {}'.format(self.receivedTopic, self.receivedMessage)) + + def OnTestLog(self, client, userdata, level, buf): + print('OnTestLog: {}'.format(buf)) + + def testPublish(self): + #Ignore warning caused by paho mqtt not closing some sockets in the destructor + with warnings.catch_warnings(): + warnings.simplefilter('ignore', ResourceWarning) + sentTopic = self.mqttClient.get_topic_string(cayennemqtt.DATA_TOPIC) + sentMessage = '{"publish_test":"data"}' + self.mqttClient.publish_packet(cayennemqtt.DATA_TOPIC, sentMessage) + sleep(0.5) + self.assertEqual(sentTopic, self.receivedTopic) + self.assertEqual(sentMessage, self.receivedMessage) + + def testCommand(self): + sentTopic = self.mqttClient.get_topic_string(cayennemqtt.COMMAND_TOPIC + '/' + cayennemqtt.SYS_POWER) + sentMessage = 'reset' #'{"command_test":"data"}' + self.testClient.publish(sentTopic, sentMessage) + sleep(0.5) + # sentMessage = loads(sentMessage) + self.assertEqual(cayennemqtt.SYS_POWER, self.receivedMessage['channel']) + self.assertEqual(sentMessage, self.receivedMessage['payload']) + + +if __name__ == "__main__": + unittest.main() diff --git a/myDevices/test/gpio_test.py b/myDevices/test/gpio_test.py new file mode 100644 index 0000000..6e69df1 --- /dev/null +++ b/myDevices/test/gpio_test.py @@ -0,0 +1,38 @@ +import unittest +from myDevices.utils.logger import exception, setDebug, info, debug, error, logToFile, setInfo +from myDevices.devices.digital.gpio import NativeGPIO + +class GpioTest(unittest.TestCase): + def setUp(self): + self.gpio = NativeGPIO() + + def testGPIO(self): + pins = [] + for header in self.gpio.MAPPING: + pins.extend([pin['gpio'] for pin in header['map'] if 'gpio' in pin and 'alt0' not in pin and 'overlay' not in pin]) + for pin in pins: + info('Testing pin {}'.format(pin)) + function = self.gpio.setFunctionString(pin, 'OUT') + if function == 'UNKNOWN': + info('Pin {} function UNKNOWN, skipping'.format(pin)) + continue + self.assertEqual('OUT', function) + value = self.gpio.digitalWrite(pin, 1) + self.assertEqual(value, 1) + value = self.gpio.digitalWrite(pin, 0) + self.assertEqual(value, 0) + + def testPinStatus(self): + pin_status = self.gpio.wildcard() + info(pin_status) + self.assertEqual(set(self.gpio.pins + self.gpio.overlay_pins), set(pin_status.keys())) + for key, value in pin_status.items(): + self.assertCountEqual(value.keys(), ('function', 'value')) + if key in self.gpio.pins: + self.assertGreaterEqual(value['value'], 0) + self.assertLessEqual(value['value'], 1) + + +if __name__ == '__main__': + setInfo() + unittest.main() diff --git a/myDevices/test/hardware_test.py b/myDevices/test/hardware_test.py new file mode 100644 index 0000000..8e5e997 --- /dev/null +++ b/myDevices/test/hardware_test.py @@ -0,0 +1,43 @@ +import unittest +from myDevices.utils.logger import setInfo, info +from myDevices.system.hardware import Hardware, BOARD_REVISION, CPU_REVISION, CPU_HARDWARE + +class HarwareTest(unittest.TestCase): + def setUp(self): + setInfo() + self.hardware = Hardware() + + def testGetManufacturer(self): + manufacturer = self.hardware.getManufacturer() + info(manufacturer) + self.assertNotEqual(manufacturer, '') + + def testGetModel(self): + model = self.hardware.getModel() + info(model) + self.assertNotEqual(model, 'Unknown') + + def testGetMac(self): + mac = self.hardware.getMac() + info(mac) + self.assertRegex(mac, '^([0-9a-fA-F]{2}[:-]){5}([0-9a-fA-F]{2})$') + + def testBoardRevision(self): + info(BOARD_REVISION) + self.assertGreaterEqual(BOARD_REVISION, 0) + self.assertLessEqual(BOARD_REVISION, 3) + + def testCpuRevision(self): + info(CPU_REVISION) + self.assertNotEqual(CPU_REVISION, '0') + + def testCpuHardware(self): + info(CPU_HARDWARE) + self.assertNotEqual(CPU_HARDWARE, '') + + def testDeviceVerification(self): + device_checks = (self.hardware.isRaspberryPi(), self.hardware.isTinkerBoard(), self.hardware.isBeagleBone()) + self.assertEqual(device_checks.count(True), 1) + +if __name__ == '__main__': + unittest.main() \ No newline at end of file diff --git a/myDevices/test/client_test.py b/myDevices/test/sensors_test.py similarity index 50% rename from myDevices/test/client_test.py rename to myDevices/test/sensors_test.py index d62c360..50ab93c 100644 --- a/myDevices/test/client_test.py +++ b/myDevices/test/sensors_test.py @@ -2,7 +2,6 @@ import os import grp from myDevices.sensors import sensors -from myDevices.utils.version import MAPPING from myDevices.devices import manager from myDevices.utils.config import Config from myDevices.utils import types @@ -13,6 +12,7 @@ from time import sleep from json import loads, dumps + class SensorsClientTest(unittest.TestCase): @classmethod def setUpClass(cls): @@ -22,57 +22,81 @@ def setUpClass(cls): def tearDownClass(cls): cls.client.StopMonitoring() del cls.client - del GPIO.instance + + def OnDataChanged(self, sensor_data): + self.previousSystemData = self.currentSystemData + self.currentSystemData = sensor_data + if self.previousSystemData: + self.done = True + + def testMonitor(self): + self.previousSystemData = None + self.currentSystemData = None + self.done = False + SensorsClientTest.client.SetDataChanged(self.OnDataChanged) + for i in range(25): + sleep(1) + if self.done: + break + info('Changed items: {}'.format([x for x in self.currentSystemData if x not in self.previousSystemData])) + self.assertNotEqual(self.previousSystemData, self.currentSystemData) def testBusInfo(self): - bus = SensorsClientTest.client.BusInfo() - # # Compare our GPIO function values with the ones from RPi.GPIO library - # import RPi.GPIO - # RPi.GPIO.setmode(RPi.GPIO.BCM) - # port_use = {0:"GPIO.OUT", 1:"GPIO.IN", 40:"GPIO.SERIAL", 41:"GPIO.SPI", 42:"GPIO.I2C", 43:"GPIO.HARD_PWM", -1:"GPIO.UNKNOWN"} - # for gpio in range(GPIO.GPIO_COUNT): - # try: - # print('{}: {} | {}'.format(gpio, bus['GPIO'][gpio]['function'], port_use[RPi.GPIO.gpio_function(gpio)])) - # except ValueError as error: - # print('{}: {}'.format(error, gpio)) - self.assertEqual(set(bus.keys()), set(['GpioMap', 'SPI', 'GPIO', 'ONEWIRE', 'I2C', 'UART'])) + bus = {item['channel']:item['value'] for item in SensorsClientTest.client.BusInfo()} + info('Bus info: {}'.format(bus)) + for pin in GPIO().pins: + self.assertIn('sys:gpio:{};function'.format(pin), bus) + self.assertIn('sys:gpio:{};value'.format(pin), bus) + + def testSensorsInfo(self): + sensors = SensorsClientTest.client.SensorsInfo() + info('Sensors info: {}'.format(sensors)) + for sensor in sensors: + self.assertEqual('dev:', sensor['channel'][:4]) + self.assertIn('value', sensor) def testSetFunction(self): - self.setChannelFunction(5, 'IN') - self.setChannelFunction(5, 'OUT') + self.setChannelFunction(GPIO().pins[7], 'IN') + self.setChannelFunction(GPIO().pins[7], 'OUT') def testSetValue(self): - self.setChannelValue(5, 1) - self.setChannelValue(5, 0) + self.setChannelFunction(GPIO().pins[7], 'OUT') + self.setChannelValue(GPIO().pins[7], 1) + self.setChannelValue(GPIO().pins[7], 0) def testSensors(self): #Test adding a sensor - testSensor = {'description': 'Digital Input', 'device': 'DigitalSensor', 'args': {'gpio': 'GPIO', 'invert': False, 'channel': 12}, 'name': 'testdevice'} + channel = GPIO().pins[8] + testSensor = {'description': 'Digital Input', 'device': 'DigitalSensor', 'args': {'gpio': 'GPIO', 'invert': False, 'channel': channel}, 'name': 'testdevice'} + SensorsClientTest.client.RemoveSensor(testSensor['name']) #Attempt to remove device if it already exists from a previous test compareKeys = ('args', 'description', 'device') retValue = SensorsClientTest.client.AddSensor(testSensor['name'], testSensor['description'], testSensor['device'], testSensor['args']) self.assertTrue(retValue) - retrievedSensor = next(obj for obj in SensorsClientTest.client.GetDevices() if obj['name'] == testSensor['name']) + retrievedSensor = next(obj for obj in manager.getDeviceList() if obj['name'] == testSensor['name']) for key in compareKeys: self.assertEqual(testSensor[key], retrievedSensor[key]) #Test updating a sensor editedSensor = testSensor - editedSensor['args']['channel'] = 13 + editedSensor['args']['channel'] = GPIO().pins[5] retValue = SensorsClientTest.client.EditSensor(editedSensor['name'], editedSensor['description'], editedSensor['device'], editedSensor['args']) self.assertTrue(retValue) - retrievedSensor = next(obj for obj in SensorsClientTest.client.GetDevices() if obj['name'] == editedSensor['name']) + retrievedSensor = next(obj for obj in manager.getDeviceList() if obj['name'] == editedSensor['name']) for key in compareKeys: self.assertEqual(editedSensor[key], retrievedSensor[key]) #Test removing a sensor - retValue = SensorsClientTest.client.DeleteSensor(testSensor['name']) + retValue = SensorsClientTest.client.RemoveSensor(testSensor['name']) self.assertTrue(retValue) - deviceNames = [device['name'] for device in SensorsClientTest.client.GetDevices()] + deviceNames = [device['name'] for device in manager.getDeviceList()] self.assertNotIn(testSensor['name'], deviceNames) def testSensorInfo(self): - sensors = {'actuator' : {'description': 'Digital Output', 'device': 'DigitalActuator', 'args': {'gpio': 'GPIO', 'invert': False, 'channel': 16}, 'name': 'test_actuator'}, - 'light_switch' : {'description': 'Light Switch', 'device': 'LightSwitch', 'args': {'gpio': 'GPIO', 'invert': True, 'channel': 15}, 'name': 'test_light_switch'}, + actuator_channel = GPIO().pins[9] + light_switch_channel = GPIO().pins[9] + sensors = {'actuator' : {'description': 'Digital Output', 'device': 'DigitalActuator', 'args': {'gpio': 'GPIO', 'invert': False, 'channel': actuator_channel}, 'name': 'test_actuator'}, + 'light_switch' : {'description': 'Light Switch', 'device': 'LightSwitch', 'args': {'gpio': 'GPIO', 'invert': True, 'channel': light_switch_channel}, 'name': 'test_light_switch'}, 'MCP3004' : {'description': 'MCP3004', 'device': 'MCP3004', 'args': {'chip': '0'}, 'name': 'test_MCP3004'}, - 'distance' : {'description': 'Analog Distance Sensor', 'device': 'DistanceSensor', 'args': {'adc': 'test_MCP3004', 'channel': 0}, 'name': 'test_distance'}} + 'distance' : {'description': 'Analog Distance Sensor', 'device': 'DistanceSensor', 'args': {'adc': 'test_MCP3004', 'channel': 0}, 'name': 'test_distance'} + } for sensor in sensors.values(): SensorsClientTest.client.AddSensor(sensor['name'], sensor['description'], sensor['device'], sensor['args']) SensorsClientTest.client.SensorsInfo() @@ -82,29 +106,28 @@ def testSensorInfo(self): self.setSensorValue(sensors['light_switch'], 1) self.setSensorValue(sensors['light_switch'], 0) #Test getting analog value - retrievedSensorInfo = next(obj for obj in SensorsClientTest.client.SensorsInfo() if obj['name'] == sensors['distance']['name']) - self.assertEqual(retrievedSensorInfo['float'], 0.0) + channel = 'dev:{}'.format(sensors['distance']['name']) + retrievedSensorInfo = next(obj for obj in SensorsClientTest.client.SensorsInfo() if obj['channel'] == channel) + self.assertGreaterEqual(retrievedSensorInfo['value'], 0.0) + self.assertLessEqual(retrievedSensorInfo['value'], 1.0) for sensor in sensors.values(): - self.assertTrue(SensorsClientTest.client.DeleteSensor(sensor['name'])) - - def testSystemInfo(self): - system_info = SensorsClientTest.client.SystemInformation() - self.assertEqual(set(system_info.keys()), set(['Storage', 'Cpu', 'CpuLoad', 'Uptime', 'Network', 'Memory'])) + self.assertTrue(SensorsClientTest.client.RemoveSensor(sensor['name'])) def setSensorValue(self, sensor, value): - SensorsClientTest.client.SensorCommand('integer', sensor['name'], sensor['device'], None, None, None, value) - sensorInfo = next(obj for obj in SensorsClientTest.client.SensorsInfo() if obj['name'] == sensor['name']) + SensorsClientTest.client.SensorCommand('integer', sensor['name'], None, value) + channel = 'dev:{}'.format(sensor['name']) + sensorInfo = next(obj for obj in SensorsClientTest.client.SensorsInfo() if obj['channel'] == channel) self.assertEqual(value, sensorInfo['value']) def setChannelFunction(self, channel, function): - SensorsClientTest.client.gpio.setFunctionString(channel, function) - bus = SensorsClientTest.client.BusInfo() - self.assertEqual(function, bus['GPIO'][channel]['function']) + SensorsClientTest.client.GpioCommand('function', channel, function) + bus = {item['channel']:item['value'] for item in SensorsClientTest.client.BusInfo()} + self.assertEqual(function, bus['sys:gpio:{};function'.format(channel)]) def setChannelValue(self, channel, value): - SensorsClientTest.client.gpio.digitalWrite(channel, value) - bus = SensorsClientTest.client.BusInfo() - self.assertEqual(value, bus['GPIO'][channel]['value']) + SensorsClientTest.client.GpioCommand('value', channel, value) + bus = {item['channel']:item['value'] for item in SensorsClientTest.client.BusInfo()} + self.assertEqual(value, bus['sys:gpio:{};value'.format(channel)]) if __name__ == '__main__': setInfo() diff --git a/myDevices/test/systemconfig_test.py b/myDevices/test/systemconfig_test.py new file mode 100644 index 0000000..ef0c9ba --- /dev/null +++ b/myDevices/test/systemconfig_test.py @@ -0,0 +1,17 @@ +import unittest +from myDevices.system.systemconfig import SystemConfig +from myDevices.utils.logger import setInfo, info + + +class SystemConfigTest(unittest.TestCase): + def testSystemConfig(self): + config = SystemConfig.getConfig() + info(config) + if config: + for item in ('DeviceTree', 'Serial', 'I2C', 'SPI', 'OneWire'): + self.assertIn(item, config) + + +if __name__ == '__main__': + setInfo() + unittest.main() \ No newline at end of file diff --git a/myDevices/test/systeminfo_test.py b/myDevices/test/systeminfo_test.py index 3225ce2..ad19abc 100644 --- a/myDevices/test/systeminfo_test.py +++ b/myDevices/test/systeminfo_test.py @@ -1,52 +1,30 @@ import unittest -from myDevices.os.systeminfo import SystemInfo +from myDevices.system.systeminfo import SystemInfo from myDevices.utils.logger import setInfo, info class SystemInfoTest(unittest.TestCase): def setUp(self): - # setInfo() + setInfo() system_info = SystemInfo() - self.info = system_info.getSystemInformation() - - def testCpuInfo(self): - cpu_info = self.info['Cpu'] - info(cpu_info) - self.assertEqual(set(cpu_info.keys()), set(('loadavg', 'usage', 'temperature'))) - self.assertEqual(set(cpu_info['loadavg'].keys()), set(('one', 'five', 'ten'))) - self.assertGreaterEqual(set(cpu_info['usage'].keys()), set(('user', 'nice', 'system', 'idle', 'iowait', 'irq', 'softirq', 'total'))) - - def testCpuLoadInfo(self): - cpu_load_info = self.info['CpuLoad'] - info(cpu_load_info) - self.assertGreaterEqual(set(cpu_load_info.keys()), set(('cpu',))) - - def testMemoryInfo(self): - memory_info = self.info['Memory'] - info(memory_info) - self.assertEqual(set(memory_info.keys()), set(('total', 'free', 'used', 'buffers', 'cached', 'processes', 'swap'))) - self.assertEqual(set(memory_info['swap'].keys()), set(('total', 'free', 'used'))) - - def testUptimeInfo(self): - uptime_info = self.info['Uptime'] - info(uptime_info) - self.assertEqual(set(uptime_info.keys()), set(('uptime',))) - - def testStorageInfo(self): - storage_info = self.info['Storage'] - info(storage_info) - self.assertEqual(set(storage_info.keys()), set(('list',))) - for item in storage_info['list']: - self.assertLessEqual(set(('device', 'filesystem', 'mount')), set(item.keys())) - - def testNetworkInfo(self): - network_info = self.info['Network'] - info(network_info) - self.assertGreaterEqual(set(network_info.keys()), set(('list',))) - for key, value in network_info['list'].items(): - self.assertTrue(value) - self.assertLessEqual(set(value.keys()), set(('ip', 'ipv6', 'mac'))) - self.assertIsNotNone(network_info['list']['eth0']['ip']['address']) + self.info = {item['channel']:item for item in system_info.getSystemInformation()} + info(self.info) + + def testSystemInfo(self): + self.assertIn('sys:cpu;load', self.info) + self.assertEqual(self.info['sys:cpu;load']['type'], 'cpuload') + self.assertEqual(self.info['sys:cpu;load']['unit'], 'p') + self.assertIn('sys:cpu;temp', self.info) + self.assertEqual(self.info['sys:cpu;temp']['type'], 'temp') + self.assertEqual(self.info['sys:cpu;temp']['unit'], 'c') + self.assertIn('sys:ram;usage', self.info) + self.assertEqual(self.info['sys:ram;usage']['type'], 'memory') + self.assertEqual(self.info['sys:ram;usage']['unit'], 'b') + self.assertIn('sys:storage:/;usage', self.info) + self.assertEqual(self.info['sys:storage:/;usage']['type'], 'memory') + self.assertEqual(self.info['sys:storage:/;usage']['unit'], 'b') + self.assertIn('sys:net;ip', self.info) + # self.assertIn('sys:net;ssid', self.info) if __name__ == '__main__': diff --git a/myDevices/test/updater_test.py b/myDevices/test/updater_test.py index 65d570b..b0b3bfa 100644 --- a/myDevices/test/updater_test.py +++ b/myDevices/test/updater_test.py @@ -8,14 +8,13 @@ def setUp(self): setDebug() self.config = Config('/etc/myDevices/AppSettings.ini') self.updater = Updater(self.config) + def tearDown(self): self.updater = None + def testCheckUpdate(self): self.updater.CheckUpdate() print('After CheckUpdate') - # def testStart(self): - # self.updater.start() - # print('After Updater') def main(): unittest.main() diff --git a/myDevices/utils/config.py b/myDevices/utils/config.py index 405e54e..5092ab3 100644 --- a/myDevices/utils/config.py +++ b/myDevices/utils/config.py @@ -7,12 +7,12 @@ def __init__(self, path): self.path = path self.config = RawConfigParser() self.config.optionxform = str - self.cloudConfig = {} try: with open(path) as fp: - self.config.readfp(fp) + self.config.read_file(fp) except: pass + def set(self, section, key, value): with self.mutex: try: @@ -20,13 +20,18 @@ def set(self, section, key, value): except NoSectionError: self.config.add_section(section) self.config.set(section, key, value) - return self.save() + self.save() def get(self, section, key, fallback=_UNSET): return self.config.get(section, key, fallback=fallback) def getInt(self, section, key, fallback=_UNSET): return self.config.getint(section, key, fallback=fallback) + + def remove(self, section, key): + with self.mutex: + result = self.config.remove_option(section, key) + self.save() def save(self): with self.mutex: @@ -36,7 +41,4 @@ def save(self): def sections(self): return self.config.sections() - def setCloudConfig(self, cloudConfig): - self.cloudConfig = cloudConfig - diff --git a/myDevices/utils/daemon.py b/myDevices/utils/daemon.py new file mode 100644 index 0000000..a85075f --- /dev/null +++ b/myDevices/utils/daemon.py @@ -0,0 +1,64 @@ +""" +This module provides a class for restarting the agent if errors occur and exiting on critical failures. +""" +from sys import exit +from datetime import datetime +from myDevices.utils.logger import exception, info, warn, error, debug +from myDevices.utils.subprocess import executeCommand + +#defining reset timeout in seconds +RESET_TIMEOUT = 30 +FAILURE_COUNT = 1000 +failureCount = {} +startFailure = {} +errorList = (-3, -2, 12, 9, 24) + + +class Daemon: + """class for restarting the agent if errors occur and exiting on critical failures.""" + + @staticmethod + def OnFailure(component, error=0): + """Handle error in component and restart the agent if necessary""" + #-3=Temporary failure in name resolution + info('Daemon failure handling ' + str(error)) + if error in errorList: + Daemon.Restart() + if component not in failureCount: + Daemon.Reset(component) + failureCount[component] += 1 + now = datetime.now() + if startFailure[component] == 0: + startFailure[component] = now + elapsedTime = now - startFailure[component] + if (elapsedTime.total_seconds() >= RESET_TIMEOUT) or (failureCount[component] > FAILURE_COUNT): + warn('Daemon::OnFailure myDevices is going to restart after ' + str(component) + ' failed: ' + str(elapsedTime.total_seconds()) + ' seconds and ' + str(failureCount) + ' times') + Daemon.Restart() + + @staticmethod + def Reset(component): + """Reset failure count for component""" + startFailure[component] = 0 + failureCount[component] = 0 + + @staticmethod + def Restart(): + """Restart the agent daemon""" + try: + info('Daemon restarting myDevices' ) + (output, returncode) = executeCommand('sudo service myDevices restart') + debug(str(output) + ' ' + str(returncode)) + del output + except: + exception("Daemon::Restart unexpected error") + Daemon.Exit() + + @staticmethod + def Exit(): + """Stop the agent daemon""" + error('Critical failure. Closing myDevices process.') + raise SystemExit + + + + diff --git a/myDevices/utils/history.py b/myDevices/utils/history.py index 73b4cb7..30f9b4b 100644 --- a/myDevices/utils/history.py +++ b/myDevices/utils/history.py @@ -192,8 +192,8 @@ def CalculateSensorsInfoAverages(self, current_avgs, new_sample, count_sensor): count_sensor['SensorsInfo'][value['sensor']] += 1 if value['type'] == 'Temperature': averageSensorsDictionary[value['sensor']]['Celsius'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Celsius'], value['Celsius'], count_sensor['SensorsInfo'][value['sensor']] ) - averageSensorsDictionary[value['sensor']]['Fahrenheit'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Fahrenheit'], value['Fahrenheit'], count_sensor['SensorsInfo'][value['sensor']] ) - averageSensorsDictionary[value['sensor']]['Kelvin'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Kelvin'], value['Kelvin'], count_sensor['SensorsInfo'][value['sensor']] ) + # averageSensorsDictionary[value['sensor']]['Fahrenheit'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Fahrenheit'], value['Fahrenheit'], count_sensor['SensorsInfo'][value['sensor']] ) + # averageSensorsDictionary[value['sensor']]['Kelvin'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Kelvin'], value['Kelvin'], count_sensor['SensorsInfo'][value['sensor']] ) if value['type'] == 'Pressure': averageSensorsDictionary[value['sensor']]['Pascal'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['Pascal'], value['Pascal'], count_sensor['SensorsInfo'][value['sensor']] ) if value['type'] == 'Luminosity': @@ -224,8 +224,8 @@ def CalculateSensorsInfoAverages(self, current_avgs, new_sample, count_sensor): averageSensorsDictionary[value['sensor']]['value'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['value'], value['value'], count_sensor['SensorsInfo'][value['sensor']] ) if value['type'] == 'AnalogSensor': averageSensorsDictionary[value['sensor']]['float'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['float'], value['float'], count_sensor['SensorsInfo'][value['sensor']] ) - averageSensorsDictionary[value['sensor']]['integer'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['integer'], value['integer'], count_sensor['SensorsInfo'][value['sensor']] ) - averageSensorsDictionary[value['sensor']]['volt'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['volt'], value['volt'], count_sensor['SensorsInfo'][value['sensor']] ) + # averageSensorsDictionary[value['sensor']]['integer'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['integer'], value['integer'], count_sensor['SensorsInfo'][value['sensor']] ) + # averageSensorsDictionary[value['sensor']]['volt'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['volt'], value['volt'], count_sensor['SensorsInfo'][value['sensor']] ) if value['type'] == 'ServoMotor': averageSensorsDictionary[value['sensor']]['angle'] = self.CalculateAverage(averageSensorsDictionary[value['sensor']]['angle'], value['angle'], count_sensor['SensorsInfo'][value['sensor']] ) if value['type'] == 'AnalogActuator': diff --git a/myDevices/utils/logger.py b/myDevices/utils/logger.py index c7da470..30a2bd8 100644 --- a/myDevices/utils/logger.py +++ b/myDevices/utils/logger.py @@ -7,7 +7,7 @@ from datetime import datetime from hashlib import sha256 import time -from myDevices.os.threadpool import ThreadPool +from myDevices.utils.threadpool import ThreadPool from glob import iglob MAX_JSON_ITEMS_PER_CATEGORY = 100 @@ -18,19 +18,10 @@ LOG_FORMATTER = Formatter(fmt='%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt="%Y-%m-%d %H:%M:%S") LOGGER = getLogger("myDevices") LOGGER.setLevel(WARN) -#Now setup the json logger -# JsonLogger = getLogger("myDevicesData") CONSOLE_HANDLER = StreamHandler() CONSOLE_HANDLER.setFormatter(LOG_FORMATTER) LOGGER.addHandler(CONSOLE_HANDLER) -# memoryhandler = MemoryHandler(capacity=1024*10, target=CONSOLE_HANDLER) -# LOGGER.addHandler(memoryhandler) - -# JsonLogger.addHandler(CONSOLE_HANDLER) -# JsonLogger.addHandler(memoryhandler) -# rotatingFileHandler = RotatingFileHandler(JSON_FILE_LOGGER, mode='a', maxBytes=1000000) -# JsonLogger.addHandler(rotatingFileHandler) jsonData = {} messageFrequence={} @@ -53,7 +44,7 @@ def rotator(source, dest): tar.close() remove(source) # Remove old myDevices.log backups if they are older than a week. This code can be removed - # in later versions if myDevices.log files have been replaced with cayenne.log. + # in later versions if myDevices.log files have been replaced with cayenne.log. for old_file in iglob('/var/log/myDevices/myDevices.log*'): if path.getmtime(old_file) + 604800 < time.time(): remove(old_file) @@ -83,10 +74,6 @@ def rotatorJson(): debug('rotatorJson called') lastRotate = datetime.now() ThreadPool.Submit(rotatorJsonMp) - # p = Process(target=rotatorJsonMp, args=()) - # p.start() - # p.join() - #no wait for process -> p.join() except: exception('rotatorJson exception') diff --git a/myDevices/utils/subprocess.py b/myDevices/utils/subprocess.py new file mode 100644 index 0000000..eb0c914 --- /dev/null +++ b/myDevices/utils/subprocess.py @@ -0,0 +1,39 @@ +""" +This module contains functions for launching subprocesses and returning output from them. +""" +from subprocess import Popen, PIPE, DEVNULL +from myDevices.utils.logger import debug, info, error, exception + +def setMemoryLimits(): + """Set memory limit when launching a process to the default maximum""" + try: + from resource import getrlimit, setrlimit, RLIMIT_AS + soft, hard = getrlimit(RLIMIT_AS) + setrlimit(RLIMIT_AS, (hard, hard)) + except: + pass + +def executeCommand(command, increaseMemoryLimit=False, disablePipe=False): + """Execute a specified command, increasing the processes memory limits if specified""" + debug('executeCommand: ' + command) + output = '' + returncode = 1 + try: + preexec = None + pipe = PIPE + if increaseMemoryLimit: + preexec = setMemoryLimits + if disablePipe: + debug('Disable pipe to prevent child exiting when parent exits') + pipe = DEVNULL + process = Popen(command, stdout=pipe, stderr=pipe, shell=True, preexec_fn=preexec) + (stdout_data, stderr_data) = process.communicate() + returncode = process.wait() + returncode = process.returncode + # debug('executeCommand: stdout_data {}, stderr_data {}'.format(stdout_data, stderr_data)) + if stdout_data: + output = stdout_data.decode('utf-8') + stdout_data = None + except: + exception('executeCommand failed: ' + command) + return (output, returncode) diff --git a/myDevices/utils/threadpool.py b/myDevices/utils/threadpool.py new file mode 100644 index 0000000..b0f59c0 --- /dev/null +++ b/myDevices/utils/threadpool.py @@ -0,0 +1,20 @@ +""" +This module provides a singleton thread pool class +""" +from concurrent.futures import ThreadPoolExecutor +from myDevices.utils.singleton import Singleton + +executor = ThreadPoolExecutor(max_workers=4) +class ThreadPool(Singleton): + """Singleton thread pool class""" + + @staticmethod + def Submit(func): + """Submit a function for the thread pool to run""" + executor.submit(func) + + @staticmethod + def Shutdown(): + """Shutdown the thread pool""" + executor.shutdown() + \ No newline at end of file diff --git a/myDevices/utils/version.py b/myDevices/utils/version.py deleted file mode 100644 index 67a8436..0000000 --- a/myDevices/utils/version.py +++ /dev/null @@ -1,47 +0,0 @@ -import re -import sys - -PYTHON_MAJOR = sys.version_info.major -BOARD_REVISION = 0 - -OS_VERSION = 0 -OS_RASPBIAN_WHEEZY = 1 -OS_RASPBIAN_JESSIE = 2 - -_MAPPING = [[], [], [], []] -_MAPPING[1] = ["V33", "V50", 0, "V50", 1, "GND", 4, 14, "GND", 15, 17, 18, 21, "GND", 22, 23, "V33", 24, 10, "GND", 9, 25, 11, 8, "GND", 7] -_MAPPING[2] = ["V33", "V50", 2, "V50", 3, "GND", 4, 14, "GND", 15, 17, 18, 27, "GND", 22, 23, "V33", 24, 10, "GND", 9, 25, 11, 8, "GND", 7] -_MAPPING[3] = ["V33", "V50", 2, "V50", 3, "GND", 4, 14, "GND", 15, 17, 18, 27, "GND", 22, 23, "V33", 24, 10, "GND", 9, 25, 11, 8, "GND", 7, "DNC", "DNC" , 5, "GND", 6, 12, 13, "GND", 19, 16, 26, 20, "GND", 21] - -try: - with open("/proc/cpuinfo") as f: - rc = re.compile("Revision\s*:\s(.*)\n") - info = f.read() - result = rc.search(info) - if result != None: - hex_cpurev = result.group(1) - if hex_cpurev.startswith("1000"): - hex_cpurev = hex_cpurev[-4:] - cpurev = int(hex_cpurev, 16) - if cpurev < 0x04: - BOARD_REVISION = 1 - elif cpurev < 0x10: - BOARD_REVISION = 2 - else: - BOARD_REVISION = 3 - -except: - pass - -try: - with open("/etc/apt/sources.list") as f: - sources = f.read() - if "wheezy" in sources: - OS_VERSION = OS_RASPBIAN_WHEEZY - elif "jessie" in sources: - OS_VERSION = OS_RASPBIAN_JESSIE - -except: - pass - -MAPPING = _MAPPING[BOARD_REVISION] diff --git a/myDevices/wifi/WifiManager.py b/myDevices/wifi/WifiManager.py index 1d1e5e3..637c798 100644 --- a/myDevices/wifi/WifiManager.py +++ b/myDevices/wifi/WifiManager.py @@ -1,31 +1,31 @@ from myDevices.wifi.WirelessLib import Wireless from json import dumps, loads, JSONEncoder, JSONDecoder from myDevices.utils.logger import exception, info, warn, error, debug -from myDevices.os.services import ServiceManager +from myDevices.utils.subprocess import executeCommand -class Network(): - def GetNetworkId(): - ip = None - network = None - returnValue = {} - try: - import netifaces - gws=netifaces.gateways() - defaultNet = gws['default'].values() - for key, val in defaultNet: - ip = key - network = val - command = 'arp -n ' + ip + ' | grep ' + network + ' | awk \'{print $3}\'' - (output, retCode) = ServiceManager.ExecuteCommand(command) - if int(retCode) > 0: - return None - returnValue['Ip'] = ip - returnValue['Network'] = network - returnValue['MAC'] = output.strip() - del output - except Exception as ex: - debug('Could not initialize netifaces module: ' + str(ex)) - return returnValue +# class Network(): +# def GetNetworkId(): +# ip = None +# network = None +# returnValue = {} +# try: +# import netifaces +# gws=netifaces.gateways() +# defaultNet = gws['default'].values() +# for key, val in defaultNet: +# ip = key +# network = val +# command = 'arp -n ' + ip + ' | grep ' + network + ' | awk \'{print $3}\'' +# (output, retCode) = executeCommand(command) +# if int(retCode) > 0: +# return None +# returnValue['Ip'] = ip +# returnValue['Network'] = network +# returnValue['MAC'] = output.strip() +# del output +# except Exception as ex: +# debug('Could not initialize netifaces module: ' + str(ex)) +# return returnValue #{'stats': {'updated': 75, 'noise': 0, 'quality': 67, 'level': 213}, 'Frequency': b'2.412 GHz', 'Access Point': b'B8:55:10:AC:8F:D8', 'Mode': b'Master', 'Key': b'off', 'BitRate': b'54 Mb/s', 'ESSID': b'SRX-WR300WH'} class WifiEndpoint(object): @@ -81,7 +81,6 @@ def Search(self, interface): exception('Wifi search address') return None - def Setup(self, ssid, password, interface): if interface in self.wirelessModules: status = self.wirelessModules[interface].connect(ssid, password) @@ -97,22 +96,23 @@ def GetIpAddress(self, interface): exception('GetIpAddress failed') return ip_addr + def GetCurretSSID(self, interface): if interface in self.wirelessModules: return self.wirelessModules[interface].current() return None + def GetDriver(self, interface): if interface in self.wirelessModules: return self.wirelessModules[interface].driver() return None + def GetPowerStatus(self, interface): if interface in self.wirelessModules: return self.wirelessModules[interface].power() return None - - - def GetStatus(self): - + + def GetStatus(self): try: jsonDictionary = {} interfaces = self.Interfaces() @@ -137,8 +137,7 @@ def GetStatus(self): driver = self.GetDriver(i) bitRate = "" stats = "" - frequency = "" - + frequency = "" if ssid is None: ssid = "" else: @@ -147,7 +146,7 @@ def GetStatus(self): if endpoint["ESSID"].decode('ascii') == ssid: frequency = endpoint["Frequency"].decode('ascii') bitRate = endpoint["BitRate"].decode('ascii') - stats = ToJson(endpoint["stats"]) + stats = endpoint["stats"] jsonDictionary[str(i)]["ssid"] = ssid jsonDictionary[str(i)]["PowerStatus"] = str(powerStatus) jsonDictionary[str(i)]["Frequency"] = str(frequency) @@ -155,7 +154,7 @@ def GetStatus(self): jsonDictionary[str(i)]["stats"] = str(stats) except Exception as ex: debug('GetStatus: failed address: ' + str(ex)) - return ToJson(jsonDictionary) + return jsonDictionary def GetWirelessNetworks(self): jsonDictionary = {} @@ -171,32 +170,32 @@ def GetWirelessNetworks(self): jsonDictionary[str(i)] = wifiEndpoints except Exception as ex: debug('GetWirelessNetworks failed: ' + str(ex)) - return ToJson(jsonDictionary) + return jsonDictionary -def ToJson(object): - returnValue = "{}" - try: - import jsonpickle - returnValue = jsonpickle.encode(object) - except: - exception('Json encoding failed') - return returnValue +# def ToJson(object): +# returnValue = "{}" +# try: +# import jsonpickle +# returnValue = jsonpickle.encode(object) +# except: +# exception('Json encoding failed') +# return returnValue -def testWifiManager(): - wifiManager = WifiManager() - info(ToJson(wifiManager.Interfaces())) - info(str(wifiManager.GetWirelessNetworks())) - info(str(wifiManager.GetCurretSSID('wlan0'))) - info(str(wifiManager.GetDriver('wlan0'))) - info(str(wifiManager.GetPowerStatus('wlan0'))) - info(str(wifiManager.GetStatus())) +# def testWifiManager(): +# wifiManager = WifiManager() +# info(wifiManager.Interfaces()) +# info(str(wifiManager.GetWirelessNetworks())) +# info(str(wifiManager.GetCurretSSID('wlan0'))) +# info(str(wifiManager.GetDriver('wlan0'))) +# info(str(wifiManager.GetPowerStatus('wlan0'))) +# info(str(wifiManager.GetStatus())) - SetBadNetwork(wifiManager) +# SetBadNetwork(wifiManager) -def SetBadNetwork(wifiManager): - info('============SETUP TESTS============') - info('Bad password test: ' + str(wifiManager.Setup('Lizuca&Patrocle', 'badpasswd'))) - info('Bad network test: ' + str(wifiManager.Setup('None', 'badpasswd'))) - info('Success setup test: ' + str(wifiManager.Setup('Lizuca&Patrocle', 'fatadraganufitrista'))) +# def SetBadNetwork(wifiManager): +# info('============SETUP TESTS============') +# info('Bad password test: ' + str(wifiManager.Setup('Lizuca&Patrocle', 'badpasswd'))) +# info('Bad network test: ' + str(wifiManager.Setup('None', 'badpasswd'))) +# info('Success setup test: ' + str(wifiManager.Setup('Lizuca&Patrocle', 'fatadraganufitrista'))) #testWifiManager() diff --git a/myDevices/wifi/WirelessLib.py b/myDevices/wifi/WirelessLib.py index 42af226..1acb35c 100644 --- a/myDevices/wifi/WirelessLib.py +++ b/myDevices/wifi/WirelessLib.py @@ -1,12 +1,11 @@ from abc import ABCMeta, abstractmethod -import subprocess +# import subprocess from time import sleep -from myDevices.os.services import ServiceManager +from myDevices.utils.subprocess import executeCommand # send a command to the shell and return the result def cmd(cmd): response = "" - try: #response = subprocess.Popen( # cmd, shell=True, @@ -14,7 +13,7 @@ def cmd(cmd): #process.stdout.close() #process.stdin.close() #process.stderr.close() - response, returncode = ServiceManager.ExecuteCommand(cmd) + response, returncode = executeCommand(cmd) except: reponse = "Error" return response diff --git a/scripts/config.sh b/scripts/config.sh index 9003be1..b7370f3 100644 --- a/scripts/config.sh +++ b/scripts/config.sh @@ -254,6 +254,7 @@ do_change_hostname() { sed -i "s/127.0.1.1.*$CURRENT_HOSTNAME/127.0.1.1\t$NEW_HOSTNAME/g" /etc/hosts ASK_TO_REBOOT=1 } + # $1 is 0 to enable overscan, 1 to disable it set_overscan() { # Stop if /boot is not a mountpoint @@ -270,6 +271,7 @@ set_overscan() { set_config_var disable_overscan 0 $CONFIG fi } + get_memory_split(){ if [ -e /boot/start_cd.elf ]; then # New-style memory split setting @@ -285,6 +287,7 @@ get_memory_split(){ echo "${MEMSPLIT_DESCRIPTION}" fi } + do_overscan() { RET=${args[1]} if [ $RET -eq 0 ] || [ $RET -eq 1 ]; then @@ -294,6 +297,7 @@ do_overscan() { return 1 fi } + do_ssh() { if [ -e /var/log/regen_ssh_keys.log ] && ! grep -q "^finished" /var/log/regen_ssh_keys.log; then echo "Initial ssh key generation still running. Please wait and try again." @@ -309,43 +313,46 @@ do_ssh() { return $RET fi } + do_devicetree() { - CURRENT_SETTING="enabled" # assume not disabled - DEFAULT= - if [ -e $CONFIG ] && grep -q "^device_tree=$" $CONFIG; then - CURRENT_SETTING="disabled" - DEFAULT=--defaultno - fi - RET=${args[1]} - if [ $RET -eq 0 ]; then - sed $CONFIG -i -e "s/^\(device_tree=\)$/#\1/" - sed $CONFIG -i -e "s/^#\(device_tree=.\)/\1/" - SETTING=enabled - elif [ $RET -eq 1 ]; then - sed $CONFIG -i -e "s/^#\(device_tree=\)$/\1/" - sed $CONFIG -i -e "s/^\(device_tree=.\)/#\1/" - if ! grep -q "^device_tree=$" $CONFIG; then - printf "device_tree=\n" >> $CONFIG - fi - SETTING=disabled - else - return 0 - fi - TENSE=is - REBOOT= - if [ $SETTING != $CURRENT_SETTING ]; then - TENSE="will be" - REBOOT=" after a reboot" - ASK_TO_REBOOT=1 - fi - + return 1 # No longer allow disabling the device tree since it is required and can prevent the Pi from booting + # CURRENT_SETTING="enabled" # assume not disabled + # DEFAULT= + # if [ -e $CONFIG ] && grep -q "^device_tree=$" $CONFIG; then + # CURRENT_SETTING="disabled" + # DEFAULT=--defaultno + # fi + # RET=${args[1]} + # if [ $RET -eq 0 ]; then + # sed $CONFIG -i -e "s/^\(device_tree=\)$/#\1/" + # sed $CONFIG -i -e "s/^#\(device_tree=.\)/\1/" + # SETTING=enabled + # elif [ $RET -eq 1 ]; then + # sed $CONFIG -i -e "s/^#\(device_tree=\)$/\1/" + # sed $CONFIG -i -e "s/^\(device_tree=.\)/#\1/" + # if ! grep -q "^device_tree=$" $CONFIG; then + # printf "device_tree=\n" >> $CONFIG + # fi + # SETTING=disabled + # else + # return 0 + # fi + # TENSE=is + # REBOOT= + # if [ $SETTING != $CURRENT_SETTING ]; then + # TENSE="will be" + # REBOOT=" after a reboot" + # ASK_TO_REBOOT=1 + # fi } + get_devicetree() { - CURRENT_SETTING="1" # assume not disabled - if [ -e $CONFIG ] && grep -q "^device_tree=$" $CONFIG; then - CURRENT_SETTING="0" - fi - echo "${CURRENT_SETTING}" + echo 0 # assume enabled since recent kernels don't appear to allow disabling the device tree anymore + # CURRENT_SETTING="1" # assume not disabled + # if [ -e $CONFIG ] && grep -q "^device_tree=$" $CONFIG; then + # CURRENT_SETTING="0" + # fi + # echo "${CURRENT_SETTING}" } #arg[1] enable I2C 1/0 arg[2] load by default 1/0 @@ -425,11 +432,17 @@ do_i2c() { return 0 fi - - - exit } + +get_i2c() { + if grep -q -E "^(device_tree_param|dtparam)=([^,]*,)*i2c(_arm)?(=(on|true|yes|1))?(,.*)?$" $CONFIG; then + echo 0 + else + echo 1 + fi +} + #arg[1] enable SPI 1/0 arg[2] load by default 1/0 #again 0 enable 1 disable do_spi() { @@ -503,27 +516,85 @@ do_spi() { fi } -do_serial() { - CURRENT_STATUS="yes" # assume ttyAMA0 output enabled - if ! grep -q "^T.*:.*:respawn:.*ttyAMA0" /etc/inittab; then - CURRENT_STATUS="no" +get_spi() { + if grep -q -E "^(device_tree_param|dtparam)=([^,]*,)*spi(=(on|true|yes|1))?(,.*)?$" $CONFIG; then + echo 0 + else + echo 1 fi +} - #"Would you like a login shell to be accessible over serial?" +get_serial() { + if grep -q -E "console=(serial0|ttyAMA0|ttyS0)" $CMDLINE ; then + echo 0 + else + echo 1 + fi +} + +get_serial_hw() { + if grep -q -E "^enable_uart=1" $CONFIG ; then + echo 0 + elif grep -q -E "^enable_uart=0" $CONFIG ; then + echo 1 + elif [ -e /dev/serial0 ] ; then + echo 0 + else + echo 1 + fi +} + +do_serial() { + CMDLINE=/boot/cmdline.txt + DEFAULTS=--defaultno + DEFAULTH=--defaultno + CURRENTS=0 + CURRENTH=0 + + if [ $(get_serial) -eq 0 ]; then + DEFAULTS= + CURRENTS=1 + fi + if [ $(get_serial_hw) -eq 0 ]; then + DEFAULTH= + CURRENTH=1 + fi RET=${args[1]} - if [ $RET -eq 1 ]; then - sed -i /etc/inittab -e "s|^.*:.*:respawn:.*ttyAMA0|#&|" - sed -i /boot/cmdline.txt -e "s/console=ttyAMA0,[0-9]\+ //" - #"Serial is now disabled" - elif [ $RET -eq 0 ]; then - sed -i /etc/inittab -e "s|^#\(.*:.*:respawn:.*ttyAMA0\)|\1|" - if ! grep -q "^T.*:.*:respawn:.*ttyAMA0" /etc/inittab; then - printf "T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100\n" >> /etc/inittab + if [ $RET -eq $CURRENTS ]; then + ASK_TO_REBOOT=1 + fi + + if [ $RET -eq 0 ]; then + if grep -q "console=ttyAMA0" $CMDLINE ; then + if [ -e /proc/device-tree/aliases/serial0 ]; then + sed -i $CMDLINE -e "s/console=ttyAMA0/console=serial0/" + fi + elif ! grep -q "console=ttyAMA0" $CMDLINE && ! grep -q "console=serial0" $CMDLINE ; then + if [ -e /proc/device-tree/aliases/serial0 ]; then + sed -i $CMDLINE -e "s/root=/console=serial0,115200 root=/" + else + sed -i $CMDLINE -e "s/root=/console=ttyAMA0,115200 root=/" + fi fi - if ! grep -q "console=ttyAMA0" /boot/cmdline.txt; then - sed -i /boot/cmdline.txt -e "s/root=/console=ttyAMA0,115200 root=/" + set_config_var enable_uart 1 $CONFIG + SSTATUS=enabled + HSTATUS=enabled + elif [ $RET -eq 1 ]; then + sed -i $CMDLINE -e "s/console=ttyAMA0,[0-9]\+ //" + sed -i $CMDLINE -e "s/console=serial0,[0-9]\+ //" + SSTATUS=disabled + if [ $RET -eq $CURRENTH ]; then + ASK_TO_REBOOT=1 + fi + if [ $RET -eq 0 ]; then + set_config_var enable_uart 1 $CONFIG + HSTATUS=enabled + elif [ $RET -eq 1 ]; then + set_config_var enable_uart 0 $CONFIG + HSTATUS=disabled + else + return $RET fi - #"Serial is now enabled" else return $RET fi @@ -540,23 +611,15 @@ get_camera() { echo $OUTPUT } -get_serial() { - if ! grep -q "^T.*:.*:respawn:.*ttyAMA0" /etc/inittab; then - echo 0 - return 0 - fi - echo 1 - return 0 -} get_w1(){ - output=$( cat $CONFIG | grep ' *dtoverlay*=*w1-gpio' ) - if [ -z "$output" ]; then - echo 0 - else - echo 1 - fi - return 0 + if grep -q -E "^dtoverlay=w1-gpio" $CONFIG; then + echo 0 + else + echo 1 + fi + return 0 } + do_w1(){ RET=${args[1]} CURRENT=$(get_w1) @@ -593,9 +656,11 @@ case "$FUN" in 15) get_timezone ;; 16) do_timezone ;; 17) get_camera ;; - 18) get_serial ;; + 18) get_serial_hw ;; 19) do_w1 ;; 20) get_w1 ;; + 21) get_i2c ;; + 22) get_spi ;; *) echo 'N/A' && exit 1;; esac || echo "There was an error running option $FUN" if [ $ASK_TO_REBOOT = 1 ]; then diff --git a/setup.py b/setup.py index 644f4e7..3be63ce 100644 --- a/setup.py +++ b/setup.py @@ -1,35 +1,45 @@ from setuptools import setup, Extension import os import pwd +import grp +from myDevices import __version__ +from myDevices.system.hardware import Hardware -classifiers = ['Development Status :: 1 - Alpha', + +classifiers = ['Development Status :: 5 - Production/Stable', 'Operating System :: POSIX :: Linux', 'License :: OSI Approved :: MIT License', 'Intended Audience :: Developers', 'Programming Language :: Python :: 3', 'Topic :: Software Development', 'Topic :: Home Automation', - 'Topic :: System :: Hardware'] + 'Topic :: System :: Hardware', + 'Topic :: System :: Monitoring'] try: - # Try to install under the cayenne user, if it exists. - user = pwd.getpwnam('cayenne') - user_id = user.pw_uid - group_id = user.pw_gid + # Try to install under the cayenne user, if it exists. + username = 'cayenne' + user = pwd.getpwnam(username) + user_id = user.pw_uid + group_id = user.pw_gid except KeyError: - # Otherwise install under the user that ran sudo. - user_id = int(os.environ['SUDO_UID']) - group_id = int(os.environ['SUDO_GID']) -directories = ('/etc/myDevices', '/var/log/myDevices', '/var/run/myDevices') + # Otherwise install under the user that ran sudo. + user_id = int(os.environ['SUDO_UID']) + group_id = int(os.environ['SUDO_GID']) + username = pwd.getpwuid(user_id).pw_name +directories = ('/etc/myDevices', '/etc/myDevices/scripts', '/var/log/myDevices', '/var/run/myDevices') for directory in directories: - try: - os.makedirs(directory) - except FileExistsError: - pass - os.chown(directory, user_id, group_id) + try: + os.makedirs(directory) + except FileExistsError: + pass + os.chown(directory, user_id, group_id) + +# Touch config file so it overwrites older versions +os.utime('scripts/config.sh', None) setup(name = 'myDevices', - version = '0.2.1', + version = __version__, author = 'myDevices', author_email = 'N/A', description = 'myDevices Cayenne agent', @@ -38,9 +48,56 @@ keywords = 'myDevices Cayenne IoT', url = 'https://www.mydevices.com/', classifiers = classifiers, - packages = ["myDevices", "myDevices.ipgetter", "myDevices.cloud", "myDevices.utils", "myDevices.os", "myDevices.sensors" , "myDevices.wifi", "myDevices.schedule", "myDevices.requests_futures", "myDevices.devices", "myDevices.devices.analog", "myDevices.devices.digital", "myDevices.devices.sensor", "myDevices.devices.shield", "myDevices.decorators"], - install_requires = ['enum34', 'iwlib', 'jsonpickle', 'netifaces', 'psutil >= 0.7.0', 'requests'], + packages = ["myDevices", "myDevices.cloud", "myDevices.utils", "myDevices.system", "myDevices.sensors" , "myDevices.wifi", "myDevices.schedule", "myDevices.requests_futures", "myDevices.devices", "myDevices.devices.analog", "myDevices.devices.digital", "myDevices.devices.sensor", "myDevices.devices.shield", "myDevices.decorators"], + install_requires = ['enum34', 'iwlib', 'jsonpickle', 'netifaces >= 0.10.5', 'psutil >= 0.7.0', 'requests', 'paho-mqtt'], data_files = [('/etc/myDevices/scripts', ['scripts/config.sh'])] ) os.chmod('/etc/myDevices/scripts/config.sh', 0o0755) + +# Add conf file to create /var/run/myDevices at boot +with open('/usr/lib/tmpfiles.d/cayenne.conf', 'w') as tmpfile: + tmpfile.write('d /run/myDevices 0744 {0} {0} -\n'.format(username)) + +relogin = False +# Add user to the i2c group if it isn't already a member +user_groups = [g.gr_name for g in grp.getgrall() if username in g.gr_mem] +if not 'i2c' in user_groups: + os.system('adduser {} i2c'.format(username)) + relogin = True + +if Hardware().isTinkerBoard(): + # Add spi group if it doesn't exist + all_groups = [g.gr_name for g in grp.getgrall()] + if not 'spi' in all_groups: + os.system('groupadd -f -f spi') + os.system('adduser {} spi'.format(username)) + with open('/etc/udev/rules.d/99-com.rules', 'w') as spirules: + spirules.write('SUBSYSTEM=="spidev", GROUP="spi", MODE="0660"\n') + os.system('udevadm control --reload-rules && udevadm trigger') + relogin = True + # Install GPIO library if it doesn't exist + try: + import ASUS.GPIO + except: + current_dir = os.getcwd() + try: + TEMP_FOLDER = '/tmp/GPIO_API_for_Python' + GPIO_API_ZIP = TEMP_FOLDER + '.zip' + import urllib.request + print('Downloading ASUS.GPIO library') + urllib.request.urlretrieve('http://dlcdnet.asus.com/pub/ASUS/mb/Linux/Tinker_Board_2GB/GPIO_API_for_Python.zip', GPIO_API_ZIP) + import zipfile + with zipfile.ZipFile(GPIO_API_ZIP, 'r') as lib_zip: + lib_zip.extractall(TEMP_FOLDER) + os.chdir(TEMP_FOLDER) + import distutils.core + print('Installing ASUS.GPIO library') + distutils.core.run_setup(TEMP_FOLDER + '/setup.py', ['install']) + except Exception as ex: + print('Error installing ASUS.GPIO library: {}'.format(ex)) + finally: + os.chdir(current_dir) + +if relogin: + print('\nYou may need to re-login in order to use I2C or SPI devices')