uploading new js script doesn't kill current running script in RAM. - callbacks not called even if function completes

[espruino-discuss3]

Posted at 2020-09-08 by dgk

Running an esp32 wroom with latest firmware. I am using the cli not the webide.

I seem be running into an issue that involves my ignorance of how the espurino firmware manages the running js script process.

I've been uploading my code via the cli and just assumed that there can be only one js process running at a time so after upload I assumed that the current running js scripts is killed and the newly uploaded one is started. That doesn't seem to be the case (e.g. all the callbacks for the newly upload code are never called). Note: when programming with arduino or espessif firmware/framework via platformio when I upload a new ccp program the old one is killed.

So I tried to first upload a simple reset() script which supposedly purges the script in RAM then upload my code, nope that doesn't work either.

So then I wiped the flash by uploading reset(true). Then I did a hard reset (button of esp dev board) so I know (via terminal) no script is running then upload my code and it works fine.

So now I am forced to press the hard reset every time before uploading a new version of my code to RAM (during development). BTW if I then save code to flash and press hard reset it runs fine each time (because there was no already running js script).

Thus it seems the former js code is still running even after a new upload, even after a soft reset?? Many times when programming in node the node process doesn't get killed (with cntrl-c) when script is exited and either have to include some shutdown code when SIGTERM is called (like stopping listeners) or I have to use the linux 'kill' command but. There is no process 'kill' nor linux SIGTERM for the esp32 (i.e. via espurino 'os') as far as I know

So how does one manage (kill) the running js script during development? I suppose I just need a better explanation about how the espruino firmware manages RAM and running the running js process(es ?) therein. I just assumed given the limited resources that only one js script process at a time can be run.

[espruino-discuss3]

Posted at 2020-09-08 by AkosLukacs

reset(1) should erase all saved code and kill all running code as well. The only thing that stays the same should be wifi connection status.
Can you create a minimal repro that shows this behaviour?

---

Posted at 2020-09-09 by dgk

Although I am using the cli with rollup if I use the webide with this code below I get the same issue (callbacks are never called) even with "reset before send" in the IDE set.

If I send

reset(1)  // reset ram, clear flash
require('Wifi').save('clear') // clear wifi credentials
console.log('RAM,flash, and wifi settings have been cleared')

separately before uploading same issue, no callbacks called.

if I press my hard reset button on esp board then send/upload callbacks are called.
if I then send/upload the same program then the callbacks are not called

this behavior is totally consistent. That soft reset(1) never fixes the issue which I assume is that the previous code is running or for whatever reason no callbacks are called.
I put in a console.log('udpdated code 2'); // increment for each send and I can see each upload that the just sent update is running but the callbacks never get called until I first do a hard reset.

So what about my question can the firmware be running multiple scripts? or otherwise what would be causing the callbacks to be silent until a hard reset?? I have not investigated further because as I said unlike linux I just don't understand how espruino firmware manages memory and js processes.

var wifi = require('Wifi');

var data = { wifi:{ ssid:'zoesplace',
  password:'PalDgk19591960',
  hostname:'esp32test',
  'static':{ ip:'10.0.0.45',
    netmask:'255.255.255.0',
    gw:'10.0.0.1' } } };

  function connect(opts) {

    wifi.on('connected', () => {
      console.log('ESP connected to AP');
      console.log('udpdated code 2'); // increment for each send
       wifi.ping('10.0.0.1',function (res) {console.log('in ping callback', res.respTime,'ms');});
       setTimeout(function (val) {console.log('test timeout done');},5000);
    });

    wifi.on('disconnected', () => {
      console.log('ESP disconnected from AP');
    });

    var state = wifi.getStatus();
    console.log('determining wifi status');

    if (state.ssid === opts.ssid & state.station === 'connected') {
      console.log('already connected to', state.ssid);
      wifi.emit('connected');
    } else {
      console.log('not connected...connecting');
      wifi.connect(opts.ssid, {
        password: opts.password
      }, function (err) {
        console.log('in connect callback');
        if (err) { console.log('wifi connection error', err);}
      });
    }
    }
   
connect(data.wifi);

---

Posted at 2020-09-12 by AkosLukacs

Wait a moment, IIRC ESP32/8266 connect to Wifi automatically, if wifi credentials were saved. And IIRC reset() will not disconnect from wifi, just resets Espruino interpreter. So yes, your wifi.on('connected' will not fire, because it's already connected to wifi by the time your code is executed.
So what I did is check in my startup code if it's connected to wifi. If not, than start connecting. If already connected, just run wifi releated startup code immediately.

---

Posted at 2020-09-12 by @fanoush

just o to add - to do full soft reset there is E.reboot(), reset() just reinitializes the javascript interpreter, clears variables etc. The idea of reset() is that it restarts without closing console connection over bluetooth/wifi.

---

Posted at 2020-09-12 by @MaBecker

I noticed on ESP32, that sometimes a wifi.restore() is helpfull.

---

Posted at 2020-09-14 by dgk

I'm beginning to suspect that is the esp wroom. It only has 512K RAM.

I changed the code above to do 3 manual manual disconnects and you can see (below) both the disconnect and connect callbacks were called the first time but then not called afterwards (I used a timeout) even though it was really disconnected and connected. Sometimes though when I run it they may get all called for all three trials. The callback failure is not particularly consistent unless I fail to do a hard reset in which case then never get called.

I know that for low.js the minimum requirement is wrover (has extra (ps)ram and flash) maybe that is also true for espruino? @MaBecker? Have others had similar issues issues using a wroom??

Explicit board JSON supplied: "../boards/ESP32.json"
Connecting to '/dev/ttyUSB0'
Connected
environment { 
  "VERSION": "2v06.122", 
  "GIT_COMMIT": "eca6aae8", 
  "BOARD": "ESP32", 
  "FLASH": 0, "STORAGE": 262144, "RAM": 524288, 
  "SERIAL": "246f2855-5a80", 
  "CONSOLE": "Serial1", 
  "MODULES": "Flash,Storage,heatshrink,fs,net,dgram,tls,http,NetworkJS,Wifi,TelnetServer,crypto,neopixel", 
  "EXPTR": 1073484860, "PRODUCTION": false }
determining wifi status
not connected...connecting
ERROR: Wifi: event_handler STA_START: esp_wifi_connect: 12298(SSID is invalid)
WARNING: Wifi:startMDNS - espressif
wifi connect callback
ESP connected to AP
application online at 10.0.0.225
---------------- manually disconnect/reconnect test -------------- 1
forcing disconnect now
wifi disconnect callback
ESP disconnected from AP
attempting reconnect in 5
application offline
determining wifi status
not connected...connecting
WARNING: Wifi:startMDNS - espressif
WARNING: Wifi:stopMDNS
wifi connect callback
ESP connected to AP
application online at 10.0.0.225
---------------- manually disconnect/reconnect test -------------- 2
forcing disconnect now
disconnect timeout
disconnect callback failed but device is disconnected!!!
ESP disconnected from AP
attempting reconnect in 5
application offline
determining wifi status
not connected...connecting
true timed out trying to connect
connect callback failed but device is connected!!!
ESP connected to AP
application online at 10.0.0.225
---------------- manually disconnect/reconnect test -------------- 3
forcing disconnect now
disconnect timeout
disconnect callback failed but device is disconnected!!!
ESP disconnected from AP
attempting reconnect in 5
application offline
determining wifi status
not connected...connecting
true timed out trying to connect
connect callback failed but device is connected!!!
ESP connected to AP
application online at 10.0.0.225
done disconnect trial of 3 times
>

---

Posted at 2020-09-22 by dgk

Well I bought a wrover and tried my code and this issue persists. Since my code looks "fine" I can only assume this is a bug in espruino.

Looks like since the esp32 is not an official board so this bug may not be addressed.

I was hoping to use espruino on an esp32 (even written a starter repo using rollup) and I'd like to make a donation but it's a catch22. I'd like to donate but only if it can be used. If I can't get a simple program running with stable behavior (callbacks called dependably) well then it's not useful :(.

---

Posted at 2020-09-22 by @MaBecker

Looks like since the esp32 is not an official board so this bug may not be addressed.

but you can file a issue for this at https://github.com/espruino/Espruino/issues

---

Posted at 2020-09-22 by parasquid

So what about my question can the firmware be running multiple scripts?

The firmware doesn't run multiple scripts; there is just one forever loop running and when you "upload" you execute the code you are uploading line by line, as if you typed them in the left hand side (except for things like modules and inline c/asm which are preprocessed or minified by the web ide then uploaded).

Like everyone said, reset should have fixed this but maybe it is a bug in the ESP32 port? If it's just a matter of resetting, you can always wire up something to the reset pin and trigger that through software when you want to reset. This would make you lose connection, but then again when your project is deployed you don't always need to be connected.

---

Posted at 2020-09-23 by dgk

this is more than just resetting. I reset and upload/run my script and initially the callbacks are called but then fail per the output I supplied. Why would it initially call a callback but on the second call of the function fail to call the passed callback even though the function has clearly completed its task (e.g. disconnect). That seems like a bug to me (coming from nodejs world that has never happened to me) although I'm still a noob when it comes to javascript/espruino on esp32

---

Posted at 2020-09-23 by parasquid

Not sure if this would work but can you try putting the wifi.on stuff outside of the connect function? (I'm assuming this is what you were referring to as the callback)

Actually nvm, consider me confused which callback you're referring to :D I saw an emit somewhere and I'm not sure what's happening anymore.

I personally just do simple things like the ones found here: https://www.espruino.com/ESP32#connecting-to-wifi

Maybe try this one first and see if the wifi.connect callback runs? (again, I'm not really sure which callback you're referring to).

---

Posted at 2020-09-23 by parasquid

I just flashed an ESP32 I have with 2.07 and uploaded this code:

var ssid = 'Mi A3';
var password = 'parasquid';

var wifi = require('Wifi');

wifi.on('disconnected', () => {
  console.log('disconnected!');
  setInterval(connectToWifi, 1000);
});
wifi.on('connected', () => console.log('connected!'));

const connectToWifi = () => {
  wifi.connect(ssid, {password: password}, function() {
      console.log('Connected to Wifi.  IP address is:', wifi.getIP().ip);
      wifi.save(); // Next reboot will auto-connect
  });
};

connectToWifi();

I then setup a hotspot on my phone to control the wifi connection and disconnection outside of the ESP32.

The callbacks are running, but it looks like the issue is that disconnect status isn't immediately detected. I can force the detection of the disconnect (and the callback) by doing something on the network (like getting a url)

Connected to Wifi.  IP address is: 192.168.43.111
connected!
connected!
>require("http").get("http://www.pur3.co.uk/hello.txt")
Uncaught InternalError: Unable to locate host
 at line 1 col 54
require("http").get("http://www.pur3.co.uk/hello.txt")
                                                     ^
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
disconnected!
WARNING: Wifi:startMDNS - espressif
WARNING: Wifi:stopMDNS
Connected to Wifi.  IP address is: 192.168.43.111
connected!
connected!
>require("http").get("http://www.pur3.co.uk/hello.txt")
=httpCRq: { type: 1,
  res: httpCRs: {  },
  opt: {
    protocol: "http:",
    method: "GET",
    host: "www.pur3.co.uk",
    path: "/hello.txt",
    pathname: "/hello.txt",
    search: null, port: null, query: null },
  dSnd: "GET /hello.txt HT" ... "ww.pur3.co.uk\r\n\r\n",
  sckt: 56, cls: true }
> 

Sometimes though, disconnection is immediately detected and the callback runs (again tested by turning on or off the hotspot on my phone). And if you wait long enough, the ESP32 detects the disconnection and then runs the callback.

So maybe the best solution for now is to have a setInterval that pings a known good url so as to trigger a disconnect if it goes down?

---

Posted at 2020-09-23 by parasquid

Just to add, connecting and disconnecting the hotspot in quick succession will sometims leave the wifi status as AUTH_LEAVE and will not call the callback (because it technically isn't disconnected yet?)

>wifi.getStatus()
={
  station: "AUTH_LEAVE",
  mode: "APSTA",
  powersave: "modem"
 }

---

Posted at 2020-09-23 by parasquid

Manually calling disconnect does call the callback (and in this case, also reconnect)

Connected to Wifi.  IP address is: 192.168.43.151
connected!
>wifi.getStatus()
={
  station: "connected",
  ssid: "Mi A3",
  bssid: "2e:1d:1f:e3:55:21",
  channel: 8, rssi: -18,
  htMode: "HT20",
  authMode: "wpa2",
  mode: "APSTA",
  powersave: "modem"
 }
>http.get("http://www.pur3.co.uk/hello.txt");
=httpCRq: { type: 1,
  res: httpCRs: {  },
  opt: {
    protocol: "http:",
    method: "GET",
    host: "www.pur3.co.uk",
    path: "/hello.txt",
    pathname: "/hello.txt",
    search: null, port: null, query: null },
  dSnd: "GET /hello.txt HT" ... "ww.pur3.co.uk\r\n\r\n",
  sckt: 59, cls: true }
>wifi.disconnect()
=undefined
disconnected!
WARNING: Wifi:startMDNS - espressif
WARNING: Wifi:stopMDNS
Connected to Wifi.  IP address is: 192.168.43.151
connected!

---

Posted at 2020-09-23 by dgk

@parasquid this "no callback" issue wasn't restricted to the connect and disconnect functions it was doing the same for wifi.ping, likely any callback.

The code I posted at the top was not the same code that generated the output above . The code whose output is above is from my starter repo that uses rollup and the listeners are not in the connect function. So rather than try to post all the pieces here is the link. I just committed the version of the "app.js" that does the three trials.

https://github.com/dkebler/espruino-starter.git

You'd be able to clone it and try my actual code. Directions are in the readme. It's swift to get this running. Maybe running the exact same code we can track down my issue.

In my wifi class there is a getter for a complete status which can be used at any time to see what state/status the wifi is in. I'll check again but I don't recall the disconnect being left in station: "AUTH_LEAVE" when the the callback was never called.

one thing to note is that station might be "connected but the ssid is empty which is really a disconnected state (as I can't ping the esp like this). That's why my connected getter checks both station and ssid.

---

Posted at 2020-09-23 by @MaBecker

check this test code that was used to verify the functionality of wifi events

https://github.com/espruino/Espruino/blob/master/targets/esp32/tests/wifiEvents.js