ESP Modem for Rust (LTE / GSM / Sim Card) #468
Comments
|
Unfortunately I haven't played with these modems yet, so below my very limited understanding:
Probably this is a start? Also, are you only interested in modems for internet connectivity? I assume modems can also be used for simple SMS send/receive? Not sure how calling / receiving voice calls work over these, if supported at all... Anyway, these are probably corner cases... Yes the project is large-ish, but maybe not that much after all... |
|
Oh. One more thing but maybe important - given that ALL communication with the modem is basically over a UART channel (sending and receiving bytes of data) - is there any pure-Rust crate that implements this? Even if this crate (obivously) does not talk to ESP Netif's PPP out of the box, it can be made to do so. I'm mentioning this because if there exists a well-supported Rust crate that does the AT comm with the modem, it might be easier and cleaner to integrate with it instead, rather than going down the ESP IDF C route again. |
|
Interesting. And sounds rather simple. |
|
Also this is interesting. Given that @diondokter is active in the Embassy Matrix channel, why don't you ping him in there? Given his library, I'm sure he had dealt with a modem or two and their pesky AT-command interface. :) @diondokter Sorry maybe even better to also ping you here (as I actually already did, from above): The TL;DR is - we are contemplating wrapping in Rust an ESP IDF C library called "ESP Modem" that does the heavy lifting of configuring a modem via AT commands so that eventually it is switched to PPP at the end (this always happens, right?) and then once the UART channel is in "PPP mode" so to say, we we can further hook it up with the ESP IDF networking layer (this is not However - this all might be unnecessary, and might just bring more unsafe C code and a lot of work for us, I fear.
... the one remaining challenge is how to configure the modem via AT commands. Given your crate I referenced above, you've surely dealt with this in the past. Do you have any link / example code how this is typically done? (Ideally, with your "at-commands" crate :) ? If the example is with - say - |
|
Hi, it really depends on your modem! With nrf-modem however, there are actual APIs that you can use. So a socket is just a socket and that doesn't use the at commands. At-commands are used only for things where no API is available. Example for building a command: https://github.com/diondokter/nrf-modem/blob/ec738c3762974bfd3559becdc785907cafff07e4/src/lib.rs#L305-L314 I'm not familiar with the ESP modem though, so the only tip I can give is to make your interface to the modem nice to use. Modems are already notoriously frustrating at times, so don't add to that with a cumbersome interface. I'm not quite sure what the embassy-net-ppp interface is like, but I'd try to make the modem device struct be able to split into two channels:
The AT channel you can then use to setup the modem and query it for information. The PPP channel can then be handed to the embassy-net-ppp library. The channels themselves then coordinate the state of the uart and the switching of modes of the modem. Not sure what else I can add... But feel free to ask anything |
That was already plenty, thanks a lot! Regarding PPP, this is just a way to tunnel IP traffic (IP packets) over a point-to-point connection, which in our case would be between the host (The ESP MCU) and the modem itself (which is whatever modem is connected over UART to the ESP). I was imagining that's all there is to it (1- AT commands to setup the modem and then 2- a PPP channel for encapsulating the IP traffic/packets)... but then, looking at your Nordic modem wrapper, you have a lot of code in there, including stuff like TCP socket struct, UDP socket struct and whatnot. ... which brings the question...
... so in a way you won't need to model your own TCP or UDP socket. That would come from whatever IP/TCP/UDP stack library you choose to use over the PPP tunnel...? So where is the complexity in your case coming from? |
In your case that is indeed all there is to it.
Yep, you're right
While it's possible to use PPP on the nRF91, it's not needed. It provides its own socket APIs with which you can do TCP, UDP and more. This TCP/IP stack then runs on the modem itself. This has two advantages:
If I ran the TCP/IP on the host core, then that takes away compute and it makes the firmware bigger. There still might be good reason to want to run your own stack, for example when the built-in stack is buggy or lacks some feature, but I've seen no issue in using the existing socket APIs. For you though, you don't seem to have a choice since the ESP modem doesn't list anything about a built-in TCP/IP stack. |
|
@diondokter Crystal clear, thanks a ton! @DaneSlattery I am somehow leaning towards seriously evaluating first if we can implement the modem in pure Rust. (I'm already regretting a bit that for the one-wire we did not instead expose the new RMT driver in Rust and then just write on top a pure-Rust one-wire impl that can simply operate on top of two APIs: Also in general, my feeling w.r.t. ESP IDF is that:
|
|
@ivmarkov I agree that the As for a pure-rust implementation, I think given the information:
|
If I recall correctly, the approach where the new RMT driver needed to be exposed was simply deemed much more effort to implement by you. Hence why we ended up where we are. Which is OK, we still did a good progress! But we definitely did not have analysis-paralysis, I believe.
There is no "steering committee" of sorts, because Espressif is taking a rather opportunistic / tactical approach towards its Rust-related portfolio. This is also valid for the bare-metal crates which do have paid developers working on those.
Do I miss a more "architectural" / "bigger-horizon" approach which can be discussed among more community members? I certainly do, but this requires a lot of prior exposure to the As in, there are bi-weekly meetings of the Anyway, I digress.
Can you elaborate what cmux and vfs support in the ESP Modem C driver is about, if you've looked into it?
I did not mention
|
|
Thank you for the detail on the management structures. It is interesting to see. It seems like Here's what I have on cmux, which seems useful for keeping the UART in AT and PPP mode simultaneously. And VFS: Here are some more relevant crates: |
It is. The ESP-IDF is viewed as a stop-gap solution, and obviously the solution if you already have a large C codebase that you can't or don't want to migrate - in one go or at all. With that said, it's been like that for many years. What I'm personally doing is trying to abstract from the underlying "OS" as much as possible - with e-hal, or by rolling your own traits etc. And by using async IO as much as possible as no other embedded platform besides Hence - the more code we have in Rust itself - the better for everyone!
Agreed that none of these is essential. Even CMUX - at least initially.
Yes this is the PPP impl for embassy. If your AT-commands code is in pure Rust (you can even abstract the UART by using just the async IO Read/Write traits), we can even write connectors for your code to embassy-PPP.
Sounds like a plan! See, if everything goes the way we hope it goes, we might not even have to PR anything in |
|
I have started writing out some AT commands over a UART device and getting replies. I will borrow a few commands from What I am not sure of is how to glue the modem when it is in
|
Great progress!
I don't think this would be necessary, because the machinery around With that said, your mileage may vary, and we have to see how it is shaping up. Two reasons for that:
It should boil down to the following:
As per above, once you do the above ^^^ then you have a netif (network interface) which is attached to the ESP IDF
Let's wait with this.
I suggest not to use the ESP Modem component for anything but inspiration (copy-paste) of C code and then translation of that code to Rust unsafe calls into Yes, exactly! This is from where you need to "copy-paste" code into Rust. :) |
|
By the way... (and no offense to the After all, all that you need is a utility, that converts an AT command to a sequence of bytes that you can then send via All of these extra layers of abstractions in
So what is the point? |
One thing I really like from for example: /// 4.1 Manufacturer identification +CGMI
///
/// Text string identifying the manufacturer.
#[derive(Clone, AtatCmd)]
#[at_cmd("+CGMI", ManufacturerId)]
pub struct GetManufacturerId;
/// 4.1 Manufacturer identification
/// Text string identifying the manufacturer.
#[derive(Clone, Debug, AtatResp)]
pub struct ManufacturerId {
pub id: String<64>,
}
impl<'d, T> EspModem<'d, T>
where
T: embedded_svc::io::Read + embedded_svc::io::Write,
{
pub fn new(serial: &'d mut T) -> Self {
Self {
serial,
_d: PhantomData,
}
}
pub fn send_cmd<CMD: AtatCmd>(&mut self, cmd: &CMD) -> Result<CMD::Response, atat::Error> {
let mut buff = [0u8; 64];
// flush the channel
while self
.serial
.read(&mut buff)
.map_err(|_err| atat::Error::Read)?
> 0
{}
// write the command to the uart
let len = cmd.write(&mut buff);
self.serial
.write(&buff[..len])
.map_err(|_err| atat::Error::Write)?;
// now read the uart to get the response
let len = self
.serial
.read(&mut buff)
.map_err(|_err| atat::Error::Read)?;
cmd.parse(Ok(&buff[..len]))
}
}This gives us strong typing of the commands and responses, but the trait is a bit bloated: I feel like the at-commands library could perhaps get it's own traits. |
|
Kept lurking... Probably not the place to discuss it, but the I'd be open to make it use the |
|
Beauty is in the eye of the beholder, I guess, because I like this builder pattern much more. You can just call it from your modem code, and it will just serialize the commands directly in the buff you supply. Incrementally. No intermediate (potentially large-ish) objects, no nothing. But it is you who are implementing it, so it is your choice of course. |
I - personally - like the Now, if communicating with the AT modem requires a complex re-try logic, then it might make sense, as this can't be modeled without introducing |
|
I have worked it for the day and come up with an idea for the The traits I was talking about was not about the transport layer, more along these lines. I like the fact that commands and their responses are linked in pub trait Cmd {
/// The type of the response. Must implement the `Resp` trait.
type Response: Resp;
/// Write the command and return the number of written bytes.
fn write(&self, buf: &mut [u8]) -> usize;
/// Parse the response into a `Self::Response` or `Error` instance.
fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error>;
}
pub trait Resp {}
pub struct GetSignalQuality;
pub struct SignalQuality{
ber: i32,
rssi: i32,
};
impl Resp for SignalQuality {}
impl Cmd for GetSignalQuality {
type Response = SignalQuality;
fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error> {
let (rssi, ber) = CommandParser::parse(&buff[..len])
.expect_identifier(b"\r\n+CSQ: ")
.expect_int_parameter()
.expect_int_parameter()
.expect_identifier(b"\r\n\r\nOK\r\n")
.finish()
.unwrap();
Ok(SignalQuality{rssi,ber})
}
fn write(&self, buf: &mut [u8]) -> usize {
CommandBuilder::create_execute(&mut buf, true)
.named("+CSQ")
.finish()
.unwrap();
}
}The part I seem to be after is more const functions, especially for builders. I don't think I should build a new signal quality command every time if the named value never changes. The command I build then is also not so easily shared with another device that might have the same command name but a different It might also be interesting to have the ability to configure delimiters for parsers and builders. For example, every response on the simcom7600 series replies with It might also be interesting to support a That said, none of this is really hindering me. I will create a PR for this so that we may see some code |
So I am completely stuck here with |
I had to shoot in the dark a bit as I weren't sure what exactly is the problematic aspect. Hopefully I was right and this feedback would be helpful. If not (or if you have additional questions), I would gladly address those. |
|
Also see this (NOTE: not typechecked!) which is what I now believe we are ultimately trying to achieve over here! :) |
|
@DaneSlattery I decided it would be fair to at least typecheck my code, so you can (try to) use my (We have to prohibit direct pushes to To use the ...and then in your binary crate you need to replace the ref to ... or just also add ... to your ==== Basically, you need to call To push RX packet into the driver, you need to call |
|
Unfortunately, I cannot test this code for real, as I'm away from home, so I'm not having access even to my measly GPRS modem which is lurking around (actually not even sure this cheapo thing supports PPP, as I never tried it but...). |
|
@DaneSlattery If you are already using my changes, please do |
|
Hi @ivmarkov Thank you for the assistance on this! The stuff you did boggles the mind, and I will continue to look around at the various comments. Just wanted you to know I have attempted this now, and I'm currently debugging this error. Please note that my binary is actually the Update, ran this without |
Yes, I commented on it.
Let's keep
Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our One inconvenience is that we don't have the |
|
You can also spawn another thread which is examining the |
Now that I think of it, I'm not sure what protocol is used over PPP so that the ESP will get an IP address (that is, assuming you don't just assign a fixed IP). It can't use DHCP, as DHCP lives on the border between Ethernet and IP, and does assume the medium below IP is Ethernet. So what is it? Maybe something within PPP itself? |
I decided to just follow the c code while I try get this to work: I think it would be good to map the events. |
PPP itself is only Layer2, so only works with MAC addresses. E.g no routing. PPP brings NCP's( Network control protocols) for every higher level supported protocol to the table. They are encapsulations to configure this higher level protocols. For the mention IP case its the IPCP ( Internet protocol control protocol) |
|
I just got myself an IP! I need to:
|
Were you able - in the meantime - to also open a TCP/UDP socket on top? Does it work?
Not strictly necessary as that's an internal API inside
As per above - up to you for the inside of
I think PPP somehow has special frames which designate "end of data mode". Depending on where we go with the If you decide to go in the direction where EspModem=SimModem is not concerned with the PPP (data) mode, then you don't even need to exit the "SimModem::run" function as that function would only be responsible for setting up the data mode and will exit when the data mode is ready to be established (but won't run the data mode).
Yes.
That's actually easier than what you probably think it would take. But let's solve everything else first, then I'll try to provide a code snippet as to how that would work.
Not necessary in my opinion. |
I did it already. I think if we don't use the event loop, we must use In However, we still need to subscribe to
I think lost-ip is good enough, or any |
Sure, but my point is, don't do this in the modem code, driver etc. Just do it in your example.
It would be good if you expose a type-safe wrapper for the PPP events, similar to
Might be the case yes. |
Amazing, it works. It takes a really long time (5 minutes now) to connect to the network, and I don't know if it's using 4G,3G or 2G, but I managed to make a network request with the example in @diondokter , do you know if there is a way to tell what network a modem is connected to? |
|
@DaneSlattery There might be an AT command to find that out |
|
@DaneSlattery I assume the slowdown is before your log snippet, as the action in your log snippet seems to take only 13 seconds or so. Even if it is 2g, this (5 minutes |
|
@DaneSlattery Just wondering where we are with this? |
|
Hi @ivmarkov , I am still going to put some time into this, but I have just received a new batch of PCBs with a few differences that I need to work through before I can pick up the modem support. I am still waiting for some modems to be built as well. I am currently blocked in the last read before switching to PPP mode, ensuring that I do not ingest any bytes beyond the AT command reply. I also need to implement some digester that reads until a terminator |
Cough.... BufRead.... Cough... that you then continue to use for the actual PPP stream with whatever remained inside... |
|
Ok I have given this another try, and I welcome a review. The design has changed slightly:
I do think the I do think , in part because the events we subscribe to for the As for the pure functionality of this, I will need to plug in a sim card tomorrow and give this a try. Will report back. |
The ESP-Modem Component is not supported.
It would enable SMS+GSM +LTE connections to work with the current Netif interface.
I have started calling into the the esp-modem component, and figured it would be good to wrap this up for use in
esp-idf-svc.The code in
[netif.rs](https://github.com/esp-rs/esp-idf-svc/blob/bf47d2bcee6e1bc71530a23264eae63a7af84aae/src/netif.rs#L34C5-L36C9)suggests support forPPPmode, but currently there are no ways to create that interface.I am happy to start development, but I think this may be a pretty big project and would appreciate some help fitting this into current components. The official
esp-modemcomponent is quite chunky: https://github.com/espressif/esp-protocols/tree/master/components/esp_modem.The text was updated successfully, but these errors were encountered: