[Bug]: wolfSSL_free crash on ESP32s3 #6637
Comments
|
Hi @PaulMartinsen - thank you for reporting this issue. There's definitely an improvement in my #6624 regarding properly locking and releasing the hardware, and falling back to SW as needed. Given that, if you disable the HW acceleration, do you see the same problem? Note that I've recently refactored the macros in #6587, removing the Coincidentally I am now working on updating the ESP32 client and server examples. See also wolfssl-examples/ESP32. There are no changes to these apps in the open PR. The work on I am doing is with regards to supporting a new cipher suite, but I've confirmed at least my current server works properly with a Linux client. If you use my example, just make sure you don't enable the SM Ciphers. If you have some example code to reproduce the error, that would be very helpful. I didn't immediately see it in your fork. Regarding your JTAG problems, I can certainly relate! I've found Zadig to be quite helpful when Windows decides to change the drivers. I set
You are still using VisualGDB, eh? For reference, here's my debug setting dialog box:
Other ideas: different USB cable, restart Windows, power cycle board. Exit/restart Visual Studio. VM running? Make sure the device does not get used there (yes, I've had that happen more than once!). Perhaps also try using Windows to completely remove the USB driver. Which exact board model are you using? Thanks again for all your input! Cheers |
|
Thanks @gojimmypi . I was a little worried reporting the problem with so little info, but some very helpful suggestions there as always so I'm glad I did. I'm using the ESP32-S3-DevkitC-1 v1.0 board with the P2N8R2 variant on it. I always overlook testing w/o the hardware acceleration, but that sounds like a good first step. And your new PR. This is part of a much larger project now, but I will try to create a simplified example that illustrates the problem, if the PR doesn't resolve, based on the library examples. One odd thing I notice with zadig: I get two devices when I connect the boards jtag port: |
|
Hi @PaulMartinsen, I'm glad you found the suggestions helpful.
Ah yes, excellent, I have exactly the same board. The There are several options for allocating the external PSRAM. Which are you using?
hm, that does sound odd. I only see one JTAG device. The UART is on a second physical connector. In contrast, my Tigard shows both USB devices on one USB connector. If you just cannot get it working, I'd suggest going into Device Manager and removing the driver. Be sure to unplug the device & allow Windows to reinstall the driver. I've also seen VisualGDB itself detect bad driver problems and offer to correct automatically with their UsbDriverTool.
I'll be looking into this more in the coming days. One thing I've found helpful when troubleshooting odd problems is the ESP stack high water mark. I think it is much better to monitor the actual level, rather than just rely on the stack smashing monitor & compiler options. I have an example in my latest benchmark app. Have a great weekend. Cheers |
|
Hi @gojimmypi , Just confirming I do not see the cross-task crash if I disable hardware acceleration (that is Currently I don't think I'm using the PSRAM at all (i.e., not configured, free ram reports 73k). Next stage of the project is going to be the memory intensive bit and I'll investigate the different options for that. Do you have recommendations? I vaguely recall reading the jtag connector can support serial comms too so I'm assuming that's why two devices are showing up with just that USB port plugged in. I know I spent a lot of time trying to get it going on my old machine; then it just worked on the new one through VisualGDB. Until now. Thanks for the reminder about the high-water mark. That's a great tool too add to my health monitoring service. You have a great weekend too. |
|
Quick update: I tried PR6624, but the cross-task crashing problem with hardware acceleration enabled still occurs in that branch unfortunately. |
|
Hi @gojimmypi . Turns out it is an id10t programmer error. While building a example to demonstrate I noticed I'd failed to call I'll mark this closed with apologies for the distraction, thanks for your help and thanks to whomever wrote the comment in |
|
Hi @gojimmypi . It looks like I jumped to conclusions closing this one. The problem resurfaced when I tidied up my fix and remains persistent. I haven't been able to create a simplified version to share yet, but instrumented a couple of files (attached) to:
What I have noticed is that a lock on the sha hardware is taken but never released during I've attached the log output I captured, the instrumented files and wolf config files. I'm using your PR6624 version of the WolfSSL library. In this scenario:
I turn on logging immediately after creating the TLS handshake at step 2, which is in the connection service task: I also noticed when the SSL session for the server request is released, the lock depth is 1 one (line 62 in the log). In this case, Let me know if you'd like to do a video chat to discuss. It seems quite complex! |
It would be quite interesting to see the diff of before and after of the tidy. What changed after the problem went away?
Let me explain in more detail some of the challenges. Perhaps I can inspire an "aha" moment and we can better isolate the root cause without me actually seeing your code. The interesting thing in a multi-thread environment with multiple processes both between threads and within a given thread is the That's what the initializer attempts to address: who am I and am I the one that called the HW encryption lock. I keep track of the address of the object that was currently active when the lock was placed. That way, any copy of the object that may think it was doing HW encryption (due to object property settings), should be gracefully and quietly falling back to SW. What's interesting is when a copy is made of the SHA ctx object. I suppose there's a non-zero chance of an in-progress SHA process that is abandoned before completion. I would think that the worst possible case scenario here though, would be an infinitely long lock. This seems quite unlikely in normal circumstances. What might be interesting though is if in one thread with The other place to look is why the unlock failed. It is very important to only call the wolfSSL locking. See the hack alert notes. See in particular the ESP-IDF There's no protective logic in the ESP-IDF functions to prevent the caller from exceeding or omitting a call. For example, see the periph_module_disable that will happily decrement their lock depth counter, even if What's even more curious is the tail end of your log. The "Bad lock depth" sounds like something I would have written, but I was not able to find those words in the code. Is that something you added? It does sound like something was not keeping track of who is actually in control of the hardware when the lock was released. How to detect this? So we need to be looking at code that may for some reason think it is in charge of the HW, but isn't. We should be looking for copies of the
yes, I agree that might help. I'm available later today (late afternoon, early evening Pacific time), or we can schedule something during the week. In the meantime I will study the information you provided in more detail & try to reproduce the problem. btw, have you tried btw (2) - if you are using my branch, there's a new
And now that I look at that code, I'm wondering if there's a missing
|
|
@gojimmypi , removing the jtag driver & letting VisualGDB install a new one resolved the debugging issue for me. Sometimes the simple solutions are best. This let me look into why the assert was failing in xGenericQueueSend. After a bit of mucking about I got the debugger to stop in the right place to inspect the values in the assert and found: This mutex comes from line 67 in Previously I thought we might have two different problems: something not releasing the hw and the cleanup not working. But the Call stack to failing mutex (line number in |
|
Hi @PaulMartinsen and thanks very much for all the detail. You have an excellent question.
Oh, that's great! I'm glad to hear your JTAG issues are resolved.
Please confirm if you are sharing a I reached out internally and asked about a given thread taking a mutex and releasing on a different thread. The response was:
I looked at the Espressif docs and there are relatively few references to the term "inversion". I'm not sure to what extent the Espressif FreeRTOS implemented the handing of priority inversion. See FreeRTOS Mutexes. In particular:
There's a comment on the wolfSSL forum:
See the referenced example tls/client-tls-writedup.c and Chapter 9 of the docs on Thread Safety. As the problem here seems to be the HW mutex across threads, I'm wondering how that would occur in the first place. I would have thought that when given a mutex in one task that is locked, any other thread that attempts a lock should "see" that and fall back to SW. The curiosity is how a different thread ended up trying to unlock the mutex. I'm going to spend some time today trying to force this situation & see what happens. If you are sharing a I would have expected the processing of HW to have released the mutex upon completion, and certainly not waiting until calling |
The My working theory is that
Grasping straws I know. Any idea where the lock taken in Log snippet from earlier: |
|
Hi @PaulMartinsen and thank you for the clarifications.
Interesting, The general consensus is to use a But I believe that was in the context of not sharing concurrent operations and not what you are doing: fulling handing off the object to a new thread and never again touching it in the originating thread. I'll need to do some more homework on that.
Oh! That's a wonderful question. I've had regular discussions with folks on social media that claim to never need to use JTAG for debugging. Here's an excellent example of how to answer your question and why JTAG debug is so valuable. I placed a single breakpoint on the first statement of
In particular, note the call stack. We ended up in
As for the question:
The role of the mutex is to ensure only one SHA calculation is using the hardware at any given time. Interestingly, the wolfSSL implementation knows that the SHA calculations can take some time, so more than one can be started and they run concurrently. This is most evident during the TLS handshake. It's really quite clever. Of course for hardware, at least on the ESP32, this is impossible as the HW only supports one full SHA calculation at a time, and it must complete before starting the next one. Thus if a second SHA calculation is requested and the mutex indicates the HW is busy, we fall back to SW calcs. In your output example above, I see the most recent message is On my to-do list is the interleaved message digest capability available on the ESP32-S3 that you are using. This method does allow concurrent HW SHA hashes to be calculated. See the bottom of page 825 in the Technical Reference Manual:
I think this is ultimately the answer. I'll try to demonstrate this before our call tomorrow. |
|
Thanks for clarifying mutex use. I was confused between the mutex and the I had a few minutes over lunch, so taking another look at log with better understanding... Second when decremented. This is in I'm using the Screenshot below lists all the instances the context at address Actually, both my instrumented lines in If |
|
Thanks so much for your time this morning @gojimmypi . I think we much much faster progress understanding the problem than trying to digest log files. Perhaps github needs to had video meetings to issue tracking! I roughed out the concept we talked about. My testing has been limited to subscribing to an event like I demoed in our discussion. It didn't crash! Tail end of the log with the interesting bits below.
So I think this might address the crash; though still we have the issue with the hardware being hogged by one calculation in some cases. As you noted, interleaving message digesting could address that. At least for the s3. Maybe for teh ESP32 as well. To me it looks like the differnce between the ESP32 & s3 might be only the s3 has separate input and output registers (which would make sequential operations with dma easier) while the ESP32 uses the same registers for input and output. On page 575 of they data sheet they talk about initializing with the first message block but I can't see why that couldn't be the last message block of a partial calculation with an extra step to write to the LOAD_REG to get the partial result out: I noticed a couple of things that surprised me:
|
|
Actually maybe we could use a critical section to protect Do you think this might solve the problem or have I missed something? |
|
Hi @PaulMartinsen and thank you for your time as well on Friday & all the great input. I agree the discussion was very productive. It was good to talk with you and learn more about your project. It looks very cool! First, for the record & as discussed: the official response from wolfSSL is that the That said, I understand your desire to open a TLS connection is one task, and manage data exchange over TLS in a separate task. If care is taken in your code to ensure the wolfssl I took a look at your suggested changes. I was not able to see the benefit of using the Today, if the hardware is busy, we immediately fall back to software, as we are assuming only one HW has can be in progress at any given time. In the future, we'll want to actually share the HW between multiple, concurrent SHA calculations, such as that in the int HashRaw() in internal.c. We'll want to take advantage of the mutex waiting capabilities when the HW is busy with a different hash, and only fall back when that wait takes too long or the HW is otherwise not available. I did implement several new features in my dev branch that allows better visibility into multi-threaded HW locking. See the new I've also implemented a feature that cleans up HW lock when free is called before I have a specific wolfssl_client_6637 project for this issue, and I've been testing with the corresponding updated wolfssl_server. The code still need a bit of work, but I'd be interested in the results as related to the assert failure that we've seen in your code. See also the client 6637 sample output, in particular the esp_sha_release_unfinished_lock 3fcc9ff8 on line 2474. |
|
Thanks @gojimmypi . Threading is a tricky topic, so not surprising there could be dragons lurking if I'm not very careful. For posterity, we're looking at this sequence, where | ↣ | is the (one way) boundary where responsibility of the Motivation, again for posterity: I could have a dozen open connections to send notifications to, most of them using TLSv1.3, adding new ones at any time. And I want to minimize latency for sending notifications. On the ESP32s3 it takes around one second to do a TLSv1.3 handshake. So notifications would stall for a second every time I need to send a message if I have to open a connection on the same thread. A couple of other solutions I considered:
An important part of my project is understanding the constraints and possibilities of the ESP32s3 environment, so very much appreciate your help navigating through. Happy to do whatever I can to find a solution. |
|
I figured a new comment to address the other details... I'm not sure if you saw the two versions, one with critical sections (the last change), the other with a mutex.
The main point of critical sections was to use static initialization to avoid the race condition here (from That might be me over thinking things though. Maybe with a critical section to protect its initialization if the race condition was a worry? What I like about this is that responsibility for locking and unlocking the synchronization object remains entirely within But I'm a bit confused about:
I don't see how waiting helps us in Has the hardware accelerated calculation finished when, for example, I spent too much time thinking about this tonight; I'll have to try the 6637 version tomorrow night. |
|
Hi @gojimmypi , I tried the dev branch but got stuck subscribing to the event. That is, when the server on the ESP32s3 receives a message from my .net client to start sending the notifications. I suspect this is related to our problem though; recall unwinding showed up in the log for the server message as well. And the issue seems to be happening in hashing functions. But we've been looking at the client side because we see the whole transaction. Anyway, I attached the new log output. In short, while trying to secure the TLS session for my .NET client, Stepping through, it went into I did have a bit of trouble with not being able to set breakpoints where I wanted and getting missed which makes me think there is some optimization going on even though I have selected the debug optoin for that. |
|
Hi @PaulMartinsen thanks for all the details. I think we're making good progress,
I realize you've gone pretty far down the road of custom TCP payloads over TLS1.3, but I keep thinking about the benefits of using something like wolfMQTT :)
Just for reference: In the absence of your source code I am using the same codebase for both ESP32 based client and server.
Yes, my Tigard is considerably more robust on the standard ESP32 as compared to the ESP32-S3 with the on-board JTAG. Frequently there are times that the S3 JTAG becomes unresponsive. It seems to be related to the running code. In particular some sort of rapid panic reboot causes subsequent JTAG flashing to be problematic. I have a wolfssl-examples/ESP32-hello-world expressly for this situation.
Oh yes, absolutely! Once we have asynchronous calls to the HW hash, combining DMA would definitely provide a noticeable performance improvement. Even just the DMA option alone would have benefit.
Yes, you are correct. Recall this is what we found in your prior log with the mutex With my recent changes to In your latest log, I'm happy to see that at least the nasty assert no longer occurs: This is exactly why we need a proper FreeRTOS mutex to manage the HW between tasks once we start using the SHA interleave capabilities. It's really a shame that the current implementation grabs the first TLS hash (when multiple hash algorithms are enabled). For example the int HashRaw() that I mentioned above that may do this with all the SHA types enabled: Without interleaving, only the first and simplest
You are correct. There's no benefit. Today However, once we have hash interleaving (perhaps even asynchronously), we'll have multiple different tasks using the HW concurrently on different hashes. Even is there's an Bottom line is that I'd like to ensure the ESP32 HW hash library is fully thread safe. To do this, we need FreeRTOS mutexes and the ability to wait for the resource to be freed.
So at this point, I think we are only dealing with TLS Session Resumption. I've been reviewing the comments in TLS1.3 Session Resumption #6172 this morning. Can you post some sample code of how you are doing session resumption? More specifically: are you doing this in your connection task, or your data exchange task? My thought at this time is that your Do you have any Thanks again for all your input. Cheers. |
|
A couple of things I can respond to quickly before I start the rest of the day; more later... I'm pretty confident that assert, & subsequent crash, happen because This arises because This also means that the server code doesn't crash because the HW lock unroll is happening on a single thread. I'm 80% confident the only wolf library functions I'm calling on the notify thread are:
There's possibly a dozen that are called on the connection thread to set everything up; I'll have to work through the code to gather that list.
My session resumption setup currently is: A later version replaces that with a I'll get a list of functions I'm calling from the connect thread and address other questions a bit later. |
|
continuing on... Wish I could use mqtt; unfortunately the standard we are following calls for challenging, memory hungry, complexity. Thread safety
The
I do have an in use flag, its even called Here BTW, while trying to learn about lwip timeouts I found Espressif sockets are "thread-safe", so that's nice. InterleavingI'm still a bit confused about the future interleaving, so working it through so see what I missed... It seems to me there are two 'in use' flags that are important here:
The first one indicates whether the hardware is occupied with a calculation and, if I understood correctly, the second one indicates whether there is (or soon will be) a calculation result in the hardware sha registers that has not been moved into application memory (i.e., the
It seems to me that this difference in the behaviour of the Update function leads to the problem in So I'm struggling a bit to see how waiting on a mutex that needs to be released in the same task that took it will be helpful since the The big advantage of having If the CPU calculation takes much longer than the HW one though, I wonder if it could be faster to simply wait for the accelerator to finish and store the intermediate result in application memory? Maybe with a token to detect sequential hash updates to save re-initializing the accelerator unnecessarily? If we can wait in a task friendly way, then other tasks could do stuff while the calculation is finishing up too, which is another way to keep the processors busy. I'm not sure if this means the behaviour of Do we need to park this bug until the strategy for interleaving is clearer? That stone might kill this bird too. |
Yes, I think we agree: This is a register than only means "the SHA calculation has not completed and is not yet available to read". It is not at all a hardware-sharing semaphore.
We're not today waiting on a mutex. However in the future with interleaved HW calculations, we would likely want to wait a bit. Today, if the HW is not done (either a single digest, or a stream of multiple digests without If we were interleaving, we might have two different SHA hashes in two different tasks. Alternatively, in our TLS connection today, we are (somewhat) concurrently digesting both SHA256 and SHA512 hashes in the same thread. If the SHA256 is started first, it will use HW. When the SHA512 digest then starts, it falls back to SW. If we had the interleaving capabilities, we could have both the SHA256 and SHA512 hashes to be incrementally digested both using hardware. The wait lock would only be useful if a separate task also wanted to do interleaving hash digests. Today we only have one HW hash computed at a time. We don't allow a separate task to compute a parallel HW hash, as we don't yet support interleaving. Once the HW is locked, it stays locks (with the mutex) until it is completely finished.
I believe this is essentially the interleaving feature, no?
Yes, I do believe you are correct & that is what we are seeing. I've convinced myself of that earlier today. I've applied some changes to my dev branch that now demonstrate that problem. I have a listening TLS server and a special 6637 client. When setting I'm currently looking into possible solutions. Again, having a Thanks very much for all your help and insight. I think we are very close to a good solution. |
|
I've made a but of progress on our multi-thread lock/release problem. I have an update to my wolfssl_client_6637. See the sample_output_stray_lock_recovery.txt log file. In particular:
The code still needs a bit of TLC and cleanup (thread-safety issues), but is basically operational. There are other ways that could accomplish this same goal. For this iteration, this simple implementation should suffice. I suspect when doing interleaved hashes we may need different logic. If you get a chance, please give this initial working stray HW lock recovery commit a try with your separate Cheers |
|
Hi @gojimmypi , thanks very much for all your work on this. Sounds very promising. I'll try it out tomorrow night. |
|
@gojimmypi , I tested the lasted commit (from 7 August, 12:55 p.m., "initial working stray HW lock recovery") in the With this version I get an endless reboot cycle. Decoding the back trace shows it is failing in line 740 of sha.c This occurs when the ESP32s3 server initializes the Actually, looks like it is inside Though I can't see where that pointer would be set. Maybe I wasn't supposed to Setting |
|
Hi @PaulMartinsen and thankyou for taking a look at the interim progress. Ah yes, if you are only using the library and not my sample code, that makes sense. I should have added more details. I have a concurrent tls_peek task. Its job it to attempt to trip up the locking mechanism. It is (was) expected to be running and initializing that stray. The peek task can be turned off by setting run_peek_task = 0. My peek task initializes that stray
Hm, Maybe. But I must have missed something for your setup. In not having your code, I emulated your connection and data tasks. This option is turned on by setting test_separate_tasks = 1 in my example. I've updated the code to turn off the peek task and enabled my test for multiple ctx tasks. I've also applied a change to check for nulls during testing. As seen in the latest sample output, I successfully used two separate tasks to connect and send data. It still needs work, as after the initial exchange is successful, I see a bunch of In any case, the point of interest is the successful release of unfinished lock.
Let's go ahead and leave |
|
Ok. I updated to the latest version. And have Uploaded.... oh. Still the same problem with I put I still get the bad ticket problem without I think something must have gotten messed up at my end. I'll investigate further in a couple of days; going to be away. |
|
Hello @PaulMartinsen and thank you again for your continued cooperation on this interesting issue. Apologies for the delay. I've been busily preparing and practicing for my ESP32 webinar that was presented yesterday. I'm now back to this topic. For other readers just catching up: we're working on something not officially supported but worth investigating: using a single wolfSSL connection in multiple freeRTOS tasks. Specifically one task for managing connections, the other for exchanging data. Although this is not fully thread safe, I do believe this is possible, given the right user code implementation. It's also helped to uncover some corner cases to cleanup for the SHA accelerator as related to ongoing and later abandoned hash calculations.
I'm assuming this is still a session resumption problem as mentioned above. Please advise if you can never connect. Perhaps I'm still missing something from your setup. I'm currently working from the client-tls13-resume. example and seeing if I can replicate the problem you are still encountering without actually having all of your code. Thanks for supplying your resumption logic, above. |
|
I was travelling and wasn't able to join the webinar unfortunately, but a colleague did and found the seminar very helpful @gojimmypi . I've been doing a bit of work to quantify the benefit of this non officially supported ability to create the secure connection in one task while another handles sending data. In my test, I'm sending a message (about 1kb) over a plain TCP connection (no TLS), with a single number that increments every 50 ms. While that's going on, I start another connection that requires a ~1 second TLS handshake from the ESP32s3 sever to accept a command that starts the connection then a second ~1 second TLS handshake from a ESP32s3 client to establish a secure connection for sending a different message. When the connection and sending happens in a single task, I see the expected 1 second interruption to messaging. When TLS handshaking happen in different tasks, I can sustain the 50ms transmission rate. Preliminary data below. Top plot is the difference in transmission time-stamp on the device (i.e., should be 50 ms when the ESP32s3 isn't too busy to send messages). The bottom plot is difference between counter value received (i.e., 1 when no messages are lost). One datapoint for each message received by the client. The second connection, with TLS handshake, starts at about message 50. Only preliminary data, but here only a single one of the 50ms period messages was lost, and otherwise the ESP32s3 kept sending a message roughly every 50ms while it negotiated the second secure connection. At 50ms per message here, that negotiation would have taken about 20 messages.
This was keeping cross-thread locks inside I'm going to try to get to the bottom of the bad ticket over the weekend.
I can never connect from the .net client to the ESP32s3 server with the test version. The ESP32s3 server is configured to support session resumption, but as far as I know the .net client doesn't use it (TLSv1.3 session resumption certainly isn't used when the ESP32s3 as a client is connecting to the .NET server, but I haven't checked the other direction). |
|
Here's a breakdown of this afternoon's investigation. Test 1. Test latest version from ED25519_SHA2_fix/6cb7eeef. Test 2. Check locking Test 3. Try without This got around the bad ticket problem so I can connect to the ESP32s3 server and the ESP32s3 can create client connections to the .NET program. After checking connections worked (this is important in a second), I captured log messages while subscribing to, then sending event messages. No crashing this time when the Test 4. a fresh start About 45 minutes later, with the ESP32s3 just blinking its little light, I made another query-response connection to the ESP32s3 server. In |
|
Hi @gojimmypi , I was thinking about the missing unlock and had a go at mocking up a concept that could resolve both the missing unlock thread-safely, resolve the cross-thread crash and support overlapping updates . To keep things easy (for me), i just made a Windows console app to illustrate the idea with mocked hardware & standard C++ in visual studio. The accelerator driver (similar to what Is it worth doing another teams session to discuss the way forward? |
A recording of my Getting Started with wolfSSL on the ESP32 webinar is now available.
❤️ thank you
That's an interesting comment. Are you not using wolfSSL on the .Net side? There's a C# Wrapper complete with Client and Server. I've not personally used these yet, but this might be an excellent opportunity to take them for a test drive, I found dotnet/runtime#84136 that references a Microsoft Blog. It states:
So it sounds like if you are using
It would definitely be good if we were working on the exact same code to resolve this. I've been trying to reproduce the problem you are without using your code. I'll take a look at your accelerator-sharing example, but what I'd really like to see is the real multi-task TLS connection failure.
In theory, my idea is that when
Yes, there needs to be some mechanism to indicate an in-progress hash has been abandoned without One of the first things I thought of when I saw your example code is the magical process of garbage collection in C++. I wonder how long before the finalize / free code is called?
Yes. Let me do a little more homework and we can schedule something. |
|
Just time for a couple of comments before rest of day starts... My .net client only fails to connect with the proposed changes. Normally it connects fine. I'm using .net 4.8.2 for historical reasons. Some of microsoft documentation on tlsv1.3 resumption didn't match my tests. I eneded up using a wolfssl based server to check resumption worked for my client My concept avoids the problem of abandoned hashes locking the hardware for good, or needing something else to free the lock in a specific thread, by keeping a hash state in the hash context which tracks where that hashes result is and keeping locks within the accelerator driver. I think you already have something similar to track initialization, which is where I got the idea. C++ cleanup isn't asynchronous like garbage collection in c# . Destructor get called when stack variables go out of scope. More like the 'using' keyword in c#. I think I only take advantage of that in the concept to make sure the mutex locks are released when the function exits. In otherwords, the cleanup happens as the stack gets unwound. |
|
Hi @gojimmypi , an update on interleaving to fix this issue... I've had a first go at a C implementation of the interleaving concept for the sha accelerator on, with a VisualGDB demo/test program for the ESP32s3. Currently only the Sha1 hash is implemented so definitely a work in progress with the following still to do:
I spent a few hours trying to figure out the I found, in v5.0, Espressif have started a hardware abstraction layer. It lives in I also found a In the prototype,
I'll keep working on the above to dos over the next week or so. But I was wondering what issues you see with this approach? |
Contact Details
No response
Version
a026d84 (master branch ca 8:33 am, 15/7/2023)
Description
Calling
wolfSSL_freeto dispose of a TLS session from a 'cleanup' thread leads to an assertion failure inxQueueGenericSendinesp-idf/v5.0/components/freertos/FreeRTOS-Kernel/queue.c. This seems to occur only whenwolfSSL_freeis called from a different task to the one that created it. In both case all communication is finished and it is time to free the memory. AndwolfSSL_freeis only being called once for the session.Stack trace (below), shows this occurs when
esp_sha_hw_unlockcallsesp_CryptHwMutexUnLock(&sha_mutex). The same application code operates correctly running on Windows (however, that's not using the Espressif port).Currently Windows decided it can't find the ESP32s3 JTAG interface, so I haven't been able to look at what's happening in any more detail. Will add new info as it comes to hand.
I'm using this version which is a clean fork from a recent master branch with only trivial changes for building on Windows. @gojimmypi , do you think your new PR might make a difference?
Reproduction steps
No response
Relevant log output
The text was updated successfully, but these errors were encountered: