Memory never freed

[martinfri]

Hi!

Im currently see some issues regarding service calls and memory never gets freed.
Im running a service which will run the async service routine:
{'op': 'call_service', 'id': 'call_service:/ns/my_node/list_parameters:8836', 'service': '/ns/my_node/list_parameters', 'type': 'rcl_interfaces/srv/ListParameters', 'args': {'prefixes': [], 'depth': 0}}
I think its connected with the snippet below.
Were does this buffer get freed, which task is responsible of freeing this memory?

std::shared_ptr<void> allocate_message(const MessageMembers * members)
{
  uint8_t * buf = static_cast<uint8_t *>(malloc(members->size_of_ + MSG_MEM_BLOCK_SIZE));
  memset(buf, 0, MSG_MEM_BLOCK_SIZE);

  allocate_string_members(members, buf);

  return std::shared_ptr<void>(buf);
}

[v-kiniv]

Quick answer: it's freed by the garbage collector when no one holds a reference to the shared pointer. But I may be missing something, need to double check.
Are you seeing memory leaks every time the service gets called? How are you checking for memory leaks? What amount of memory increased with each call?

[martinfri]

Hmm is that really true, the shared pointer is freed, but the buffer allocated need to call free aswell i think?
Htop and a simple python script

import asyncio
import websockets
import json

# WebSocket server URL (replace with your server URL)
WEBSOCKET_SERVER_URL = "ws://localhost:9090"

# Request template
def create_request():
    return {
        "op": "call_service",
        "id": "call_service:/ns/my_service/list_parameters:8836",
        "service": "/ns/my_service/list_parameters",
        "type": "rcl_interfaces/srv/ListParameters",
        "args": {
            "prefixes": [],
            "depth": 0
        }
    }

async def send_requests():
    async with websockets.connect(WEBSOCKET_SERVER_URL) as websocket:
        print("Connected to the WebSocket server.")

        while True:
            try:
                # Create and send a request
                request = create_request()
                await websocket.send(json.dumps(request))
                print(f"Sent request: {request}")

                # Wait for a response
                response = await websocket.recv()
                response_data = json.loads(response)
                print(f"Received response: {response_data}")
                response = await websocket.recv()
                response_data = json.loads(response)
                print(f"Received response: {response_data}")
            except Exception as e:
                print(f"An error occurred: {e}")
                break

if __name__ == "__main__":
    asyncio.run(send_requests())

[v-kiniv]

https://en.cppreference.com/w/cpp/memory/shared_ptr

std::shared_ptr is a smart pointer that retains shared ownership of an object through a pointer. Several shared_ptr objects may own the same object. The object is destroyed and its memory deallocated when either of the following happens:

Hmm is that really true

I'm not saying that it's definitely what's happening, that's just an answer to

which task is responsible of freeing this memory?

In reality it may need custom deleter to free up buffer properly.

[martinfri]

I'm not saying that it's definitely what's happening, that's just an answer to

Sorry for sounding harsh it was meant as a discussion!

In reality it may need custom deleter to free up buffer properly.

I think so unfortunately:
https://stackoverflow.com/questions/12264701/shared-ptr-with-malloc-and-free

[v-kiniv]

Sorry for sounding harsh it was meant as a discussion!

Nah, it's ok, I'm at the end of the long day here and just wanted to quickly reply to your initial question, but don't have time right now to check the issue properly. Will try to do it tomorrow.

[v-kiniv]

The culprit was not shared_ptr vs malloc, but these allocations. Yes, proper way to create shared_ptr from malloc'ed memory is to pass free as a second argument, but as it turns out, compiler does it implicitly. I verified it by passing to it a pointer on stack(without free as the second argument) and the process crashes as it tries to free memory on stack.

The method I used to allocate the buffer for a message was hacky anyway, and I found a proper way to do it. So the PR #47 should also fix issue #27.

I tested it with my usual setup with nav2 but there aren't a lot of service calls, especially with strings and string arrays. If possible, please test the PR for such cases.

And thanks for providing python script, it was handy to test for leaks.

[martinfri]

Alright great, The compiler is getting smarter! But yes this solution looks much more robust, I guess that the compiler couldn't solve to delete on nested new's then.

I will try it out during the day!