Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fast-wrapper #1032

Open
tomuben opened this issue Dec 5, 2024 · 3 comments
Open

Add fast-wrapper #1032

tomuben opened this issue Dec 5, 2024 · 3 comments
Labels
feature Product feature

Comments

@tomuben
Copy link
Collaborator

tomuben commented Dec 5, 2024

Background

It turns out we need to manipulate the "LD_LIBRARY_PATH" of the UDF client during the runtime. More specifically, we need to extend this environment variable coming from the database with specific values of the Script-Languages-Container, i.e. the path to the conda cuda-compat libraries.
As this is not possible in the UDF client itself (subsequent loading of shared libraries ignores the newly set environment variable), we need to implement a fast wrapper, similar to the slow wrapper bash scripts, but in C++.

Acceptance Criteria

  1. Implement a thin wrapper which get 's an aribrary list of environment variables to change from preprocessor variables (or maybe some form of template file). The wrapper should then extend the environment variables by prepending the values (using ::getenv() and ::setenv())
  2. The thin wrapper should call the real exaudfclient binary then with the passed arguments, via ::execve().
  3. Extend Bazel configs to pass the environment variables easily from the Dockerfile of the build_run step.
  4. Extend Bazel BUILD file to build the new fast wrapper and pass the new environment variable and pass the info about the real exaudfclient path
  5. Use this new config in the standard-Exasol-8-cuda-ml flavor

Alternative

Manipulate /etc/ld.cache.so. See comment below

@tomuben
Copy link
Collaborator Author

tomuben commented Jan 7, 2025

Maybe we don't need a wrapper, but can manipulate the shared library path otherwise: see https://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html?m=1

Additionally, we need an integration test which verifies the correct order in /etc/ld.cache.so

@tkilias
Copy link
Collaborator

tkilias commented Feb 12, 2025

/etc/ld.cache.so probably doesn't work, because the directories are empty. However, a rpath or runpath could work.

AI says:

Yes, you can set the RUNPATH for a binary to point to a directory. The RUNPATH is an attribute used by the dynamic linker to specify a list of directories to search for shared libraries at runtime. This is different from RPATH, which is encoded into the binary during linking and cannot be overridden at runtime.

To set the RUNPATH to a directory, you can use the chrpath or patchelf tools. Here’s an example using patchelf:

patchelf --set-rpath /path/to/your/directory your_binary

Or, you can also use the -Wl,-rpath linker flag while compiling, for example:

gcc -o your_binary your_source.c -Wl,-rpath,/path/to/your/directory

Using RUNPATH is generally preferred over RPATH because it allows you to override the search path through the LD_LIBRARY_PATH environment variable at runtime, providing more flexibility.

If you have any issues or need further assistance, please feel free to ask!

@tkilias
Copy link
Collaborator

tkilias commented Feb 12, 2025

If we use the wrapper, we would need to add a test for %env LD_LIBRARY_PATH

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Product feature
Projects
None yet
Development

No branches or pull requests

2 participants