rgw: blanket apply NO_SSL_DL to civetweb sections

[ghost]

Since civetweb.c is compiled directly into the radosgw executable,
NO_SSL_DL needs to also be applied to the radosgw binary. For
completeness, also apply it to libcivetweb cxxflags.

Fixes: http://tracker.ceph.com/issues/16957

Signed-off-by: Karol Mroz [email protected]

[ghost]

Replaces: #10631

[ghost]

@cbodley - Targeting Jewel branch directly with this as master focuses on cmake.

[mattbenjamin]

@mdw-at-linuxbox I know you have work-in-progress that also addresses issues in SSL, please review this one as well? tx

[mdw-at-linuxbox]

src/civetweb.c should not be built directly as part of radosgw, it should only be built as part of civetweb_common_objs. So it's only necessary to define it there. This is the same place that USE_IPV6 should already be defined. There's no reason to define it globally for radosgw, nor for any c++ file (while civetweb does have c++ support, it's just a front end to the core C code, so no advantage to ceph to use it.)

So, besides your problem
1, NO_SSL_DL,
there are 4 further problems that need to be solved to make this useful.
2, "soname". By default, the code loads libssl.so libcrypto.so. Great for a devel machine, not so happy for a production environment (typically only files like "libssl.so.10 libcrypto.so.10" are available.) And yes people care about that -see http://tracker.ceph.com/issues/11239 .
3. "s" warning. So great, you got civetweb loaded, and you pass in "port=443s". And you see "error parsing int: 443s: The option value '443s' seems to be invalid. Yuck. At the very least, it's a pretty glaring cosmetic defect, but it can be a very attractive red herring if something else broke (bad cert, can't load libssl.so...). And that's bad. But.
4. there's a couple of less than obvious places where radosgw actually uses the port number, and that mis-parsed 443s may not do what you want. Two specific cases: s3 "v4" auth, swift "preauth" urls.
5. this results in loading openssl on top of an environment which might have nss already loaded. It would be better if civetweb used the same crytpo as the rest of ceph. Again, people complained, see http://tracker.ceph.com/issues/16535 . That one related to having libssl et al loaded in by ld.so, but using dlopen doesn't really solve the problem, just hides it from a casual use of "ldd".

There is also,
6. https and http in the same front end. http://tracker.ceph.com/issues/13718 . This matters because each front end gets its own thread pool, and the thread pool is a significant resource consumption point.

So you might want to take a look at my current attempt to solve this,
https://github.com/mdw-at-linuxbox/civetweb
wip-listen1
https://github.com/mdw-at-linuxbox/ceph
wip-rgw-openssl-4
(there is also a "wip-rgw-openssl-4" in github.com/ceph/ceph, which lacks the last commit.)
So this fixes, 1-4,6 in the above list.

Take a look and let me know what you think!

[mdw-at-linuxbox]

I've updated wip-rgw-openssl-4 in github.com/ceph/ceph with an even newer version.

[ghost]

Thanks for taking a look! Let me address your comments:

Regarding civetweb.c. From what I recall in Jewel with autotools, I initially was under the impression that civetweb was an .so linked into rgw and so passed the flag solely there. However the civetweb load_dll() symbol persisted in the rgw executable. From the logs it appeard civetweb.c was indeed getting compiled directly in the rgw executable and thus passing NO_SSL_DL globally was required. Perhaps this has changed since then?

1. The so naming convention differences between SUSE/RedHat are what prompted our original investigation in the early days of Jewel. We hoped to rely on the system to load the necessary ssl libs.
2. Yes I've seen this one also. Agree it's cosmetic, agree it's not great 😄

4/5. I've not hit this issue personally. But good to know, thanks.

I will have a look into the wip branches you mention when I get a moment. If civetweb.c is truly no longer being compiled directly into the RGW executable, then I agree this blanket apply of NO_SSL_DL would not be needed.

Cheers!

[mdw-at-linuxbox]

In ceph/cmake, civetweb.c.o is built as part of "civetweb_common_objs", which is a cmake "object library". That in turn gets linked into the radosgw binary with the string "$<TARGET_OBJECTS:civetweb_common_objs>". So in the final radosgw build, you will see "dlopen()". But, since the civetweb.c.o object file was built separately from the rest of the radosgw sources, it actually won't see any flags you set for building radosgw sources, it will only see the flags set for civetweb common objs.

You'll definitely also want to have a look at http://tracker.ceph.com/issues/16535 . The PR for that reverts a change that set NO_SSL_DL . Is the SUSE build of ceph using openssl elsewhere and not nss?

[mdw-at-linuxbox]

There's some work going on to update civetweb so I updated the ssl patches i had for that: see #11776 ceph/civetweb#15 .

[ghost]

Ancient, closing.