ENH: Don't add system library directories to rpath #1169

Job	Run time
test_cygwin (39, windows-latest)	4m 21s
test (3.8, ubuntu-latest)	38s
ci_setuptools (3.10, ubuntu-latest)	3m 54s
test (3.8, macos-latest)	1m 13s
collateral (diffcov)	45s
test (3.8, windows-latest)	2m 15s
collateral (docs)	26s
test_msvc_python_mingw	1m 43s
test (3.12, ubuntu-latest)	45s
test_msys2_mingw (mingw64, x86_64, gcc, g++)	1m 39s
test_msys2_mingw (mingw32, i686, gcc, g++)	1m 45s
test (3.12, macos-latest)	48s
test_msys2_mingw (ucrt64, ucrt-x86_64, gcc, g++)	1m 38s
test (3.12, windows-latest)	2m 42s
test (3.9, ubuntu-latest)	39s
test_msys2_mingw (clang64, clang-x86_64, clang, clang++)	1m 47s
test (3.10, ubuntu-latest)	36s
test (3.11, ubuntu-latest)	38s
test (pypy3.10, ubuntu-latest)	1m 32s
check	3s
release	0s
	29m 47s

Provide feedback