Revert "[ci] Allow parallel testing on NVIDIA GPUs" (#5496)

I'm suspicious that this is the proximate cause of crashes we're seeing
while destroying the Vulkan device. We think
#2659 and
#2900 are the root cause, but
reverting for now to try to get us back to green.

Example failure, starting at the PR being reverted:
https://source.cloud.google.com/results/invocations/a3093172-ae55-43bb-956c-256bb726006f/targets/iree%2Fgcp_ubuntu%2Fcmake-bazel%2Flinux%2Fx86-turing%2Fmain/log

Reverts #5467

build_tools/kokoro/gcp_ubuntu/cmake-bazel/linux/x86-turing/build.sh

@@ -69,7 +69,9 @@ echo "Building with Ninja"
 cd "${CMAKE_BUILD_DIR?}"
 ninja
 
-export CTEST_PARALLEL_LEVEL=${CTEST_PARALLEL_LEVEL:-$(nproc)}
+# Limit parallelism dramatically to avoid exhausting GPU memory
+# TODO(#5162): Handle this more robustly
+export CTEST_PARALLEL_LEVEL=${CTEST_PARALLEL_LEVEL:-1}
 
 # Only test drivers that use the GPU, since we run all tests on non-GPU machines
 # as well.