Build without UVM

Making the use of UVM in Trilinos optional is still very much work in progress.

Here are some notes on how to test out the current status. This seems to work on Lassen.

1. Lots of changes are collected in the tpetraCrsRefactor branch https://github.com/trilinos/Trilinos/tree/tpetraCrsRefactor
2. Configure with UVM disabled:

   -D Kokkos_ENABLE_CUDA_UVM:BOOL=OFF                                                                                                                                                                                                                                                                                               
   -D Tpetra_ENABLE_CUDA_UVM:BOOL=OFF
   

   You should see a warning about disabling UVM.
3. Link in NVIDIA profiling tools

   -D Trilinos_EXTRA_LINK_FLAGS=-lnvToolsExt
   

4. Build the nvprof-connector from the Kokkos profiling tools https://github.com/kokkos/kokkos-tools/tree/master/profiling/nvprof-connector

   export KOKKOS_PROFILE_LIBRARY=$HOME/kokkos-tools/profiling/nvprof-connector/kp_nvprof_connector.so
   

5. Run as nsys profile --trace=cuda,mpi --mpi-impl=openmpi -o %q{OMPI_COMM_WORLD_RANK} -f true ./YOUR_DRIVER.exe