Building TensorFlow

Building TensorFlow

The instructions provided below specify the steps to build TensorFlow version 2.16.1 on Linux on IBM Z for the following distributions:

• Ubuntu (20.04, 22.04, 24.04)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install TensorFlow v2.16.1

1.1) Build using script

If you want to build TensorFlow using manual steps, go to STEP 1.2.

Use the following commands to build TensorFlow using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Tensorflow/2.16.1/build_tensorflow.sh

# Build Tensorflow
bash build_tensorflow.sh [Provide -t option for executing build with tests, -p option for choosing the Python version from {3.9, 3.10, 3.11, 3.12}, if not specified, the script will use the default Python version as 3.11]

If the build completes successfully, go to STEP 2. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install the dependencies

export SOURCE_ROOT=/<source_root>/
export NUMPY_VERSION="1.23.5"
export SCIPY_VERSION="1.13.0"

• Only when choosing Python version 3.9

  PYTHON_V=3.9
  PY_VERSION=3.9.7

• Only when choosing Python version 3.10

  PYTHON_V=3.10
  PY_VERSION=3.10.6

• Only when choosing Python version 3.11

  PYTHON_V=3.11
  PY_VERSION=3.11.4
  

• Only when choosing Python version 3.12

  PYTHON_V=3.12
  PY_VERSION=3.12.0
  NUMPY_VERSION=1.26.0

• Ubuntu 20.04

  sudo apt-get update
  sudo apt-get install -y wget git unzip zip openjdk-11-jdk pkg-config libhdf5-dev libssl-dev libblas-dev liblapack-dev gfortran curl patchelf gcc-10 g++-10 libopenblas-dev libatlas-base-dev wget
  sudo ldconfig

• Ubuntu (22.04, 24.04)

  sudo apt-get update
  sudo apt-get install -y wget git unzip zip openjdk-11-jdk pkg-config libhdf5-dev libssl-dev libblas-dev liblapack-dev gfortran curl patchelf gcc g++ libopenblas-dev libatlas-base-dev wget
  sudo ldconfig   

• Build Bazel

  cd $SOURCE_ROOT
  wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/6.4.0/build_bazel.sh
  sed -i 's/6.4.0/6.5.0/g' build_bazel.sh
  sed -i 's#Bazel/${PACKAGE_VERSION}/patch#Bazel/6.4.0/patch#g' build_bazel.sh
  sed -i 's/apt-get install/DEBIAN_FRONTEND=noninteractive apt-get install/g' build_bazel.sh
  sed -i 's/23.10/24.04/g' build_bazel.sh
  bash build_bazel.sh -y
  sudo cp $SOURCE_ROOT/bazel/output/bazel /usr/local/bin/bazel	

• Setup Python

  cd $SOURCE_ROOT
  if [[ ! -z "$PYTHON_V" ]]; then
  	PYTHON_V=3.11
  	PY_VERSION=3.11.4
  fi
  wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Python3/$PY_VERSION/build_python3.sh
  sed -i 's/apt-get install/DEBIAN_FRONTEND=noninteractive apt-get install/g' build_python3.sh
  sed -i 's/ubuntu-20.04/ubuntu-22.04/g' build_python3.sh     # For Ubuntu 22.04 with Python version 3.9
  sed -i 's/ubuntu-20.04/ubuntu-24.04/g' build_python3.sh	# For Ubuntu 24.04 only	   
  bash build_python3.sh -y
  sudo update-alternatives --install /usr/local/bin/python python /usr/local/bin/python3 40  
  sudo update-alternatives --install /usr/local/bin/pip3 pip3 /usr/local/bin/pip${PYTHON_V} 50

• Install additional python packages

  sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 60  # Only for Ubuntu 20.04
  sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 60  # Only for Ubuntu 20.04
  sudo pip3 install --no-cache-dir numpy==$NUMPY_VERSION wheel packaging requests opt_einsum portpicker protobuf scipy==$SCIPY_VERSION psutil setuptools==68.2.2
  sudo pip3 install h5py==3.11.0  # Only for Ubuntu 20.04

• Build ICU data in big-endian format

  cd $SOURCE_ROOT
  export ICU_MAJOR_VERSION="69"
  export ICU_RELEASE="release-${ICU_MAJOR_VERSION}-1"
  git clone --depth 1 --single-branch --branch "$ICU_RELEASE" https://github.com/unicode-org/icu.git
  cd icu/icu4c/source/
  # create ./filters.json
  cat << 'EOF' > filters.json
  {
    "localeFilter": {
      "filterType": "language",
      "includelist": [
        "en"
      ]
    }
  }
  EOF
  ICU_DATA_FILTER_FILE=filters.json ./runConfigureICU Linux
  make clean && make
  # Workaround makefile issue where not all of the resource files may have been processed
  find data/out/build/ -name '*pool.res' -print0 | xargs -0 touch
  make
  cd data/out/tmp
  LD_LIBRARY_PATH=../../../lib ../../../bin/genccode "icudt${ICU_MAJOR_VERSION}b.dat"
  echo "U_CAPI const void * U_EXPORT2 uprv_getICUData_conversion() { return icudt${ICU_MAJOR_VERSION}b_dat.bytes; }" >> "icudt${ICU_MAJOR_VERSION}b_dat.c"
  cp icudt${ICU_MAJOR_VERSION}b_dat.c icu_conversion_data_big_endian.c
  gzip icu_conversion_data_big_endian.c
  split -a 3 -b 100000 icu_conversion_data_big_endian.c.gz icu_conversion_data_big_endian.c.gz.

• Install grpcio

  cd $SOURCE_ROOT
  export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=True
  sudo -E pip3 install grpcio

1.3) Build TensorFlow

• Download source code

  cd $SOURCE_ROOT
  git clone https://github.com/tensorflow/tensorflow
  cd tensorflow
  git checkout v2.16.1

• Apply patches

  rm -rf third_party/tf_runtime/BUILD
  curl -o tf_v2.16.1.patch https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Tensorflow/2.16.1/patch/tf_v2.16.1.patch
  patch -p1 < tf_v2.16.1.patch
  rm -f tf_v2.16.1.patch  

• Copy the big-endian version of ICU data

  cp ${SOURCE_ROOT}/icu/icu4c/source/data/out/tmp/icu_conversion_data_big_endian.c.gz.* third_party/icu/data/

• Configure

  export TF_NEED_CLANG=0
  export TF_NEED_OPENCL_SYCL=0
  export TF_NEED_CUDA=0
  export TF_NEED_MKL=0
  export TF_PYTHON_VERSION=$PYTHON_V
  export PYTHON_VERSION=$PY_VERSION
  yes "" | ./configure || true

• Build TensorFlow

   TF_SYSTEM_LIBS=boringssl bazel build --define tflite_with_xnnpack=false //tensorflow/tools/pip_package:build_pip_package

  Note:
  1. TensorFlow build is resource intensive operation. If build continues to fail try increasing the swap space and reduce the number of concurrent jobs by specifying --jobs=n in the build command above, where n is the number of concurrent jobs.

  2. Building TensorFlow from source can use a lot of RAM. If your system is memory-constrained, limit Bazel's RAM usage with: --local_ram_resources=2048.

1.4) Build tensorflow_io_gcs_filesystem wheel

cd $SOURCE_ROOT
git clone https://github.com/tensorflow/io.git
cd io/
git checkout v0.29.0
curl -o tf_io.patch https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Tensorflow/2.16.1/patch/tf_io.patch
patch -p1 < tf_io.patch
rm -rf tf_io.patch
python3 setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem          
cd dist
sudo pip3 install ./tensorflow_io_gcs_filesystem-0.29.0-cp*-cp*-linux_s390x.whl

1.5) Build and Install TensorFlow wheel

cd $SOURCE_ROOT/tensorflow
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_wheel
sudo pip3 install /tmp/tensorflow_wheel/tensorflow-2.16.1-cp*-linux_s390x.whl

Step 2: Verify TensorFlow (Optional)

• Run TensorFlow from command Line and check the installed version

   $ cd $SOURCE_ROOT
   $ python -c "import tensorflow as tf; print(tf.__version__)"
     2.16.1
  
   $ python
    >>> import tensorflow as tf
    >>> tf.add(1, 2).numpy()
    3
    >>> hello = tf.constant('Hello, TensorFlow!')
    >>> hello.numpy()
    b'Hello, TensorFlow!'
    >>>

Step 3: Execute Test Suite (Optional)

• Run complete testsuite

  cd $SOURCE_ROOT/tensorflow  
  TF_SYSTEM_LIBS=boringssl bazel test --define tflite_with_xnnpack=false --test_size_filters=small,medium --build_tests_only --keep_going --test_output=errors --verbose_failures=true --local_test_jobs=HOST_CPUS --test_env=LD_LIBRARY_PATH --test_tag_filters=-no_oss,-oss_excluded,-oss_serial,-gpu,-tpu,-benchmark-test,-v1only --build_tag_filters=-no_oss,-oss_excluded,-oss_serial,-gpu,-tpu,-benchmark-test,-v1only --test_lang_filters=cc,py -- //tensorflow/... -//tensorflow/compiler/... -//tensorflow/core/tpu/... -//tensorflow/lite/... -//tensorflow/tools/toolchains/... 

• Run individual test

  TF_SYSTEM_LIBS=boringssl bazel test --define tflite_with_xnnpack=false //tensorflow/<module_name>:<testcase_name>

  For example,

  TF_SYSTEM_LIBS=boringssl bazel test --define tflite_with_xnnpack=false //tensorflow/core/kernels:shape_ops_test

  Note:
  

  1. Below test cases will fail as it uses test data generated in LE format.

   //tensorflow/cc/saved_model:fingerprinting_chunked_test 
   //tensorflow/cc/saved_model:fingerprinting_utils_test 
   //tensorflow/tools/proto_splitter:merge_test   
   //tensorflow/tools/proto_splitter/cc:util_test
   //tensorflow/python/kernel_tests/random:stateless_random_ops_test_cpu 
  

Step 4: Model Training Verification (Optional)

Follow the instructions provided here to verify TensorFlow v2.16.1.

References:

• https://www.tensorflow.org/
• https://github.com/tensorflow/tensorflow
• http://bazel.io/