Skip to content

Latest commit

 

History

History
126 lines (96 loc) · 4.68 KB

index.rst

File metadata and controls

126 lines (96 loc) · 4.68 KB
Raw

buildtest Documentation

Welcome to buildtest documentation. buildtest is a HPC Application Testing Framework designed to build tests quickly verify an entire HPC software stack that operates in a module environment.

This documentation was last rebuild on |today| and is intended for version |version|.

This project is in development and lead by Shahzeb Siddiqui

Contents:

.. toctree::
   :glob:
   :maxdepth: 2

   Setup
   Features_of_buildtest
   Architecture
   Writing_Test_In_YAML
   How_to_use_BuildTest

   Future_Work

Description

buildtest is an Automatic Test Generating Framework for writing test cases efficiently and quickly for scientific applications. buildtest will generate test scripts for any app/version automatically and tests can be recreated as many times. buildtest makes use of EasyBuild easyconfig files to determine which module + toolchain to use. You will need module environment EnvironmentModules or Lmod on your system to use this framework

Motivation

An HPC environment has hundreds of application that running constantly, the HPC engineers would build these apps and put them on production system. Some of the application have vendor provided scripts such as make test or ctest that can test the software after make step. Unfortunately, these methods perform tests in the build directory and does not perform a Post Install Test. Changing the vendor test script to the install paths can be tedious and this would have to be done manually.

Writing test scripts manually can be tedious, also there is no sharing of code and most likely they are not compatible to work at other HPC sites. Easybuild takes a step at improving application build process by automating the entire software workflow that can be built on any HPC site .

buildtest takes a similar approach as EasyBuild but focusing on application testing.

Objectives

buildtest will focus on the following key elements

  1. Provide a test toolkit with test examples for all HPC apps
  2. Provide quick means for doing functional testing
  3. Generate tests to be run with job scheduler to verify apps on all Nodes
  4. Provide performance metrics for tests scripts
  5. Integrate application benchmarks to buildtest for stress testing

What is buildtest?

buildtest is a python script that can generate self-contained testscripts in shellscript (.sh). The test scripts can be run independently but they are designed to work in CMake CTest Framework.

buildtest can do the following:

  • Creates test for binary testing, scripting tests, and compilation tests
  • Verify modulefile can be loaded.
  • generate tests for system packages
  • List software packages provided by MODULEPATH
  • List available toolchains
  • List software packages by versions
  • Support for logging
  • Search for YAML and test scripts
  • build tests easily for scripting languages (R, Python, Perl, Ruby, Tcl)
  • Run tests through an interactive menu
  • Scan tests and report which ones can be built with buildtest

Useful Links

Indices and tables