Skip to content

Latest commit

 

History

History
501 lines (338 loc) · 11 KB

development.md

File metadata and controls

501 lines (338 loc) · 11 KB

serve-grpc development

The following instructions are useful during development.

Note: This has been tested on Linux and Darwin/macOS. It has not been tested on Windows.

Prerequisites for development

🤔 The following tasks need to be complete before proceeding. These are "one-time tasks" which may already have been completed.

  1. The following software programs need to be installed:
    1. git
    2. make
    3. docker
    4. go

Install Senzing C library

Since the Senzing library is a prerequisite, it must be installed first.

  1. Verify Senzing C shared objects, configuration, and SDK header files are installed.

    1. /opt/senzing/er/lib
    2. /opt/senzing/er/sdk/c
    3. /etc/opt/senzing

  2. If not installed, see How to Install Senzing for Go Development.

Install Git repository

  1. Identify git repository.

    export GIT_ACCOUNT=senzing-garage
export GIT_REPOSITORY=serve-grpc
export GIT_ACCOUNT_DIR=~/${GIT_ACCOUNT}.git
export GIT_REPOSITORY_DIR="${GIT_ACCOUNT_DIR}/${GIT_REPOSITORY}"

  2. Using the environment variables values just set, follow steps in clone-repository to install the Git repository.

Dependencies

  1. A one-time command to install dependencies needed for make targets. Example:

    cd ${GIT_REPOSITORY_DIR}
make dependencies-for-development

  2. Install dependencies needed for Go code. Example:

    cd ${GIT_REPOSITORY_DIR}
make dependencies

Lint

  1. Run linting. Example:

    cd ${GIT_REPOSITORY_DIR}
make lint

Build

  1. Build the binaries. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean build

  2. The binaries will be found in the ${GIT_REPOSITORY_DIR}/target directory. Example:

    tree ${GIT_REPOSITORY_DIR}/target

Run

  1. Run program. Examples:

    1. Linux

      1. ✏️ Identify a location for database. Example:

        export SENZING_TOOLS_DATABASE_FILE=/tmp/sqlite/G2C.db

      2. Copy template database and run command. Example:

        mkdir --parents ${SENZING_TOOLS_DATABASE_FILE%/*}
cp ${GIT_REPOSITORY_DIR}/testdata/sqlite/G2C.db ${SENZING_TOOLS_DATABASE_FILE}
export SENZING_TOOLS_DATABASE_URL=sqlite3://na:na@nowhere${SENZING_TOOLS_DATABASE_FILE}
${GIT_REPOSITORY_DIR}/target/linux-amd64/serve-grpc

    2. macOS

      ${GIT_REPOSITORY_DIR}/target/darwin-amd64/serve-grpc

    3. Windows

      ${GIT_REPOSITORY_DIR}/target/windows-amd64/serve-grpc

  2. Clean up. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean

Test

  1. Run tests. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean setup test

  2. Optional: View the SQLite database. Example:

    docker run \
    --env SQLITE_DATABASE=G2C.db \
    --interactive \
    --publish 9174:8080 \
    --rm \
    --tty \
    --volume /tmp/sqlite:/data \
    coleifer/sqlite-web

    Visit localhost:9174.

Coverage

Create a code coverage map.

  1. Run Go tests. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean setup coverage

    A web-browser will show the results of the coverage. The goal is to have over 80% coverage. Anything less needs to be reflected in testcoverage.yaml.

Documentation

  1. View documentation. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean documentation

  2. If a web page doesn't appear, visit localhost:6060.

  3. Senzing documentation will be in the "Third party" section. github.com > senzing-garage > serve-grpc

  4. When a versioned release is published with a v0.0.0 format tag, the reference can be found by clicking on the following badge at the top of the README.md page. Example:

    Go Reference Badge

  5. To stop the godoc server, run

    cd ${GIT_REPOSITORY_DIR}
make clean

Docker

  1. Use make target to run a docker images that builds RPM and DEB files. Example:

    cd ${GIT_REPOSITORY_DIR}
make docker-build

  2. Run docker container. Example:

    docker run \
  --interactive \
  --publish 8261:8261 \
  --read-only \
  --rm \
  --tty \
  senzing/serve-grpc

  3. Optional: Test using docker-compose. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean docker-test

    To bring the docker-compose formation, run

    cd ${GIT_REPOSITORY_DIR}
make clean

Package

Note: This only packages the serve-grpc command. It is only to be used in development and test. The actual packaging is done in the senzing-tools repository.

Package RPM and DEB files

  1. Use make target to run a docker images that builds RPM and DEB files. Example:

    cd ${GIT_REPOSITORY_DIR}
make package

  2. The results will be in the ${GIT_REPOSITORY_DIR}/target directory. Example:

    tree ${GIT_REPOSITORY_DIR}/target

Test DEB package on Ubuntu

  1. Determine if serve-grpc is installed. Example:

    apt list --installed | grep serve-grpc

  2. ✏️ Install serve-grpc. The serve-grpc-... filename will need modification. Example:

    cd ${GIT_REPOSITORY_DIR}/target
sudo apt install ./serve-grpc-0.0.0.deb

  3. ✏️ Identify a location for database. Example:

    export SENZING_TOOLS_DATABASE_FILE=/tmp/sqlite/G2C.db

  4. Copy template database and run command. Example:

    mkdir --parents ${SENZING_TOOLS_DATABASE_FILE%/*}
cp ${GIT_REPOSITORY_DIR}/testdata/sqlite/G2C.db ${SENZING_TOOLS_DATABASE_FILE}
export SENZING_TOOLS_DATABASE_URL=sqlite3://na:na@nowhere${SENZING_TOOLS_DATABASE_FILE}

  5. ✏️ Run command. Example:

    export LD_LIBRARY_PATH=/opt/senzing/er/lib/
serve-grpc

  6. Remove serve-grpc from system. Example:

    sudo apt-get remove serve-grpc

Test RPM package on Centos

  1. Determine if serve-grpc is installed. Example:

    yum list installed | grep serve-grpc

  2. ✏️ Install serve-grpc. The serve-grpc-... filename will need modification. Example:

    cd ${GIT_REPOSITORY_DIR}/target
sudo yum install ./serve-grpc-0.0.0.rpm

  3. ✏️ Identify database. One option is to bring up PostgreSql as see in Test using Docker-compose stack with PostgreSql database. Example:

    export SENZING_TOOLS_DATABASE_URL=sqlite3://na:na@nowhere/tmp/sqlite/G2C.db

  4. Run command. Example:

    export LD_LIBRARY_PATH=/opt/senzing/er/lib/
serve-grpc

  5. Remove serve-grpc from system. Example:

    sudo yum remove serve-grpc

Specialty

Test using Docker-compose stack with PostgreSql database

The following instructions show how to bring up a test stack to be used in testing the sz-sdk-go-core packages.

  1. Bring up the docker-compose stack. Example:

    cd ${GIT_REPOSITORY_DIR}
docker-compose up

  2. Optional: View the PostgreSQL database.

    Visit localhost:9171. For the initial login, review the instructions at the top of the web page. For server password information, see the POSTGRES_PASSWORD value in docker-compose.yaml. Usually, it's "postgres".

  3. Example programs using gRPC:

    1. senzing-garage/knowledge-base/proposals/quickstart-grpc

  4. Cleanup.

    cd ${SENZING_DEMO_DIR}
docker-compose down

cd ${GIT_REPOSITORY_DIR}
make clean

Test using bloomrpc

Using a (deprecated) BloomRPC client, test the Senzing gRPC Server. For other gRPC tools, visit Awesome gRPC.

  1. Install the bloomrpc gRPC test client.

    1. Example for Ubuntu.

      1. Find latest release.

      2. ✏️ Install. Example:

        wget https://github.com/bloomrpc/bloomrpc/releases/download/1.5.3/bloomrpc_1.5.3_amd64.deb
sudo apt install ./bloomrpc_1.5.3_amd64.deb

  2. Start the test server. Example:

    cd ${GIT_REPOSITORY_DIR}
make clean run-serve-grpc

  3. In a separate terminal, start the gRPC test client. Example:

    bloomrpc

  4. In bloomrpc:

    1. Near top-center, use the address of 0.0.0.0:8258 to reach the local gRPC server.
    2. In upper-left, click on plus sign ("+").
      1. Navigate to the ${GIT_REPOSITORY_DIR}/proto directory
      2. Choose one or more .proto files.
    3. In left-hand pane,

      1. Choose the Init message.

      2. Set the request values. Example:

        {
  "moduleName": "Test of gRPC",
  "iniParams": "{\"PIPELINE\":{\"CONFIGPATH\":\"/etc/opt/senzing\",\"RESOURCEPATH\":\"/opt/senzing/er/resources\",\"SUPPORTPATH\":\"/opt/senzing/data\"},\"SQL\":{\"CONNECTION\":\"sqlite3://na:na@nowhere/tmp/sqlite/G2C.db\"}}",
  "verboseLogging": 0
}

      3. Near the center, click the green "play" button.

    4. The Senzing object is initialized and other messages can be tried.

References