Vehicle Routing Meta Solvers added

.gitignore

@@ -1,10 +1,12 @@
 HELP.md
+*.DS_Store
 .gradle
 build/
 !gradle/wrapper/gradle-wrapper.jar
 !**/src/main/**/build/
 !**/src/test/**/build/
-*.DS_Store
+### Solution Files from Vehicle Routing Solver:
+*.sol
 
 ### STS ###
 .apt_generated
@@ -14,7 +16,8 @@ build/
 .settings
 .springBeans
 .sts4-cache
-bin/
+bin/main
+bin/test
 !**/src/main/**/bin/
 !**/src/test/**/bin/
 
@@ -38,6 +41,9 @@ out/
 .vscode/
 
 
+### virtual envs and python ###
+venv/
+__pycache__/
 
 ### ProvideQ toolbox-server ###
 # input, output and temporary files for solving problems with our solvers
@@ -49,5 +55,5 @@ gamslice.txt
 # Listing files compiled from our GAMS scripts
 *.lst
 *.op2
-gams/**/225a
+solvers/gams/**/225a
 *.out

Dockerfile

@@ -48,10 +48,7 @@ COPY --from=builder /app/build/jre /opt/java
 ENV PATH="${PATH}:/opt/java/bin"
 
 # Install the toolbox server and its solver scripts
-COPY gams gams
-COPY qiskit qiskit
-COPY cirq cirq
-COPY python python
+COPY solvers solvers
 RUN scripts/install-solver-dependencies.sh
 COPY --from=builder /app/build/libs/toolbox-server-*.jar toolbox-server.jar
 

README.md

@@ -7,28 +7,44 @@ A web-based user interface can be found in our
 [ProvideQ/ProvideQ repository](https://github.com/ProvideQ/ProvideQ).
 
 ## Development setup
-1. Install Java 17 (check with `java -version`)
+1. Install Java 17 or newer (check with `java -version`)
 2. Clone this repository
-3. Install a python env that works with GAMS (skip this step if you don't need GAMS)
-   1. Install GAMS.
-   2. Install miniconda (or anaconda, if you prefer that):
-      https://conda.io/projects/conda/en/stable/user-guide/install/index.html
-   3. Create a GAMS conda environment: `conda create --name gams python=3.10 --yes`
-   4. Activate your conda environment: `conda activate gams`.
-   5. Make GAMS use that python environment by setting the `GMSPYTHONLIB=<path-to-conda>/envs/gams/lib/libpython3.10.so`
-      environment variable.
-   6. Install GAMS packages to the GAMS conda env:
-      `pip install gams[core,connect] --find-links <path-to-gams>/api/python/bdist`
-      * If you get an error building `psycopg2`, try to install these postgres packages:
-        `sudo apt-get install postgresql libpq-dev` and run the `pip install ...` command again
-   7. Install the python dependencies we use in our python packages: `pip install -r gams/requirements.txt`
-4. Install solver dependencies (skip this step if you don't want to use the solvers):
-   * Note that these dependencies must be installed to the gams conda env if you want to use GAMS and other solvers from
-     the same toolbox installation!
-   1. Install GAMS solver dependencies: `pip install -r gams/requirements.txt`
-   2. Install Qiskit solver dependencies: `pip install -r qiskit/requirements.txt`
-   3. Install Cirq solver dependencies: `pip install -r cirq/requirements.txt`
-5. Run the server using `./gradlew bootRun`
+3. [Optional, Solver Installation - install the Solvers that you want/need]
+    * We hope to provide an automated script for most of this in the future.  
+   1. Python-based Solvers (Qiskit, Cirq, Dwave, Qrisp)
+      1. Install Python and the required libraries by using `pip install -r [path to requirements.txt]`
+      2. Example for Qiskit: `pip install -r solvers/qiskit/requirements.txt`
+      * Note: These dependencies must be installed to the gams conda env if you want to use GAMS and other solvers from
+        the same toolbox installation!
+   2. Compiled Solvers (e.g. used for VRP and TSP)
+      1. Solvers implemented in compiled languages must be executed via binaries that are compiled for your operating system. For those types of solvers we usually include pre-compiled binaries for windows, mac (only arm), and unix.
+      2. In case the pre-compiled versions do not work on your machine: re-compile them:
+      * LKH-3:
+        1. Build LKH-3 using the offical guide: http://webhotel4.ruc.dk/~keld/research/LKH-3/
+        2. Put the build binary in `solvers/lkh/bin`, replace the binary that matches your OS.
+      * VRP-Pipeline (used for K-means, Two Phase Clustering, VRP to QUBO convertion):
+        1. Install Rust: https://www.rust-lang.org/tools/install
+        2. Install a specific Rust nightly build (needed cause the solver uses experimental features): `rustup install nightly-2023-07-01`
+        3. Check how the nightly build is called on your machine (this is shown when running the install command, on Mac it is called *nightly-2023-07-01-aarch64-apple-darwin*)
+        4. Set the nightly build as default: `rustup default nightly-2023-07-01(... specific version name on machine)`
+        5. Download source code of the VRP-Pipeline: https://github.com/ProvideQ/hybrid-vrp-solver
+        6. build the source code using `cargo build`
+        7. Put the build binary in `solvers/berger-vrp/bin`, replace the binary that matches your OS.
+   3. GAMS (multiple solvers are build on this):
+      1. Install a python env that works with GAMS (skip this step if you don't need GAMS)
+      2. Install GAMS. (https://www.gams.com/download/)
+      3. Install miniconda (or anaconda, if you prefer that):
+         https://conda.io/projects/conda/en/stable/user-guide/install/index.html
+      4. Create a GAMS conda environment: `conda create --name gams python=3.10 --yes`
+      5. Activate your conda environment: `conda activate gams`.
+      6. Make GAMS use that python environment by setting the `GMSPYTHONLIB=<path-to-conda>/envs/gams/lib/libpython3.10.so`
+            environment variable.
+      7. Install GAMS packages to the GAMS conda env:
+            `pip install gams[core,connect] --find-links <path-to-gams>/api/python/bdist`
+            * If you get an error building `psycopg2`, try to install these postgres packages:
+              `sudo apt-get install postgresql libpq-dev` and run the `pip install ...` command again
+      8. Install the python dependencies we use in our python packages: `pip install -r gams/requirements.txt`
+4. Run the server using `./gradlew bootRun`
 
 ## Deployment
 This repository is designed to be deployed with [Dokku](https://dokku.com/), but you can also run 
@@ -83,4 +99,4 @@ To use this, enable GitHub Actions and configure the following secrets in the Gi
 ## License
 Copyright (c) 2022 - 2023 ProvideQ
 
-This project is available under the [MIT License](./LICENSE).
+This project is available under the [MIT License](./LICENSE).

scripts/install-solver-dependencies.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# this script is made for the CI-pipeline, do not use this to install dependencies on your private machine!
 
 # exit on error
 set -e
@@ -10,7 +11,12 @@ REPO_DIR=$(dirname "$(dirname "$(readlink -f "$0")")")
 source /opt/conda/bin/activate gams
 
 # install solver dependencies
-pip install -r "$REPO_DIR/gams/requirements.txt"
-pip install -r "$REPO_DIR/qiskit/requirements.txt"
-pip install -r "$REPO_DIR/cirq/requirements.txt"
-pip install -r "$REPO_DIR/python/requirements.txt"
+pip install -r "$REPO_DIR/solvers/gams/requirements.txt"
+# quantum frameworks:
+pip install -r "$REPO_DIR/solvers/qiskit/requirements.txt"
+pip install -r "$REPO_DIR/solvers/cirq/requirements.txt"
+pip install -r "$REPO_DIR/solvers/dwave/requirements.txt"
+pip install -r "$REPO_DIR/solvers/qrisp/requirements.txt"
+# custom solvers with python wrapper:
+pip install -r "$REPO_DIR/solvers/custom/hs-knapsack/requirements.txt"
+pip install -r "$REPO_DIR/solvers/custom/lkh/requirements.txt"

solvers/custom/berger-vrp/bin/README.md

@@ -0,0 +1,35 @@
+### How to Cross-compile the Rust Pipeline:
+
+Guide is written for Mac-ARM System
+
+1. Download Repo: https://github.com/ProvideQ/hybrid-vrp-solver
+2. Install Rust: https://www.rust-lang.org/tools/install
+3. Install a specific Rust nightly build (needed cause the solver uses experimental features): `rustup install nightly-2023-07-01` 
+4. Check how the nightly build is called on your machine (this is shown when running the install command, on Mac it is called *nightly-2023-07-01-aarch64-apple-darwin*)
+5. Set the nightly build as default: `rustup default nightly-2023-07-01(... specific version name on machine)`
+
+#### Mac Version: (native)
+1. build the source code using `cargo build --release`
+
+#### Windows Version (cross compilation)
+1. Add Target for Windows: `rustup target add x86_64-pc-windows-gnu`
+2. Install Cross Toolchain: `brew install mingw-w64`
+3. Create .cargo/config.toml and add the following lines: <br>
+```
+[target.x86_64-pc-windows-gnu]
+linker = "x86_64-w64-mingw32-gcc"
+```
+4. run `cargo build --target x86_64-pc-windows-gnu --release`
+
+#### Linux Version (cross compilation) 
+* only gnu is supported, Lucas Bergers code does not support static linking
+1. Target for Linux: `rustup target add 86_64-unknown-linux-gnu`
+2. Toolchain: `arch -arm64 brew install SergioBenitez/osxct/x86_64-unknown-linux-gnu`
+3. Create .cargo/config.toml and add the following lines: (Path for linker: `which x86_64-unknown-linux-gnu`)<br>
+```
+[target.x86_64-pc-windows-gnu]
+linker = "/opt/homebrew/bin/x86_64-unknown-linux-gnu-gcc"
+```
+4. run `cargo build --target x86_64-unknown-linux-gnu --release`
+5. When you have issues, check here: https://github.com/briansmith/ring/issues/1605
+

python/requirements.txt → solvers/custom/hs-knapsack/requirements.txt

@@ -1,3 +1,3 @@
 # This file describes the python package requirements for the knapsack problem solver
 
-knapsack-pip
+knapsack-pip == 0.2

solvers/custom/lkh/requirements.txt

@@ -0,0 +1,6 @@
+# This file describes the python package requirements for all LKH scripts
+# supported by ProvideQ
+
+# required for LKH VRP solver
+lkh == 1.1.1
+tsplib95 == 0.7.1

solvers/custom/lkh/vrp_lkh.py

@@ -0,0 +1,47 @@
+import argparse
+
+from lkh import LKHProblem, solve
+from tsplib95.models import StandardProblem
+
+parser = argparse.ArgumentParser(
+    prog="LKH-3 Interface",
+    description="A CLI Program to initiate solving CVRP files with LKH-3",
+    epilog="Made by Lucas Berger for scientific purposes",
+)
+
+
+parser.add_argument("tsplib_file")
+parser.add_argument("--lkh-instance", default="./bin/LKH-unix")
+parser.add_argument("--output-file")
+parser.add_argument("-t", "--max-trials", default=1000)
+parser.add_argument("-r", "--runs", default=10)
+
+args = parser.parse_args()
+
+problem = LKHProblem.load(args.tsplib_file)
+
+print(f"solving {args.tsplib_file}")
+
+if sum(problem.demands.values()) <= problem.capacity:
+    problem.type = "TSP"
+
+if len(problem.node_coords.values()) > 2:
+    extra = {}
+    tours = solve(
+        args.lkh_instance, problem=problem, max_trials=args.max_trials, **extra
+    )
+
+    tour = StandardProblem()
+
+    tour.tours = [[*problem.depots, *path] for path in tours]
+    tour.type = "TOUR"
+    tour.name = problem.name + " solution"
+else:
+    tour = StandardProblem()
+    tour.tours = [problem.node_coords.keys()]
+    tour.type = "TOUR"
+    tour.name = problem.name + " solution"
+
+
+if args.output_file is not None:
+    tour.save(args.output_file)

solvers/dwave/qubo/embeddings.py

@@ -0,0 +1 @@
+cached_embeddings = {}

solvers/dwave/qubo/main.py

@@ -0,0 +1,81 @@
+import argparse
+import os
+from datetime import datetime
+from typing import Literal
+
+from dimod import BINARY, BinaryQuadraticModel, binary, constrained, lp
+from dimod.serialization import coo
+from dwave.cloud import Client
+from solver import solve_with
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        prog="DWave QUBO solver",
+        description="A CLI Program to initiate solving COOrdinate files with DWave Systems",
+        epilog="Made by Lucas Berger for scientific purposes",
+    )
+
+    parser.add_argument("file")
+    parser.add_argument(
+        "type", default="sim", choices=["sim", "hybrid", "qbsolv", "direct"]
+    )
+    parser.add_argument("--output-file")
+
+    args = parser.parse_args()
+    type: Literal["sim", "hybrid", "qbsolv", "direct"] = args.type
+
+    bqm: BinaryQuadraticModel | None = None
+    with open(args.file) as problem:
+        bqm = coo.load(problem, vartype=BINARY)
+        if len(bqm.quadratic) == 0:
+            bqm = None
+
+    if bqm is None:
+        with open(args.file) as problem:
+            cqm = lp.load(problem)
+            converted, _ = constrained.cqm_to_bqm(cqm)
+
+            linear_conv = {
+                (int(str(x)[1:])): converted.linear[x] for x in converted.linear
+            }
+            quad_conv = {
+                (int(str(x)[1:]), int(str(y)[1:])): converted.quadratic[(x, y)]
+                for x, y in converted.quadratic
+            }
+
+            bqm = BinaryQuadraticModel(linear_conv, quad_conv, converted.offset, BINARY)
+            if len(bqm.quadratic) == 0:
+                bqm = None
+
+    filename = os.path.basename(args.file)
+
+    if bqm is None:
+        raise Exception("Could not load file")
+
+    last = datetime.now().timestamp()
+    print("started")
+
+    now = datetime.now().timestamp()
+    print(f"connected after {now - last}. starting solver")
+    sampleset = solve_with(bqm, type, filename)
+
+    # accessing the sampleset's properties await for the future
+    print(sampleset.info)
+
+    now = datetime.now().timestamp()
+    print(f"ended {now - last}")
+
+    if args.output_file:
+        with open(args.output_file, "w") as out:
+            out.writelines([f"{bin}\n" for bin in sampleset.first.sample.values()])
+    else:
+        print(sampleset.first.energy)
+        print(sampleset.first.sample)
+
+    now = datetime.now().timestamp()
+    print(f"connection closed after {now - last}")
+
+
+if __name__ == "__main__":
+    main()