docs: Benchmark Distro Readme and Licensing (#365)

.github/distro/LICENSE.md

@@ -0,0 +1,138 @@
+Deephaven Data Labs is dedicated to developing Deephaven's software in the open
+and is making software available under the following licenses:
+
+- **Deephaven Open Source** is made available under the Apache 2.0 License
+- **Deephaven Community** is provided under a source-available license, the
+Deephaven Community License Agreement
+- **Deephaven Benchmark** is provided under a source-available license, the
+Deephaven Community License Agreement
+
+To request a commercial license for software or uses not covered by these
+licenses or to ask any license-related question, please contact us at:
+[[email protected]](mailto:[email protected])
+
+
+# Deephaven Community License Agreement Version 1.0
+
+1. **ACCEPTANCE.** This Deephaven Community License Agreement is entered into by
+and between Deephaven Data Labs LLC, a Delaware Limited Liability Company, with
+an address of 2800 Niagara Lane North, Plymouth, MN 55447 ("Deephaven") and
+“You” (a) in your individual capacity, or (b) on behalf of your company if you
+are licensing the Software for a company for which or with which you work. By
+using the Software, You agree to all of the terms and conditions in this
+Deephaven Community License Agreement (the "Agreement").
+
+2. **THE SOFTWARE.** The "Software" means the version of the Deephaven Community
+Software provided to the original licensee with this Agreement. The Software may
+include third party owned code. Each third party module is subject to the terms
+of its respective license; the details of which can be found in the notices
+served at [https://github.com/deephaven/deephaven-core](https://github.com/deephaven/deephaven-core).
+Since licensees may contribute back to the Software as provided for in Section
+3(b), the Software may include any such contributions.
+
+3. **GRANT OF LICENSE.** Subject to the terms and conditions of this Agreement,
+Deephaven hereby grants You a royalty-free, worldwide, non-exclusive,
+non-transferable license to the Software (the "License"), subject, in all of the
+cases, to applicable laws and regulations, but not for the Prohibited Use (as
+provided for in Section 4), solely as follows:
+
+   - a. **Internal Use.** A license to use, copy, compile, and install the
+Software for Your internal use;
+
+   - b. **Derivative Works.** A license to (i) prepare, develop, compile, and test
+Derivative Works of the Software, (ii) use Your Derivative Works for Your
+internal use solely as expressly permitted in Section 3(a), and (iii) distribute
+Your Derivative Works back to Deephaven under a separate Deephaven Contributor
+Agreement for potential incorporation into the Software at Deephaven's sole
+discretion. A "Derivative Work" means any work that (A) is based on or derived
+from the Software including any modifications to the Software, or (B) meets the
+definition of derivative work under the United States Copyright Act of 1976, 17
+U.S.C. Section 101; and
+
+   - c. **Distribution and redistribution.** A license to distribute or
+redistribute the Software and Your Derivative Works, and copies of the Software
+and Your Derivative Works, to customers and other third parties for their use
+pursuant to this Agreement subject to the prohibitions in Section 4 and
+Agreement requirement as provided for in Section 5.
+
+4. **PROHIBITED USE.** Notwithstanding any provision of this Agreement to the
+contrary, the original licensee is prohibited from using, distributing, or
+deploying the Software or Derivative Works in such a way that a third party
+(including any recipient under Section 3(c)) can, directly or indirectly
+through the original licensee, its agents, its technology, or solutions made
+available to them, add, define, redefine, or modify the schema for any input
+tables (including source tables, or other input objects) that the Software or
+Derivative Works access. For clarification, every recipient of the Software or
+Derivative Works after the original licensee of the Software is prohibited from
+adding, defining, redefining, or modifying the schema for any input tables
+(including source tables, or other input objects) that the Software or
+Derivative Works access. This prohibition in no way affects the ability of a
+third party to add, define, redefine, or modify the schema of output tables
+(including output tables that are interim or derived). Further, this
+prohibition in no way affects the ability of a third party to add, define,
+redefine, or modify data (as opposed to schema), nor does it cause You to be
+limited in any capacity for Your internal use as permitted in Section 3(a). If
+You have any questions on this prohibition, please contact us at:
+[[email protected]](mailto:[email protected])
+
+5. **AGREEMENT.** If You distribute the Software, whether directly, as a copy,
+or via Your Derivative Works you must provide recipients a copy of this
+Agreement (which will bind each recipient directly) and You must ensure that all
+copyright, patent, trademark, and attribution notices from the Software are
+retained.
+
+6. **AFFILIATES.** You may permit Your affiliates to exercise the License,
+provided that such exercise must be solely for Your benefit and/or the benefit
+of Your affiliate, and You shall be responsible for all acts and omissions of
+such affiliates in connection with such exercise of the License, including but
+not limited to breach of any terms of this Agreement.
+
+7. **SUBLICENSE.** The License is sublicensable only to Your third-party
+subcontractors and contractors performing services on Your or Your affiliates'
+behalf, subject to the terms and conditions of this Agreement. The License is
+not sublicensable to any other third party nor is it further sublicensable. You
+shall be responsible for all acts and omissions of such subcontractor and
+contractors in connection with such exercise of the sublicense, including but
+not limited to breach of any terms of this Agreement.
+
+8. **PATENTS.** Rights to use the Software do not give You any right to
+implement or use Deephaven's patents independently of the permitted use of the
+Software.
+
+9. **RESERVATION OF RIGHTS.** Deephaven reserves all rights not granted in this
+Agreement. You have no right to use Deephaven trade names, trademarks, service
+marks, or product names except as required for reasonable and customary use in
+describing the origin of the Software.
+
+10. **DISCLAIMER OF WARRANTY.** THE SOFTWARE IS LICENSED ON AN "AS IS" BASIS
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND. DEEPHAVEN DISCLAIMS ALL
+WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF
+TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
+You are solely responsible for determining the appropriateness of using,
+distributing or redistributing the Software and Derivative Works and assume all
+risks associated with Your exercise of the License.
+
+11. **INDEMNIFICATION.** If You distribute or redistribute the Software or
+Derivative Works, You agree to defend on request, and indemnify, Deephaven and
+its affiliates, officers, directors, employees and agents from and against any
+and all losses, damages, liabilities, claims, costs and expenses (including
+reasonable attorney’s fees and expenses) incurred or arising from the
+exploitation of the Software or Derivative Works.
+
+12. **LIMITATION OF LIABILITY.** THE LICENSE IS GRANTED FOR NO FEE. IN NO EVENT
+AND UNDER NO LEGAL THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR
+OTHERWISE, SHALL DEEPHAVEN BE LIABLE FOR ANY DAMAGES ARISING OUT OF OR AS A
+RESULT OF THIS AGREEMENT, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
+OR CONSEQUENTIAL DAMAGES (INCLUDING BUT NOT LIMITED TO DAMAGES FOR LOSS OF
+GOODWILL, WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR LOSS OR CORRUPTION
+OF DATA), EVEN IF DEEPHAVEN HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+13. **TERMINATION.** The License will automatically terminate if You violate the
+use rights or prohibitions in this Agreement.
+
+14. **GOVERNING LAW.** This Agreement will be interpreted, construed and
+enforced in all respects in accordance with the laws of the State of Delaware,
+USA without reference to its choice of law rules to the contrary.
+
+
+# END OF AGREEMENT

.github/distro/NOTICE.md

@@ -0,0 +1,21 @@
+# Deephaven Community Benchmark
+Copyright 2021 Deephaven Data Lab
+
+Licensed under the Deephaven Community License Agreement 1.0 (the "License");  
+You may obtain a copy of the License at [LICENSE.md](LICENSE.md).
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+--------------------------------------------------------------------------------
+
+Specific directories in this archive are released under licenses other than 
+Deephaven Community License.  See the `LICENSE` and `NOTICE` files in 
+these directories for more information.
+
+Examples:
+* `libs/arrow-vector-<version>.jar` has Apache 2.0 License in META-INF/LICENSE
+* `libs/junit-<version>.jar` has Eclipse Public License 1.0 in LICENSE-junit.txt

.github/distro/README.md

@@ -0,0 +1,7 @@
+# Deephaven Benchmark Distribution
+
+The Benchmark distribution is a self-contained mechanism for running existing Deephaven operational benchmarks against Deephaven Community Core (DHC) without the need for checking out the Benchmark project or running from Github workflows.
+
+More information about the Deephaven Community Core Benchmark project, including running this distribution, can be found in the [Benchmark Project](https://github.com/deephaven/benchmark) on Github.
+
+

README.md

@@ -28,6 +28,7 @@ Resources:
 - [Getting Started](docs/GettingStarted.md) - Getting set up to run benchmarks against Deephaven Community Core
 - [Test-writing Basics](docs/TestWritingBasics.md) - How to generate data and use it for tests
 - [Collected Results](docs/CollectedResults.md) - What's in the benchmark results
+- [Running the Release Distribution](docs/distro/BenchmarkDistribution.md) - How to run Deephaven benchmarks from a release tar file
 - [Running from the Command Line](docs/CommandLine.md) - How to run the benchmark jar with a test package
 - [Published Results Storage](docs/PublishedResults.md) - How to grab and use Deephaven's published benchmarks
 

docs/distro/BenchmarkDistribution.md

@@ -0,0 +1,50 @@
+# Deephaven Benchmark Distribution
+
+The Benchmark distribution is a self-contained mechanism for running existing Deephaven operational benchmarks against Deephaven Community Core (DHC) without the need for checking out the Benchmark project or running from Github workflows.
+
+Prerequisites
+- [Benchmark Distribution Tar](https://github.com/deephaven/benchmark/releases/latest/)
+- [Docker](https://docs.docker.com/engine/install/)
+- [Linux Operating System](https://www.linux.com/what-is-linux/)
+- [Java 21+](https://adoptium.net/temurin/releases/)
+
+Notes
+- Benchmarks are only tested and run on [Ubuntu Linux](https://ubuntu.com/server). Other Operating Systems may work but may not be supported
+- Variability for Rates between runs for the same benchmark is likely, even on the same hardware
+- The base scale for Deephaven's nightly Benchmark runs is 10mm rows
+- Running all Deephaven benchmarks, like those done every night, takes over 7.5 hours
+- The included benchmarks run only on Deephaven with Python queries
+
+> [!WARNING]   
+> If other docker containers are running on the same system, there may be conflicts.
+
+## Running the Benchmarks
+
+Each Benchmark release includes a tar asset in the [Github Releases](https://github.com/deephaven/benchmark/releases).  This can be downloaded, unpacked into a directory, and run with the provided script.
+
+- Download the Benchmark distribution tar into an empty directory.  ex. `wget https://github.com/deephaven/benchmark/releases/download/v0.36.1/deephaven-benchmark-0.36.1.tar`
+- From that directory, unpack the tar file. ex. `tar xvf deephaven-benchmark-0.36.1.tar`
+- Test to make sure things work. ex. `./benchmark.sh 1 "Avg*"`
+- When the tests are finished, check the results. ex. `cat results/benchmark-summary-results.csv`
+- Try running the same set as before at higher scale and more iterations
+  - In *benchmark.properties*, overwrite exsiting row count with `scale.row.count=10000000`
+  - Run the script again.  It will take much longer. ex. `./benchmark.sh 3 "Avg*"`
+  - The results will contain 3 runs, each with a `benchmark-results.csv` file
+- If the host system has enough memory, try increasing DHC memory and runnning at even higher scale
+  - Edit *docker-compose.yml* and change `-Xmx24G` to `-Xmx48G`
+  - In *benchmark.properties*, set `scale.row.count` higher
+
+> [!WARNING]  
+> Setting `scale.row.count` to a higher value will effect memory usage in DHC.  If set too high, DHC may crash with an "Out of Memory" error.
+
+## Benchmarking like Deephaven
+
+If you've gotten this far, you are now using the same software Deephaven uses to benchmark DHC.  However, the configuration of the Benchmark distribution is not necessarily the same as what is used every night.  It is also difficult to reproduce the same Rates without running on exactly the same hardware.  So expect differences. See the [full documentation in Github](https://github.com/deephaven/benchmark) for more information on Benchmark concepts, configuration, running, recent results, and more.
+
+> [!NOTE]  
+> More information on Deephaven Community Core and Deephaven Enterprise can be found at [Deephaven IO](https://deephaven.io/)
+
+
+
+
+

src/it/java/io/deephaven/benchmark/tests/internal/generator/ProtobufGeneratorTest.java

@@ -1,6 +1,5 @@
 package io.deephaven.benchmark.tests.internal.generator;
 
-import static org.junit.jupiter.api.Assertions.*;
 import org.junit.jupiter.api.*;
 import io.deephaven.benchmark.api.Bench;