Add Gymnasium client #18

siboehm · 2024-01-17T23:04:37Z

closes #12

garymm

Went through the first two commits only.

garymm · 2024-01-18T18:46:37Z

.gitmodules

@@ -0,0 +1,7 @@
+[submodule "lib/SDL"]


Can you use CMake to fetch these instead?
An example from a CMakeLists.txt file I wrote for something else:

include(FetchContent) FetchContent_Declare( ut GIT_REPOSITORY https://github.com/boost-ext/ut GIT_TAG 20f2c3f811e25b8c398c6e98545dbf07d72eb4e8 ) FetchContent_MakeAvailable(ut)

I don't have a lot experience with it but it seems maybe better if we don't expect users to edit the source of these at all. I guess at least for SDL we don't expect to edit it at all. irrlictmt maybe we will need to edit it?
Thoughts?

CMakeLists.txt

doc/compiling/linux.md

doc/compiling/macos.md

src/client/render/pipeline.h

src/client/render/plain.cpp

src/client/renderingengine.cpp

garymm

Went only through 3rd commit adding remote client.

Can you add a README.astera.md in the root that summarizes the changes we've made in our fork?

garymm · 2024-01-19T02:46:30Z

.gitmodules

@@ -1,3 +1,6 @@
+[submodule "lib/zmqpp"]


this one also seems better to use CMake since we don't expect to modify it

garymm · 2024-01-19T02:48:10Z

src/CMakeLists.txt

+    include_directories(${CAPNP_INCLUDE_DIRS})
+    get_filename_component(PROTO_PARENT ${PROJECT_SOURCE_DIR}/.. ABSOLUTE)
+    file(GLOB PROTO_FILES ${PROTO_PARENT}/src/network/proto/*.capnp)
+	# set(CAPNPC_OUTPUT_DIR "${CMAKE_CURRENT_SOURCE_DIR}")


garymm · 2024-01-19T02:48:14Z

src/CMakeLists.txt

+    # TODO error handling
+    message(STATUS "Cap'n Proto enabled.")
+    find_package(CapnProto CONFIG REQUIRED)
+    message(STATUS "Cap'n Proto include dir: ${CAPNP_INCLUDE_DIRS}")


src/CMakeLists.txt

garymm · 2024-01-19T02:50:59Z

cmake/Modules/FindZmqpp.cmake

@@ -0,0 +1,41 @@
+option(ENABLE_SYSTEM_ZMQPP "Use ZMQPP from system" TRUE)


I don't think we should to support both system and submodule. I think the submodule way is better in general, but most other dependencies are system deps so maybe more consistent for now to just support a system dep?

I agree. Thing to note here is that zmqpp is not packaged for Ubuntu, but exists in brew. So this makes the installation process on infra somewhat more annoying, since you have to build & install from source.

src/client/remoteinputhandler.h

garymm

OK, done. That was a lot of code. So sad I won't be reviewing any more of your code 😢

.github/workflows/linux.yml

garymm · 2024-01-19T03:35:04Z

games/minetest_game

Can we use the devtest game instead?

gym_client.py

gym_client_headless.py

python/minetester/minetest_env.py

python/tests/minetest_test.py

python/tests/worlds/test_world_minetestenv/worldmods/reward/init.lua

Change-Id: Ibaca219398508582bf0f89baabbab23cb307fa60

Change-Id: I2ff2531aa1b90a450a5c8e904c808d8c2ea1a4be

Change-Id: Ifdaeb008b7cdc789044a7c49baa528df8962e70f

Change-Id: Ib31ff36503d7c5bc8f13f5f3811bd5dd7b6ee5d8

Change-Id: Ied9c594630d586b9ad397d9bb8e8fa8783f40591

Change-Id: I960b7c451c95f5ce958a5301f825a113cf363f29

Change-Id: Ia6580817d446f9863f8a3586f84afa1e8867b244

Best to not have it as default as I don't understand everything it does.

It seems there is some asynchronous initialization that sometimes results in observations with mostly grey background and no reward. Work-around by attempting to detect and retry in MinetestEnv.reset(). Also in the mod have the reward initialized 1 in the mod because sometimes we were seeing the first observation have it set to 0.

Get almost all dependencies from anaconda rather than apt-get. I think this will make it easier to build an anaconda package.

siboehm added 4 commits January 17, 2024 14:36

Rm Android & Docker & windows CI tests

26f0d23

Add headless mode

be27afd

Add remote client

3c9ac52

Add Python minetest client

09b99aa

siboehm requested a review from garymm January 17, 2024 23:14

garymm requested changes Jan 18, 2024

View reviewed changes

siboehm added 6 commits January 18, 2024 15:29

rm obsolete SDL_Target cmake settings

1e8d03a

Docs: Add instr for faster build using Ninja & mold

d9a9b44

Rm commented out code in Cmake

cfe0958

Fix macos test failure by installing devtest game

13b3d01

re-add delete line in renderingengine

9218ddb

simplify pytest setup on macos

b18fb18

garymm requested changes Jan 19, 2024

View reviewed changes

siboehm and others added 8 commits January 19, 2024 11:20

Minor changes to minetest gym env

a0ac991

Clean up Python helper scripts

ecd7e47

Linux CI: Pytest doesn't need gdb

0d76a01

ruff formatting

bdfb2d3

minor comments

ec16498

Minor refactorings in remote input handling

5686a52

comment change in minetest.conf.example

441bc23

Change-Id: Ibaca219398508582bf0f89baabbab23cb307fa60

always enable capnproto

1cb167a

Change-Id: I2ff2531aa1b90a450a5c8e904c808d8c2ea1a4be

garymm mentioned this pull request Jan 24, 2024

Use CapnProto RPC instead of ZeroMQ #19

Closed

garymm added 4 commits January 30, 2024 21:48

Include local python code in conda environment

8b7dfa7

Change-Id: Ifdaeb008b7cdc789044a7c49baa528df8962e70f

add devcontainer

eafe572

Change-Id: Ib31ff36503d7c5bc8f13f5f3811bd5dd7b6ee5d8

specify gym environment version

a5a49f5

Change-Id: Ied9c594630d586b9ad397d9bb8e8fa8783f40591

python VS Code settings

4020a3a

Change-Id: I960b7c451c95f5ce958a5301f825a113cf363f29

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch 3 times, most recently from 12beed5 to 2a0af30 Compare January 31, 2024 23:53

garymm added 3 commits February 2, 2024 23:32

move python tools into python/tools

d18eec2

devcontainer: add ruff, run container priveleged

ada76bb

vscode: format python and jsonc on save

7f90675

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch from fb8fbd1 to 7f90675 Compare February 2, 2024 23:37

improve setup instructions

becd62b

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch from 3db6865 to 0ff052b Compare February 3, 2024 00:27

garymm and others added 3 commits February 6, 2024 23:26

gym: support connecting to running minetest

02071a6

Change-Id: Ia6580817d446f9863f8a3586f84afa1e8867b244

gym test: close env on test failure. dump images

21e9ea6

devcontainer: disable priveleged mode

e4abb75

Best to not have it as default as I don't understand everything it does.

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch from 0ff052b to e4abb75 Compare February 6, 2024 23:41

garymm added 3 commits February 8, 2024 00:55

suppress pygame log spam

bfe6332

headless: minor naming and RAII

e5e7a31

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch 13 times, most recently from 10db0fd to 62a7646 Compare February 13, 2024 23:49

apt-get -> conda

bbbe3f4

Get almost all dependencies from anaconda rather than apt-get. I think this will make it easier to build an anaconda package.

garymm force-pushed the siboehm/gymInterfaceSAVEGAME2 branch from 62a7646 to bbbe3f4 Compare February 14, 2024 00:06

garymm merged commit b4f270c into master Feb 14, 2024
20 checks passed

garymm deleted the siboehm/gymInterfaceSAVEGAME2 branch February 14, 2024 00:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Gymnasium client #18

Add Gymnasium client #18

siboehm commented Jan 17, 2024 •

edited

Loading

garymm left a comment

garymm Jan 18, 2024

garymm left a comment

garymm Jan 19, 2024

garymm Jan 19, 2024

garymm Jan 19, 2024

garymm Jan 19, 2024

siboehm Jan 19, 2024 •

edited

Loading

garymm left a comment

garymm Jan 19, 2024

		@@ -0,0 +1,41 @@
		option(ENABLE_SYSTEM_ZMQPP "Use ZMQPP from system" TRUE)

Add Gymnasium client #18

Add Gymnasium client #18

Conversation

siboehm commented Jan 17, 2024 • edited Loading

garymm left a comment

Choose a reason for hiding this comment

garymm Jan 18, 2024

Choose a reason for hiding this comment

garymm left a comment

Choose a reason for hiding this comment

garymm Jan 19, 2024

Choose a reason for hiding this comment

garymm Jan 19, 2024

Choose a reason for hiding this comment

garymm Jan 19, 2024

Choose a reason for hiding this comment

garymm Jan 19, 2024

Choose a reason for hiding this comment

siboehm Jan 19, 2024 • edited Loading

Choose a reason for hiding this comment

garymm left a comment

Choose a reason for hiding this comment

garymm Jan 19, 2024

Choose a reason for hiding this comment

siboehm commented Jan 17, 2024 •

edited

Loading

siboehm Jan 19, 2024 •

edited

Loading