Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Support for macOS #1

Closed
wants to merge 3 commits into from
Closed

WIP: Support for macOS #1

wants to merge 3 commits into from

Conversation

lifeiteng
Copy link
Owner

No description provided.

@lifeiteng
Copy link
Owner Author

pip install tensorflow==2.3.0
bash oss_build.sh --python 3.6

INFO: Elapsed time: 132.871s, Critical Path: 45.42s
INFO: 1018 processes: 1018 darwin-sandbox.
INFO: Build completed successfully, 1074 total actions
//reverb/cc:chunk_store_test                                             PASSED in 0.7s
//reverb/cc:client_test                                                  PASSED in 0.1s
//reverb/cc:rate_limiter_test                                            PASSED in 1.5s
//reverb/cc:reverb_service_impl_test                                     PASSED in 1.0s
//reverb/cc:sampler_test                                                 PASSED in 3.7s
//reverb/cc:table_test                                                   PASSED in 2.5s
//reverb/cc:tensor_compression_test                                      PASSED in 0.2s
//reverb/cc:writer_test                                                  PASSED in 1.1s
//reverb/cc/platform:net_test                                            PASSED in 0.1s
//reverb/cc/platform:server_test                                         PASSED in 0.9s
//reverb/cc/platform:tfrecord_checkpointer_test                          PASSED in 0.9s
//reverb/cc/platform:thread_test                                         PASSED in 0.2s
//reverb/cc/selectors:fifo_test                                          PASSED in 0.2s
//reverb/cc/selectors:heap_test                                          PASSED in 0.2s
//reverb/cc/selectors:lifo_test                                          PASSED in 0.1s
//reverb/cc/selectors:prioritized_test                                   PASSED in 0.2s
//reverb/cc/selectors:uniform_test                                       PASSED in 1.2s
//reverb/cc/support:cleanup_test                                         PASSED in 0.2s
//reverb/cc/support:intrusive_heap_test                                  PASSED in 0.3s
//reverb/cc/support:periodic_closure_test                                PASSED in 0.3s
//reverb/cc/support:queue_test                                           PASSED in 0.2s

INFO: Build completed successfully, 1074 total actions
INFO: Analyzed target //reverb/pip_package:build_pip_package (2 packages loaded, 66 targets configured).
INFO: Found 1 target...
Target //reverb/pip_package:build_pip_package up-to-date:
  bazel-bin/reverb/pip_package/build_pip_package
INFO: Elapsed time: 15.193s, Critical Path: 14.84s
INFO: 2 processes: 2 darwin-sandbox.
INFO: Build completed successfully, 6 total actions
2020年11月27日 星期五 19时58分41秒 CST : === Preparing sources in dir: /var/folders/y3/1s2lqlds29x_nrklc5vftyt00000gn/T/tmp.XXXXXXXXXX.PCMezZP9
/var/folders/y3/1s2lqlds29x_nrklc5vftyt00000gn/T/tmp.XXXXXXXXXX.PCMezZP9/reverb/cc/libreverb_service_cc_proto.so
/var/folders/y3/1s2lqlds29x_nrklc5vftyt00000gn/T/tmp.XXXXXXXXXX.PCMezZP9/reverb/cc/libschema_cc_proto.so
/var/folders/y3/1s2lqlds29x_nrklc5vftyt00000gn/T/tmp.XXXXXXXXXX.PCMezZP9/reverb/cc/ops
/var/folders/y3/1s2lqlds29x_nrklc5vftyt00000gn/T/tmp.XXXXXXXXXX.PCMezZP9/reverb/cc/checkpointing/libcheckpoint_cc_proto.so
===== DIR = _U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc
===== DIR = _U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc
===== DIR = _U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing
2020年11月27日 星期五 19时58分41秒 CST Setting PYTHON_BIN_PATH equal to what was set with configure.py.
2020年11月27日 星期五 19时58分41秒 CST : === Building wheel
copying reverb/libpybind.so -> build/lib/reverb
copying reverb/cc/libreverb_service_cc_proto.so -> build/lib/reverb/cc
copying reverb/cc/libschema_cc_proto.so -> build/lib/reverb/cc
copying reverb/cc/checkpointing/libcheckpoint_cc_proto.so -> build/lib/reverb/cc/checkpointing
copying reverb/cc/ops/libgen_client_ops_gen_op.so -> build/lib/reverb/cc/ops
copying reverb/cc/ops/libgen_dataset_op_gen_op.so -> build/lib/reverb/cc/ops
creating build/lib/_solib_darwin
creating build/lib/_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing
copying reverb/../_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing/libcheckpoint_cc_proto.so -> build/lib/reverb/../_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing
creating build/lib/_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc
copying reverb/../_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc/libreverb_service_cc_proto.so -> build/lib/reverb/../_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc
creating build/lib/_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc
copying reverb/../_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc/libschema_cc_proto.so -> build/lib/reverb/../_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc
copying build/lib/reverb/libpybind.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb
copying build/lib/reverb/cc/libreverb_service_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb/cc
copying build/lib/reverb/cc/libschema_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb/cc
copying build/lib/reverb/cc/ops/libgen_dataset_op_gen_op.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb/cc/ops
copying build/lib/reverb/cc/ops/libgen_client_ops_gen_op.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb/cc/ops
copying build/lib/reverb/cc/checkpointing/libcheckpoint_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/reverb/cc/checkpointing
creating build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin
creating build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing
copying build/lib/_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing/libcheckpoint_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing
creating build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc
copying build/lib/_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc/libreverb_service_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc
creating build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc
copying build/lib/_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc/libschema_cc_proto.so -> build/bdist.macosx-10.9-x86_64/wheel/_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc
adding '_solib_darwin/_U_S_Sreverb_Scc_Creverb_Uservice_Ucc_Uproto___Ureverb_Scc/libreverb_service_cc_proto.so'
adding '_solib_darwin/_U_S_Sreverb_Scc_Cschema_Ucc_Uproto___Ureverb_Scc/libschema_cc_proto.so'
adding '_solib_darwin/_U_S_Sreverb_Scc_Scheckpointing_Ccheckpoint_Ucc_Uproto___Ureverb_Scc_Scheckpointing/libcheckpoint_cc_proto.so'
adding 'reverb/libpybind.so'
adding 'reverb/cc/libreverb_service_cc_proto.so'
adding 'reverb/cc/libschema_cc_proto.so'
adding 'reverb/cc/checkpointing/libcheckpoint_cc_proto.so'
adding 'reverb/cc/ops/libgen_client_ops_gen_op.so'
adding 'reverb/cc/ops/libgen_dataset_op_gen_op.so'
2020年11月27日 星期五 19时58分43秒 CST : === Output wheel file is in: /tmp/reverb/dist/
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Processing /tmp/reverb/dist/dm_reverb_nightly-0.2.0.dev20201127-cp36-cp36m-macosx_10_15_x86_64.whl
Processing /Users/feiteng/Library/Caches/pip/wheels/d1/16/ff/0566c4ef305f6e4629fe7aba838f1f5aa42b5003ae3ea6a10d/portpicker-1.3.1-py3-none-any.whl
Collecting dm-tree
  Using cached https://mirrors.aliyun.com/pypi/packages/e0/af/d60d32acf5e9c3511cfffbd2bf72d55fdf3ca01c116631120addfda70226/dm_tree-0.1.5-cp36-cp36m-macosx_10_9_x86_64.whl (94 kB)
Collecting six>=1.12.0
  Using cached https://mirrors.aliyun.com/pypi/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: portpicker, six, dm-tree, dm-reverb-nightly
  Attempting uninstall: portpicker
    Found existing installation: portpicker 1.3.1
    Uninstalling portpicker-1.3.1:
      Successfully uninstalled portpicker-1.3.1
  Attempting uninstall: six
    Found existing installation: six 1.15.0
    Uninstalling six-1.15.0:
      Successfully uninstalled six-1.15.0
  Attempting uninstall: dm-tree
    Found existing installation: dm-tree 0.1.5
    Uninstalling dm-tree-0.1.5:
      Successfully uninstalled dm-tree-0.1.5
  Attempting uninstall: dm-reverb-nightly
    Found existing installation: dm-reverb-nightly 0.2.0.dev20201127
    Uninstalling dm-reverb-nightly-0.2.0.dev20201127:
      Successfully uninstalled dm-reverb-nightly-0.2.0.dev20201127
Successfully installed dm-reverb-nightly-0.2.0.dev20201127 dm-tree-0.1.5 portpicker-1.3.1 six-1.15.0
Run Python tests...
===========Running Python tests============
####=======Testing reverb//platform/checkpointers_lib_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55914 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//platform/checkpointers_lib_test.py
####=======Testing reverb//dataset_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55925 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//dataset_test.py
####=======Testing reverb//tf_client_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55936 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//tf_client_test.py
####=======Testing reverb//pybind_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55939 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//pybind_test.py
####=======Testing reverb//server_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55942 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//server_test.py
####=======Testing reverb//rate_limiters_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55945 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//rate_limiters_test.py
####=======Testing reverb//client_test.py=======####
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/descriptor_database.cc:118] File already exists in database: reverb/cc/schema.proto
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1379] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
run_python_tests.sh: line 32: 55948 Abort trap: 6           ${PYTHON_BIN_PATH} "${test_file}"
FAIL: reverb//client_test.py



===========Error Summary============
FAIL: reverb//platform/checkpointers_lib_test.py
FAIL: reverb//dataset_test.py
FAIL: reverb//tf_client_test.py
FAIL: reverb//pybind_test.py
FAIL: reverb//server_test.py
FAIL: reverb//rate_limiters_test.py
FAIL: reverb//client_test.py

@lifeiteng
Copy link
Owner Author

bazel https://github.com/bazelbuild/bazel/releases/download/3.1.0/bazel-3.1.0-installer-darwin-x86_64.sh

@lifeiteng
Copy link
Owner Author

bazel test -c opt --copt=-mavx --verbose_failures --sandbox_debug reverb:pybind_test

FAIL: //reverb:pybind_test (see /private/var/tmp/_bazel_feiteng/25f22fa3cbf40674101bed76ec656768/execroot/reverb/bazel-out/darwin-opt/testlogs/reverb/pybind_test/test.log)
Target //reverb:pybind_test up-to-date:
  bazel-bin/reverb/pybind_test
INFO: Elapsed time: 3.308s, Critical Path: 3.12s
INFO: 3 processes: 3 darwin-sandbox.
INFO: Build completed, 1 test FAILED, 3 total actions
//reverb:pybind_test                                                     FAILED in 2.6s
  /private/var/tmp/_bazel_feiteng/25f22fa3cbf40674101bed76ec656768/execroot/reverb/bazel-out/darwin-opt/testlogs/reverb/pybind_test/test.log

error info

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //reverb:pybind_test
-----------------------------------------------------------------------------
Fatal Python error: PyThreadState_Get: no current thread

@lifeiteng lifeiteng mentioned this pull request Dec 3, 2020
Closed
4 tasks
@lifeiteng
Copy link
Owner Author

google-deepmind#24

@lifeiteng lifeiteng closed this Dec 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@lifeiteng