Skip to content

Commit

Permalink
limit cs factors only on linux due to multiprocessing limitation
Browse files Browse the repository at this point in the history
  • Loading branch information
qianyun210603 committed Dec 12, 2023
1 parent 0c7e2e7 commit 741c3f7
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ I will try to merge some non-debateable features/bugfix to the original qlib. Bu
# Features
| Feature | Description | Status |
|--------------------------------|--------------------------------------------------------------------------| -- |
| Cross Sectional Factor | Add cross-sectional factors such as cross-sectional ranking; average etc | Done |
| Cross Sectional Factor | Add cross-sectional factors such as cross-sectional ranking; average etc | Done (Linux only) |
| Orthogonalization preprocesser | Add preprocessers to do Schimit and Symetric Orthogonalization | Done |
| Support non-adjusted data | Add support for non-adjusted data | Done |
| Enhanced plotting I | Use rangebreak to allow Datetime axis in plottings | [Merged](https://github.com/microsoft/qlib/pull/1390) |
Expand Down
7 changes: 6 additions & 1 deletion qlib/data/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import bisect
import copy
import multiprocessing
import platform
import queue
from collections import deque
from typing import List, Optional, Union
Expand Down Expand Up @@ -662,7 +663,11 @@ def __call__(self):
return [self.col_names[this_idx]] + col_names
return self.col_names

if len(cs_levels) > 1 and C["joblib_backend"] == "multiprocessing": # pylint: disable=R1702
if len(cs_levels) > 1:
if C["joblib_backend"] != "multiprocessing": # pylint: disable=R1702
raise RuntimeError("only multiprocessing backend is supported for cross-section data")
if platform.system() == "Windows":
raise SystemError("Cross-section data not supported due to the limitation of multiprocessing.")
get_module_logger("data").info("shared memory created")
shared_mgr = multiprocessing.Manager()
shared_data_cache = shared_mgr.dict()
Expand Down

0 comments on commit 741c3f7

Please sign in to comment.