Skip to content

Commit

Permalink
optimize get_data code
Browse files Browse the repository at this point in the history
  • Loading branch information
Linlang committed Mar 6, 2024
1 parent e855806 commit 467b352
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 42 deletions.
2 changes: 1 addition & 1 deletion examples/orderbook_data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ pip install arctic # NOTE: pip may fail to resolve the right package dependency
2. Please follow following steps to download example data
```bash
cd examples/orderbook_data/
python get_data.py other_data --target_dir ~/.qlib/other_data/orderbook_data --name highfreq_orderbook_example_data.zip
python get_data.py download_data --target_dir ~/.qlib/orderbook_data --file_name highfreq_orderbook_example_data.zip
```

3. Please import the example data to your mongo db
Expand Down
41 changes: 0 additions & 41 deletions qlib/tests/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,47 +130,6 @@ def _delete_qlib_data(file_dir: Path):
logger.warning(f"delete: {_p}")
shutil.rmtree(_p)

def other_data(
self,
target_dir="~/.qlib/other_data",
name=None,
version=None,
delete_old=True,
):
"""Specifies a file name to download a file from a remote location.
Parameters
----------
target_dir: str
data save directory
name: str
filename, by default None
version: str
data version, value from [v1, ...], by default None(use script to specify version)
delete_old: bool
delete an existing directory, by default True
Examples
---------
# get orderbook data
python get_data.py other_data --target_dir ~/.qlib/other_data/orderbook_data --name highfreq_orderbook_example_data.zip
When this command is run, the data will be downloaded from this link: https://qlibpublic.blob.core.windows.net/data/default/stock_data/highfreq_orderbook_example_data.zip?{token}
-------
"""
if name == None:
logger.warning("Specify the name of the file to be downloaded.")
return

file_name = f"{version}/{name}"
file_name = name if version == None else file_name

if not self.check_dataset(file_name):
logger.warning("The file you specified does not exist in the remote repository.")
return

self.download_data(file_name.lower(), target_dir, delete_old)

def qlib_data(
self,
name="qlib_data",
Expand Down

0 comments on commit 467b352

Please sign in to comment.