Skip to content

FRAMEWORK DEVELOPMENT

Jump to bottom
Andrew edited this page Jan 20, 2018 · 2 revisions
Questions related to btgym framework itself: features, issues and open development directions.

Data input pipe:

Issue 1: No input ecosystem is developed, only CSV files input is supported, parsing options are is somewhat obscure; no predefined templates except for initially built-in 1 min Forex; no live data input;

Comment: There exists generic data iterator class BTGymBaseData providing core functionality for:

  1. loading data from sources (read_csv())
  2. shaping various distributions over data provided
  3. sampling and returning child data iterators from above-mentioned distributions (_sample_interval(), _sample_random() etc.)
  4. providing environment simulation engine with stream of data in its native format (to_btfeed())
  5. some service functions (describe(), reset() )

Pp. 2 and 3 are implementations of data abstractions outlined here and related to my attempts to properly formulate algo-trading problem in terms of learning and meta-learning in changing POMDP environments. All of them internally relies on pandas data frames methods.
P.1 is pure input pipeline question, closely tied with backtrader as later serves as core environment simulation engine.
To implement new input pipe (either it another CSV, or direct pandas frames or whatever) one should subclass generic BTGymBaseData and provide it with extended loading/parsing functionality, such as implement .load_pandas() or load_xls() or load_yahoo_finance() methods so data provided are to be converted into pandas data frames and stored in MyDataset.data attribute, leaving along all data sampling and iterating methods (if there is no need to bring another distribution or sampling method).
Same is true for designing live feeds, though later obviously should be subclass if SequentialDataIterator.

Related: #31, #25, #8, #32

Todo: develop hierarchy of input methods and parsing templates; clear guidelines for implementing own input pipeline;

Platform compatibility:

Issue 1: btgym doesn't seems to fully work under Windows.

Comment: main known issue is due to fact backtrader-based rendering implementation is somewhat hacky due to matplotlib memory leak issues

Related: #17

Todo: override base backtrader render module.

Clone this wiki locally