Skip to content

Commit

Permalink
Open-source FlexIO.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 616150571
  • Loading branch information
AlexeyG authored and Scenic Authors committed Mar 15, 2024
1 parent 193af73 commit 9bf9ad2
Show file tree
Hide file tree
Showing 3 changed files with 782 additions and 0 deletions.
16 changes: 16 additions & 0 deletions scenic/dataset_lib/flexio/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# FlexIO

Flexible IO that supports reading data from multiple sources.

Contact: [email protected] [email protected] [email protected]

# Motivation
Deliver a flexible lightweight research-friendly input pipeline that enables quick hacking and experimentation, and can be used for many projects and tasks.

# Key requirements / features
* Support common image and video dataset sources
* Support extensible (per-dataset) pre-processing
* Provide a clear overview of the pre-processing ops
* Support fully deterministic pre-processing
* Rely on a well-tested pre-processing op library
* Support dataset mixing (e.g. for co-training on several datasets)
Loading

0 comments on commit 9bf9ad2

Please sign in to comment.