Skip to content

Commit

Permalink
Moves documentation to tensorflow.org and updates README.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 450076353
Change-Id: I3020fa832fed08607ff059f178a32e66c04d8627
  • Loading branch information
Johannes Ballé authored and copybara-github committed May 20, 2022
1 parent ea3e8f5 commit 29b276e
Show file tree
Hide file tree
Showing 58 changed files with 33 additions and 52,384 deletions.
57 changes: 33 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,42 @@ TensorFlow Compression (TFC) contains data compression tools for TensorFlow.
You can use this library to build your own ML models with end-to-end optimized
data compression built in. It's useful to find storage-efficient representations
of your data (images, features, examples, etc.) while only sacrificing a small
fraction of model performance.

Specifically, the entropy model classes in this library simplify the process of
designing rate–distortion optimized codes. During training, they act like
likelihood models. Once training is completed, they encode floating point
tensors into optimized bit sequences by automating the design of probability
tables and calling a range coder implementation behind the scenes.

The library implements range coding (a.k.a. arithmetic coding) using a set of
flexible TF ops written in C++. These include an optional "overflow"
functionality that embeds an Elias gamma code into the range encoded bit
sequence, making it possible to encode the entire set of signed integers rather
than just a finite range.

The main novelty of the learned approach over traditional transform coding is
the stochastic minimization of the rate-distortion Lagrangian, and using
nonlinear transforms implemented by neural networks. For an introduction to
this from a data compression perspective, consider our [paper on nonlinear
transform coding](https://arxiv.org/abs/2007.03034), or watch @jonycgn's [talk
on learned image compression](https://www.youtube.com/watch?v=x_q7cZviXkY). For
an introduction to lossy data compression from a machine learning perspective,
take a look at @yiboyang's [review paper](https://arxiv.org/abs/2202.06533).
fraction of model performance. Take a look at the [lossy data compression
tutorial](https://www.tensorflow.org/tutorials/generative/data_compression) to
get started.

For a more in-depth introduction from a classical data compression perspective,
consider our [paper on nonlinear transform
coding](https://arxiv.org/abs/2007.03034), or watch @jonycgn's [talk on learned
image compression](https://www.youtube.com/watch?v=x_q7cZviXkY). For an
introduction to lossy data compression from a machine learning perspective, take
a look at @yiboyang's [review paper](https://arxiv.org/abs/2202.06533).

The library contains (see the [API
docs](https://www.tensorflow.org/api_docs/python/tfc) for details):

- Range coding (a.k.a. arithmetic coding) implementations in the form of
flexible TF ops written in C++. These include an optional "overflow"
functionality that embeds an Elias gamma code into the range encoded bit
sequence, making it possible to encode alphabets containing the entire set of
signed integers rather than just a finite range.

- Entropy model classes which simplify the process of designing rate–distortion
optimized codes. During training, they act like likelihood models. Once
training is completed, they encode floating point tensors into optimized bit
sequences by automating the design of range coding tables and calling the
range coder implementation behind the scenes.

- Additional TensorFlow functions and Keras layers that are useful in the
context of learned data compression, such as methods to numerically find
quantiles of density functions, take expectations with respect to dithering
noise, convolution layers with more flexible padding options, and an
implementation of generalized divisive normalization (GDN).


## Documentation & getting help

Refer to [the API
documentation](https://tensorflow.github.io/compression/docs/api_docs/python/tfc.html)
Refer to [the API documentation](https://www.tensorflow.org/api_docs/python/tfc)
for a complete description of the classes and functions this package implements.

Please post all questions or comments on
Expand Down
140 changes: 0 additions & 140 deletions docs/api_docs/python/tfc.md

This file was deleted.

Loading

0 comments on commit 29b276e

Please sign in to comment.