Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/jalammar/ecco into main
Browse files Browse the repository at this point in the history
  • Loading branch information
Haziqa-coder committed Jan 2, 2022
2 parents 566af9c + b40a028 commit ccfd42b
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 4 deletions.
18 changes: 17 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,22 @@ Ecco runs inside Jupyter notebooks. It is built on top of [pytorch](https://pyto

Ecco is not concerned with training or fine-tuning models. Only exploring and understanding existing pre-trained models.

## Installation

You can install `ecco` either with `pip` or with `conda`.

**with pip**

```sh
pip install ecco
```

**with conda**

```sh
conda install -c conda-forge ecco
```

## Tutorials
- Video: [Take A Look Inside Language Models With Ecco](https://www.youtube.com/watch?v=rHrItfNeuh0). \[<a href="https://colab.research.google.com/github/jalammar/ecco/blob/main/notebooks/Language_Models_and_Ecco_PyData_Khobar.ipynb">Colab Notebook</a>]

Expand Down Expand Up @@ -51,4 +67,4 @@ The [API reference](api/ecco) and the [architecture](architecture) page explain
Having trouble?

- The [Discussion](https://github.com/jalammar/ecco/discussions) board might have some relevant information. If not, you can post your questions there.
- Report bugs at Ecco's [issue tracker](https://github.com/jalammar/ecco/issues)
- Report bugs at Ecco's [issue tracker](https://github.com/jalammar/ecco/issues)
34 changes: 31 additions & 3 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,20 @@
<br />
<br />

[![PyPI Package latest release](https://img.shields.io/pypi/v/ecco.svg)](https://pypi.org/project/ecco)
[![Supported versions](https://img.shields.io/pypi/pyversions/ecco.svg)](https://pypi.org/project/ecco)
<!--- BADGES: START --->
[![GitHub - License](https://img.shields.io/github/license/jalammar/ecco?logo=github&style=flat&color=green)][#github-license]
[![PyPI - Latest Package Version](https://img.shields.io/pypi/v/ecco?logo=pypi&style=flat&color=orange)][#pypi-package]
[![PyPI - Supported Python Versions](https://img.shields.io/pypi/pyversions/ecco?logo=pypi&style=flat&color=blue)][#pypi-package]
[![Conda - Platform](https://img.shields.io/conda/pn/conda-forge/ecco?logo=anaconda&style=flat)][#conda-forge-package]
[![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/ecco?logo=anaconda&style=flat&color=orange)][#conda-forge-package]
[![Docs - GitHub.io](https://img.shields.io/static/v1?logo=readthedocs&style=flat&color=pink&label=docs&message=ecco)][#docs-package]


[#github-license]: https://github.com/jalammar/ecco/blob/main/LICENSE
[#pypi-package]: https://pypi.org/project/ecco/
[#conda-forge-package]: https://anaconda.org/conda-forge/ecco
[#docs-package]: https://ecco.readthedocs.io/
<!--- BADGES: END --->


Ecco is a python library for exploring and explaining Natural Language Processing models using interactive visualizations.
Expand All @@ -27,11 +39,27 @@ Documentation: [ecco.readthedocs.io](https://ecco.readthedocs.io/)
- **Feature attribution** ([IntegratedGradients](https://arxiv.org/abs/1703.01365), [Saliency](https://arxiv.org/abs/1312.6034), [InputXGradient](https://arxiv.org/abs/1412.6815), [DeepLift](https://arxiv.org/abs/1704.02685), [DeepLiftShap](https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html), [GuidedBackprop](https://arxiv.org/abs/1412.6806), [GuidedGradCam](https://arxiv.org/abs/1610.02391), [Deconvolution](https://arxiv.org/abs/1311.2901), and [LRP](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140) via [Captum](https://captum.ai/))
- **Capture neuron activations** in the FFNN layer in the Transformer block
- Identify and **visualize neuron activation patterns** (via Non-negative Matrix Factorization)
- Examine neuron activations via comparisons of activations spaces using [SVCCA](https://arxiv.org/abs/1706.05806), [PWCCA](https://arxiv.org/abs/1806.05759), and [CKA](https://arxiv.org/abs/1905.00414)
- Examine neuron activations via comparisons of activations spaces using [SVCCA](https://arxiv.org/abs/1706.05806), [PWCCA](https://arxiv.org/abs/1806.05759), and [CKA](https://arxiv.org/abs/1905.00414) (See [this video on inspecting neural networks with CCA](https://www.youtube.com/watch?v=u7Dvb_a1D-0))
- Visualizations for:
- Evolution of processing a token through the layers of the model ([Logit lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens))
- Candidate output tokens and their probabilities (at each layer in the model)

## Installation

You can install `ecco` either with `pip` or with `conda`.

**with pip**

```sh
pip install ecco
```

**with conda**

```sh
conda install -c conda-forge ecco
```

## Examples:

### What is the sentiment of this film review?
Expand Down

0 comments on commit ccfd42b

Please sign in to comment.