Skip to content

Commit

Permalink
FEAT: Add spectral decomposition demo and documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@MKaczkow
MKaczkow committed Feb 14, 2025
1 parent 157a7cb commit b362726
Show file tree
Hide file tree
Showing 3 changed files with 285 additions and 0 deletions.
37 changes: 37 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,40 @@ Repo for learning tools and ideas of ML/AI
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
[![CI](https://github.com/MKaczkow/ml_concepts/actions/workflows/ci.yml/badge.svg)](https://github.com/MKaczkow/ml_concepts/actions/workflows/ci.yml)

---

## Contents
* [graph neural networks](./ideas/graph_neural_networks)
* [lstm](./ideas/lstm)
* [metric_learning](./ideas/metric_learning)
* [seq2seq](./ideas/seq2seq)
* [ssl](./ideas/ssl)
* [transformers](./ideas/transformers_from_scratch)
* [vae](./ideas/vae)
* [xai](./ideas/xai)
* [xbgoost](./ideas/xgboost)
* [eye biometrics](./tasks/eye_biometrics/)
* [optimal urban route choice](./tasks/optimal_urban_route_choice/)
* [sentiment analysis](./tasks/sentiment_analysis/)
* [speech analysis](./tasks/speech_analysis/)
* [style transfer](./tasks/style_transfer/)
* [timeseries](./tasks/timeseries/)
* [cross entropy](./theory/cross_entropy)
* [einsum](./theory/einsum)
* [kulback leibler divergence](./theory/kulback_leibler_divergence)
* [lipschitz normalization](./theory/lipschitz_normalization)
* [logits](./theory/logits)
* [semantic search](./theory/semantic_search/)
* [softmax vs argmax](./theory/softmax_vs_argmax/)
* [spectral decomposition](./theory/spectral_decomposition/)
* [transformer](./theory/transformer/)
* [vector similarity](./theory/vector_similarity/)
* [wasserstein distance](./theory/wasserstein_distance/)
* [apache airflow](./tools/apache_airflow)
* [data visualisation](./tools/data_visualisation)
* [gradio demos](./tools/gradio_demos)
* [pyspark](./tools/pyspark/)
* [pytorch lightning](./tools/pytorch_lightning)
* [rasa framework](./tools/rasa_framework)
* [word2vec embeddings](./tools/word2vec_embeddings/)
239 changes: 239 additions & 0 deletions theory/spectral_decomposition/DEMO.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from numpy import linalg as LA"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[1 0]\n",
" [0 1]]\n"
]
}
],
"source": [
"A = np.array([[1, 0], [0, 1]])\n",
"print(A)"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"eigenvalues\n",
" [1. 1.]\n",
"eigenvectors\n",
" [[1. 0.]\n",
" [0. 1.]]\n"
]
}
],
"source": [
"eigenvalues, eigenvectors = LA.eig(A)\n",
"print(\"eigenvalues\\n\", eigenvalues)\n",
"print(\"eigenvectors\\n\", eigenvectors)"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"P\n",
" [[1. 0.]\n",
" [0. 1.]]\n",
"D\n",
" [[1. 0.]\n",
" [0. 1.]]\n",
"P_inv\n",
" [[1. 0.]\n",
" [0. 1.]]\n"
]
}
],
"source": [
"# P -> composed of eigenvectors\n",
"# D -> composed of eigenvalues\n",
"P = eigenvectors\n",
"D = np.diag(eigenvalues)\n",
"P_inv = np.linalg.inv(P)\n",
"print(\"P\\n\", P)\n",
"print(\"D\\n\", D)\n",
"print(\"P_inv\\n\", P_inv)"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1., 0.],\n",
" [0., 1.]])"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P @ D @ P_inv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## From chapter 25"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[-3 5]\n",
" [ 4 -2]]\n"
]
}
],
"source": [
"A = np.array([[-3, 5], [4, -2]])\n",
"print(A)"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"eigenvalues\n",
" [-7. 2.]\n",
"eigenvectors\n",
" [[-0.78086881 -0.70710678]\n",
" [ 0.62469505 -0.70710678]]\n"
]
}
],
"source": [
"eigenvalues, eigenvectors = LA.eig(A)\n",
"print(\"eigenvalues\\n\", eigenvalues)\n",
"print(\"eigenvectors\\n\", eigenvectors)"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"P\n",
" [[-0.78086881 -0.70710678]\n",
" [ 0.62469505 -0.70710678]]\n",
"D\n",
" [[-7. 0.]\n",
" [ 0. 2.]]\n",
"P_inv\n",
" [[-0.71145825 0.71145825]\n",
" [-0.62853936 -0.7856742 ]]\n"
]
}
],
"source": [
"# P -> composed of eigenvectors\n",
"# D -> composed of eigenvalues\n",
"P = eigenvectors\n",
"D = np.diag(eigenvalues)\n",
"P_inv = np.linalg.inv(P)\n",
"print(\"P\\n\", P)\n",
"print(\"D\\n\", D)\n",
"print(\"P_inv\\n\", P_inv)"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[-3., 5.],\n",
" [ 4., -2.]])"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P @ D @ P_inv"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
9 changes: 9 additions & 0 deletions theory/spectral_decomposition/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Spectral Decomposition

## General
* given matrix A find matrix P and D such that `A = PDP^-1`
* many use-cases, like PCA, signal processing, etc.

## References
* [some write up](https://zief0002.github.io/matrix-algebra/spectral-decompostion.html)
* [stackexchange](https://math.stackexchange.com/questions/2568305/find-the-spectral-decomposition-of-a)

0 comments on commit b362726

Please sign in to comment.