随机梯度下降法实现

cr-mao · Aug 26, 2024 · 522e6a5 · 522e6a5
1 parent 59e6d6b
commit 522e6a5
Show file tree

Hide file tree

Showing 5 changed files with 302 additions and 230 deletions.
diff --git a/README.md b/README.md
@@ -51,6 +51,7 @@ jupyter notebook ,numpy,pandas,matplotlib
   - [模拟实现梯度下降法(单变量)](machinelearning/gradientDescent/01-GradientDescentSimulations/01-GradientDescentSimulations.ipynb)
   - [在线性回归中实现梯度下降法](machinelearning/gradientDescent/02-ImplementGradientDescentInLinearRegression/02-ImplementGradientDescentInLinearRegression.ipynb)
   - [梯度下降向量化公式及性能和正规方程对比](machinelearning/gradientDescent/03-VectorizeGradientDescent/03-VectorizeGradientDescent.ipynb)
+  - [随机梯队下降法](machinelearning/gradientDescent/04-StochasticGradientDescent/04-StochasticGradientDescent.ipynb)
 
 
 

diff --git a/machinelearning/03梯度下降法.md b/machinelearning/03梯度下降法.md
@@ -141,4 +141,19 @@ def dJ(theta, X_b, y):
 
 
 
+### 随机梯度下降法
+
+上面的梯度是对所有样本ｍ 进行 梯度计算。 如果ｍ 很大，那么 这个算法可能也是很费时的。
+
+随机梯度下降算法(SG), 这里是取一个样本，公式变乘下图所示
+
+![](images/ml_21.png)
+
+
+模拟退火的思想 
+
+学习率 ＝ t0/  （i_iters (迭代次数) +t1)  
+
+经验值t0=5, t1=50
+
 
diff --git a/...elearning/gradientDescent/04-StochasticGradientDescent/04-StochasticGradientDescent.ipynb b/...elearning/gradientDescent/04-StochasticGradientDescent/04-StochasticGradientDescent.ipynb
diff --git a/...rning/gradientDescent/06-Stochastic-Gradient-Descent/06-Stochastic-Gradient-Descent.ipynb b/...rning/gradientDescent/06-Stochastic-Gradient-Descent/06-Stochastic-Gradient-Descent.ipynb
diff --git a/machinelearning/images/ml_21.png b/machinelearning/images/ml_21.png