Skip to content

Commit

Permalink
Consistency
Browse files Browse the repository at this point in the history
  • Loading branch information
christiangnrd committed Dec 4, 2024
1 parent 9a65a19 commit 0a7b233
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 14 deletions.
14 changes: 7 additions & 7 deletions docs/src/tutorials/linear_regression.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Flux is a pure Julia ML stack that allows you to build predictive models. Here a
- Build a model with configurable parameters to make predictions
- Iteratively train the model by tweaking the parameters to improve predictions
- Verify your model

Under the hood, Flux uses a technique called automatic differentiation to take gradients that help improve predictions. Flux is also fully written in Julia so you can easily replace any layer of Flux with your own code to improve your understanding or satisfy special requirements.

The following page contains a step-by-step walkthrough of the linear regression algorithm in `Julia` using `Flux`! We will start by creating a simple linear regression model for dummy data and then move on to a real dataset. The first part would involve writing some parts of the model on our own, which will later be replaced by `Flux`.
Expand Down Expand Up @@ -115,7 +115,7 @@ julia> custom_loss(W, b, x, y)

Calling the loss function on our `x`s and `y`s shows how far our predictions (`ŷ`) are from the real labels. More precisely, it calculates the sum of the squares of residuals and divides it by the total number of data points.

We have successfully defined our model and the loss function, but surprisingly, we haven't used `Flux` anywhere till now. Let's see how we can write the same code using `Flux`.
We have successfully defined our model and the loss function, but surprisingly, we haven't used `Flux` anywhere till now. Let's see how we can write the same code using `Flux`.

```jldoctest linear_regression_simple
julia> flux_model = Dense(1 => 1)
Expand Down Expand Up @@ -214,8 +214,8 @@ The loss went down! This means that we successfully trained our model for one ep
Let's plug our super training logic inside a function and test it again -

```jldoctest linear_regression_simple; filter = r"[+-]?([0-9]*[.])?[0-9]+(f[+-]*[0-9])?"
julia> function train_custom_model!(f_loss, weights, biases, x_train, y_train)
dLdW, dLdb, _, _ = gradient(f_loss, weights, biases, x_train, y_train)
julia> function train_custom_model!(f_loss, weights, biases, X, y)
dLdW, dLdb, _, _ = gradient(f_loss, weights, biases, X, y)
@. weights = weights - 0.1 * dLdW
@. biases = biases - 0.1 * dLdb
end;
Expand Down Expand Up @@ -266,7 +266,7 @@ julia> using Flux, Statistics, MLDatasets, DataFrames
```

## Gathering real data
Let's start by initializing our dataset. We will be using the [`BostonHousing`](https://juliaml.github.io/MLDatasets.jl/stable/datasets/misc/#MLDatasets.BostonHousing) dataset consisting of `506` data points. Each of these data points has `13` features and a corresponding label, the house's price. The `x`s are still mapped to a single `y`, but now, a single `x` data point has 13 features.
Let's start by initializing our dataset. We will be using the [`BostonHousing`](https://juliaml.github.io/MLDatasets.jl/stable/datasets/misc/#MLDatasets.BostonHousing) dataset consisting of `506` data points. Each of these data points has `13` features and a corresponding label, the house's price. The `x`s are still mapped to a single `y`, but now, a single `x` data point has 13 features.

```jldoctest linear_regression_complex
julia> dataset = BostonHousing();
Expand Down Expand Up @@ -385,9 +385,9 @@ The loss is not as small as the loss of the training data, but it looks good! Th

---

Summarising this tutorial, we started by generating a random yet correlated dataset for our `custom model`. We then saw how a simple linear regression model could be built with and without `Flux`, and how they were almost identical.
Summarising this tutorial, we started by generating a random yet correlated dataset for our `custom model`. We then saw how a simple linear regression model could be built with and without `Flux`, and how they were almost identical.

Next, we trained the model by manually writing down the Gradient Descent algorithm and optimising the loss. We also saw how `Flux` provides various wrapper functionalities and keeps the API extremely intuitive and simple for the users.
Next, we trained the model by manually writing down the Gradient Descent algorithm and optimising the loss. We also saw how `Flux` provides various wrapper functionalities and keeps the API extremely intuitive and simple for the users.

After getting familiar with the basics of `Flux` and `Julia`, we moved ahead to build a machine learning model for a real dataset. We repeated the exact same steps, but this time with a lot more features and data points, and by harnessing `Flux`'s full capabilities. In the end, we developed a training loop that was smarter than the hardcoded one and ran the model on our normalised dataset to conclude the tutorial.

Expand Down
14 changes: 7 additions & 7 deletions docs/src/tutorials/logistic_regression.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Logistic Regression

The following page contains a step-by-step walkthrough of the logistic regression algorithm in Julia using Flux. We will then create a simple logistic regression model without any usage of Flux and compare the different working parts with Flux's implementation.
The following page contains a step-by-step walkthrough of the logistic regression algorithm in Julia using Flux. We will then create a simple logistic regression model without any usage of Flux and compare the different working parts with Flux's implementation.

Let's start by importing the required Julia packages.

Expand Down Expand Up @@ -141,7 +141,7 @@ julia> flux_model = Chain(Dense(4 => 3), softmax)
Chain(
Dense(4 => 3), # 15 parameters
softmax,
)
)
```

A [`Dense(4 => 3)`](@ref Dense) layer denotes a layer with four inputs (four features in every data point) and three outputs (three classes or labels). This layer is the same as the mathematical model defined by us above. Under the hood, Flux too calculates the output using the same expression, but we don't have to initialize the parameters ourselves this time, instead Flux does it for us.
Expand Down Expand Up @@ -313,8 +313,8 @@ julia> custom_loss(W, b, x, custom_y_onehot)
The loss went down! Let's plug our super training logic inside a function.

```jldoctest logistic_regression
julia> function train_custom_model!(f_loss, weights, biases, x, y)
dLdW, dLdb, _, _ = gradient(f_loss, weights, biases, x, y)
julia> function train_custom_model!(f_loss, weights, biases, X, y)
dLdW, dLdb, _, _ = gradient(f_loss, weights, biases, X, y)
weights .= weights .- 0.1 .* dLdW
biases .= biases .- 0.1 .* dLdb
end;
Expand All @@ -327,7 +327,7 @@ julia> for i = 1:500
train_custom_model!(custom_loss, W, b, x, custom_y_onehot);
custom_accuracy(W, b, x, y) >= 0.98 && break
end
julia> @show custom_accuracy(W, b, x, y);
custom_accuracy(W, b, x, y) = 0.98
```
Expand All @@ -347,8 +347,8 @@ We can write a similar-looking training loop for our `flux_model` and train it s
julia> flux_loss(flux_model, x, flux_y_onehot)
1.215731131385928
julia> function train_flux_model!(f_loss, model, x, y)
dLdm, _, _ = gradient(f_loss, model, x, y)
julia> function train_flux_model!(f_loss, model, X, y)
dLdm, _, _ = gradient(f_loss, model, X, y)
@. model[1].weight = model[1].weight - 0.1 * dLdm[:layers][1][:weight]
@. model[1].bias = model[1].bias - 0.1 * dLdm[:layers][1][:bias]
end;
Expand Down

0 comments on commit 0a7b233

Please sign in to comment.