Boltzmann Machines

A Boltzmann machine is a type of stochastic recurrent neural network (and Markov Random Field) invented by Geoffrey Hinton and Terry Sejnowski in 1985. Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets. They were one of the first examples of a neural network capable of learning internal representations, and are able to represent and (given sufficient time) solve difficult combinatoric problems.

They are theoretically intriguing because of the locality and Hebbian nature of their training algorithm, and because of their parallelism and the resemblance of their dynamics to simple physical processes. Due to a number of issues discussed below, Boltzmann machines with unconstrained connectivity have not proven useful for practical problems in machine learning or inference, but if the connectivity is properly constrained, the learning can be made efficient enough to be useful for practical problems.They are named after the Boltzmann distribution in statistical mechanics, which is used in their sampling function.

Boltzmann Machines to create a Recomender System

Code

Used Pytorch Library for Building this system
Install Ubuntu on a Virtual Machine --> Build an Anaconda Environment with required Libraries for the same

**Creating the architecture of the Neural Network **
class RBM():
def __init__(self, nv, nh):
#nv - no of visible variable
#nh - no of hidden variable
self.W = torch.randn(nh, nv)
self.a = torch.randn(1, nh)
self.b = torch.randn(1, nv)

def sample_h(self,x):
wx = torch.nn(x, self.W.t())
activation = wx + self.a.expand_as(wx)
p_h_given_v = torch.sigmoid(activation)
return p_h_given_v, torch.bernoulli(p_h_given_v)

def sample_v(self,y):
wx = torch.nn(y, self.W)
activation = wy + self.b.expand_as(wy)
p_v_given_h = torch.sigmoid(activation)
return p_v_given_h, torch.bernoulli(p_v_given_h)

def train(self, v0, vk, ph0, phk):
self.W += torch.nn(v0.t(), ph0) - torch.nn(vk.t(), phk)
self.b += torch.sum((v0 - vk), 0)
self.a += torch.sum((ph0 - phk), 0)

Output

Train Loss = 0.2518
Test Loss = 0.2429