-
Notifications
You must be signed in to change notification settings - Fork 0
/
diff.html
241 lines (228 loc) · 24.5 KB
/
diff.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>Automatic Differentiation
</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="description" content="Deep.Net machine learning framework"/>
<meta name="author" content="Deep.Net developers"/>
<script src="https://code.jquery.com/jquery-1.8.0.js"></script>
<script src="https://code.jquery.com/ui/1.8.23/jquery-ui.js"></script>
<script src="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/js/bootstrap.min.js"></script>
<link href="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/css/bootstrap-combined.min.css" rel="stylesheet"/>
<link type="text/css" rel="stylesheet" href="http://www.deepml.net/content/style.css" />
<script type="text/javascript" src="http://www.deepml.net/content/tips.js"></script>
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" async
src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
</head>
<body>
<div class="container">
<div class="masthead">
<ul class="nav nav-pills pull-right">
<li><a href="http://github.com/DeepMLNet/DeepNet">github page</a></li>
</ul>
<h3 class="muted"><a href="http://www.deepml.net/index.html">Deep.Net</a></h3>
</div>
<hr />
<div class="row">
<div class="span9" id="main">
<h1><a name="Automatic-Differentiation" class="anchor" href="#Automatic-Differentiation">Automatic Differentiation</a></h1>
<p>Deep.Net performs automatic reverse accumulation differentiation on symbolic expressions to calculate the derivatives of the user-specified model.</p>
<p>In most cases, the differentiation functions are invoked by the optimizer.
However, sometimes it is desired to obtain an expression for the derivative.</p>
<h3><a name="A-sample-expression" class="anchor" href="#A-sample-expression">A sample expression</a></h3>
<p>We define an expression</p>
<p><span class="math">\[\mathbf{f}(\mathbf{x}, \mathbf{y}) = \frac{1}{(\sin x)^2 + y} \,.\]</span></p>
<p>Here we do not use the model builder, because our intent is not to build a full model with parameter and optimizer support.
Instead we define the symbolic sizes directly using the <code>SizeSpec.symbol</code> function and declare variables using <code>Expr.var</code>.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
<span class="l">8: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">open</span> <span class="i">ArrayNDNS</span>
<span class="k">open</span> <span class="i">SymTensor</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs1', 1)" onmouseover="showTip(event, 'fs1', 1)" class="i">n</span> <span class="o">=</span> <span class="i">SizeSpec</span><span class="o">.</span><span class="i">symbol</span> <span class="s">"n"</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs2', 2)" onmouseover="showTip(event, 'fs2', 2)" class="i">x</span> <span class="o">=</span> <span class="i">Expr</span><span class="o">.</span><span class="i">var</span> <span class="s">"x"</span> [<span onmouseout="hideTip(event, 'fs1', 3)" onmouseover="showTip(event, 'fs1', 3)" class="i">n</span>]
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs3', 4)" onmouseover="showTip(event, 'fs3', 4)" class="i">y</span> <span class="o">=</span> <span class="i">Expr</span><span class="o">.</span><span class="i">var</span> <span class="s">"y"</span> [<span onmouseout="hideTip(event, 'fs1', 5)" onmouseover="showTip(event, 'fs1', 5)" class="i">n</span>]
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs4', 6)" onmouseover="showTip(event, 'fs4', 6)" class="i">f</span> <span class="o">=</span> <span class="n">1.0</span> <span class="o">/</span> ((<span onmouseout="hideTip(event, 'fs5', 7)" onmouseover="showTip(event, 'fs5', 7)" class="f">sin</span> <span onmouseout="hideTip(event, 'fs2', 8)" onmouseover="showTip(event, 'fs2', 8)" class="i">x</span>) <span class="o">**</span> <span class="n">2.0</span> <span class="o">+</span> <span onmouseout="hideTip(event, 'fs3', 9)" onmouseover="showTip(event, 'fs3', 9)" class="i">y</span>)
</code></pre></td>
</tr>
</table>
<h2><a name="Computing-derivatives" class="anchor" href="#Computing-derivatives">Computing derivatives</a></h2>
<p>We can now compute the derivatives of <span class="math">\(\mathbf{f}(\mathbf{x}, \mathbf{y})\)</span>.
To do so, we call the <code>Deriv.compute</code> function with the expression we want to differentiate.
The function value and the input variables can be of any dimensionality and shape.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs6', 10)" onmouseover="showTip(event, 'fs6', 10)" class="i">df</span> <span class="o">=</span> <span class="i">Deriv</span><span class="o">.</span><span class="i">compute</span> <span onmouseout="hideTip(event, 'fs4', 11)" onmouseover="showTip(event, 'fs4', 11)" class="i">f</span>
</code></pre></td>
</tr>
</table>
<p>The derivative object <code>df</code> now contains the derivative of <code>f</code> w.r.t. all variables that occur in that expression.</p>
<p>To access a specific derivative use the <code>Deriv.ofVar</code> function on <code>df</code> and pass the requested variable.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs7', 12)" onmouseover="showTip(event, 'fs7', 12)" class="i">dfdx</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs6', 13)" onmouseover="showTip(event, 'fs6', 13)" class="i">df</span> <span class="o">|></span> <span class="i">Deriv</span><span class="o">.</span><span class="i">ofVar</span> <span onmouseout="hideTip(event, 'fs2', 14)" onmouseover="showTip(event, 'fs2', 14)" class="i">x</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs8', 15)" onmouseover="showTip(event, 'fs8', 15)" class="i">dfdy</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs6', 16)" onmouseover="showTip(event, 'fs6', 16)" class="i">df</span> <span class="o">|></span> <span class="i">Deriv</span><span class="o">.</span><span class="i">ofVar</span> <span onmouseout="hideTip(event, 'fs3', 17)" onmouseover="showTip(event, 'fs3', 17)" class="i">y</span>
</code></pre></td>
</tr>
</table>
<p>We now have expressions for <span class="math">\(\partial f / \partial x\)</span> and <span class="math">\(\partial f / \partial y\)</span>.</p>
<h2><a name="Evaluating-the-expressions" class="anchor" href="#Evaluating-the-expressions">Evaluating the expressions</a></h2>
<p>We evaluate the expressions using the (slow) host interpreter.
Because we do not use the model builder, we have to invoke <code>Func.make</code> directly to create a callable function from an expression.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs9', 18)" onmouseover="showTip(event, 'fs9', 18)" class="i">cmplr</span> <span class="o">=</span> <span class="i">DevHost</span><span class="o">.</span><span class="i">Compiler</span>, <span class="i">CompileEnv</span><span class="o">.</span><span class="i">empty</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs10', 19)" onmouseover="showTip(event, 'fs10', 19)" class="f">fnF</span> <span class="o">=</span> <span class="i">Func</span><span class="o">.</span><span class="i">make</span> <span onmouseout="hideTip(event, 'fs9', 20)" onmouseover="showTip(event, 'fs9', 20)" class="i">cmplr</span> <span onmouseout="hideTip(event, 'fs4', 21)" onmouseover="showTip(event, 'fs4', 21)" class="i">f</span> <span class="o">|></span> <span class="i">arg2</span> <span onmouseout="hideTip(event, 'fs2', 22)" onmouseover="showTip(event, 'fs2', 22)" class="i">x</span> <span onmouseout="hideTip(event, 'fs3', 23)" onmouseover="showTip(event, 'fs3', 23)" class="i">y</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs11', 24)" onmouseover="showTip(event, 'fs11', 24)" class="f">fnDfdx</span> <span class="o">=</span> <span class="i">Func</span><span class="o">.</span><span class="i">make</span> <span onmouseout="hideTip(event, 'fs9', 25)" onmouseover="showTip(event, 'fs9', 25)" class="i">cmplr</span> <span onmouseout="hideTip(event, 'fs7', 26)" onmouseover="showTip(event, 'fs7', 26)" class="i">dfdx</span> <span class="o">|></span> <span class="i">arg2</span> <span onmouseout="hideTip(event, 'fs2', 27)" onmouseover="showTip(event, 'fs2', 27)" class="i">x</span> <span onmouseout="hideTip(event, 'fs3', 28)" onmouseover="showTip(event, 'fs3', 28)" class="i">y</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs12', 29)" onmouseover="showTip(event, 'fs12', 29)" class="f">fnDfdy</span> <span class="o">=</span> <span class="i">Func</span><span class="o">.</span><span class="i">make</span> <span onmouseout="hideTip(event, 'fs9', 30)" onmouseover="showTip(event, 'fs9', 30)" class="i">cmplr</span> <span onmouseout="hideTip(event, 'fs8', 31)" onmouseover="showTip(event, 'fs8', 31)" class="i">dfdy</span> <span class="o">|></span> <span class="i">arg2</span> <span onmouseout="hideTip(event, 'fs2', 32)" onmouseover="showTip(event, 'fs2', 32)" class="i">x</span> <span onmouseout="hideTip(event, 'fs3', 33)" onmouseover="showTip(event, 'fs3', 33)" class="i">y</span>
</code></pre></td>
</tr>
</table>
<p><code>Func.make</code> expects two arguments: the first is the compiler (or interpreter) to use to transform the expression into a function.
We use the host interpreter (<code>DevHost.Compiler</code>) without any optional options (<code>CompileEnv.empty</code>).
The second argument is the expression to compile.</p>
<p><code>Func.make</code> returns a function taking a variable environment <code>VarEnvT</code> (essentially a map from variable names to values) and returning a tensor value.
To avoid having to build the variable environment explicitly, we use the <code>arg2</code> function that modifies the resulting function to take two tensor arguments instead.</p>
<p>We can now generate some test values for the variables <span class="math">\(\mathbf{x}\)</span> and <span class="math">\(\mathbf{y}\)</span>.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs13', 34)" onmouseover="showTip(event, 'fs13', 34)" class="i">valX</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs14', 35)" onmouseover="showTip(event, 'fs14', 35)" class="i">seq</span> { <span class="n">0.1</span> <span class="o">..</span> <span class="n">0.2</span> <span class="o">..</span> <span class="n">1.0</span> } <span class="o">|></span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ofSeq</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs15', 36)" onmouseover="showTip(event, 'fs15', 36)" class="i">valY</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs14', 37)" onmouseover="showTip(event, 'fs14', 37)" class="i">seq</span> { <span class="n">1.1</span> <span class="o">..</span> <span class="n">0.2</span> <span class="o">..</span> <span class="n">2.0</span> } <span class="o">|></span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ofSeq</span>
</code></pre></td>
</tr>
</table>
<p>And compute the function values as well as derivatives.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs16', 38)" onmouseover="showTip(event, 'fs16', 38)" class="f">printfn</span> <span class="s">"Using x = </span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs13', 39)" onmouseover="showTip(event, 'fs13', 39)" class="i">valX</span>
<span onmouseout="hideTip(event, 'fs16', 40)" onmouseover="showTip(event, 'fs16', 40)" class="f">printfn</span> <span class="s">"Using y = </span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs15', 41)" onmouseover="showTip(event, 'fs15', 41)" class="i">valY</span>
<span onmouseout="hideTip(event, 'fs16', 42)" onmouseover="showTip(event, 'fs16', 42)" class="f">printfn</span> <span class="s">"f(x, y) = </span><span class="pf">%A</span><span class="s">"</span> (<span onmouseout="hideTip(event, 'fs10', 43)" onmouseover="showTip(event, 'fs10', 43)" class="f">fnF</span> <span onmouseout="hideTip(event, 'fs13', 44)" onmouseover="showTip(event, 'fs13', 44)" class="i">valX</span> <span onmouseout="hideTip(event, 'fs15', 45)" onmouseover="showTip(event, 'fs15', 45)" class="i">valY</span>)
<span onmouseout="hideTip(event, 'fs16', 46)" onmouseover="showTip(event, 'fs16', 46)" class="f">printfn</span> <span class="s">""</span>
<span onmouseout="hideTip(event, 'fs16', 47)" onmouseover="showTip(event, 'fs16', 47)" class="f">printfn</span> <span class="s">"J_x f = </span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> (<span onmouseout="hideTip(event, 'fs11', 48)" onmouseover="showTip(event, 'fs11', 48)" class="f">fnDfdx</span> <span onmouseout="hideTip(event, 'fs13', 49)" onmouseover="showTip(event, 'fs13', 49)" class="i">valX</span> <span onmouseout="hideTip(event, 'fs15', 50)" onmouseover="showTip(event, 'fs15', 50)" class="i">valY</span>)
<span onmouseout="hideTip(event, 'fs16', 51)" onmouseover="showTip(event, 'fs16', 51)" class="f">printfn</span> <span class="s">"J_y f = </span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> (<span onmouseout="hideTip(event, 'fs12', 52)" onmouseover="showTip(event, 'fs12', 52)" class="f">fnDfdy</span> <span onmouseout="hideTip(event, 'fs13', 53)" onmouseover="showTip(event, 'fs13', 53)" class="i">valX</span> <span onmouseout="hideTip(event, 'fs15', 54)" onmouseover="showTip(event, 'fs15', 54)" class="i">valY</span>)
</code></pre></td>
</tr>
</table>
<p>This prints</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l"> 1: </span>
<span class="l"> 2: </span>
<span class="l"> 3: </span>
<span class="l"> 4: </span>
<span class="l"> 5: </span>
<span class="l"> 6: </span>
<span class="l"> 7: </span>
<span class="l"> 8: </span>
<span class="l"> 9: </span>
<span class="l">10: </span>
<span class="l">11: </span>
<span class="l">12: </span>
<span class="l">13: </span>
<span class="l">14: </span>
<span class="l">15: </span>
<span class="l">16: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">Using</span> <span class="i">x</span> <span class="o">=</span> [ <span class="n">0.1000</span> <span class="n">0.3000</span> <span class="n">0.5000</span> <span class="n">0.7000</span> <span class="n">0.9000</span>]
<span class="i">Using</span> <span class="i">y</span> <span class="o">=</span> [ <span class="n">1.1000</span> <span class="n">1.3000</span> <span class="n">1.5000</span> <span class="n">1.7000</span> <span class="n">1.9000</span>]
<span class="i">f</span>(<span class="i">x</span>, <span class="i">y</span>) <span class="o">=</span> [ <span class="n">0.9009</span> <span class="n">0.7208</span> <span class="n">0.5781</span> <span class="n">0.4728</span> <span class="n">0.3978</span>]
<span class="i">J_x</span> <span class="i">f</span> <span class="o">=</span>
[[ <span class="o">-</span><span class="n">0.1613</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.2934</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.2812</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.2203</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.1541</span>]]
<span class="i">J_y</span> <span class="i">f</span> <span class="o">=</span>
[[ <span class="o">-</span><span class="n">0.8117</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.5196</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.3342</span> <span class="n">0.0000</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.2235</span> <span class="n">0.0000</span>]
[ <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="n">0.0000</span> <span class="o">-</span><span class="n">0.1583</span>]]
</code></pre></td>
</tr>
</table>
<p>As expected, the Jacobians are diagonal because we computed the derivatives of an element-wise function.</p>
<h2><a name="Meaning-of-the-derivative-matrix" class="anchor" href="#Meaning-of-the-derivative-matrix">Meaning of the derivative matrix</a></h2>
<p>The derivative is always returned in the shape of a Jacobian, i.e. the derivative is always a matrix.
If <span class="math">\(\mathbf{f}\)</span> and <span class="math">\(\mathbf{x}\)</span> are vectors, this means</p>
<p><span class="math">\[(J_\mathbf{x} \mathbf{f})_{ij} = \frac{\partial f_i}{\partial x_j}\]</span></p>
<p>and <span class="math">\(J_\mathbf{x} \mathbf{f}\)</span> will be an <span class="math">\(n \times m\)</span> matrix where <span class="math">\(n\)</span> is the length of <span class="math">\(\mathbf{f}\)</span> and <span class="math">\(m\)</span> is the length of <span class="math">\(\mathbf{y}\)</span>.</p>
<p>If the function or an argument has the value of a matrix, the Jacobian will still be a matrix.
Consider, for instance, that <span class="math">\(X\)</span> is a <span class="math">\(k \times l\)</span> matrix and <span class="math">\(G(X)\)</span> is an <span class="math">\(n \times m\)</span> matrix-valued function.
Then the Jacobian <span class="math">\(J_G X\)</span> computed by Deep.Net will be a matrix of shape <span class="math">\(k l \times n m\)</span>.
The derivative is computed as if <span class="math">\(G\)</span> and <span class="math">\(X\)</span> were flattened into vectors (using row-major order).
Thus the derivatives of the individual elements are given by</p>
<p><span class="math">\[\frac{\partial G_{i,j}}{\partial X_{v,w}} = (J_\mathbf{X} \mathbf{G})_{im + j, vl + w} \]</span></p>
<p>This is also true for higher-order tensors, i.e. the derivative will be computed as if any higher order tensor were flattened into a vector using row-major order.
Likewise, a scalar-valued function will produce a Jacobian matrix with one row.</p>
<h3><a name="Chain-rule" class="anchor" href="#Chain-rule">Chain rule</a></h3>
<p>Using matrices to store the derivatives has the advantage that the <a href="https://en.wikipedia.org/wiki/Chain_rule">chain rule</a> is always valid.</p>
<p>Consider a vector-valued function <span class="math">\(\mathbf{f} (G (\mathbf{x}))\)</span>.
Given the derivatives <span class="math">\(J_\mathbf{x} G\)</span> and <span class="math">\(J_G \mathbf{f}\)</span> we can compute the derivative <span class="math">\(J_\mathbf{x} \mathbf{f}\)</span> by</p>
<p><span class="math">\[J_\mathbf{x} \mathbf{f} = J_G \mathbf{f} \cdot J_\mathbf{x} G \]</span></p>
<p>where <span class="math">\(\cdot\)</span> represent the matrix dot product.</p>
<div class="tip" id="fs1">val n : obj<br /><br />Full name: Diff.n</div>
<div class="tip" id="fs2">val x : float<br /><br />Full name: Diff.x</div>
<div class="tip" id="fs3">val y : float<br /><br />Full name: Diff.y</div>
<div class="tip" id="fs4">val f : float<br /><br />Full name: Diff.f</div>
<div class="tip" id="fs5">val sin : value:'T -> 'T (requires member Sin)<br /><br />Full name: Microsoft.FSharp.Core.Operators.sin</div>
<div class="tip" id="fs6">val df : obj<br /><br />Full name: Diff.df</div>
<div class="tip" id="fs7">val dfdx : obj<br /><br />Full name: Diff.dfdx</div>
<div class="tip" id="fs8">val dfdy : obj<br /><br />Full name: Diff.dfdy</div>
<div class="tip" id="fs9">val cmplr : obj * obj<br /><br />Full name: Diff.cmplr</div>
<div class="tip" id="fs10">val fnF : (obj -> obj -> obj)<br /><br />Full name: Diff.fnF</div>
<div class="tip" id="fs11">val fnDfdx : (obj -> obj -> obj)<br /><br />Full name: Diff.fnDfdx</div>
<div class="tip" id="fs12">val fnDfdy : (obj -> obj -> obj)<br /><br />Full name: Diff.fnDfdy</div>
<div class="tip" id="fs13">val valX : obj<br /><br />Full name: Diff.valX</div>
<div class="tip" id="fs14">Multiple items<br />val seq : sequence:seq<'T> -> seq<'T><br /><br />Full name: Microsoft.FSharp.Core.Operators.seq<br /><br />--------------------<br />type seq<'T> = System.Collections.Generic.IEnumerable<'T><br /><br />Full name: Microsoft.FSharp.Collections.seq<_></div>
<div class="tip" id="fs15">val valY : obj<br /><br />Full name: Diff.valY</div>
<div class="tip" id="fs16">val printfn : format:Printf.TextWriterFormat<'T> -> 'T<br /><br />Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn</div>
</div>
<div class="span3">
<!-- <img src="http://www.deepml.net/img/logo.png" alt="Deep.Net logo" style="width:150px;margin:10px" /> -->
<ul class="nav nav-list" id="menu" style="margin-top: 20px;">
<li class="nav-header">Deep.Net</li>
<li><a href="http://www.deepml.net/index.html">Home page</a></li>
<li class="divider"></li>
<li><a href="http://nuget.org/packages/DeepNet">Get Library via NuGet</a></li>
<li><a href="http://github.com/DeepMLNet/DeepNet">Source Code on GitHub</a></li>
<li><a href="http://www.deepml.net/release-notes.html">Release Notes</a></li>
<li class="nav-header">Basics</li>
<li><a href="http://www.deepml.net/tensor.html">Working with Tensors</a></li>
<li><a href="http://www.deepml.net/model.html">Model Definition</a></li>
<li><a href="http://www.deepml.net/components.html">Model Components</a></li>
<li><a href="http://www.deepml.net/dataset.html">Dataset Handling</a></li>
<li><a href="http://www.deepml.net/training.html">Training</a></li>
<li class="nav-header">Advanced</li>
<li><a href="http://www.deepml.net/diff.html">Automatic Differentiation</a></li>
<li class="nav-header">Documentation</li>
<li><a href="http://www.deepml.net/reference/index.html">API Reference</a></li>
</ul>
</div>
</div>
</div>
<a href="http://github.com/DeepMLNet/DeepNet"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" alt="Fork me on GitHub"/></a>
</body>
</html>