-
Notifications
You must be signed in to change notification settings - Fork 0
/
tensor.html
892 lines (883 loc) · 84.3 KB
/
tensor.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>Working with Tensors
</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="description" content="Deep.Net machine learning framework"/>
<meta name="author" content="Deep.Net developers"/>
<script src="https://code.jquery.com/jquery-1.8.0.js"></script>
<script src="https://code.jquery.com/ui/1.8.23/jquery-ui.js"></script>
<script src="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/js/bootstrap.min.js"></script>
<link href="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/css/bootstrap-combined.min.css" rel="stylesheet"/>
<link type="text/css" rel="stylesheet" href="http://www.deepml.net/content/style.css" />
<script type="text/javascript" src="http://www.deepml.net/content/tips.js"></script>
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" async
src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
</head>
<body>
<div class="container">
<div class="masthead">
<ul class="nav nav-pills pull-right">
<li><a href="http://github.com/DeepMLNet/DeepNet">github page</a></li>
</ul>
<h3 class="muted"><a href="http://www.deepml.net/index.html">Deep.Net</a></h3>
</div>
<hr />
<div class="row">
<div class="span9" id="main">
<h1><a name="Working-with-Tensors" class="anchor" href="#Working-with-Tensors">Working with Tensors</a></h1>
<p>In Deep.Net a <em>tensor</em> is an n-dimensional array of an arbitrary data type (for example <code>single</code>, <code>double</code> or <code>System.Numerics.Complex</code>).
Tensors are implemented by the <code>ArrayNDT<'T></code> class and its derivatives <code>ArrayNDHostT</code> (for tensors stored in host memory) and <code>ArrayNDCudaT</code> (for tensors stored in CUDA GPU memory).
The Deep.Net tensor provides functionality similar to <a href="http://docs.scipy.org/doc/numpy-1.10.0/reference/arrays.html">Numpy's Ndarray</a> and <a href="http://www.mathworks.com/help/matlab/matrices-and-arrays.html">MATLAB arrays</a>, including vector-wise operations, reshaping, slicing, broadcasting, masked assignment, reduction operations and BLAS operations.
The API for host and GPU stored tensors is equal, thus a program can make use of GPU accelerated operations without much porting effort.</p>
<p>You can run this example by executing <code>FsiAnyCPU.exe docs\content\tensor.fsx</code> after cloning the Deep.Net repository.
You can move your mouse over any symbol in the code samples to see the full signature.</p>
<h2><a name="Architecture" class="anchor" href="#Architecture">Architecture</a></h2>
<p>To work with the tensor library, open the <code>ArrayNDNS</code> namespace.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">open</span> <span class="i">ArrayNDNS</span>
</code></pre></td>
</tr>
</table>
<p>The tensor library consists of three primary modules: ArrayND, ArrayNDHost, ArrayNDCuda.
ArrayND contains functions to work with existing tensors (regardless of their storage location) such as <code>reshape</code>, <code>sum</code>, and <code>copy</code>.
ArrayNDHost contains functions to create new tensors in host memory, for example <code>zeros</code>, <code>ones</code> and <code>ofList</code>.
ArrayNDCuda contains functions to create new tensors in CUDA GPU memory and to facilitate the transfer of tensors between host and GPU memory, for example <code>zeros</code>, <code>toDev</code> and <code>fromDev</code>.</p>
<p>A tensor stored in host memory is of type <code>ArrayNDHostT<'T></code> where <code>'T</code> is the contained data type.
Correspondingly a tensor stored in CUDA GPU memory is of type <code>ArrayNDCudaT<'T></code>.
Both types are derived from <code>ArrayNDT<'T></code>.</p>
<h2><a name="Creating-tensors" class="anchor" href="#Creating-tensors">Creating tensors</a></h2>
<p>Let us create a simple <span class="math">\(7 \times 5\)</span> matrix, i.e. two-dimensional tensor, in host memory.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs1', 1)" onmouseover="showTip(event, 'fs1', 1)" class="i">a</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">7</span>; <span class="n">5</span>] (<span class="k">fun</span> [<span class="i">i</span>; <span class="i">j</span>] <span class="k">-></span> <span class="n">5.0</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs2', 2)" onmouseover="showTip(event, 'fs2', 2)" class="i">float</span> <span class="i">i</span> <span class="o">+</span> <span onmouseout="hideTip(event, 'fs2', 3)" onmouseover="showTip(event, 'fs2', 3)" class="i">float</span> <span class="i">j</span>)
</code></pre></td>
</tr>
</table>
<p>The first argument to the <code>ArrayNDHost.initIndexed</code> function is an <a href="https://en.wikibooks.org/wiki/F_Sharp_Programming/Lists">F# list</a> specifies the shape of the tensor.
The second argument is a function that takes the n-dimensional index (zero-based) of an entry and computes its initial value; here we use the formula <span class="math">\(5i + j\)</span> where <span class="math">\(i\)</span> is the row and <span class="math">\(j\)</span> is the column of the matrix.
The data type (here float) is automatically inferred from the return type of the initialization function.
By default row-major storage (C storage order) is used.</p>
<p>The resulting tensor has the following entries.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0.0000</span> <span class="n">1.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span> <span class="n">4.0000</span>]
[ <span class="n">5.0000</span> <span class="n">6.0000</span> <span class="n">7.0000</span> <span class="n">8.0000</span> <span class="n">9.0000</span>]
[ <span class="n">10.0000</span> <span class="n">11.0000</span> <span class="n">12.0000</span> <span class="n">13.0000</span> <span class="n">14.0000</span>]
[ <span class="n">15.0000</span> <span class="n">16.0000</span> <span class="n">17.0000</span> <span class="n">18.0000</span> <span class="n">19.0000</span>]
[ <span class="n">20.0000</span> <span class="n">21.0000</span> <span class="n">22.0000</span> <span class="n">23.0000</span> <span class="n">24.0000</span>]
[ <span class="n">25.0000</span> <span class="n">26.0000</span> <span class="n">27.0000</span> <span class="n">28.0000</span> <span class="n">29.0000</span>]
[ <span class="n">30.0000</span> <span class="n">31.0000</span> <span class="n">32.0000</span> <span class="n">33.0000</span> <span class="n">34.0000</span>]]
</code></pre></td>
</tr>
</table>
<p>We can also create a tensor filled with zeros using the <code>ArrayNDHost.zeros</code> function.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs3', 4)" onmouseover="showTip(event, 'fs3', 4)" class="i">z1</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs4', 5)" onmouseover="showTip(event, 'fs4', 5)" class="i">int</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">zeros</span> [<span class="n">3</span>]
</code></pre></td>
</tr>
</table>
<p>In this case we must specify the data type (int) explicitly, since there is no information in the function call to infer it automatically.</p>
<p>We can use the <code>ArrayNDHost.ones</code> function to obtain a tensor filled with ones.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs5', 6)" onmouseover="showTip(event, 'fs5', 6)" class="i">o1</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs6', 7)" onmouseover="showTip(event, 'fs6', 7)" class="i">single</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ones</span> [<span class="n">3</span>]
</code></pre></td>
</tr>
</table>
<p>The <code>ArrayNDHost.identity</code> function creates an identity matrix of the given size.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs7', 8)" onmouseover="showTip(event, 'fs7', 8)" class="i">id1</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs2', 9)" onmouseover="showTip(event, 'fs2', 9)" class="i">float</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">identity</span> <span class="n">6</span>
</code></pre></td>
</tr>
</table>
<p>This created a <span class="math">\(6 \times 6\)</span> identity matrix.</p>
<p>A tensor of rank zero can be created with the <code>ArrayNDHost.scalar</code> function.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs8', 10)" onmouseover="showTip(event, 'fs8', 10)" class="i">s1</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">scalar</span> <span class="n">33.2</span>
</code></pre></td>
</tr>
</table>
<p>The numeric value of a zero-rank tensor can be extracted using the <code>ArrayND.value</code> function.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs9', 11)" onmouseover="showTip(event, 'fs9', 11)" class="f">printfn</span> <span class="s">"The numeric value of s1 is </span><span class="pf">%f</span><span class="s">."</span> (<span class="i">ArrayND</span><span class="o">.</span><span class="i">value</span> <span onmouseout="hideTip(event, 'fs8', 12)" onmouseover="showTip(event, 'fs8', 12)" class="i">s1</span>)
</code></pre></td>
</tr>
</table>
<h3><a name="Creation-from-F-sequences-lists-and-arrays" class="anchor" href="#Creation-from-F-sequences-lists-and-arrays">Creation from F# sequences, lists and arrays</a></h3>
<p>The <code>ArrayNDHost.ofSeq</code> converts a <a href="https://en.wikibooks.org/wiki/F_Sharp_Programming/Sequences">F# sequence</a> of finite length into a one-dimensional tensor.
For example, the code</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs10', 13)" onmouseover="showTip(event, 'fs10', 13)" class="i">seq1</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 14)" onmouseover="showTip(event, 'fs11', 14)" class="i">seq</span> { <span class="k">for</span> <span onmouseout="hideTip(event, 'fs12', 15)" onmouseover="showTip(event, 'fs12', 15)" class="i">i</span><span class="o">=</span><span class="n">0</span> <span class="k">to</span> <span class="n">20</span> <span class="k">do</span> <span class="k">if</span> <span onmouseout="hideTip(event, 'fs12', 16)" onmouseover="showTip(event, 'fs12', 16)" class="i">i</span> <span class="o">%</span> <span class="n">3</span> <span class="o">=</span> <span class="n">0</span> <span class="k">then</span> <span class="k">yield</span> <span onmouseout="hideTip(event, 'fs12', 17)" onmouseover="showTip(event, 'fs12', 17)" class="i">i</span> } <span class="o">|></span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ofSeq</span>
</code></pre></td>
</tr>
</table>
<p>creates a vector of all multiplies of 3 in the range between 0 and 20:</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">seq1</span> <span class="o">=</span> [ <span class="n">0</span> <span class="n">3</span> <span class="n">6</span> <span class="n">9</span> <span class="n">12</span> <span class="n">15</span> <span class="n">18</span>]
</code></pre></td>
</tr>
</table>
<p>A list can be converted into a one-dimensional tensor using the <code>ArrayNDHost.ofList</code> function.</p>
<p>To convert an array into a tensor use the <code>ArrayNDHost.ofArray</code> function.
The <code>ArrayNDHost.ofArray2D</code> and <code>ArrayNDHost.ofArray3D</code> take two-dimensional and three-dimensional arrays and convert them into tensors of respective shapes.</p>
<h3><a name="Conversion-to-F-lists-and-arrays" class="anchor" href="#Conversion-to-F-lists-and-arrays">Conversion to F# lists and arrays</a></h3>
<p>Use the <code>ArrayNDHost.toList</code> function to convert a tensor into a list.
A multi-dimensional tensor is flattened before the conversion.</p>
<p>Use the <code>ArrayNDHost.toArray</code>, <code>ArrayNDHost.toArray2D</code> and <code>ArrayNDHost.toArray3D</code> to convert a tensor into an array of respective dimensionality.</p>
<h2><a name="Printing-tensors" class="anchor" href="#Printing-tensors">Printing tensors</a></h2>
<p>Tensors can be printed using the <code>%A</code> format specifier of the standard <code>printf</code> function.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs9', 18)" onmouseover="showTip(event, 'fs9', 18)" class="f">printfn</span> <span class="s">"The tensor a is</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs1', 19)" onmouseover="showTip(event, 'fs1', 19)" class="i">a</span>
<span onmouseout="hideTip(event, 'fs9', 20)" onmouseover="showTip(event, 'fs9', 20)" class="f">printfn</span> <span class="s">"The tensor z1 is</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs3', 21)" onmouseover="showTip(event, 'fs3', 21)" class="i">z1</span>
<span onmouseout="hideTip(event, 'fs9', 22)" onmouseover="showTip(event, 'fs9', 22)" class="f">printfn</span> <span class="s">"The tensor o1 is</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs5', 23)" onmouseover="showTip(event, 'fs5', 23)" class="i">o1</span>
<span onmouseout="hideTip(event, 'fs9', 24)" onmouseover="showTip(event, 'fs9', 24)" class="f">printfn</span> <span class="s">"The tensor id1 is</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs7', 25)" onmouseover="showTip(event, 'fs7', 25)" class="i">id1</span>
</code></pre></td>
</tr>
</table>
<p>The output of large tensors is automatically truncated to a reasonable size.</p>
<h2><a name="Accessing-elements" class="anchor" href="#Accessing-elements">Accessing elements</a></h2>
<p>Individual elements of a tensor can be accessed using the <code>tensor.[[idx0; idx1; ...; idxN]]</code> notation.
Zero-based indexing is used.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs1', 26)" onmouseover="showTip(event, 'fs1', 26)" class="i">a</span><span class="o">.</span>[[<span class="n">1</span>; <span class="n">1</span>]]
</code></pre></td>
</tr>
</table>
<p>accesses the element at index <span class="math">\(1,1\)</span> and returns <code>5.0</code>.
Note that the indices are specified with double brackets ([[ and ]]) and separated using a semicolon.</p>
<p>Tensors are mutable objects.
An element can be changed using the <code>tensor.[[idx0; idx1; ...; idxN]] <- newValue</code> notation.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs1', 27)" onmouseover="showTip(event, 'fs1', 27)" class="i">a</span><span class="o">.</span>[[<span class="n">2</span>; <span class="n">2</span>]] <span class="o"><-</span> <span class="n">55.</span>
</code></pre></td>
</tr>
</table>
<p>changes the tensor to</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0.0000</span> <span class="n">1.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span> <span class="n">4.0000</span>]
[ <span class="n">5.0000</span> <span class="n">6.0000</span> <span class="n">7.0000</span> <span class="n">8.0000</span> <span class="n">9.0000</span>]
[ <span class="n">10.0000</span> <span class="n">11.0000</span> <span class="n">55.0000</span> <span class="n">13.0000</span> <span class="n">14.0000</span>]
[ <span class="n">15.0000</span> <span class="n">16.0000</span> <span class="n">17.0000</span> <span class="n">18.0000</span> <span class="n">19.0000</span>]
[ <span class="n">20.0000</span> <span class="n">21.0000</span> <span class="n">22.0000</span> <span class="n">23.0000</span> <span class="n">24.0000</span>]
[ <span class="n">25.0000</span> <span class="n">26.0000</span> <span class="n">27.0000</span> <span class="n">28.0000</span> <span class="n">29.0000</span>]
[ <span class="n">30.0000</span> <span class="n">31.0000</span> <span class="n">32.0000</span> <span class="n">33.0000</span> <span class="n">34.0000</span>]]
</code></pre></td>
</tr>
</table>
<h2><a name="Slicing" class="anchor" href="#Slicing">Slicing</a></h2>
<p>Slicing creates a new view into an existing tensor.
Slicing is done using the <code>tensor.[rng0, rng1, ..., rngN]</code> notation.
Note that the ranges are specified within single brackets and separated using commas.</p>
<p>Let us select the first row of tensor a.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs13', 28)" onmouseover="showTip(event, 'fs13', 28)" class="i">a1</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs1', 29)" onmouseover="showTip(event, 'fs1', 29)" class="i">a</span><span class="o">.</span>[<span class="n">0</span>, <span class="o">*</span>]
</code></pre></td>
</tr>
</table>
<p>The asterisk selects all elements of the specified dimension.
The result is a tensor of rank one (i.e. a vector) with the entries</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[ <span class="n">0.0000</span> <span class="n">1.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span> <span class="n">4.0000</span>]
</code></pre></td>
</tr>
</table>
<p>Since <code>a1</code> is a view of <code>a</code> it shares the same memory.
Changing an element of <code>a1</code> by assigning a new value to it</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs13', 30)" onmouseover="showTip(event, 'fs13', 30)" class="i">a1</span><span class="o">.</span>[[<span class="n">1</span>]] <span class="o"><-</span> <span class="n">99.</span>
</code></pre></td>
</tr>
</table>
<p>changes the tensor <code>a</code> as well.
This can be seen by outputting <code>a1</code> and <code>a</code>.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs9', 31)" onmouseover="showTip(event, 'fs9', 31)" class="f">printfn</span> <span class="s">"a1 is now</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs13', 32)" onmouseover="showTip(event, 'fs13', 32)" class="i">a1</span>
<span onmouseout="hideTip(event, 'fs9', 33)" onmouseover="showTip(event, 'fs9', 33)" class="f">printfn</span> <span class="s">"a is now</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs1', 34)" onmouseover="showTip(event, 'fs1', 34)" class="i">a</span>
</code></pre></td>
</tr>
</table>
<p>The corresponding output is</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l"> 1: </span>
<span class="l"> 2: </span>
<span class="l"> 3: </span>
<span class="l"> 4: </span>
<span class="l"> 5: </span>
<span class="l"> 6: </span>
<span class="l"> 7: </span>
<span class="l"> 8: </span>
<span class="l"> 9: </span>
<span class="l">10: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">a1</span> <span class="i">is</span> <span class="i">now</span>
[ <span class="n">0.0000</span> <span class="n">99.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span> <span class="n">4.0000</span>]
<span class="i">a</span> <span class="i">is</span> <span class="i">now</span>
[[ <span class="n">0.0000</span> <span class="n">99.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span> <span class="n">4.0000</span>]
[ <span class="n">5.0000</span> <span class="n">6.0000</span> <span class="n">7.0000</span> <span class="n">8.0000</span> <span class="n">9.0000</span>]
[ <span class="n">10.0000</span> <span class="n">11.0000</span> <span class="n">55.0000</span> <span class="n">13.0000</span> <span class="n">14.0000</span>]
[ <span class="n">15.0000</span> <span class="n">16.0000</span> <span class="n">17.0000</span> <span class="n">18.0000</span> <span class="n">19.0000</span>]
[ <span class="n">20.0000</span> <span class="n">21.0000</span> <span class="n">22.0000</span> <span class="n">23.0000</span> <span class="n">24.0000</span>]
[ <span class="n">25.0000</span> <span class="n">26.0000</span> <span class="n">27.0000</span> <span class="n">28.0000</span> <span class="n">29.0000</span>]
[ <span class="n">30.0000</span> <span class="n">31.0000</span> <span class="n">32.0000</span> <span class="n">33.0000</span> <span class="n">34.0000</span>]]
</code></pre></td>
</tr>
</table>
<p>The slicing notation can also be used for changing multiple elements of a tensor at once.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs14', 35)" onmouseover="showTip(event, 'fs14', 35)" class="i">a2</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs2', 36)" onmouseover="showTip(event, 'fs2', 36)" class="i">float</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ones</span> [<span class="n">5</span>]
<span onmouseout="hideTip(event, 'fs1', 37)" onmouseover="showTip(event, 'fs1', 37)" class="i">a</span><span class="o">.</span>[<span class="n">0</span>, <span class="o">*</span>] <span class="o"><-</span> <span onmouseout="hideTip(event, 'fs14', 38)" onmouseover="showTip(event, 'fs14', 38)" class="i">a2</span>
</code></pre></td>
</tr>
</table>
<p>sets all elements of the first row of <code>a</code> to 1.
The tensor <code>a</code> is now</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">1.0000</span> <span class="n">1.0000</span> <span class="n">1.0000</span> <span class="n">1.0000</span> <span class="n">1.0000</span>]
[ <span class="n">5.0000</span> <span class="n">6.0000</span> <span class="n">7.0000</span> <span class="n">8.0000</span> <span class="n">9.0000</span>]
[ <span class="n">10.0000</span> <span class="n">11.0000</span> <span class="n">55.0000</span> <span class="n">13.0000</span> <span class="n">14.0000</span>]
[ <span class="n">15.0000</span> <span class="n">16.0000</span> <span class="n">17.0000</span> <span class="n">18.0000</span> <span class="n">19.0000</span>]
[ <span class="n">20.0000</span> <span class="n">21.0000</span> <span class="n">22.0000</span> <span class="n">23.0000</span> <span class="n">24.0000</span>]
[ <span class="n">25.0000</span> <span class="n">26.0000</span> <span class="n">27.0000</span> <span class="n">28.0000</span> <span class="n">29.0000</span>]
[ <span class="n">30.0000</span> <span class="n">31.0000</span> <span class="n">32.0000</span> <span class="n">33.0000</span> <span class="n">34.0000</span>]]
</code></pre></td>
</tr>
</table>
<h3><a name="Slicing-operations" class="anchor" href="#Slicing-operations">Slicing operations</a></h3>
<p>Consider the two-dimensional tensor <code>a</code> of shape <span class="math">\(7 \times 5\)</span> and the four-dimensional tensor <code>b</code> of shape <span class="math">\(1 \times 2 \times 3 \times 4\)</span>.
A slice range can be one of the following.</p>
<ul>
<li>A range, e.g. <code>1..3</code>. Selects the specified elements in the corresponding dimension. For example <code>a.[1..3, 0..2]</code> is the <span class="math">\(3 \times 3\)</span> sub-tensor of <code>a</code> containing rows 1, 2, 3 and columns 0, 1, 2. <strong>The ending index is inclusive.</strong></li>
<li>A partial range, e.g. <code>1..</code> or <code>..3</code>. This selects all elements in the corresponding dimension to the end or from the beginning respectively. Thus <code>a.[1.., ..3]</code> is equivalent to <code>a.[1..6, 0..3]</code>.</li>
<li>An asterisk <code>*</code>. Selects all elements in the corresponding dimension. For example <code>a.[1..3, *]</code> is equivalent to <code>a.[1..3, 0..4]</code>.</li>
<li>An integer. The corresponding dimension collapses, e.g. <code>a.[*, 0]</code> specifies a one-dimensional tensor of shape <span class="math">\(7\)</span> corresponding to the first column of <code>a</code>.</li>
<li>The special identifier <code>NewAxis</code>. It inserts a new axis of size one at the given position. For example <code>a.[*, NewAxis, *]</code> produces a view of shape <span class="math">\(7 \times 1 \times 5\)</span>.</li>
<li>The special identifier <code>Fill</code>. It fills any dimensions not specified (if any) with an asterisk <code>*</code>. For example <code>b.[0, Fill, 2]</code> is equivalent to <code>b.[0, *, *, 4]</code> and results into a two-dimensional view into tensor <code>b</code>.</li>
</ul>
<p>All slice range operators can be combined arbitrarily.</p>
<p>The reader should note that the result of selecting a single element using the slicing operator, e.g. <code>a.[1,1]</code>, is a <em>tensor</em> of dimension zero sharing the same memory as <code>a</code>.</p>
<h2><a name="Shape-operations" class="anchor" href="#Shape-operations">Shape operations</a></h2>
<h3><a name="Getting-the-shape" class="anchor" href="#Getting-the-shape">Getting the shape</a></h3>
<p>The shape of a tensor can be accessed using the function <code>ArrayND.shape</code> or using the <code>Shape</code> property.
Both methods return a list.
The rank (number of dimensions) can be accessed using the function <code>ArrayND.nDims</code> or with the <code>NDims</code> property.
The number of elements can be accessed using the function <code>ArrayND.nElems</code> or with the <code>NElems</code> property.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs9', 39)" onmouseover="showTip(event, 'fs9', 39)" class="f">printfn</span> <span class="s">"a has shape </span><span class="pf">%A</span><span class="s">, rank </span><span class="pf">%d</span><span class="s"> and </span><span class="pf">%d</span><span class="s"> elements."</span> <span onmouseout="hideTip(event, 'fs1', 40)" onmouseover="showTip(event, 'fs1', 40)" class="i">a</span><span class="o">.</span><span class="i">Shape</span> <span onmouseout="hideTip(event, 'fs1', 41)" onmouseover="showTip(event, 'fs1', 41)" class="i">a</span><span class="o">.</span><span class="i">NDims</span> <span onmouseout="hideTip(event, 'fs1', 42)" onmouseover="showTip(event, 'fs1', 42)" class="i">a</span><span class="o">.</span><span class="i">NElems</span>
</code></pre></td>
</tr>
</table>
<p>prints</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">a</span> <span class="i">has</span> <span class="i">shape</span> [<span class="n">7</span>; <span class="n">5</span>], <span class="i">rank</span> <span class="n">2</span> <span class="k">and</span> <span class="n">35</span> <span class="i">elements</span><span class="o">.</span>
</code></pre></td>
</tr>
</table>
<h3><a name="Reshaping" class="anchor" href="#Reshaping">Reshaping</a></h3>
<p>Reshaping changes the shape of the tensor while keeping the number of elements constant.</p>
<p>For example consider the <span class="math">\(4 \times 4\)</span> matrix <code>b</code>,</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs15', 43)" onmouseover="showTip(event, 'fs15', 43)" class="i">b</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">4</span>; <span class="n">4</span>] (<span class="k">fun</span> [<span class="i">y</span>; <span class="i">x</span>] <span class="k">-></span> <span class="n">4</span> <span class="o">*</span> <span class="i">y</span> <span class="o">+</span> <span class="i">x</span>)
</code></pre></td>
</tr>
</table>
<p>with value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">4</span> <span class="n">5</span> <span class="n">6</span> <span class="n">7</span>]
[ <span class="n">8</span> <span class="n">9</span> <span class="n">10</span> <span class="n">11</span>]
[ <span class="n">12</span> <span class="n">13</span> <span class="n">14</span> <span class="n">15</span>]]
</code></pre></td>
</tr>
</table>
<p>We can use the <code>ArrayND.reshape</code> function to transform this matrix into a vector of length 16.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs16', 44)" onmouseover="showTip(event, 'fs16', 44)" class="i">b1</span> <span class="o">=</span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">reshape</span> [<span class="n">16</span>] <span onmouseout="hideTip(event, 'fs15', 45)" onmouseover="showTip(event, 'fs15', 45)" class="i">b</span>
</code></pre></td>
</tr>
</table>
<p>Now <code>b1</code> has the value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span> <span class="n">4</span> <span class="n">5</span> <span class="o">..</span><span class="o">.</span> <span class="n">12</span> <span class="n">13</span> <span class="n">14</span> <span class="n">15</span>]
</code></pre></td>
</tr>
</table>
<p>We can also specify -1 for the new size of at most one dimension.
In this case its size will be chosen automatically (so that the number of elements does not change).
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs17', 46)" onmouseover="showTip(event, 'fs17', 46)" class="i">b2</span> <span class="o">=</span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">reshape</span> [<span class="n">4</span>; <span class="n">2</span>; <span class="o">-</span><span class="n">1</span>] <span onmouseout="hideTip(event, 'fs15', 47)" onmouseover="showTip(event, 'fs15', 47)" class="i">b</span>
</code></pre></td>
</tr>
</table>
<p>reshapes <code>b</code> into a three dimensional tensor of shape <span class="math">\(4 \times 2 \times 2\)</span> with the value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
<span class="l">8: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[[ <span class="n">0</span> <span class="n">1</span>]
[ <span class="n">2</span> <span class="n">3</span>]]
[[ <span class="n">4</span> <span class="n">5</span>]
[ <span class="n">6</span> <span class="n">7</span>]]
[[ <span class="n">8</span> <span class="n">9</span>]
[ <span class="n">10</span> <span class="n">11</span>]]
[[ <span class="n">12</span> <span class="n">13</span>]
[ <span class="n">14</span> <span class="n">15</span>]]]
</code></pre></td>
</tr>
</table>
<h4><a name="View-or-copy" class="anchor" href="#View-or-copy">View or copy?</a></h4>
<p>If the tensor to reshape has row-major order (C order), then <code>ArrayND.reshape</code> creates a new view into the existing tensor.
Otherwise the tensor is copied during the reshape operation.
If you need to ensure that no copy is performed, i.e. the original and reshaped tensor share the same memory, use the <code>ArrayND.reshapeView</code> function instead.
It will raise an error if the original tensor is not in row-major order.</p>
<h3><a name="Reordering-axes-and-transposing" class="anchor" href="#Reordering-axes-and-transposing">Reordering axes and transposing</a></h3>
<p>The <code>ArrayND.swapDim</code> function creates a new view of a tensor with the given dimensions swapped.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs18', 48)" onmouseover="showTip(event, 'fs18', 48)" class="i">b3</span> <span class="o">=</span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">swapDim</span> <span class="n">0</span> <span class="n">1</span> <span onmouseout="hideTip(event, 'fs15', 49)" onmouseover="showTip(event, 'fs15', 49)" class="i">b</span>
</code></pre></td>
</tr>
</table>
<p>transpose the matrix <code>b</code> into</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0</span> <span class="n">4</span> <span class="n">8</span> <span class="n">12</span>]
[ <span class="n">1</span> <span class="n">5</span> <span class="n">9</span> <span class="n">13</span>]
[ <span class="n">2</span> <span class="n">6</span> <span class="n">10</span> <span class="n">14</span>]
[ <span class="n">3</span> <span class="n">7</span> <span class="n">11</span> <span class="n">15</span>]]
</code></pre></td>
</tr>
</table>
<p>The original and tensor with swapped axes share the same memory and modifications made to one of them will affect the other one.
A matrix can also be transposed using the <code>ArrayND.transpose</code> function or the <code>.T</code> method, i.e. <code>ArrayND.transpose b</code> and <code>b.T</code> both transpose the matrix <code>b</code>.</p>
<p>The <code>ArrayND.reorderAxes</code> function can reorder axes arbitrarily.
It takes a list (of length equal to the rank of the tensor) with each element specifying the new position of the corresponding axis.
The list must be a permutation of the axes indices, i.e. duplicating or leaving out axes is not permitted.
Consider the rank three tensor <code>b2</code> from above; then</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs19', 50)" onmouseover="showTip(event, 'fs19', 50)" class="i">b4</span> <span class="o">=</span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">reorderAxes</span> [<span class="n">2</span>; <span class="n">0</span>; <span class="n">1</span>] <span onmouseout="hideTip(event, 'fs17', 51)" onmouseover="showTip(event, 'fs17', 51)" class="i">b2</span>
</code></pre></td>
</tr>
</table>
<p>creates the view <code>b4</code> with shape <span class="math">\(2 \times 2 \times 4\)</span>.
It is important to understand that each list entry specifies where the axis <em>moves to</em>, not where it is coming from.
Thus, in this example, axis 0 becomes axis 2, axis 1 becomes axis 0 and axis 2 becomes axis 1.</p>
<h3><a name="Adding-axes" class="anchor" href="#Adding-axes">Adding axes</a></h3>
<p>The <code>ArrayND.padLeft</code> and <code>ArrayND.padRight</code> functions add a new axis of size one on the left or right respectively.
If you need to add an axis at another position, use the slicing operator with the <code>NewAxis</code> identifier.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs20', 52)" onmouseover="showTip(event, 'fs20', 52)" class="i">b5</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs19', 53)" onmouseover="showTip(event, 'fs19', 53)" class="i">b4</span><span class="o">.</span>[<span class="o">*</span>, <span class="o">*</span>, <span class="i">NewAxis</span>, <span class="o">*</span>]
</code></pre></td>
</tr>
</table>
<p>creates a view of shape <span class="math">\(2 \times 2 \times 1 \times 4\)</span>.</p>
<h3><a name="Broadcasting" class="anchor" href="#Broadcasting">Broadcasting</a></h3>
<p>An axis of size one can be repeated multiple times with the same value.
This is called broadcasting.
Consider the <span class="math">\(1 \times 4\)</span> matrix</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs21', 54)" onmouseover="showTip(event, 'fs21', 54)" class="i">c</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">1</span>; <span class="n">4</span>] (<span class="k">fun</span> [_; <span class="i">i</span>] <span class="k">-></span> <span class="i">i</span>)
</code></pre></td>
</tr>
</table>
<p>with value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]]
</code></pre></td>
</tr>
</table>
<p>We can use the <code>ArrayND.broadcastToShape</code> function,</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs22', 55)" onmouseover="showTip(event, 'fs22', 55)" class="i">c1</span> <span class="o">=</span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">broadcastToShape</span> [<span class="n">3</span>; <span class="n">4</span>] <span onmouseout="hideTip(event, 'fs21', 56)" onmouseover="showTip(event, 'fs21', 56)" class="i">c</span>
</code></pre></td>
</tr>
</table>
<p>to obtain the tensor</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]]
</code></pre></td>
</tr>
</table>
<p>Broadcasting creates a view of the original tensor, thus the repeated dimensions do not use additional memory and changing the broadcasted view will also change the original as well as all indices of a broadcasted dimension.
Thus, in this example, executing</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs22', 57)" onmouseover="showTip(event, 'fs22', 57)" class="i">c1</span><span class="o">.</span>[[<span class="n">1</span>; <span class="n">1</span>]] <span class="o"><-</span> <span class="n">11</span>
<span onmouseout="hideTip(event, 'fs9', 58)" onmouseover="showTip(event, 'fs9', 58)" class="f">printfn</span> <span class="s">"c1 is now</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs22', 59)" onmouseover="showTip(event, 'fs22', 59)" class="i">c1</span>
<span onmouseout="hideTip(event, 'fs9', 60)" onmouseover="showTip(event, 'fs9', 60)" class="f">printfn</span> <span class="s">"c is now</span><span class="e">\n</span><span class="s"></span><span class="pf">%A</span><span class="s">"</span> <span onmouseout="hideTip(event, 'fs21', 61)" onmouseover="showTip(event, 'fs21', 61)" class="i">c</span>
</code></pre></td>
</tr>
</table>
<p>prints</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">c1</span> <span class="i">is</span> <span class="i">now</span>
[[ <span class="n">0</span> <span class="n">11</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">0</span> <span class="n">11</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">0</span> <span class="n">11</span> <span class="n">2</span> <span class="n">3</span>]]
<span class="i">c</span> <span class="i">is</span> <span class="i">now</span>
[[ <span class="n">0</span> <span class="n">11</span> <span class="n">2</span> <span class="n">3</span>]]
</code></pre></td>
</tr>
</table>
<p>Broadcasting is also performed automatically when performing element-wise operations between two tensors of different, but compatible, shapes.
This will be explained in the section about element-wise tensor operations.</p>
<h2><a name="Tensor-operations" class="anchor" href="#Tensor-operations">Tensor operations</a></h2>
<p>The tensor type supports most standard arithmetic operators and arithmetic functions.</p>
<h3><a name="Element-wise-binary-arithmetic-operators" class="anchor" href="#Element-wise-binary-arithmetic-operators">Element-wise binary arithmetic operators</a></h3>
<p>The elementary operators (<code>+</code>, <code>-</code>, <code>*</code>, <code>/</code>, <code>%</code>, <code>**</code>) are executed element-wise.
For example, consider the vectors <code>a</code> and <code>b</code>,</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs23', 62)" onmouseover="showTip(event, 'fs23', 62)" class="i">d</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">4</span>] (<span class="k">fun</span> [<span class="i">i</span>] <span class="k">-></span> <span onmouseout="hideTip(event, 'fs2', 63)" onmouseover="showTip(event, 'fs2', 63)" class="i">float</span> <span class="i">i</span>)
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs24', 64)" onmouseover="showTip(event, 'fs24', 64)" class="i">e</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">4</span>] (<span class="k">fun</span> [<span class="i">i</span>] <span class="k">-></span> <span class="n">10.</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs2', 65)" onmouseover="showTip(event, 'fs2', 65)" class="i">float</span> <span class="i">i</span>)
</code></pre></td>
</tr>
</table>
<p>with values</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">d</span> <span class="o">=</span> [ <span class="n">0.0000</span> <span class="n">1.0000</span> <span class="n">2.0000</span> <span class="n">3.0000</span>]
<span class="i">e</span> <span class="o">=</span> [ <span class="n">0.0000</span> <span class="n">10.0000</span> <span class="n">20.0000</span> <span class="n">30.0000</span>]
</code></pre></td>
</tr>
</table>
<p>Then vector <code>f</code>,</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs25', 66)" onmouseover="showTip(event, 'fs25', 66)" class="i">f</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs23', 67)" onmouseover="showTip(event, 'fs23', 67)" class="i">d</span> <span class="o">+</span> <span onmouseout="hideTip(event, 'fs24', 68)" onmouseover="showTip(event, 'fs24', 68)" class="i">e</span>
</code></pre></td>
</tr>
</table>
<p>has the value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[ <span class="n">0.0000</span> <span class="n">11.0000</span> <span class="n">22.0000</span> <span class="n">33.0000</span>]
</code></pre></td>
</tr>
</table>
<p>It is also possible to apply an operator to a tensor and a scalar of the same data type.
In this case the scalar is broadcasted (repeated) to the size of the given tensor.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs26', 69)" onmouseover="showTip(event, 'fs26', 69)" class="i">d1</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs23', 70)" onmouseover="showTip(event, 'fs23', 70)" class="i">d</span> <span class="o">*</span> <span class="n">100.</span>
</code></pre></td>
</tr>
</table>
<p>results in</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">d1</span> <span class="o">=</span> [ <span class="n">0.0000</span> <span class="n">100.0000</span> <span class="n">200.0000</span> <span class="n">300.0000</span>]
</code></pre></td>
</tr>
</table>
<h4><a name="Automatic-broadcasting" class="anchor" href="#Automatic-broadcasting">Automatic broadcasting</a></h4>
<p>If a binary operator (for example <code>+</code>) is applied to two tensors of different shapes, the library tries to automatically broadcast both tensors to a compatible shape using the following rules.</p>
<ol>
<li>
If the tensors have different ranks, the tensor with the lower rank is padded from the left with axes of size one until both tensors have the same rank.
For example, if <code>a</code> is of shape <span class="math">\(4 \times 3 \times 1\)</span> and <code>b</code> is of shape <span class="math">\(2\)</span>, then <code>b</code> is padded to the shape <span class="math">\(\mathbf{1} \times \mathbf{1} \times 2\)</span>.
</li>
<li>
For each dimension that has different size in both tensors and size one in one of the tensors, this dimension of the tensor with size one is broadcasted to the corresponding dimension of the other tensor.
Thus, in our example, the last dimension of <code>a</code> is broadcasted resulting in the shape <span class="math">\(4 \times 3 \times \mathbf{2}\)</span> and the first and second dimensions of <code>b</code> are broadcasted resulting in the shape <span class="math">\(\mathbf{4} \times \mathbf{3} \times 2\)</span>.
</li>
</ol>
<p>If the shapes still differ after applying the above rules, the operation fails.</p>
<h3><a name="Element-wise-arithmetic-functions" class="anchor" href="#Element-wise-arithmetic-functions">Element-wise arithmetic functions</a></h3>
<p>The standard F# arithmetic functions, such as <code>sin</code>, <code>cos</code>, <code>exp</code>, <code>log</code>, can also be applied to tensors.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs27', 71)" onmouseover="showTip(event, 'fs27', 71)" class="i">f2</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs28', 72)" onmouseover="showTip(event, 'fs28', 72)" class="f">sin</span> <span onmouseout="hideTip(event, 'fs25', 73)" onmouseover="showTip(event, 'fs25', 73)" class="i">f</span>
</code></pre></td>
</tr>
</table>
<p>calculates the sine of <code>f</code> element-wise.</p>
<h3><a name="Element-wise-function-application" class="anchor" href="#Element-wise-function-application">Element-wise function application</a></h3>
<p>It is also possible to apply an arbitrary function element-wise using the <code>ArrayND.map</code> function.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">ArrayND</span><span class="o">.</span><span class="i">map</span> (<span class="k">fun</span> <span class="i">x</span> <span class="k">-></span> <span class="k">if</span> <span class="i">x</span> <span class="o">></span> <span class="n">15.</span> <span class="k">then</span> <span class="n">7.</span> <span class="o">+</span> <span class="i">x</span> <span class="k">else</span> <span class="o">-</span><span class="n">1.</span>) <span onmouseout="hideTip(event, 'fs25', 74)" onmouseover="showTip(event, 'fs25', 74)" class="i">f</span>
</code></pre></td>
</tr>
</table>
<p>produces the tensor</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[ <span class="o">-</span><span class="n">1.0000</span> <span class="o">-</span><span class="n">1.0000</span> <span class="n">29.0000</span> <span class="n">40.0000</span>]
</code></pre></td>
</tr>
</table>
<p>An in-place variant that overwrites the original tensor is the <code>ArrayND.mapInplace</code> function.
The <code>ArrayND.map2</code> function takes two tensors and applies a binary function on their elements.</p>
<h3><a name="Arithmetic-reduction-functions" class="anchor" href="#Arithmetic-reduction-functions">Arithmetic reduction functions</a></h3>
<p>The <code>ArrayND.sum</code> function computes the sum of all elements of a tensor and returns a zero-rank tensor.
Thus, in our example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs9', 75)" onmouseover="showTip(event, 'fs9', 75)" class="f">printfn</span> <span class="s">"The sum of f is </span><span class="pf">%.3f</span><span class="s">."</span> (<span class="i">ArrayND</span><span class="o">.</span><span class="i">sum</span> <span onmouseout="hideTip(event, 'fs25', 76)" onmouseover="showTip(event, 'fs25', 76)" class="i">f</span> <span class="o">|></span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">value</span>)
</code></pre></td>
</tr>
</table>
<p>prints</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">The</span> <span class="i">sum</span> <span class="k">of</span> <span class="i">f</span> <span class="i">is</span> <span class="n">66.000</span><span class="o">.</span>
</code></pre></td>
</tr>
</table>
<p>The <code>ArrayND.sumAxis</code> function computes the sum over the given axis.
Thus</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs29', 77)" onmouseover="showTip(event, 'fs29', 77)" class="i">g</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">4</span>; <span class="n">4</span>] (<span class="k">fun</span> [<span class="i">y</span>; <span class="i">x</span>] <span class="k">-></span> <span class="n">4</span> <span class="o">*</span> <span class="i">y</span> <span class="o">+</span> <span class="i">x</span>)
<span class="i">ArrayND</span><span class="o">.</span><span class="i">sumAxis</span> <span class="n">0</span> <span onmouseout="hideTip(event, 'fs29', 78)" onmouseover="showTip(event, 'fs29', 78)" class="i">g</span>
</code></pre></td>
</tr>
</table>
<p>computes the sums of all columns of the matrix</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[[ <span class="n">0</span> <span class="n">1</span> <span class="n">2</span> <span class="n">3</span>]
[ <span class="n">4</span> <span class="n">5</span> <span class="n">6</span> <span class="n">7</span>]
[ <span class="n">8</span> <span class="n">9</span> <span class="n">10</span> <span class="n">11</span>]
[ <span class="n">12</span> <span class="n">13</span> <span class="n">14</span> <span class="n">15</span>]]
</code></pre></td>
</tr>
</table>
<p>resulting in the vector</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[ <span class="n">24</span> <span class="n">28</span> <span class="n">32</span> <span class="n">36</span>]
</code></pre></td>
</tr>
</table>
<p>Likewise the <code>ArrayND.product</code> and <code>ArrayND.productAxis</code> compute the product of a tensor.</p>
<h3><a name="Maximum-and-minimum" class="anchor" href="#Maximum-and-minimum">Maximum and minimum</a></h3>
<p>The <code>ArrayND.min</code> and <code>ArrayND.max</code> compute the minimum and maximum of a tensor and return a zero-rank tensor.
Analogously <code>ArrayND.minAxis</code> and <code>ArrayND.maxAxis</code> compute the minimum and maximum over the given axis.</p>
<h3><a name="Matrix-multiplication" class="anchor" href="#Matrix-multiplication">Matrix multiplication</a></h3>
<p>Matrix multiplication (dot product) is implemented using the <code>.*</code> operator.
This operator can be used to calculate a vector/vector product resulting in a scalar, a matrix/vector product resulting in a vector and a matrix/matrix product resulting in a matrix.</p>
<p>For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs30', 79)" onmouseover="showTip(event, 'fs30', 79)" class="i">h</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">5</span>; <span class="n">3</span>] (<span class="k">fun</span> [<span class="i">i</span>; <span class="i">j</span>] <span class="k">-></span> <span class="n">3.</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs2', 80)" onmouseover="showTip(event, 'fs2', 80)" class="i">float</span> <span class="i">i</span> <span class="o">+</span> <span onmouseout="hideTip(event, 'fs2', 81)" onmouseover="showTip(event, 'fs2', 81)" class="i">float</span> <span class="i">j</span>)
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs31', 82)" onmouseover="showTip(event, 'fs31', 82)" class="i">i</span> <span class="o">=</span> <span class="n">0.1</span> <span class="o">+</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">identity</span> <span class="n">3</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs32', 83)" onmouseover="showTip(event, 'fs32', 83)" class="i">hi</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs30', 84)" onmouseover="showTip(event, 'fs30', 84)" class="i">h</span> <span class="o">.</span><span class="o">*</span> <span onmouseout="hideTip(event, 'fs31', 85)" onmouseover="showTip(event, 'fs31', 85)" class="i">i</span>
</code></pre></td>
</tr>
</table>
<p>computes the matrix product of the <span class="math">\(5 \times 3\)</span> matrix <code>h</code> with the <span class="math">\(3 \times 3\)</span> matrix <code>i</code> resulting in the <code>5 \times 3</code> matrix <code>hi</code>.</p>
<h3><a name="Tensor-product" class="anchor" href="#Tensor-product">Tensor product</a></h3>
<p>The tensor product between two tensors can be obtained using the <code>%*</code> operator or the <code>ArrayND.tensorProduct</code> function.</p>
<h3><a name="Element-wise-comparison-operators" class="anchor" href="#Element-wise-comparison-operators">Element-wise comparison operators</a></h3>
<p>Element-wise comparisons are performed using the <code>====</code> (element-wise equal), <code><<<<</code> (element-wise smaller than), <code>>>>></code> (element-wise larger than) and <code><<>></code> (element-wise not equal) operators.
They return a tensor of equal shape and boolean data type.
For example</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span class="i">j</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs23', 86)" onmouseover="showTip(event, 'fs23', 86)" class="i">d</span> <span class="o">====</span> <span onmouseout="hideTip(event, 'fs24', 87)" onmouseover="showTip(event, 'fs24', 87)" class="i">e</span>
</code></pre></td>
</tr>
</table>
<p>has the value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="i">d</span> <span class="o">=</span> [<span class="k">true</span>; <span class="k">false</span>; <span class="k">false</span>; <span class="k">false</span>;]
</code></pre></td>
</tr>
</table>
<h3><a name="Logic-reduction-operations" class="anchor" href="#Logic-reduction-operations">Logic reduction operations</a></h3>
<p>To check whether all elements satisfy a condition, use the <code>ArrayND.all</code> function after applying the element-wise comparison operator.
To check whether at least one element satisfies a condition, use the <code>ArrayND.any</code> function after applying the element-wise comparison operator.
Thus</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs23', 88)" onmouseover="showTip(event, 'fs23', 88)" class="i">d</span> <span class="o">====</span> <span onmouseout="hideTip(event, 'fs24', 89)" onmouseover="showTip(event, 'fs24', 89)" class="i">e</span> <span class="o">|></span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">all</span>
</code></pre></td>
</tr>
</table>
<p>has the value <code>false</code>, but</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span onmouseout="hideTip(event, 'fs23', 90)" onmouseover="showTip(event, 'fs23', 90)" class="i">d</span> <span class="o">====</span> <span onmouseout="hideTip(event, 'fs24', 91)" onmouseover="showTip(event, 'fs24', 91)" class="i">e</span> <span class="o">|></span> <span class="i">ArrayND</span><span class="o">.</span><span class="i">any</span>
</code></pre></td>
</tr>
</table>
<p>returns <code>true</code>.</p>
<h3><a name="Element-wise-logic-operators" class="anchor" href="#Element-wise-logic-operators">Element-wise logic operators</a></h3>
<p>Element-wise logic operations are performed using the <code>~~~~</code> (element-wise negation), <code>&&&&</code> (element-wise and) and <code>||||</code> (element-wise or) operators.
Thus</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="o">~~~~</span><span class="i">j</span>
</code></pre></td>
</tr>
</table>
<p>has value</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp">[<span class="k">false</span>; <span class="k">true</span>; <span class="k">true</span>; <span class="k">true</span>;]
</code></pre></td>
</tr>
</table>
<h2><a name="Disk-storage-in-HDF5-format" class="anchor" href="#Disk-storage-in-HDF5-format">Disk storage in HDF5 format</a></h2>
<p>Tensors can be stored in industry-standard <a href="https://en.wikipedia.org/wiki/Hierarchical_Data_Format">HDF5 files</a>.
Multiple tensors can be stored in a single HDF5 file and accessed by assigning names to them.</p>
<h3><a name="Writing-tensors-to-disk" class="anchor" href="#Writing-tensors-to-disk">Writing tensors to disk</a></h3>
<p>The following code creates two tensors <code>k</code> and <code>l</code> and writes them into the HDF5 file <code>tensors.h5</code> in the current directory.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
<span class="l">5: </span>
<span class="l">6: </span>
<span class="l">7: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">open</span> <span class="i">Basics</span> <span class="c">// TODO: remove after refactoring</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs33', 92)" onmouseover="showTip(event, 'fs33', 92)" class="i">k</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">5</span>; <span class="n">3</span>] (<span class="k">fun</span> [<span onmouseout="hideTip(event, 'fs31', 93)" onmouseover="showTip(event, 'fs31', 93)" class="i">i</span>; <span class="i">j</span>] <span class="k">-></span> <span class="n">3.</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs2', 94)" onmouseover="showTip(event, 'fs2', 94)" class="i">float</span> <span onmouseout="hideTip(event, 'fs31', 95)" onmouseover="showTip(event, 'fs31', 95)" class="i">i</span> <span class="o">+</span> <span onmouseout="hideTip(event, 'fs2', 96)" onmouseover="showTip(event, 'fs2', 96)" class="i">float</span> <span class="i">j</span>)
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs34', 97)" onmouseover="showTip(event, 'fs34', 97)" class="i">l</span> <span class="o">=</span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">initIndexed</span> [<span class="n">5</span>] (<span class="k">fun</span> [<span onmouseout="hideTip(event, 'fs31', 98)" onmouseover="showTip(event, 'fs31', 98)" class="i">i</span>] <span class="k">-></span> <span class="n">2.</span> <span class="o">*</span> <span onmouseout="hideTip(event, 'fs2', 99)" onmouseover="showTip(event, 'fs2', 99)" class="i">float</span> <span onmouseout="hideTip(event, 'fs31', 100)" onmouseover="showTip(event, 'fs31', 100)" class="i">i</span>)
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs35', 101)" onmouseover="showTip(event, 'fs35', 101)" class="i">hdfFile</span> <span class="o">=</span> <span class="i">HDF5</span><span class="o">.</span><span class="i">OpenWrite</span> <span class="s">"tensors.h5"</span>
<span class="i">ArrayNDHDF</span><span class="o">.</span><span class="i">write</span> <span onmouseout="hideTip(event, 'fs35', 102)" onmouseover="showTip(event, 'fs35', 102)" class="i">hdfFile</span> <span class="s">"k"</span> <span onmouseout="hideTip(event, 'fs33', 103)" onmouseover="showTip(event, 'fs33', 103)" class="i">k</span>
<span class="i">ArrayNDHDF</span><span class="o">.</span><span class="i">write</span> <span onmouseout="hideTip(event, 'fs35', 104)" onmouseover="showTip(event, 'fs35', 104)" class="i">hdfFile</span> <span class="s">"l"</span> <span onmouseout="hideTip(event, 'fs34', 105)" onmouseover="showTip(event, 'fs34', 105)" class="i">l</span>
<span onmouseout="hideTip(event, 'fs35', 106)" onmouseover="showTip(event, 'fs35', 106)" class="i">hdfFile</span><span class="o">.</span><span class="i">Dispose</span> ()
</code></pre></td>
</tr>
</table>
<p>The resulting file can be viewed using any HDF5 viewer, for example using the free, cross-platform <a href="https://www.hdfgroup.org/products/java/hdfview/">HDFView</a> application as shown below.</p>
<p><img src="img/hdfview.png" alt="HDFView screenshot" /></p>
<h3><a name="Loading-tensors-from-disk" class="anchor" href="#Loading-tensors-from-disk">Loading tensors from disk</a></h3>
<p>The following code loads the tensors <code>k</code> and <code>l</code> from the previously created HDF5 file <code>tensors.h5</code> and stores them in the variables <code>k2</code> and <code>l2</code>.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
<span class="l">3: </span>
<span class="l">4: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs36', 107)" onmouseover="showTip(event, 'fs36', 107)" class="i">hdfFile2</span> <span class="o">=</span> <span class="i">HDF5</span><span class="o">.</span><span class="i">OpenRead</span> <span class="s">"tensors.h5"</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs37', 108)" onmouseover="showTip(event, 'fs37', 108)" class="i">k2</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs2', 109)" onmouseover="showTip(event, 'fs2', 109)" class="i">float</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHDF</span><span class="o">.</span><span class="i">read</span> <span onmouseout="hideTip(event, 'fs36', 110)" onmouseover="showTip(event, 'fs36', 110)" class="i">hdfFile2</span> <span class="s">"k"</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs38', 111)" onmouseover="showTip(event, 'fs38', 111)" class="i">l2</span> <span class="o">:</span> <span class="i">ArrayNDHostT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs2', 112)" onmouseover="showTip(event, 'fs2', 112)" class="i">float</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDHDF</span><span class="o">.</span><span class="i">read</span> <span onmouseout="hideTip(event, 'fs36', 113)" onmouseover="showTip(event, 'fs36', 113)" class="i">hdfFile2</span> <span class="s">"l"</span>
<span onmouseout="hideTip(event, 'fs36', 114)" onmouseover="showTip(event, 'fs36', 114)" class="i">hdfFile2</span><span class="o">.</span><span class="i">Dispose</span> ()
</code></pre></td>
</tr>
</table>
<p>The data types of <code>k2</code> and <code>l2</code> must be declared explicitly, since they must be known at compile-time.
If the declared data type does not match the data type encountered in the HDF5, an error will be raised.</p>
<h2><a name="Reading-npy-and-npz-files-produced-by-Numpy" class="anchor" href="#Reading-npy-and-npz-files-produced-by-Numpy">Reading .npy and .npz files produced by Numpy</a></h2>
<p>For compatibility, it is possible to read <code>.npy</code> and <code>.npz</code> files produced by Numpy.
Not all features of the format are supported.
Writing <code>.npy</code> and <code>.npz</code> files is not possible; use the HDF5 format instead.</p>
<p>Use the <code>NPYFile.load</code> function to read an <code>.npy</code> file and return its contents as an <code>ArrayNDHostT</code>.
Use the <code>NPZFile.Open</code> function to open an <code>.npz</code> file and the <code>Get</code> method of the resulting object to obtain individual entries as <code>ArrayNDHostT</code>.</p>
<h2><a name="CUDA-GPU-support" class="anchor" href="#CUDA-GPU-support">CUDA GPU support</a></h2>
<p>If your workstation is equipped with a <a href="https://en.wikipedia.org/wiki/CUDA">CUDA</a> <a href="https://developer.nvidia.com/cuda-gpus">capable GPU</a>, you can transfer tensors to GPU memory and perform operations on the GPU.
GPU tensors are instances of the generic type <code>ArrayNDCudaT<'T></code> where <code>'T</code> is the contained data type.</p>
<p><strong>Note: While tensors can be created on or transferred to the GPU, <em>currently</em> there is no supported for accelerated operations on the GPU <em>when using tensor directly</em>. Thus executing tensor operations (except slicing and view operations) on the GPU will be very slow <em>at the moment</em> and should be avoided. This is supposed to change with future versions and does <em>not</em> affect compiled functions created by the Deep.Net library.</strong></p>
<h3><a name="Data-transfer" class="anchor" href="#Data-transfer">Data transfer</a></h3>
<p>Tensors can be transferred to the GPU by using the <code>ArrayNDCuda.toDev</code> function.
Transfer back to host memory is done using the <code>ArrayNDCuda.toHost</code> function.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
<span class="l">2: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs43', 119)" onmouseover="showTip(event, 'fs43', 119)" class="i">m</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs11', 120)" onmouseover="showTip(event, 'fs11', 120)" class="i">seq</span> {<span class="n">1</span> <span class="o">..</span> <span class="n">10</span>} <span class="o">|></span> <span class="i">ArrayNDHost</span><span class="o">.</span><span class="i">ofSeq</span>
<span class="k">let</span> <span onmouseout="hideTip(event, 'fs44', 121)" onmouseover="showTip(event, 'fs44', 121)" class="i">mGpu</span> <span class="o">=</span> <span class="i">ArrayNDCuda</span><span class="o">.</span><span class="i">toDev</span> <span onmouseout="hideTip(event, 'fs43', 122)" onmouseover="showTip(event, 'fs43', 122)" class="i">m</span>
</code></pre></td>
</tr>
</table>
<p><code>mGpu</code> is now a copy of <code>m</code> in GPU memory.</p>
<h3><a name="Tensor-creation" class="anchor" href="#Tensor-creation">Tensor creation</a></h3>
<p>It is possible to create tensors directly in GPU memory.
The function <code>ArrayNDCuda.zeros</code>, <code>ArrayNDCuda.ones</code>, <code>ArrayNDCuda.identity</code> and <code>ArrayNDCuda.scalar</code> behave like their <code>ArrayNDHost</code> counterparts, except that the result is stored on the GPU.</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs45', 123)" onmouseover="showTip(event, 'fs45', 123)" class="i">n</span> <span class="o">:</span> <span class="i">ArrayNDCudaT</span><span class="o"><</span><span onmouseout="hideTip(event, 'fs2', 124)" onmouseover="showTip(event, 'fs2', 124)" class="i">float</span><span class="o">></span> <span class="o">=</span> <span class="i">ArrayNDCuda</span><span class="o">.</span><span class="i">identity</span> <span class="n">4</span>
</code></pre></td>
</tr>
</table>
<p>Here we created a <span class="math">\(4 \times 4\)</span> identity matrix of data type <code>float</code> on the GPU.</p>
<h3><a name="Operators-and-functions" class="anchor" href="#Operators-and-functions">Operators and functions</a></h3>
<p>All functions and operators described in previous section can be equally applied to GPU tensors.
For example, the code</p>
<table class="pre"><tr><td class="lines"><pre class="fssnip"><span class="l">1: </span>
</pre></td>
<td class="snippet"><pre class="fssnip highlighted"><code lang="fsharp"><span class="k">let</span> <span onmouseout="hideTip(event, 'fs46', 125)" onmouseover="showTip(event, 'fs46', 125)" class="i">mGpuSq</span> <span class="o">=</span> <span onmouseout="hideTip(event, 'fs44', 126)" onmouseover="showTip(event, 'fs44', 126)" class="i">mGpu</span><span class="o">.</span>[<span class="n">3..</span><span class="n">5</span>] <span class="o">*</span> <span class="n">3</span>
</code></pre></td>
</tr>
</table>
<p>takes three elements from the GPU tensor and multiplies them by 3 on the GPU.</p>
<h2><a name="Conclusion" class="anchor" href="#Conclusion">Conclusion</a></h2>
<p>We presented an introduction to working with host and GPU tensors in Deep.Net.
Direct tensor manipulation will be used mostly for dataset and results handling.
The machine learning model will instead be defined as a symbolic computation graph, that supports automatic differentiation and compilation to optimized CUDA code.</p>
<div class="tip" id="fs1">val a : obj<br /><br />Full name: Tensor.a</div>
<div class="tip" id="fs2">Multiple items<br />val float : value:'T -> float (requires member op_Explicit)<br /><br />Full name: Microsoft.FSharp.Core.Operators.float<br /><br />--------------------<br />type float = System.Double<br /><br />Full name: Microsoft.FSharp.Core.float<br /><br />--------------------<br />type float<'Measure> = float<br /><br />Full name: Microsoft.FSharp.Core.float<_></div>
<div class="tip" id="fs3">val z1 : obj<br /><br />Full name: Tensor.z1</div>
<div class="tip" id="fs4">Multiple items<br />val int : value:'T -> int (requires member op_Explicit)<br /><br />Full name: Microsoft.FSharp.Core.Operators.int<br /><br />--------------------<br />type int = int32<br /><br />Full name: Microsoft.FSharp.Core.int<br /><br />--------------------<br />type int<'Measure> = int<br /><br />Full name: Microsoft.FSharp.Core.int<_></div>
<div class="tip" id="fs5">val o1 : obj<br /><br />Full name: Tensor.o1</div>
<div class="tip" id="fs6">Multiple items<br />val single : value:'T -> single (requires member op_Explicit)<br /><br />Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.single<br /><br />--------------------<br />type single = System.Single<br /><br />Full name: Microsoft.FSharp.Core.single</div>
<div class="tip" id="fs7">val id1 : obj<br /><br />Full name: Tensor.id1</div>
<div class="tip" id="fs8">val s1 : obj<br /><br />Full name: Tensor.s1</div>
<div class="tip" id="fs9">val printfn : format:Printf.TextWriterFormat<'T> -> 'T<br /><br />Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn</div>
<div class="tip" id="fs10">val seq1 : obj<br /><br />Full name: Tensor.seq1</div>
<div class="tip" id="fs11">Multiple items<br />val seq : sequence:seq<'T> -> seq<'T><br /><br />Full name: Microsoft.FSharp.Core.Operators.seq<br /><br />--------------------<br />type seq<'T> = System.Collections.Generic.IEnumerable<'T><br /><br />Full name: Microsoft.FSharp.Collections.seq<_></div>
<div class="tip" id="fs12">val i : int</div>
<div class="tip" id="fs13">val a1 : obj<br /><br />Full name: Tensor.a1</div>
<div class="tip" id="fs14">val a2 : obj<br /><br />Full name: Tensor.a2</div>
<div class="tip" id="fs15">val b : obj<br /><br />Full name: Tensor.b</div>
<div class="tip" id="fs16">val b1 : obj<br /><br />Full name: Tensor.b1</div>
<div class="tip" id="fs17">val b2 : obj<br /><br />Full name: Tensor.b2</div>
<div class="tip" id="fs18">val b3 : obj<br /><br />Full name: Tensor.b3</div>
<div class="tip" id="fs19">val b4 : obj<br /><br />Full name: Tensor.b4</div>
<div class="tip" id="fs20">val b5 : obj<br /><br />Full name: Tensor.b5</div>
<div class="tip" id="fs21">val c : obj<br /><br />Full name: Tensor.c</div>
<div class="tip" id="fs22">val c1 : obj<br /><br />Full name: Tensor.c1</div>
<div class="tip" id="fs23">val d : float<br /><br />Full name: Tensor.d</div>
<div class="tip" id="fs24">val e : float<br /><br />Full name: Tensor.e</div>
<div class="tip" id="fs25">val f : float<br /><br />Full name: Tensor.f</div>
<div class="tip" id="fs26">val d1 : float<br /><br />Full name: Tensor.d1</div>
<div class="tip" id="fs27">val f2 : float<br /><br />Full name: Tensor.f2</div>
<div class="tip" id="fs28">val sin : value:'T -> 'T (requires member Sin)<br /><br />Full name: Microsoft.FSharp.Core.Operators.sin</div>
<div class="tip" id="fs29">val g : 'a<br /><br />Full name: Tensor.g</div>
<div class="tip" id="fs30">val h : 'a (requires member ( .* ))<br /><br />Full name: Tensor.h</div>
<div class="tip" id="fs31">val i : float<br /><br />Full name: Tensor.i</div>
<div class="tip" id="fs32">val hi : '_arg3<br /><br />Full name: Tensor.hi</div>
<div class="tip" id="fs33">val k : 'a<br /><br />Full name: Tensor.k</div>
<div class="tip" id="fs34">val l : 'a<br /><br />Full name: Tensor.l</div>
<div class="tip" id="fs35">val hdfFile : 'a<br /><br />Full name: Tensor.hdfFile</div>
<div class="tip" id="fs36">val hdfFile2 : 'a<br /><br />Full name: Tensor.hdfFile2</div>
<div class="tip" id="fs37">val k2 : 'a<br /><br />Full name: Tensor.k2</div>
<div class="tip" id="fs38">val l2 : 'a<br /><br />Full name: Tensor.l2</div>
<div class="tip" id="fs39">namespace System</div>
<div class="tip" id="fs40">namespace System.IO</div>
<div class="tip" id="fs41">type File =<br />  static member AppendAllLines : path:string * contents:IEnumerable<string> -> unit + 1 overload<br />  static member AppendAllText : path:string * contents:string -> unit + 1 overload<br />  static member AppendText : path:string -> StreamWriter<br />  static member Copy : sourceFileName:string * destFileName:string -> unit + 1 overload<br />  static member Create : path:string -> FileStream + 3 overloads<br />  static member CreateText : path:string -> StreamWriter<br />  static member Decrypt : path:string -> unit<br />  static member Delete : path:string -> unit<br />  static member Encrypt : path:string -> unit<br />  static member Exists : path:string -> bool<br />  ...<br /><br />Full name: System.IO.File</div>
<div class="tip" id="fs42">System.IO.File.Delete(path: string) : unit</div>
<div class="tip" id="fs43">val m : 'a<br /><br />Full name: Tensor.m</div>
<div class="tip" id="fs44">val mGpu : 'a<br /><br />Full name: Tensor.mGpu</div>
<div class="tip" id="fs45">val n : 'a<br /><br />Full name: Tensor.n</div>
<div class="tip" id="fs46">val mGpuSq : int<br /><br />Full name: Tensor.mGpuSq</div>
</div>
<div class="span3">
<!-- <img src="http://www.deepml.net/img/logo.png" alt="Deep.Net logo" style="width:150px;margin:10px" /> -->
<ul class="nav nav-list" id="menu" style="margin-top: 20px;">
<li class="nav-header">Deep.Net</li>
<li><a href="http://www.deepml.net/index.html">Home page</a></li>
<li class="divider"></li>
<li><a href="http://nuget.org/packages/DeepNet">Get Library via NuGet</a></li>
<li><a href="http://github.com/DeepMLNet/DeepNet">Source Code on GitHub</a></li>
<li><a href="http://www.deepml.net/release-notes.html">Release Notes</a></li>
<li class="nav-header">Basics</li>
<li><a href="http://www.deepml.net/tensor.html">Working with Tensors</a></li>
<li><a href="http://www.deepml.net/model.html">Model Definition</a></li>
<li><a href="http://www.deepml.net/components.html">Model Components</a></li>
<li><a href="http://www.deepml.net/dataset.html">Dataset Handling</a></li>
<li><a href="http://www.deepml.net/training.html">Training</a></li>
<li class="nav-header">Advanced</li>
<li><a href="http://www.deepml.net/diff.html">Automatic Differentiation</a></li>
<li class="nav-header">Documentation</li>
<li><a href="http://www.deepml.net/reference/index.html">API Reference</a></li>
</ul>
</div>
</div>
</div>
<a href="http://github.com/DeepMLNet/DeepNet"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" alt="Fork me on GitHub"/></a>
</body>
</html>