-
Notifications
You must be signed in to change notification settings - Fork 33
/
fibers.texi
1577 lines (1310 loc) · 63.9 KB
/
fibers.texi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\input texinfo @c -*-texinfo-*-
@c %**start of header
@setfilename fibers.info
@settitle Fibers
@c %**end of header
@set VERSION 1.3.1
@set UPDATED 22 August 2023
@copying
This manual is for Fibers (version @value{VERSION}, updated
@value{UPDATED})
@noindent
Copyright 2016-2023 Andy Wingo@*
Copyright 2021, 2023 Maxime Devos@*
Copyright 2022, 2023 Aleix Conchillo Flaqué
@quotation
@c For more information, see COPYING.docs in the fibers
@c distribution.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
@end quotation
@end copying
@dircategory The Algorithmic Language Scheme
@direntry
* Fibers: (fibers.info). Lightweight concurrency for Guile.
@end direntry
@titlepage
@title Fibers
@subtitle version @value{VERSION}, updated @value{UPDATED}
@author Andy Wingo
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage
@ifnottex
@node Top
@top Fibers
@insertcopying
@menu
* Introduction:: What's this all about?
* Reference:: API reference.
* Pitfalls:: Stay on the happy path.
* Examples:: Starting points for a hack.
* Status:: Fibers is a work in progress.
* Index:: Index of procedures.
@end menu
@end ifnottex
@iftex
@shortcontents
@end iftex
@node Introduction
@chapter Introduction
Fibers is a facility for lightweight concurrency in Guile.
@menu
* Context:: How do other systems handle concurrency?
* Design:: Fibers' point in the design space.
* Parallelism:: Faster throughput via more cores.
@end menu
@node Context
@section A brief history of language facilities for concurrency
Modern machines have the raw capability to serve hundreds of thousands
of simultaneous long-lived connections, but it's often hard to manage
this at the software level. Fibers tries to solve this problem in a
nice way. Before discussing the approach taken in Fibers, it's worth
spending some time on history to see how we got here.
One of the most dominant patterns for concurrency these days is
``callbacks'', notably in the Twisted library for Python and the
Node.js run-time for JavaScript. The basic observation in the
callback approach to concurrency is that the efficient way to handle
tens of thousands of connections at once is with low-level operating
system facilities like @code{poll}, @code{epoll} or @code{kqueue}. By
default, Fibers will try to use the native events system facility
implementation (e.g. @code{epoll} on Linux systems) but
@code{libevent} is also supported for portability reasons. You add
all of the file descriptors that you are interested in to a ``poll
set'' and then ask the operating system which ones are readable or
writable, as appropriate. Once the operating system says ``yes, file
descriptor 7145 is readable'', you can do something with that socket;
but what? With callbacks, the answer is ``call a user-supplied
closure'': a callback, representing the continuation of the
computation on that socket.
Building a network service with a callback-oriented concurrency system
means breaking the program into little chunks that can run without
blocking. Wherever a program could block, instead of just continuing
the program, you register a callback. Unfortunately this requirement
permeates the program, from top to bottom: you always pay the mental
cost of inverting your program's control flow by turning it into
callbacks, and you always incur run-time cost of closure creation,
even when the particular I/O could proceed without blocking. It's a
somewhat galling requirement, given that this contortion is required
of the programmer, but could be done by the compiler. We Schemers
demand better abstractions than manual, obligatory
continuation-passing-style conversion.
Callback-based systems also encourage unstructured concurrency, as in
practice callbacks are not the only path for data and control flow in
a system: usually there is mutable global state as well. Without
strong patterns and conventions, callback-based systems often exhibit
bugs caused by concurrent reads and writes to global state.
Some of the problems of callbacks can be mitigated by using
``promises'' or other library-level abstractions; if you're a Haskell
person, you can think of this as lifting all possibly-blocking
operations into a monad. If you're not a Haskeller, that's cool,
neither am I! But if your typey spidey senses are tingling, it's for
good reason: with promises, your whole program has to be transformed
to return promises-for-values instead of values anywhere it would
block.
An obvious solution to the control-flow problem of callbacks is to use
threads. In the most generic sense, a thread is a language feature
which denotes an independent computation. Threads are created by
other threads, but fork off and run independently instead of returning
to their caller. In a system with threads, there is implicitly a
scheduler somewhere that multiplexes the threads so that when one
suspends, another can run.
In practice, the concept of threads is often conflated with a
particular implementation, @dfn{kernel threads}. Kernel threads are
very low-level abstractions that are provided by the operating system.
The nice thing about kernel threads is that they can use any CPU that
is the kernel knows about. That's an important factor in today's
computing landscape, where Moore's law seems to be giving us more
cores instead of more gigahertz.
However, as a building block for a highly concurrent system, kernel
threads have a few important problems.
One is that kernel threads simply aren't designed to be allocated in
huge numbers, and instead are more optimized to run in a
one-per-CPU-core fashion. Their memory usage is relatively high for
what should be a lightweight abstraction: some 10 kilobytes at least
and often some megabytes, in the form of the thread's stack. There
are ongoing efforts to reduce this for some systems but we cannot
expect wide deployment in the next 5 years, if ever. Even in the best
case, a hundred thousand kernel threads will take at least a gigabyte
of memory, which seems a bit excessive for book-keeping overhead.
Kernel threads can be a bit irritating to schedule, too: when one
thread suspends, it's for a reason, and it can be that user-space
knows a good next thread that should run. However because kernel
threads are scheduled in the kernel, it's rarely possible for the
kernel to make informed decisions. There are some ``user-mode
scheduling'' facilities that are in development for some systems, but
again only for some systems.
The other significant problem is that building non-crashy systems on
top of kernel threads is hard to do, not to mention ``correct''
systems. It's an embarrassing situation. For one thing, the
low-level synchronization primitives that are typically provided with
kernel threads, mutexes and condition variables, are not composable.
Also, as with callback-oriented concurrency, one thread can silently
corrupt another via unstructured mutation of shared state. It's worse
with kernel threads, though: a kernel thread can be interrupted at any
point, not just at I/O. And though callback-oriented systems can
theoretically operate on multiple CPUs at once, in practice they
don't. This restriction is sometimes touted as a benefit by
proponents of callback-oriented systems, because in such a system, the
callback invocations have a single, sequential order. With multiple
CPUs, this is not the case, as multiple threads can run at the same
time, in parallel.
Kernel threads can work. The Java virtual machine does at least
manage to prevent low-level memory corruption and to do so with high
performance, but still, even Java-based systems that aim for maximum
concurrency avoid using a thread per connection because threads use
too much memory.
In this context it's no wonder that there's a third strain of
concurrency: shared-nothing message-passing systems like Erlang.
Erlang isolates each thread (called @dfn{processes} in the Erlang
world), giving each it its own heap and ``mailbox''. Processes can
spawn other processes, and the concurrency primitive is
message-passing. A process that tries receive a message from an empty
mailbox will ``block'', from its perspective. In the meantime the
system will run other processes. Message sends never block, oddly;
instead, sending to a process with many messages pending makes it more
likely that Erlang will pre-empt the sending process. It's a strange
trade off, but it makes sense when you realize that Erlang was designed
for network transparency: the same message send/receive interface can
be used to send messages to processes on remote machines as well.
No network is truly transparent, however. At the most basic level,
the performance of network sends should be much slower than local
sends. Whereas a message sent to a remote process has to be written
out byte-by-byte over the network, there is no need to copy immutable
data within the same address space. The complexity of a remote
message send is O(n) in the size of the message, whereas a local
immutable send is O(1). This suggests that hiding the different
complexities behind one operator is the wrong thing to do. And
indeed, given byte read and write operators over sockets, it's
possible to implement remote message send and receive as a process
that serializes and parses messages between a channel and a byte sink
or source. In this way we get cheap local channels, and network shims
are under the programmer's control. This is the approach that the Go
language takes, and is the one we use in Fibers.
Structuring a concurrent program as separate threads that communicate
over channels is an old idea that goes back to Tony Hoare's work on
``Communicating Sequential Processes'' (CSP). CSP is an elegant tower
of mathematical abstraction whose layers form a pattern language for
building concurrent systems that you can still reason about.
Interestingly, it does so without any concept of time at all, instead
representing a thread's behavior as a @dfn{trace} of instantaneous
events. Threads themselves are like functions that unfold over the
possible events to produce the actual event trace seen at run-time.
This view of events as instantaneous happenings extends to
communication as well. In CSP, one communication between two threads
is modelled as an instantaneous event, partitioning the traces of the
two threads into ``before'' and ``after'' segments.
Practically speaking, this has ramifications in the Go language, which
was heavily inspired by CSP. You might think that a channel is just a
an asynchronous queue that blocks when writing to a full queue, or
when reading from an empty queue. That's a bit closer to the Erlang
conception of how things should work, though as we mentioned, Erlang
simply slows down writes to full mailboxes rather than blocking them
entirely. However, that's not what Go and other systems in the CSP
family do; sending a message on a channel will block until there is a
receiver available, and vice versa. The threads are said to
``rendezvous'' at the event.
Unbuffered channels have the interesting property that you can
@code{select} between sending a message on channel @var{a} or channel
@var{b}, and in the end only one message will be sent; nothing happens
until there is a receiver ready to take the message. In this way
messages are really owned by threads and never by the channels
themselves. You can of course add buffering if you like, simply by
making a thread that waits on either sends or receives on a channel,
and which buffers sends and makes them available to receives. It's
also possible to add explicit support for buffered channels, as Go
does, which can reduce the number of context switches as there is no
explicit buffer thread.
Whether to buffer or not to buffer is a tricky choice. It's possible
to implement singly-buffered channels in a system like Erlang via an
explicit send/acknowledge protocol, though it seems difficult to
implement completely unbuffered channels. As we mentioned, it's
possible to add buffering to an unbuffered system by the introduction
of explicit buffer threads. In the end though in Fibers we follow
CSP's lead so that we can implement the nice @code{select} behavior
that we mentioned above.
As a final point, @code{select} is OK but is not a great language
abstraction. Say you call a function and it returns some kind of
asynchronous result which you then have to @code{select} on. It could
return this result as a channel, and that would be fine: you can add
that channel to the other channels in your @code{select} set and you
are good. However, what if what the function does is receive a
message on a channel, then do something with the message? In that
case the function should return a channel, plus a continuation (as a
closure or something). If @code{select} results in a message being
received over that channel, then we call the continuation on the
message. Fine. But, what if the function itself wanted to
@code{select} over some channels? It could return multiple channels
and continuations, but that becomes unwieldy.
What we need is an abstraction over asynchronous operations, and that
is the main idea of a CSP-derived system called ``Concurrent ML''
(CML). Originally implemented as a library on top of Standard ML of
New Jersey by John Reppy, CML provides this abstraction, which in
Fibers is called an @dfn{operation}@footnote{CML uses the term
@dfn{event}, but we find this to be a confusing name.}. Calling
@code{put-operation} on a channel returns an operation, which is just
a value. Operations are like closures in a way; a closure wraps up
code in its environment, which can be later called many times or not
at all. Operations likewise can be @dfn{performed}@footnote{In CML,
@dfn{synchronized}.} many times or not at all; performing an operation
is like calling a function. The interesting part is that you can
compose operations via the @code{wrap-operation} and
@code{choice-operation} combinators. The former lets you bundle up an
operation and a continuation. The latter lets you construct an
operation that chooses over a number of operations. Calling
@code{perform-operation} on a choice operation will perform one and
only one of the choices. Performing an operation will call its
@code{wrap-operation} continuation on the resulting values.
While it's possible to implement Concurrent ML in terms of Go's
channels and baked-in @code{select} statement, it's more expressive to
do it the other way around, as that also lets us implement other
operations types besides channel send and receive, for example
timeouts and condition variables.
@node Design
@section Fibers design
In Fibers, the unit of computation is the @dfn{fiber}, a lightweight
thread managed by Guile. A fiber communicates with the outside world
via normal Guile ports: @code{get-bytevector}, @code{put-string}, and
all that. Within a single Guile process fibers communicate by sending
and receiving Scheme values over @dfn{channels}.
Whenever a fiber tries to read but no data is available, or tries to
write but no data can be written, Guile will suspend the fiber and
arrange for it to be resumed when the port or channel operation can
proceed. In the meantime, Guile will run other fibers. When no fiber
is runnable, Guile will use efficient system facilities to sleep until
input or output can proceed.
When a fiber would block, it suspends to the scheduler from the
current thread. The scheduler will arrange to re-start the fiber when
the port or channel becomes readable or writable, as appropriate. For
ports, the scheduler adds the file descriptor associated with the port
as a pending event. In either case, the scheduler remembers which
fibers are waiting and for what, so that the user can inspect the
state of their system.
Currently in Fibers there is no ambient scheduler running; an error is
signalled if a user calls @code{spawn-fiber} while not inside a
@code{run-fibers} invocation. However it is possible to communicate
with fibers via channels or other Concurrent ML-like operations, even
outside of a @code{run-fibers} invocation. If an operation would
block, it suspends the entire kernel thread until the operation can
proceed.
On the Scheme level, a fiber is a delimited continuation. When a
scheduler runs a fiber, it does so within a prompt; when the fiber
suspends, it suspends to the prompt. The scheduler saves the
resulting continuation as part of the fiber's state. In this way the
per-fiber computational state overhead is just the size of the pending
stack frames of the fiber, which can be just a handful of words.
By default, Fibers takes advantage of all available cores on your
system. @xref{Parallelism}, for full details.
Ports are how fibers communicate with the world; channels are how
fibers communicate with each other. Channels are meeting places
between fibers, or between threads. A fiber or thread that goes to
send a message over a channel will block until there is a fiber or
thread ready to receive the message, and vice versa. Once both
parties are ready, the message is exchanged and both parties resume.
There can be multiple fibers and threads waiting to read and write on
a channel, allowing channels to express not only pipelines but also
common concurrency patterns such as fan-in and fan-out.
Unlike Erlang channels, channels in Fibers are purely local and do not
attempt to provide the illusion of network transparency. This does
have the positive advantage that we are able to provide better
backpressure support than Erlang, blocking when no receiver is
available to handle a message instead of letting the sender keep
sending many messages.
To avoid starvation, a fiber can only run once within a ``turn''.
Each turn starts with a poll on file descriptors of interest and marks
the associated fibers as runnable. If no fiber is runnable at the
start of the poll, the poll call will ask the kernel to wait for a
runnable descriptor. Otherwise the poll call will still check for
runnable file descriptors, but also ask the kernel to return
immediately. There is an additional FD added to the poll set that is
used to interrupt a blocking poll, for example if a fiber becomes
runnable due to I/O on a channel from a separate kernel thread while
the first scheduler was still polling.
If a fiber runs for too long (by default, 10 milliseconds), it will be
@dfn{preempted}: interrupted and rescheduled for the next turn. The
preemption frequency can be tuned by the user or turned off for a
fully cooperative scheduling model.
To enable expressive cross-kernel-thread communications, channel sends
and receives are atomic and thread-safe.
@node Parallelism
@section Parallelism
By default, Fibers will take advantage of all CPU cores available to
it. The degree of parallelism is controlled by the
@code{#:parallelism} keyword argument to @code{run-fibers}, which
defaults to @code{(current-processor-count)}.
@xref{Threads,,,guile.info,Guile Reference Manual}, for more
information on @code{current-processor-count}. Pass a different
argument to @code{#:parallelism} to choose a different degree of
parallelism, for example @code{1} for single-threaded operation. To
allocate specific cores to a Guile process, use the @code{taskset}
command-line utility.
A newly spawned fiber will be scheduled on the kernel thread in which
it was created, unless @code{#:parallel? #t} was passed to the
@code{spawn-fiber} invocation, in which case its initial kernel thread
will be selected at random. In this way the default is to preserve
locality of memory access and minimize cross-thread coordination.
Additionally, after a scheduler has exhausted its run queue for the
current turn, if it has nothing scheduled for the next turn it will
try to steal work from other schedulers. This @dfn{work stealing}
allows a set of parallel schedulers to automatically rebalance and
burn through the current global run queue as fast as possible.
After processing its current run queue, possibly including stolen work
if its next run queue was empty, a scheduler will then ask the
operating system for any file descriptors that have pending activity.
The scheduler puts a time limit on this sleep phase if there are
pending timeouts, but otherwise the sleep will only wake up when a
file descriptor becomes readable or writable, or if another thread
wakes up the scheduler. Schedulers that are sleeping do not
participate in work stealing. For this reason there is another source
of work rebalancing in Fibers, @dfn{work sharing}. As mentioned
above, to schedule a fiber on a random remote scheduler, use
@code{spawn-fiber} with the @code{#:parallel? #t} keyword argument.
The specifics of the scheduling algorithm may change, and it may be
that there is no global ``best scheduler''. We look forward to
experimenting and finding not only a good default algorithm, but also
a library that you can use to find your own local maximum in the
scheduling space.
As far as performance goes, we have found that computationally
intensive tasks parallelize rather well. Expect near-linear speedup
as you make more cores available to fibers.
On the other hand, although allocation rate improves with additional
cores, it currently does not scale linearly, and works best when all
cores are on the same NUMA node. This is due to details about how
Guile manages its memory.
In general there may be many bottlenecks that originate in Guile,
Fibers, and in your application, and these bottlenecks constrain the
ability of an application to scale linearly.
Probably the best way to know if Fibers scales appropriately for your
use case is to make some experiments. To restrict the set of cores
available to Guile, run Guile from within @code{taskset -c}. See
@code{taskset}'s manual page. For machines with multiple sockets you
will probably want to use @code{numactl --membind} as well. Then to
test scalability on your machine, run @code{./env guile
tests/speedup.scm} from within your Fibers build directory, or
benchmark your application directly. In time we should be able to
develop some diagnostic facilities to help the Fibers user determine
where a scaling bottleneck is in their application.
@node Reference
@chapter API reference
Fibers is a library built on Guile. It consists of a public
interface, base support for asynchronous operations, implementations
of operations for channels and timers, and an internals interface.
@menu
* Using Fibers:: User-facing interface to fibers
* Operations:: Composable abstractions for concurrency.
* Channels:: Share memory by communicating.
* Timers:: Operations on time.
* Conditions:: Waiting for simple state changes.
* Port Readiness:: Waiting until a port is ready for I/O.
* REPL Commands:: Experimenting with Fibers at the console.
* Schedulers and Tasks:: Fibers are built from lower-level primitives.
@end menu
@node Using Fibers
@section Using Fibers
The public interface of fibers right now is quite minimal. To use it,
import the @code{(fibers)} module:
@example
(use-modules (fibers))
@end example
To create a new fibers scheduler and run it in the current Guile
thread, use @code{run-fibers}.
@defun run-fibers [init-thunk=@code{#f}] @
[#:install-suspendable-ports?=@code{#t}] @
[#:scheduler=@code{#f}] @
[#:parallelism=@code{(current-processor-count)}] @
[#:cpus=@code{(getaffinity 0)}] @
[#:hz=@code{100}] [#:drain?=@code{#f}]
Run @var{init-thunk} within a fiber in a fresh scheduler, blocking
until @var{init-thunk} returns. Return the value(s) returned by the
call to @var{init-thunk}.
For example:
@example
(run-fibers (lambda () 1))
@result{} 1
(run-fibers
(lambda ()
(spawn-fiber (lambda () (display "hey!\n"))))
#:drain? #t)
@print{} hey!
@end example
Calling @code{run-fibers} will ensure that Guile's port implementation
allows fibers to suspend if a read or a write on a port would block.
@xref{Non-Blocking I/O,,,guile.info,Guile Reference Manual}, for more
details on suspendable ports. If for some reason you want port reads
or writes to prevent other fibers from running, pass @code{#f} as the
@code{#:install-suspendable-ports?} keyword argument.
By default, @code{run-fibers} will create a fresh scheduler, and
destroy it after @code{run-fibers} finishes. If you happen to have a
pre-existing scheduler (because you used the low-level scheduler
interface to create one), you can pass it to @code{run-fibers} using
the @code{#:scheduler} keyword argument. In that case the scheduler
will not be destroyed when @code{run-fibers} finishes.
@code{run-fibers} will return when the @var{init-thunk} call returns.
To make it additionally wait until there are no more runnable fibers
or pending timeouts, specify the @code{#:drain? #t} keyword argument.
If @code{run-fibers} creates a scheduler on your behalf, it will
arrange for a number of ``peer'' schedulers to also be created, up to
a total scheduler count controlled by the @var{parallelism} keyword
argument. These peer schedulers will be run in separate threads and
will participate in work rebalancing. The fibers will be run on the
CPUs specified by @var{cpus}. @xref{Parallelism}.
By default @var{hz} is 100, indicating that running fibers should be
preempted 100 times per every second of CPU time (not wall-clock
time). Note that preemption will only occur if the fiber can actually
be suspended; @xref{Barriers}, for more information. Pass @code{0}
for @var{hz} to disable preemption, effectively making scheduling
fully cooperative.
@end defun
@defun spawn-fiber thunk [scheduler=@code{(require-current-scheduler)}] @
[#:parallel?=@code{#f}]
Spawn a new fiber that will run @var{thunk}. Return the new fiber.
The new fiber will run concurrently with other fibers.
The fiber will be added to the current scheduler, which is usually
what you want. It's also possible to spawn the fiber on a specific
scheduler, which is useful to ensure that the fiber runs on a
different kernel thread. In that case, pass the optional
@code{scheduler} argument.
If @var{parallel?} is true, the fiber will be started not
(necessarily) on @var{scheduler}, but on a random member of the peer
set of @var{scheduler}. @xref{Parallelism}. Note that every
scheduler is a member of its own peer set.
The fiber will inherit the fluid--value associations (the dynamic
state) in place when @code{spawn-fiber} is called. Any
@code{fluid-set!} or parameter set within the fiber will not affect
fluid or parameter bindings outside the fiber.
@end defun
@defun sleep seconds
Wake up the current fiber after @var{seconds} of wall-clock time have
elapsed. This definition will replace the binding for @code{sleep} in
the importing module, effectively overriding Guile's ``core''
definition.
@end defun
@node Operations
@section Operations
Operations are first-class abstractions for asynchronous events.
There are primitive operation types, such as waiting for a timer
(@pxref{Timers}) or waiting for a message on a channel
(@pxref{Channels}). Operations can also be combined and transformed
using the @code{choice-operation} and @code{wrap-operation} from this module:
@example
(use-modules (fibers operations))
@end example
@defun wrap-operation op f
Given the operation @var{op}, return a new operation that, if and when
it succeeds, will apply @var{f} to the values yielded by performing
@var{op}, and yield the result as the values of the wrapped operation.
@end defun
@defun choice-operation . ops
Given the operations @var{ops}, return a new operation that if it
succeeds, will succeed with one and only one of the sub-operations
@var{ops}.
@end defun
Finally, once you have an operation, you can perform it using
@code{perform-operation}.
@defun perform-operation op
Perform the operation @var{op} and return the resulting values. If the
operation cannot complete directly, block until it can complete.
@end defun
@xref{Introduction}, for more on the ``Concurrent ML'' system that
introduced the concept of the operation abstraction. In the context
of Fibers, ``blocking'' means to suspend the current fiber, or to
suspend the current kernel thread if the operation is performed
outside of a fiber.
There is also a low-level constructor for other modules that implement
primitive operation types:
@defun make-base-operation wrap-fn try-fn block-fn
Make a fresh base operation.
@end defun
This is a low-level constructor, though; if you ever feel the need to
call @code{make-base-operation}, make sure you're familiar with the
Concurrent ML literature. Godspeed!
@node Channels
@section Channels
Channels are the way to communicate between fibers. To use them, load
the channels module:
@example
(use-modules (fibers channels))
@end example
@defun make-channel
Make a fresh channel.
@end defun
@defun channel? obj
Return @code{#t} if @var{obj} is a channel, or @code{#f} otherwise.
@end defun
@defun put-operation channel message
Make an operation that if and when it completes will rendezvous with a
receiving operation to send @var{message} over @var{channel}.
@end defun
@defun get-operation channel
Make an operation that if and when it completes will rendezvous with a
sending operation to receive one value from @var{channel}.
@end defun
@defun put-message channel message
Send @var{message} on @var{channel}, and return zero values. If there
is already a receiver waiting to receive a message on this channel,
give it our message and continue. Otherwise, block until a receiver
becomes available.
Equivalent to:
@example
(perform-operation (put-operation channel message))
@end example
@end defun
@defun get-message channel
Receive a message from @var{channel} and return it. If there is
already a sender waiting to send a message on this channel, take its
message directly. Otherwise, block until a sender becomes available.
Equivalent to:
@example
(perform-operation (get-operation channel))
@end example
@end defun
Channels are thread-safe; you can use them to send and receive values
between fibers on different kernel threads.
@node Timers
@section Timers
Timers are a kind of operation that, you guessed it, let you sleep
until a certain time.
@example
(use-modules (fibers timers))
@end example
@defun sleep-operation seconds
Make an operation that will succeed with no values when @var{seconds}
have elapsed.
@end defun
@defun timer-operation expiry
Make an operation that will succeed when the current time is greater
than or equal to @var{expiry}, expressed in internal time units. The
operation will succeed with no values.
@end defun
@defun sleep seconds
Block the calling fiber or kernel thread until @var{seconds} have
elapsed.
@end defun
@node Conditions
@section Conditions
Condition variables are a simple one-bit form of concurrent
communication. A condition variable has two states: it starts in the
@dfn{unsignalled} state and later may transition to the
@dfn{signalled} state. When a condition becomes signalled, any
associated waiting operations complete.
@example
(use-modules (fibers conditions))
@end example
@defun make-condition
Make a new condition variable.
@end defun
@defun condition? obj
Return @code{#t} if @var{obj} is a condition variable, or @code{#f}
otherwise.
@end defun
@defun signal-condition! cvar
Signal @var{cvar}, notifying all waiting fibers and preventing
blocking of future fibers waiting on this condition.
@end defun
@defun wait-operation cvar
Make an operation that will succeed with no values when @var{cvar}
becomes signalled.
@end defun
@defun wait cvar
Block the calling fiber or kernel thread until @var{cvar} becomes
signalled. Equivalent to @code{(perform-operation (wait-operation
cvar))}.
@end defun
@node Port Readiness
@section Port Readiness
These two operations can be used on file ports to wait until
they are readable or writable. Spurious wake-ups are possible.
This is complementary to Guile's suspendable ports.
@example
(use-modules (fibers io-wakeup))
@end example
@defun wait-until-port-readable-operation port
Make an operation that will succeed with no values when the input
port @var{port} becomes readable. For passive sockets, this operation
succeeds when a connection becomes available.
@end defun
@defun wait-until-port-writable-operation port
Make an operation that will succeed with no values when the output
port @var{port} becomes writable.
@end defun
Usually, you don't only want to detect readiness, but also act upon
it. While this can be accomplished by combining one of the previous
two procedures with @code{wrap-operation}, the spurious wake-ups can
be inconvenient. In that case, the following two procedures may be
useful:
@defun make-read-operation try-fn port
Make an operation that tries @var{try-fn}, and when @var{try-fn}
fails, blocks until @var{port} is readable. @var{try} is a thunk that
either returns @code{#false}, indicating failure, or a thunk, whose
return values are the result of the operation.
@end defun
@defun make-write-operation try-fn port
Make an operation that tries @var{try-fn}, and when @var{try-fn}
fails, blocks until @var{port} is writable. @var{try} is a thunk that
either returns @code{#false}, indicating failure, or a thunk, whose
return values are the result of the operation.
@end defun
Often, failure is indicated by a procedure calling the thunk of
@code{current-read-waiter} or @code{current-write-waiter}. In that
case, the following two procedures can be useful:
@defun with-read-waiting-is-failure port try-fn
Return a thunk like @var{try-fn}, except that it also fails when
@var{try-fn} the procedure from the parameter
@code{current-write-waiter} is invoked on @var{port}.
The read waiter at the current depth (relative to the invocation of
the returned thunk) may not be changed while the thunk is being invoked.
@end defun
@defun with-write-waiting-is-failure port try-fn
Return a thunk like @var{try-fn}, except that it also fails when
@var{try-fn} the procedure from the parameter
@code{current-write-waiter} is invoked on @var{port}.
The write waiter at the current depth (relative to the invocation of
the returned thunk) may not be changed while the thunk is being invoked.
@end defun
These two procedures need to install a new read/write waiter, but this
waiter doesn't cover all situations and as such might need to call the
‘previous’ waiter in turn. The waiter has two options for accessing
the previous waiter:
@itemize
@item the constructed thunk could save the at the time current waiter
(this is what's done currently)
@item the read waiter could look ‘deeper’ in the fluid stack by using
@code{fluid-ref*}
@end itemize
The first option also has a problem: if prompts are used, it is
possible that the ‘less deep’ waiter changes after the old ‘less deep’
waiter was saved.
The second option would be ideal, but the problem is that the waiter
doesn't know how deep it is in the stack -- while it could iterate the
fluid stack and use @code{eq?} what depths are possible, it might
exist at multiple depths at the same time even though the waiter is
‘fresh’ because of weird usage of prompts.
Perhaps the solution is to add a ‘depth’ argument to the procedures of
@code{current-read-waiter} and @code{current-write-waiter} or allow
the waiter to return true or false to indicate whether it handled the
situation or whether the previous waiter should be tried, but neither
of these is implemented yet at time of writing.
In the meantime, don't change the ‘less deep’ waiter while
@code{with-read-waiter} / @code{with-write-waiter} is doing its thing!
@defun accept-operation port [flags=(logior O_NONBLOCK O_CLOEXEC)]
Like @code{(accept port flags)}, but as an operation. Unlike
@code{accept}, @code{O_NONBLOCK} is included in the default flags,
which makes the accepted port non-blocking and hence suitable for
use with Fibers.
The flag @code{O_CLOEXEC} is also included (if available). Without
this flag, ports would be leaked to subprocesses by default, which
usually is at best inefficient and at worst a security problem. But
rarely, ports are intended to be leaked to subprocesses, in which
case you could remove this flag.
Fibers doesn't emulate @code{O_CLOEXEC} when unavailable, because it
would not be entirely equivalent in case of parallelism.
@end defun
@node REPL Commands
@section REPL Commands
Fibers implements some basic extensions to the Guile command-line
interface (its Read-Eval-Print Loop, or the REPL). Prefix these
commands with a comma (@code{,}) to run them at the REPL; see
@code{,help fibers} for full details, once you have loaded the
@code{(fibers)} module of course.
@deffn {REPL Command} scheds
Show a list of all schedulers.
@end deffn
@deffn {REPL Command} spawn-sched
Create a new scheduler for fibers, and run it on a new kernel thread.
@end deffn
@deffn {REPL Command} kill-sched name
Shut down the scheduler named @var{name}. Use @code{,scheds} to list
scheduler names.
@end deffn
@deffn {REPL Command} spawn-fiber exp [sched]
Spawn a new fiber that runs @var{exp}. If @var{sched} is given, the
fiber will be spawned on the given scheduler.
@end deffn
@node Schedulers and Tasks
@section Schedulers and Tasks
Internally, fibers are built on top of schedulers and tasks.
A scheduler runs tasks. A task is just a thunk (a function of no
arguments) whose return value is ignored. A scheduler runs tasks in
batches, once per turn. Each turn, a scheduler takes all tasks from
its ``next'' run-queue and adds them to its ``current'' run-queue, and
then runs the tasks on the current run-queue in order. The scheduler
then goes to the next turn, unless its ``finished?'' function returns
true.
Both the ``next'' and the ``current'' run-queues are public atomic
data structures. Scheduling a task adds it to the ``next'' run-queue.
Scheduling a task is a thread-safe operation; it can be done by any
thread. Scheduling a task on a scheduler running on a remote thread
will ensure that the thread wakes up from any sleeping operation it
might be currently engaged in.
There is normally just one scheduler for each kernel thread that runs
fibers. Several schedulers can be made aware of each other so that
they can one can spread out the load when spawning tasks that should
run in parallel. Also, before moving to the next turn, a scheduler
will try to steal work from other schedulers that it knows about,
popping off an item from the remote scheduler's ``current'' run-queue.
There are two additional sources of tasks for a scheduler: file
descriptor events and timers. When gathering tasks to schedule for
the next turn, a scheduler will call the native events system faciliy
implementation to be notified of activity on file descriptors of
interest. If there are no pending tasks on the next run-queue, the
call to the events system facility will sleep until the scheduler is
woken up, or until a timer expires.
The capability that allows fibers to be built on schedulers is that
tasks can suspend. Suspending a task calls a user-supplied
after-suspend handler that is passed the continuation of the task.
The user can then schedule that continuation at some later time. In
this way a fiber starts as a single task run by a scheduler, but each
time it suspends and is resumed, a fresh task consisting of the
fiber's is scheduled. The fibers layer also uses other Guile
mechanisms to isolate fibers from each other, such as dynamic states.
All interfaces in this module are thread-safe except where marked
otherwise.
@example
(use-modules (fibers scheduler))
@end example
@defun make-scheduler [#:parallelism=@code{#f}] @
[#:prompt-tag=@code{(make-prompt-tag "fibers")}]
Make a new scheduler in which to run fibers. If @var{parallelism} is
true, it should be an integer indicating the number of schedulers to
make. The resulting schedulers will all share the same prompt tag and
will steal and share out work from among themselves.
@end defun
@defun run-scheduler sched finished?
Run @var{sched} until calling the supplied @var{finished?} thunk
returns true. Return zero values. Signal an error if @var{scheduler}
is already running in some other kernel thread.
@end defun
@defun current-scheduler
Return the current scheduler, or @code{#f} if no scheduler is current.
@end defun
@defun scheduler-kernel-thread sched
Return the kernel thread that @var{sched} is running on, or @code{#f}
if it is not currently running.
@end defun
@defun scheduler-runcount sched
Return the number of tasks that have been run on @var{sched}, modulo
@math{2^{32}}. This interface is useful as a lightweight check to see if
a remote scheduler is making progress.
@end defun
@defun scheduler-remote-peers sched
Return a list of peer schedulers of @var{sched}, not including
@var{sched} itself.
@end defun
@defun scheduler-work-pending? sched
Return @code{#t} if @var{sched} has any work pending: any runnable
tasks or any pending timeouts.
@end defun
@defun choose-parallel-scheduler sched
Return a random scheduler from @var{sched}'s peer set. Note that
@var{sched}'s peer set includes @var{sched} itself.
@end defun
@defun destroy-scheduler sched
Release any resources associated with @var{sched}.
@end defun
@defun schedule-task sched task
Arrange to run @var{task}, a procedure of no arguments, on the next
turn of @var{sched}. If @var{sched} is remote and sleeping, it will
be woken up.
@end defun
@defun schedule-task-when-fd-readable sched fd task
Arrange to schedule @var{task} when the file descriptor @var{fd}
becomes readable. @emph{Not thread-safe.}
@end defun
@defun schedule-task-when-fd-writable sched fd task
Arrange to schedule @var{task} on @var{sched} when the file descriptor
@var{fd} becomes writable. @emph{Not thread-safe.}
@end defun
@defun schedule-task-at-time sched expiry task
Arrange to schedule @var{task} on @var{sched} when the absolute real
time is greater than or equal to @var{expiry}, expressed in internal
time units. @emph{Not thread-safe.}
@end defun
@defun suspend-current-task after-suspend
Suspend the current task to the current scheduler. After suspending,
call the @var{after-suspend} callback with two arguments: the current
scheduler, and the continuation of the current task. The continuation
passed to the @var{after-suspend} handler is the continuation of the
@code{suspend-current-task} call.
@end defun