1 \input texinfo @c -*-texinfo-*-
3 @setfilename 8sync.info
7 Copyright @copyright{} 2015 Christopher Allan Webber @email{cwebber@@dustycloud.org}
11 Permission is granted to copy, distribute and/or modify this document
12 under the terms of the GNU Free Documentation License, Version 1.3
13 or any later version published by the Free Software Foundation;
14 with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
15 A copy of the license is included in the section entitled ``GNU
16 Free Documentation License''.
18 A copy of the license is also available from the Free Software
19 Foundation Web site at @url{http://www.gnu.org/licenses/fdl.html}.
21 Alternately, this document is also available under the Lesser General
22 Public License, version 3 or later, as published by the Free Software
25 A copy of the license is also available from the Free Software
26 Foundation Web site at @url{http://www.gnu.org/licenses/lgpl.html}.
32 @subtitle 8sync, asynchronous actors for Guile
33 @author Christopher Allan Webber
35 @vskip 0pt plus 1filll
39 @c Output the table of the contents at the beginning.
43 @node Top, Preface, (dir), (dir)
49 @c Generate the nodes for this menu with `C-c C-u C-m'.
52 @c Update all node entries with `C-c C-u C-n'.
53 @c Insert new nodes with `C-c C-c n'.
61 * Copying This Manual::
70 Welcome to 8sync's documentation!
71 8sync is an asynchronous programming environment for GNU Guile.
72 (Get it? 8sync? Async??? Quiet your groans, it's a great name!)
74 8sync has some nice properties:
78 8sync uses the actor model as its fundamental concurrency
79 synchronization mechanism.
80 Since the actor model is a "shared nothing" asynchronous
81 environment, you don't need to worry about deadlocks or other
82 tricky problems common to other asynchronous models.
83 Actors are modular units of code and state which communicate
84 by sending messages to each other.
86 If you've done enough asynchronous programming, you're probably
87 familiar with the dreaded term "callback hell".
88 Getting around callback hell usually involves a tradeoff of other,
89 still rather difficult to wrap your brain around programming
91 8sync uses some clever tricks involving "delimited continuations"
92 under the hood to make the code you write look familiar and
94 When you need to send a request to another actor and get some
95 information back from it without blocking, there's no need
96 to write a separate procedure@dots{} 8sync's scheduler will suspend
97 your procedure and wake it back up when a response is ready.
99 Even nonblocking I/O code is straightforward to write.
100 Thanks to the "suspendable ports" code introduced in Guile 2.2,
101 writing asynchronous, nonblocking networked code looks mostly
102 like writing the same synchronous code.
103 8sync's scheduler handles suspending and resuming networked
104 code that would otherwise block.
106 8sync aims to be "batteries included".
107 Useful subsystems for IRC bots, HTTP servers, and so on are
108 included out of the box.
110 8sync prioritizes live hacking.
111 If using an editor like Emacs with a nice mode like Geiser,
112 an 8sync-using developer can change and fine-tune the behavior
113 of code @emph{while it runs}.
114 This makes both debugging and development much more natural,
115 allowing the right designs to evolve under your fingertips.
116 A productive hacker is a happy hacker, after all!
119 In the future, 8sync will also provide the ability to spawn and
120 communicate with actors on different threads, processes, and machines,
121 with most code running the same as if actors were running in the same
122 execution environment.
124 But as a caution, 8sync is still very young.
125 The API is stabilizing, but not yet stable, and it is not yet well
128 But, consider this as much an opportunity as a warning.
129 8sync is in a state where there is much room for feedback and
133 And now, into the wild, beautiful frontier.
140 * A silly little IRC bot::
141 * Writing our own actors::
142 * Writing our own network-enabled actor::
143 * An intermission on live hacking::
148 @node A silly little IRC bot
149 @section A silly little IRC bot
151 IRC! Internet Relay Chat!
152 The classic chat protocol of the Internet.
153 And it turns out, one of the best places to learn about networked
154 programming.@footnote{In the 1990s I remember stumbling into some funky IRC chat rooms and
155 being astounded that people there had what they called "bots" hanging
157 From then until now, I've always enjoyed encountering bots whose range
158 of functionality has spanned from saying absurd things, to taking
159 messages when their "owners" were offline, to reporting the weather,
160 to logging meetings for participants.
161 And it turns out, IRC bots are a great way to cut your teeth on
162 networked programming; since IRC is a fairly simple line-delineated
163 protocol, it's a great way to learn to interact with sockets.
164 (My first IRC bot helped my team pick a place to go to lunch, previously
165 a source of significant dispute!)
166 At the time of writing, venture capital awash startups are trying to
167 turn chatbots into "big business"@dots{} a strange (and perhaps absurd)
168 thing given chat bots being a fairly mundane novelty amongst hackers
169 and teenagers everywhere a few decades ago.}
170 We ourselves are going to explore chat bots as a basis for getting our
173 First of all, we're going to need to import some modules. Put this at
174 the top of your file:
177 (use-modules (8sync) ; 8sync's agenda and actors
178 (8sync systems irc) ; the irc bot subsystem
179 (oop goops) ; 8sync's actors use GOOPS
180 (ice-9 format) ; basic string formatting
181 (ice-9 match)) ; pattern matching
184 Now we need to add our bot. Initially, it won't do much.
187 (define-class <my-irc-bot> (<irc-bot>))
189 (define-method (handle-line (irc-bot <my-irc-bot>) message
190 speaker channel line emote?)
192 (format #t "~a emoted ~s in channel ~a\n"
193 speaker line channel)
194 (format #t "~a said ~s in channel ~a\n"
195 speaker line channel)))
198 We've just defined our own IRC bot!
199 This is an 8sync actor.
200 (8sync uses GOOPS to define actors.)
201 We extended the handle-line generic method, so this is the code that
202 will be called whenever the IRC bot "hears" anything.
203 This method is itself an action handler, hence the second argument
204 for @verb{~message~}, which we can ignore for now.
205 Pleasantly, the message's argument body is passed in as the rest of
208 For now the code is pretty basic: it just outputs whatever it "hears"
209 from a user in a channel to the current output port.
211 But it should help us make sure we have things working when we kick
214 Speaking of, even though we've defined our actor, it's not running
215 yet. Time to fix that!
218 (define* (run-bot #:key (username "examplebot")
219 (server "irc.freenode.net")
220 (channels '("##botchat")))
221 (define hive (make-hive))
223 (bootstrap-actor hive <my-irc-bot>
226 #:channels channels))
230 Actors are connected to something called a "hive", which is a
231 special kind of actor that runs and manages all the other actors.
232 Actors can spawn other actors, but before we start the hive we use
233 this special @verb{~bootstrap-actor~} method.
234 It takes the hive as its first argument, the actor class as the second
235 argument, and the rest are initialization arguments to the
237 @verb{~bootstrap-actor~} passes back not the actor itself (we don't
238 get access to that usually) but the @strong{id} of the actor.
239 (More on this later.)
240 Finally we run the hive with run-hive and pass it a list of
241 "bootstrapped" messages.
242 Normally actors send messages to each other (and sometimes themselves),
243 but we need to send a message or messages to start things or else
244 nothing is going to happen.
249 (run-bot #:username "some-bot-name") ; be creative!
252 Assuming all the tubes on the internet are properly connected, you
253 should be able to join the "##botchat" channel on irc.freenode.net and
254 see your bot join as well.
255 Now, as you probably guessed, you can't really @emph{do} much yet.
256 If you talk to the bot, it'll send messages to the terminal informing
257 you as such, but it's hardly a chat bot if it's not chatting yet.
259 So let's do the most boring (and annoying) thing possible.
260 Let's get it to echo whatever we say back to us.
261 Change handle-line to this:
264 (define-method (handle-line (irc-bot <my-irc-bot>) message
265 speaker channel line emote?)
266 (<- (actor-id irc-bot) 'send-line channel
267 (format #f "Bawwwwk! ~a says: ~a" speaker line)))
270 This will do exactly what it looks like: repeat back whatever anyone
271 says like an obnoxious parrot.
272 Give it a try, but don't keep it running for too long@dots{} this
273 bot is so annoying it's likely to get banned from whatever channel
276 This method handler does have the advantage of being simple though.
277 It introduces a new concept simply@dots{} sending a message!
278 Whenever you see "<-", you can think of that as saying "send this
280 The arguments to "<-" are as follows: the actor sending the message,
281 the id of the actor the message is being sent to, the "action" we
282 want to invoke (a symbol), and the rest are arguments to the
283 "action handler" which is in this case send-line (with itself takes
284 two arguments: the channel our bot should send a message to, and
285 the line we want it to spit out to the channel).@footnote{8sync's name for sending a message, "<-", comes from older,
286 early lisp object oriented systems which were, as it turned out,
287 inspired by the actor model!
288 Eventually message passing was dropped in favor of something called
289 "generic functions" or "generic methods"
290 (you may observe we made use of such a thing in extending
292 Many lispers believe that there is no need for message passing
293 with generic methods and some advanced functional techniques,
294 but in a concurrent environment message passing becomes useful
295 again, especially when the communicating objects / actors are not
296 in the same address space.}
298 Normally in the actor model, we don't have direct references to
299 an actor, only an identifier.
300 This is for two reasons: to quasi-enforce the "shared nothing"
301 environment (actors absolutely control their own resources, and
302 "all you can do is send a message" to request that they modify
303 them) and because@dots{} well, you don't even know where that actor is!
304 Actors can be anything, and anywhere.
305 It's possible in 8sync to have an actor on a remote hive, which means
306 the actor could be on a remote process or even remote machine, and
307 in most cases message passing will look exactly the same.
308 (There are some exceptions; it's possible for two actors on the same
309 hive to "hand off" some special types of data that can't be serialized
310 across processes or the network, eg a socket or a closure, perhaps even
311 one with mutable state.
312 This must be done with care, and the actors should be careful both
313 to ensure that they are both local and that the actor handing things
314 off no longer accesses that value to preserve the actor model.
315 But this is an advanced topic, and we are getting ahead of ourselves.)
316 We have to supply the id of the receiving actor, and usually we'd have
318 But since in this case, since the actor we're sending this to is
319 ourselves, we have to pass in our identifier, since the Hive won't
320 deliver to anything other than an address.
322 Astute readers may observe, since this is a case where we are just
323 referencing our own object, couldn't we just call "sending a line"
324 as a method of our own object without all the message passing?
325 Indeed, we do have such a method, so we @emph{could} rewrite handle-line
329 (define-method (handle-line (irc-bot <my-irc-bot>) message
330 speaker channel line emote?)
331 (irc-bot-send-line irc-bot channel
332 (format #f "Bawwwwk! ~a says: ~a" speaker line)))
335 @dots{} but we want to get you comfortable and familiar with message
336 passing, and we'll be making use of this same message passing shortly
337 so that @emph{other} actors may participate in communicating with IRC
340 Anyway, our current message handler is simply too annoying.
341 What we would really like to do is have our bot respond to individual
342 "commands" like this:
345 <foo-user> examplebot: hi!
346 <examplebot> Oh hi foo-user!
347 <foo-user> examplebot: botsnack
348 <examplebot> Yippie! *does a dance!*
349 <foo-user> examplebot: echo I'm a very silly bot
350 <examplebot> I'm a very silly bot
353 Whee, that looks like fun!
354 To implement it, we're going to pull out Guile's pattern matcher.
357 (define-method (handle-line (irc-bot <my-irc-bot>) message
358 speaker channel line emote?)
359 (define my-name (irc-bot-username irc-bot))
360 (define (looks-like-me? str)
361 (or (equal? str my-name)
362 (equal? str (string-concatenate (list my-name ":")))))
363 (match (string-split line #\space)
364 (((? looks-like-me? _) action action-args ...)
366 ;; The classic botsnack!
368 (<- (actor-id irc-bot) 'send-line channel
369 "Yippie! *does a dance!*"))
371 ((or "hello" "hello!" "hello." "greetings" "greetings." "greetings!"
372 "hei" "hei." "hei!" "hi" "hi!")
373 (<- (actor-id irc-bot) 'send-line channel
374 (format #f "Oh hi ~a!" speaker)))
376 (<- (actor-id irc-bot) 'send-line channel
377 (string-join action-args " ")))
379 ;; ---> Add yours here <---
383 (<- (actor-id irc-bot) 'send-line channel
384 "*stupid puppy look*"))))))
387 Parsing the pattern matcher syntax is left as an exercise for the
390 If you're getting the sense that we could make this a bit less wordy,
394 (define-method (handle-line (irc-bot <my-irc-bot>) message
395 speaker channel line emote?)
396 (define my-name (irc-bot-username irc-bot))
397 (define (looks-like-me? str)
398 (or (equal? str my-name)
399 (equal? str (string-concatenate (list my-name ":")))))
400 (define (respond respond-line)
401 (<- (actor-id irc-bot) 'send-line channel
403 (match (string-split line #\space)
404 (((? looks-like-me? _) action action-args ...)
406 ;; The classic botsnack!
408 (respond "Yippie! *does a dance!*"))
410 ((or "hello" "hello!" "hello." "greetings" "greetings." "greetings!"
411 "hei" "hei." "hei!" "hi" "hi." "hi!")
412 (respond (format #f "Oh hi ~a!" speaker)))
414 (respond (string-join action-args " ")))
416 ;; ---> Add yours here <---
420 (respond "*stupid puppy look*"))))))
423 Okay, that looks pretty good!
424 Now we have enough information to build an IRC bot that can do a lot
426 Take some time to experiment with extending the bot a bit before
427 moving on to the next section!
428 What cool commands can you add?
431 @node Writing our own actors
432 @section Writing our own actors
434 Let's write the most basic, boring actor possible.
435 How about an actor that start sleeping, and keeps sleeping?
438 (use-modules (oop goops)
441 (define-class <sleeper> (<actor>)
442 (actions #:allocation #:each-subclass
443 #:init-value (build-actions
444 (*init* sleeper-loop))))
446 (define (sleeper-loop actor message)
447 (while (actor-alive? actor)
448 (display "Zzzzzzzz....\n")
449 ;; Sleep for one second
450 (8sleep (sleeper-sleep-secs actor))))
452 (let* ((hive (make-hive))
453 (sleeper (bootstrap-actor hive <sleeper>)))
457 We see some particular things in this example.
458 One thing is that our @verb{~<sleeper>~} actor has an actions slot.
459 This is used to look up what the "action handler" for a message is.
460 We have to set the #:allocation to either @verb{~#:each-subclass~} or
461 @verb{~#:class~}.@footnote{#:class should be fine, except there is @uref{https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25211,a bug in Guile} which keeps
462 us from using it for now.}
464 The only action handler we've added is for @verb{~*init*~}, which is called
465 implicitly when the actor first starts up.
466 (This will be true whether we bootstrap the actor before the hive
467 starts or create it during the hive's execution.)
469 In our sleeper-loop we also see a call to "8sleep".
470 "8sleep" is like Guile's "sleep" method, except it is non-blocking
471 and will always yield to the scheduler.
473 Our while loop also checks "actor-alive?" to see whether or not
474 it is still registered.
475 In general, if you keep a loop in your actor that regularly yields
476 to the scheduler, you should check this.@footnote{Or rather, for now you should call @verb{~actor-alive?~} if your code
477 is looping like this.
478 In the future, after an actor dies, its coroutines will
479 automatically be "canceled".}
480 (An alternate way to handle it would be to not use a while loop at all
481 but simply send a message to ourselves with "<-" to call the
482 sleeper-loop handler again.
483 If the actor was dead, the message simply would not be delivered and
484 thus the loop would stop.)
486 It turns out we could have written the class for the actor much more
490 ;; You could do this instead of the define-class above.
491 (define-actor <sleeper> (<actor>)
492 ((*init* sleeper-loop)))
495 This is sugar, and expands into exactly the same thing as the
497 The third argument is an argument list, the same as what's passed
499 Everything after that is a slot.
500 So for example, if we had added an optional slot to specify
501 how many seconds to sleep, we could have done it like so:
504 (define-actor <sleeper> (<actor>)
505 ((*init* sleeper-loop))
506 (sleep-secs #:init-value 1
507 #:getter sleeper-sleep-secs))
510 This actor is pretty lazy though.
511 Time to get back to work!
512 Let's build a worker / manager type system.
518 (define-actor <manager> (<actor>)
519 ((assign-task manager-assign-task))
520 (direct-report #:init-keyword #:direct-report
521 #:getter manager-direct-report))
523 (define (manager-assign-task manager message difficulty)
524 "Delegate a task to our direct report"
525 (display "manager> Work on this task for me!\n")
526 (<- (manager-direct-report manager)
527 'work-on-this difficulty))
530 This manager keeps track of a direct report and tells them to start
531 working on a task@dots{} simple delegation.
532 Nothing here is really new, but note that our friend "<-" (which means
533 "send message") is back.
534 There's one difference this time@dots{} the first time we saw "<-" was in
535 the handle-line procedure of the irc-bot, and in that case we explicitly
536 pulled the actor-id after the actor we were sending the message to
537 (ourselves), which we aren't doing here.
538 But that was an unusual case, because the actor was ourself.
539 In this case, and in general, actors don't have direct references to
540 other actors; instead, all they have is access to identifiers which
541 reference other actors.
544 (define-actor <worker> (<actor>)
545 ((work-on-this worker-work-on-this))
546 (task-left #:init-keyword #:task-left
547 #:accessor worker-task-left))
549 (define (worker-work-on-this worker message difficulty)
550 "Work on one task until done."
551 (set! (worker-task-left worker) difficulty)
552 (display "worker> Whatever you say, boss!\n")
553 (while (and (actor-alive? worker)
554 (> (worker-task-left worker) 0))
555 (display "worker> *huff puff*\n")
556 (set! (worker-task-left worker)
557 (- (worker-task-left worker) 1))
561 The worker also contains familiar code, but we now see that we can
562 call 8sleep with non-integer real numbers.
564 Looks like there's nothing left to do but run it.
567 (let* ((hive (make-hive))
568 (worker (bootstrap-actor hive <worker>))
569 (manager (bootstrap-actor hive <manager>
570 #:direct-report worker)))
571 (run-hive hive (list (bootstrap-message hive manager 'assign-task 5))))
574 Unlike the @verb{~<sleeper>~}, our @verb{~<manager>~} doesn't have an implicit
575 @verb{~*init*~} method, so we've bootstrapped the calling @verb{~assign-task~} action.
578 manager> Work on this task for me!
579 worker> Whatever you say, boss!
587 "<-" pays no attention to what happens with the messages it has sent
589 This is useful in many cases@dots{} we can blast off many messages and
590 continue along without holding anything back.
592 But sometimes we want to make sure that something completes before
593 we do something else, or we want to send a message and get some sort
595 Luckily 8sync comes with an answer to that with "<-wait", which will
596 suspend the caller until the callee gives some sort of response, but
597 which does not block the rest of the program from running.
598 Let's try applying that to our own code by turning our manager
602 ;;; Update this method
603 (define (manager-assign-task manager message difficulty)
604 "Delegate a task to our direct report"
605 (display "manager> Work on this task for me!\n")
606 (<- (manager-direct-report manager)
607 'work-on-this difficulty)
609 ;; Wait a moment, then call the micromanagement loop
611 (manager-micromanage-loop manager))
613 ;;; And add the following
614 ;;; (... Note: do not model actual employee management off this)
615 (define (manager-micromanage-loop manager)
616 "Pester direct report until they're done with their task."
617 (display "manager> Are you done yet???\n")
618 (let ((worker-is-done
619 (mbody-val (<-wait (manager-direct-report manager)
622 (begin (display "manager> Oh! I guess you can go home then.\n")
623 (<- (manager-direct-report manager) 'go-home))
624 (begin (display "manager> Harumph!\n")
626 (when (actor-alive? manager)
627 (manager-micromanage-loop manager))))))
630 We've appended a micromanagement loop here@dots{} but what's going on?
631 "<-wait", as it sounds, waits for a reply, and returns a reply
633 In this case there's a value in the body of the message we want,
634 so we pull it out with mbody-val.
635 (It's possible for a remote actor to return multiple values, in which
636 case we'd want to use mbody-receive, but that's a bit more
639 Of course, we need to update our worker accordingly as well.
642 ;;; Update the worker to add the following new actions:
643 (define-actor <worker> (<actor>)
644 ((work-on-this worker-work-on-this)
646 (done-yet? worker-done-yet?)
647 (go-home worker-go-home))
648 (task-left #:init-keyword #:task-left
649 #:accessor worker-task-left))
652 (define (worker-done-yet? worker message)
653 "Reply with whether or not we're done yet."
654 (let ((am-i-done? (= (worker-task-left worker) 0)))
656 (display "worker> Yes, I finished up!\n")
657 (display "worker> No... I'm still working on it...\n"))
658 (<-reply message am-i-done?)))
660 (define (worker-go-home worker message)
661 "It's off of work for us!"
662 (display "worker> Whew! Free at last.\n")
663 (self-destruct worker))
666 (As you've probably guessed, you wouldn't normally call @verb{~display~}
667 everywhere as we are in this program@dots{} that's just to make the
668 examples more illustrative.)
670 "<-reply" is what actually returns the information to the actor
671 waiting on the reply.
672 It takes as an argument the actor sending the message, the message
673 it is in reply to, and the rest of the arguments are the "body" of
675 (If an actor handles a message that is being "waited on" but does not
676 explicitly reply to it, an auto-reply with an empty body will be
677 triggered so that the waiting actor is not left waiting around.)
679 The last thing to note is the call to "self-destruct".
680 This does what you might expect: it removes the actor from the hive.
681 No new messages will be sent to it.
684 Running it is the same as before:
687 (let* ((hive (make-hive))
688 (worker (bootstrap-actor hive <worker>))
689 (manager (bootstrap-actor hive <manager>
690 #:direct-report worker)))
691 (run-hive hive (list (bootstrap-message hive manager 'assign-task 5))))
694 But the output is a bit different:
697 manager> Work on this task for me!
698 worker> Whatever you say, boss!
701 manager> Are you done yet???
702 worker> No... I'm still working on it...
705 manager> Are you done yet???
707 worker> No... I'm still working on it...
710 manager> Are you done yet???
711 worker> Yes, I finished up!
712 manager> Oh! I guess you can go home then.
713 worker> Whew! Free at last.
717 @node Writing our own network-enabled actor
718 @section Writing our own network-enabled actor
720 So, you want to write a networked actor!
721 Well, luckily that's pretty easy, especially with all you know so far.
724 (use-modules (oop goops)
726 (ice-9 rdelim) ; line delineated i/o
727 (ice-9 match)) ; pattern matching
729 (define-actor <telcmd> (<actor>)
730 ((*init* telcmd-init)
731 (*cleanup* telcmd-cleanup)
732 (new-client telcmd-new-client)
733 (handle-line telcmd-handle-line))
734 (socket #:accessor telcmd-socket
738 Nothing surprising about the actor definition, though we do see that
739 it has a slot for a socket.
740 Unsurprisingly, that will be set up in the @verb{~*init*~} handler.
743 (define (set-port-nonblocking! port)
744 (let ((flags (fcntl port F_GETFL)))
745 (fcntl port F_SETFL (logior O_NONBLOCK flags))))
747 (define (setup-socket)
750 (socket PF_INET SOCK_STREAM 0))
751 ;; reuse port even if busy
752 (setsockopt s SOL_SOCKET SO_REUSEADDR 1)
753 ;; connect to port 8889 on localhost
754 (bind s AF_INET INADDR_LOOPBACK 8889)
755 ;; make it nonblocking and start listening
756 (set-port-nonblocking! s)
760 (define (telcmd-init telcmd message)
761 (set! (telcmd-socket telcmd) (setup-socket))
762 (display "Connect like: telnet localhost 8889\n")
763 (while (actor-alive? telcmd)
764 (let ((client-connection (accept (telcmd-socket telcmd))))
765 (<- (actor-id telcmd) 'new-client client-connection))))
767 (define (telcmd-cleanup telcmd message)
768 (display "Closing socket!\n")
769 (when (telcmd-socket telcmd)
770 (close (telcmd-socket telcmd))))
773 That @verb{~setup-socket~} code looks pretty hard to read!
774 But that's pretty standard code for setting up a socket.
775 One special thing is done though@dots{} the call to
776 @verb{~set-port-nonblocking!~} sets flags on the socket port so that,
777 you guessed it, will be a nonblocking port.
779 This is put to immediate use in the telcmd-init method.
780 This code looks suspiciously like it @emph{should} block@dots{} after
781 all, it just keeps looping forever.
782 But since 8sync is using Guile's suspendable ports code feature,
783 so every time this loop hits the @verb{~accept~} call, if that call
784 @emph{would have} blocked, instead this whole procedure suspends
785 to the scheduler@dots{} automatically!@dots{} allowing other code to run.
787 So, as soon as we do accept a connection, we send a message to
788 ourselves with the @verb{~new-client~} action.
790 Aren't actors only supposed to handle one message at a time?
791 If the telcmd-init loop just keeps on looping and looping,
792 when will the @verb{~new-client~} message ever be handled?
793 8sync actors only receive one message at a time, but by default if an
794 actor's message handler suspends to the agenda for some reason (such
795 as to send a message or on handling I/O), that actor may continue to
796 accept other messages, but always in the same thread.@footnote{This is customizable: an actor can be set up to queue messages so
797 that absolutely no messages are handled until the actor completely
798 finishes handling one message.
799 Our loop couldn't look quite like this though!}
801 We also see that we've established a @verb{~*cleanup*~} handler.
802 This is run any time either the actor dies, either through self
803 destructing, because the hive completes its work, or because
804 a signal was sent to interrupt or terminate our program.
805 In our case, we politely close the socket when @verb{~<telcmd>~} dies.
808 (define (telcmd-new-client telcmd message client-connection)
809 (define client (car client-connection))
810 (set-port-nonblocking! client)
812 (let ((line (read-line client)))
813 (cond ((eof-object? line)
816 (<- (actor-id telcmd) 'handle-line
817 client (string-trim-right line #\return))
818 (when (actor-alive? telcmd)
821 (define (telcmd-handle-line telcmd message client line)
822 (match (string-split line #\space)
823 (("") #f) ; ignore empty lines
826 (strftime "The time is: %c\n" (localtime (current-time)))
829 (format client "~a\n" (string-join rest " ")))
831 (_ (display "Sorry, I don't know that command.\n" client))))
834 Okay, we have a client, so we handle it!
835 And once again@dots{} we see this goes off on a loop of its own!
836 (Also once again, we have to do the @verb{~set-port-nonblocking!~} song and
838 This loop also automatically suspends when it would otherwise block@dots{}
839 as long as read-line has information to process, it'll keep going, but
840 if it would have blocked waiting for input, then it would suspend the
841 agenda.@footnote{If there's a lot of data coming in and you don't want your I/O loop
842 to become too "greedy", take a look at @verb{~setvbuf~}.}
844 The actual method called whenever we have a "line" of input is pretty
845 straightforward@dots{} in fact it looks an awful lot like the IRC bot
846 handle-line procedure we used earlier.
847 No surprises there!@footnote{Well, there may be one surprise to a careful observer.
848 Why are we sending a message to ourselves?
849 Couldn't we have just dropped the argument of "message" to
850 telcmd-handle-line and just called it like any other procedure?
851 Indeed, we @emph{could} do that, but sending a message to ourself has
852 an added advantage: if we accidentally "break" the
853 telcmd-handle-line procedure in some way (say we add a fun new
854 command we're playing with it), raising an exception won't break
855 and disconnect the client's main loop, it'll just break the
856 message handler for that one line, and our telcmd will happily
857 chug along accepting another command from the user while we try
858 to figure out what happened to the last one.}
863 (let* ((hive (make-hive))
864 (telcmd (bootstrap-actor hive <telcmd>)))
868 Open up another terminal@dots{} you can connect via telnet:
871 $ telnet localhost 8889
873 Connected to localhost.
874 Escape character is '^]'.
876 The time is: Thu Jan 5 03:20:17 2017
880 Sorry, I don't know that command.
884 Type @verb{~Ctrl+] Ctrl+d~} to exit telnet.
887 There's more that could be optimized, but we'll consider that to be
888 advanced topics of discussion.
890 So that's a pretty solid intro to how 8sync works!
891 Now that you've gone through this introduction, we hope you'll have fun
892 writing and hooking together your own actors.
893 Since actors are so modular, it's easy to have a program that has
894 multiple subystems working together.
895 You could build a worker queue system that displayed a web interface
896 and spat out notifications about when tasks finish to IRC, and making
897 all those actors talk to each other should be a piece of cake.
903 @node An intermission on live hacking
904 @section An intermission on live hacking
906 This section is optional, but highly recommended.
907 It requires that you're a user of GNU Emacs.
908 If you aren't, don't worry@dots{} you can forge ahead and come back in case
909 you ever do become an Emacs user.
910 (If you're more familiar with Vi/Vim style editing, I hear good things
911 about Spacemacs@dots{})
913 Remember all the way back when we were working on the IRC bot?
914 So you may have noticed while updating that section that the
915 start/stop cycle of hacking isn't really ideal.
916 You might either edit a file in your editor, then run it, or
917 type the whole program into the REPL, but then you'll have to spend
918 extra time copying it to a file.
919 Wouldn't it be nice if it were possible to both write code in a
920 file and try it as you go?
921 And wouldn't it be even better if you could live edit a program
924 Luckily, there's a great Emacs mode called Geiser which makes
925 editing and hacking and experimenting all happen in harmony.
926 And even better, 8sync is optimized for this experience.
927 8sync provides easy drop-in "cooperative REPL" support, and
928 most code can be simply redefined on the fly in 8sync through Geiser
929 and actors will immediately update their behavior, so you can test
930 and tweak things as you go.
932 Okay, enough talking. Let's add it!
933 Redefine run-bot like so:
936 (define* (run-bot #:key (username "examplebot")
937 (server "irc.freenode.net")
938 (channels '("##botchat"))
939 (repl-path "/tmp/8sync-repl"))
940 (define hive (make-hive))
942 (bootstrap-actor hive <my-irc-bot>
945 #:channels channels))
947 (bootstrap-actor hive <repl-manager>
953 If we put a call to run-bot at the bottom of our file we can call it,
954 and the repl-manager will start something we can connect to automatically.
956 Now when we run this it'll start up a REPL with a unix domain socket at
958 We can connect to it in emacs like so:
961 M-x geiser-connect-local <RET> guile <RET> /tmp/8sync-repl <RET>
965 Okay, so what does this get us?
966 Well, we can now live edit our program.
967 Let's change how our bot behaves a bit.
968 Let's change handle-line and tweak how the bot responds to a botsnack.
974 (respond "Yippie! *does a dance!*"))
978 (respond "Yippie! *catches botsnack in midair!*"))
981 Okay, now let's evaluate the change of the definition.
982 You can hit "C-M-x" anywhere in the definition to re-evaluate.
983 (You can also position your cursor at the end of the definition and press
984 "C-x C-e", but I've come to like "C-M-x" better because I can evaluate as soon
985 as I'm done writing.)
986 Now, on IRC, ask your bot for a botsnack.
987 The bot should give the new message@dots{} with no need to stop and start the
990 Let's fix a bug live.
991 Our current program works great if you talk to your bot in the same
992 IRC channel, but what if you try to talk to them over private message?
995 IRC> /query examplebot
996 <foo-user> examplebot: hi!
999 Hm, we aren't seeing any response on IRC!
1000 Huh? What's going on?
1001 It's time to do some debugging.
1002 There are plenty of debugging tools in Guile, but sometimes the simplest
1003 is the nicest, and the simplest debugging route around is good old
1004 fashioned print debugging.
1006 It turns out Guile has an under-advertised feature which makes print
1007 debugging really easy called "pk", pronounced "peek".
1008 What pk accepts a list of arguments, prints out the whole thing,
1009 but returns the last argument.
1010 This makes wrapping bits of our code pretty easy to see what's
1012 So let's peek into our program with pk.
1013 Edit the respond section to see what channel it's really sending
1017 (define-method (handle-line (irc-bot <my-irc-bot>) message
1018 speaker channel line emote?)
1020 (define (respond respond-line)
1021 (<- (actor-id irc-bot) 'send-line (pk 'channel channel)
1028 Now let's ping our bot in both the channel and over PM.
1031 ;;; (channel "##botchat")
1033 ;;; (channel "sinkbot")
1036 Oh okay, this makes sense.
1037 When we're talking in a normal multi-user channel, the channel we see
1038 the message coming from is the same one we send to.
1039 But over PM, the channel is a username, and in this case the username
1040 we're sending our line of text to is ourselves.
1041 That isn't what we want.
1042 Let's edit our code so that if we see that the channel we're sending
1043 to looks like our own username that we respond back to the sender.
1044 (We can remove the pk now that we know what's going on.)
1047 (define-method (handle-line (irc-bot <my-irc-bot>) message
1048 speaker channel line emote?)
1050 (define (respond respond-line)
1051 (<- (actor-id irc-bot) 'send-line
1052 (if (looks-like-me? channel)
1053 speaker ; PM session
1054 channel) ; normal IRC channel
1060 Re-evaluate and test.
1063 IRC> /query examplebot
1064 <foo-user> examplebot: hi!
1065 <examplebot> Oh hi foo-user!
1072 @chapter API reference
1075 @node Systems reference
1076 @chapter Systems reference
1096 * Recommended emacs additions::
1097 * 8sync and Fibers::
1098 * 8sync's license and general comments on copyleft::
1099 * Acknowledgements::
1103 @node Recommended emacs additions
1104 @section Recommended emacs additions
1106 In order for @verb{~mbody-receive~} to indent properly, put this in your
1110 (put 'mbody-receive 'scheme-indent-function 2)
1113 @node 8sync and Fibers
1114 @section 8sync and Fibers
1116 One other major library for asynchronous communication in Guile-land
1117 is @uref{https://github.com/wingo/fibers/,Fibers}.
1118 There's a lot of overlap:
1122 Both use Guile's suspendable-ports facility
1124 Both communicate between asynchronous processes using message passing;
1125 you don't have to squint hard to see the relationship between Fibers'
1126 channels and 8sync's actor inboxes.
1129 However, there are clearly differences too.
1130 There's a one to one relationship between 8sync actors and an actor inbox,
1131 whereas each Fibers fiber may read from multiple channels, for example.
1133 Luckily, it turns out there's a clear relationship, based on real,
1135 8sync is based on the @uref{https://en.wikipedia.org/wiki/Actor_model,actor model} whereas fibers follows
1136 @uref{http://usingcsp.com/,Communicating Sequential Processes (CSP)}, which is a form of
1137 @uref{https://en.wikipedia.org/wiki/Process_calculus,process calculi}.
1138 And it turns out, the
1139 @uref{https://en.wikipedia.org/wiki/Actor_model_and_process_calculi,relationship between the actor model and process calculi} is well documented,
1140 and even more precisely, the
1141 @uref{https://en.wikipedia.org/wiki/Communicating_sequential_processes#Comparison_with_the_Actor_Model,relationship between CSP and the actor model} is well understood too.
1143 So, 8sync and Fibers do take somewhat different approaches, but both
1144 have a solid theoretical backing@dots{} and their theories are well
1145 understood in terms of each other.
1146 Good news for theory nerds!
1148 (Since the actors and CSP are @uref{https://en.wikipedia.org/wiki/Dual_%28mathematics%29,dual}, maybe eventually 8sync will be
1149 implemented on top of Fibers@dots{} that remains to be seen!)
1152 @node 8sync's license and general comments on copyleft
1153 @section 8sync's license and general comments on copyleft
1155 8sync is released under the GNU LGPL (Lesser General Public License),
1156 version 3 or later, as published by the Free Software Foundation.
1157 The short version of this is that if you distribute a modifications to
1158 8sync, whether alone or in some larger combination, must release the
1159 corresponding source code.
1160 A program which uses this library, if distributed without source code,
1161 must also allow relinking with a modified version of this library.
1162 In general, it is best to contribute them back to 8sync under the same terms;
1163 we'd appreciate any enhancements or fixes to be contributed upstream to
1165 (This is an intentional oversimplification for brevity, please read the LGPL
1166 for the precise terms.)
1168 This usage of the LGPL helps us ensure that 8sync and derivatives of
1169 8sync as a library will remain free.
1170 Though it is not a requirement, we request you use 8sync to build free
1171 software rather than use it to contribute to the growing world of
1172 proprietary software.
1174 The choice of the LGPL for 8sync was a strategic one.
1175 This is not a general recommendation to use the LGPL instead of the GPL
1177 In general, we encourage stronger copyleft.
1178 (For more thinking on this position, see
1179 @uref{https://www.gnu.org/licenses/why-not-lgpl.html,
1180 Why you shouldn't use the Lesser GPL for your next library}.)
1182 Although 8sync provides some unique features, its main functionality
1183 is as an asynchronous programming environment, and there are many
1184 other asynchronous programming environments out there such as Node.js
1185 for Javascript and Asyncio for Python (there are others as well).
1186 It is popular in some of these communities to hold anti-copyleft positions,
1187 which is unfortunate, and many community members seem to be adopting
1188 these positions because other developers they look up to are holding
1190 If you have come from one of these communities and are exploring 8sync, we
1191 hope reading this will help you reconsider your position.
1193 In particular, if you are building a library or application that uses
1194 8sync in some useful way, consider releasing your program under the GNU
1196 In a world where more and more software is locked down, where software is used
1197 to restrict users, we could use every chance we can get to provide
1198 protections so that software which is free remains free, and encourages even
1199 more software freedom to be built upon it.
1201 So to answer the question, ``Can I build a proprietary program on top of
1202 8sync?'' our response is
1203 ``Yes, but please don't.
1204 Choose to release your software under a freedom-respecting license.
1205 And help us turn the tide towards greater software freedom...
1206 consider a strong copyleft license!''
1208 @node Acknowledgements
1209 @section Acknowledgements
1211 8sync has a number of inspirations:
1215 @uref{https://docs.python.org/3.5/library/asyncio.html, asyncio}
1216 for Python provides a nice asynchronous programming environment, and
1217 makes great use of generator-style coroutines.
1218 It's a bit more difficult to work with than 8sync (or so thinks the author)
1219 because you have to ``line up'' the coroutines.
1222 @uref{http://dthompson.us/pages/software/sly.html, Sly}
1223 by David Thompson is an awesome functional reactive game programming
1225 If you want to write graphical games, Sly is almost certainly a better choice
1227 Thanks to David for being very patient in explaining tough concepts;
1228 experience on hacking Sly greatly informed 8sync's development.
1229 (Check out Sly, it rocks!)
1232 Reading @uref{https://mitpress.mit.edu/sicp/, SICP}, particularly
1233 @uref{https://mitpress.mit.edu/sicp/full-text/book/book-Z-H-19.html#%_chap_3,
1234 Chapter 3's writings on concurrent systems},
1235 greatly informed 8sync's design.
1238 Finally, @uref{http://xudd.readthedocs.io/en/latest/, XUDD}
1239 was an earlier ``research project'' that preceeded 8sync.
1240 It attempted to bring an actor model system to Python.
1241 However, the author eventually grew frustrated with some of Python's
1242 limitations, fell in love with Guile, and well... now we have 8sync.
1243 Much of 8sync's actor model design came from experiments in developing
1248 The motivation to build 8sync came out of
1249 @uref{https://lists.gnu.org/archive/html/guile-devel/2015-10/msg00015.html,
1251 at the FSF 30th party between Mark Weaver, David Thompson, Andrew
1252 Engelbrecht, and Christopher Allan Webber over how to build
1253 an asynchronous event loop for Guile and just what would be needed.
1255 A little over a month after that, hacking on 8sync began!
1259 @node Copying This Manual
1260 @appendix Copying This Manual
1262 This manual is licensed under the GNU Free Documentation License, with
1263 no invariant sections. At your option, it is also available under the
1264 GNU Lesser General Public License, as published by the Free Software
1265 Foundation, version 3 or any later version.
1270 * GNU Free Documentation License:: License for copying this manual.
1273 @c Get fdl.texi from http://www.gnu.org/licenses/fdl.html
1274 @node GNU Free Documentation License
1275 @section GNU Free Documentation License
1287 @c 8sync.texi ends here