X-Git-Url: https://jxself.org/git/?p=8sync.git;a=blobdiff_plain;f=doc%2F8sync-new-manual.org;h=1caa79b8ccb15adfa5e52c4c46c17b7a9ee5ddae;hp=0115003604e2681b57c7f061f2c19c384afe3f1a;hb=39be3a1d4f667eae7dc31b7677721692aab3a319;hpb=3980bf3b22b75fa51fe514f9a38f4d7232843fe2 diff --git a/doc/8sync-new-manual.org b/doc/8sync-new-manual.org index 0115003..1caa79b 100644 --- a/doc/8sync-new-manual.org +++ b/doc/8sync-new-manual.org @@ -111,16 +111,669 @@ how easy it is to add new networked layers by tacking on a high score to our game, and as a finale we'll dive into writing our own little irc bot framework "from scratch" on top of the 8sync actor model. -Alright, let's get going. +Alright, let's get started. This should be a lot of fun! ** A silly little IRC bot -** Battle bot! +First of all, we're going to need to import some modules. Put this at +the top of your file: -** Adding a "rankings" web page +#+BEGIN_SRC scheme + (use-modules (8sync) ; 8sync's agenda and actors + (8sync systems irc) ; the irc bot subsystem + (oop goops) ; 8sync's actors use GOOPS + (ice-9 format) ; basic string formatting + (ice-9 match)) ; pattern matching +#+END_SRC + +Now we need to add our bot. Initially, it won't do much. + +#+BEGIN_SRC scheme + (define-class ()) + + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (if emote? + (format #t "~a emoted ~s in channel ~a\n" + speaker line channel) + (format #t "~a said ~s in channel ~a\n" + speaker line channel))) +#+END_SRC + +We've just defined our own IRC bot! +This is an 8sync actor. +(8sync uses GOOPS to define actors.) +We extended the handle-line generic method, so this is the code that +will be called whenever the IRC bot "hears" anything. +For now the code is pretty basic: it just outputs whatever it "hears" +from a user in a channel to the current output port. +Pretty boring! +But it should help us make sure we have things working when we kick +things off. + +Speaking of, even though we've defined our actor, it's not running +yet. Time to fix that! + +#+BEGIN_SRC scheme +(define* (run-bot #:key (username "examplebot") + (server "irc.freenode.net") + (channels '("##botchat"))) + (define hive (make-hive)) + (define irc-bot + (hive-create-actor* hive "irc-bot" + #:username username + #:server server + #:channels channels)) + (run-hive hive (list (bootstrap-message hive irc-bot 'init)))) +#+END_SRC + +Actors are connected to something called a "hive", which is a +special kind of actor that runs all the other actors. +Actors can spawn other actors, but before we start the hive we use +this special "hive-create-actor*" method. +It takes the hive as its first argument, the actor class as the second +argument, a decorative "cookie" as the third argument (this is +optional, but it helps with debugging... you can skip it by setting it +to #f if you prefer), and the rest are initialization arguments to the +actor. hive-create-actor* passes back not the actor itself (we don't +get access to that usually) but the *id* of the actor. +(More on this later.) +Finally we run the hive with run-hive and pass it a list of +"bootstrapped" messages. +Normally actors send messages to each other (and sometimes themselves), +but we need to send a message or messages to start things or else +nothing is going to happen. + +We can run it like: + +#+BEGIN_SRC scheme +(run-bot #:username "some-bot-username") ; be creative! +#+END_SRC + +Assuming all the tubes on the internet are properly connected, you +should be able to join the "##botchat" channel on irc.freenode.net and +see your bot join as well. +Now, as you probably guessed, you can't really /do/ much yet. +If you talk to the bot, it'll send messages to the terminal informing +you as such, but it's hardly a chat bot if it's not chatting yet. + +So let's do the most boring (and annoying) thing possible. +Let's get it to echo whatever we say back to us. +Change handle-line to this: + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (<- irc-bot (actor-id irc-bot) 'send-line channel + (format #f "Bawwwwk! ~a says: ~a" speaker line))) +#+END_SRC + +This will do exactly what it looks like: repeat back whatever anyone +says like an obnoxious parrot. +Give it a try, but don't keep it running for too long... this +bot is so annoying it's likely to get banned from whatever channel +you put it in. + +This method handler does have the advantage of being simple though. +It introduces a new concept simply... sending a message! +Whenever you see "<-", you can think of that as saying "send this +message". +The arguments to "<-" are as follows: the actor sending the message, +the id of the actor the message is being sent to, the "action" we +want to invoke (a symbol), and the rest are arguments to the +"action handler" which is in this case send-line (with itself takes +two arguments: the channel our bot should send a message to, and +the line we want it to spit out to the channel). + +(Footnote: 8sync's name for sending a message, "<-", comes from older, +early lisp object oriented systems which were, as it turned out, +inspired by the actor model! +Eventually message passing was dropped in favor of something called +"generic functions" or "generic methods" +(you may observe we made use of such a thing in extending +handle-line). +Many lispers believe that there is no need for message passing +with generic methods and some advanced functional techniques, +but in a concurrent environment message passing becomes useful +again, especially when the communicating objects / actors are not +in the same address space.) + +Normally in the actor model, we don't have direct references to +an actor, only an identifier. +This is for two reasons: to quasi-enforce the "shared nothing" +environment (actors absolutely control their own resources, and +"all you can do is send a message" to request that they modify +them) and because... well, you don't even know where that actor is! +Actors can be anything, and anywhere. +It's possible in 8sync to have an actor on a remote hive, which means +the actor could be on a remote process or even remote machine, and +in most cases message passing will look exactly the same. +(There are some exceptions; it's possible for two actors on the same +hive to "hand off" some special types of data that can't be serialized +across processes or the network, eg a socket or a closure, perhaps even +one with mutable state. +This must be done with care, and the actors should be careful both +to ensure that they are both local and that the actor handing things +off no longer accesses that value to preserve the actor model. +But this is an advanced topic, and we are getting ahead of ourselves.) +We have to supply the id of the receiving actor, and usually we'd have +only the identifier. +But since in this case, since the actor we're sending this to is +ourselves, we have to pass in our identifier, since the Hive won't +deliver to anything other than an address. + +Astute readers may observe, since this is a case where we are just +referencing our own object, couldn't we just call "sending a line" +as a method of our own object without all the message passing? +Indeed, we do have such a method, so we /could/ rewrite handle-line +like so: + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (irc-bot-send-line irc-bot channel + (format #f "Bawwwwk! ~a says: ~a" speaker line))) +#+END_SRC + +... but we want to get you comfortable and familiar with message +passing, and we'll be making use of this same message passing shortly +so that /other/ actors may participate in communicating with IRC +through our IRC bot. + +Anyway, our current message handler is simply too annoying. +What would be much more interesting is if we could recognize +when an actor could repeat messages /only/ when someone is speaking +to it directly. +Luckily this is an easy adjustment to make. + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (define my-name (irc-bot-username irc-bot)) + (define (looks-like-me? str) + (or (equal? str my-name) + (equal? str (string-concatenate (list my-name ":"))))) + (when (looks-like-me?) + (<- irc-bot (actor-id irc-bot) 'send-line channel + (format #f "Bawwwwk! ~a says: ~a" speaker line)))) +#+END_SRC + +This is relatively straightforward, but it isn't very interesting. +What we would really like to do is have our bot respond to individual +"commands" like this: + +#+BEGIN_SRC text + examplebot: hi! + Oh hi foo-user! + examplebot: botsnack + Yippie! *does a dance!* + examplebot: echo I'm a very silly bot + I'm a very silly bot +#+END_SRC + +Whee, that looks like fun! +To implement it, we're going to pull out Guile's pattern matcher. + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (define my-name (irc-bot-username irc-bot)) + (define (looks-like-me? str) + (or (equal? str my-name) + (equal? str (string-concatenate (list my-name ":"))))) + (match (string-split line #\space) + (((? looks-like-me? _) action action-args ...) + (match action + ;; The classic botsnack! + ("botsnack" + (<- irc-bot (actor-id irc-bot) 'send-line channel + "Yippie! *does a dance!*")) + ;; Return greeting + ((or "hello" "hello!" "hello." "greetings" "greetings." "greetings!" + "hei" "hei." "hei!" "hi" "hi!") + (<- irc-bot (actor-id irc-bot) 'send-line channel + (format #f "Oh hi ~a!" speaker))) + ("echo" + (<- irc-bot (actor-id irc-bot) 'send-line channel + (string-join action-args " "))) + + ;; ---> Add yours here <--- + + ;; Default + (_ + (<- irc-bot (actor-id irc-bot) 'send-line channel + "*stupid puppy look*")))))) +#+END_SRC + +Parsing the pattern matcher syntax is left as an exercise for the +reader. + +If you're getting the sense that we could make this a bit less wordy, +you're right: + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + (define my-name (irc-bot-username irc-bot)) + (define (looks-like-me? str) + (or (equal? str my-name) + (equal? str (string-concatenate (list my-name ":"))))) + (define (respond respond-line) + (<- irc-bot (actor-id irc-bot) 'send-line channel + respond-line)) + (match (string-split line #\space) + (((? looks-like-me? _) action action-args ...) + (match action + ;; The classic botsnack! + ("botsnack" + (respond "Yippie! *does a dance!*")) + ;; Return greeting + ((or "hello" "hello!" "hello." "greetings" "greetings." "greetings!" + "hei" "hei." "hei!" "hi" "hi." "hi!") + (respond (format #f "Oh hi ~a!" speaker))) + ("echo" + (respond (string-join action-args " "))) + + ;; ---> Add yours here <--- + + ;; Default + (_ + (respond "*stupid puppy look*")))))) +#+END_SRC + +Okay, that looks pretty good! +Now we have enough information to build an IRC bot that can do a lot +of things. +Take some time to experiment with extending the bot a bit before +moving on to the next section! +What cool commands can you add? + +** An intermission: about live hacking + +This section is optional, but highly recommended. +It requires that you're a user of GNU Emacs. +If you aren't, don't worry... you can forge ahead and come back in case +you ever do become an Emacs user. +(If you're more familiar with Vi/Vim style editing, I hear good things +about Spacemacs...) + +So you may have noticed while updating the last section that the +start/stop cycle of hacking isn't really ideal. +You might either edit a file in your editor, then run it, or +type the whole program into the REPL, but then you'll have to spend +extra time copying it to a file. +Wouldn't it be nice if it were possible to both write code in a +file and try it as you go? +And wouldn't it be even better if you could live edit a program +while it's running? + +Luckily, there's a great Emacs mode called Geiser which makes +editing and hacking and experimenting all happen in harmony. +And even better, 8sync is optimized for this experience. +8sync provides easy drop-in "cooperative REPL" support, and +most code can be simply redefined on the fly in 8sync through Geiser +and actors will immediately update their behavior, so you can test +and tweak things as you go. + +Okay, enough talking. Let's add it! +Redefine run-bot like so: + +#+BEGIN_SRC scheme + (define* (run-bot #:key (username "examplebot") + (server "irc.freenode.net") + (channels '("##botchat")) + (repl-path "/tmp/8sync-repl")) + (define hive (make-hive)) + (define irc-bot + (hive-create-actor* hive "irc-bot" + #:username username + #:server server + #:channels channels)) + (define repl-manager + (hive-create-actor* hive "repl" + #:path repl-path)) + + (run-hive hive (list (bootstrap-message hive irc-bot 'init) + (bootstrap-message hive repl-manager 'init)))) +#+END_SRC + +If we put a call to run-bot at the bottom of our file we can call it, +and the repl-manager will start something we can connect to automatically. +Horray! +Now when we run this it'll start up a REPL with a unix domain socket at +the repl-path. +We can connect to it in emacs like so: + +: M-x geiser-connect-local guile /tmp/8sync-repl + +Okay, so what does this get us? +Well, we can now live edit our program. +Let's change how our bot behaves a bit. +Let's change handle-line and tweak how the bot responds to a botsnack. +Change this part: + +#+BEGIN_SRC scheme + ;; From this: + ("botsnack" + (respond "Yippie! *does a dance!*")) + + ;; To this: + ("botsnack" + (respond "Yippie! *catches botsnack in midair!*")) +#+END_SRC + +Okay, now let's evaluate the change of the definition. +You can hit "C-M-x" anywhere in the definition to re-evaluate. +(You can also position your cursor at the end of the definition and press +"C-x C-e", but I've come to like "C-M-x" better because I can evaluate as soon +as I'm done writing.) +Now, on IRC, ask your bot for a botsnack. +The bot should give the new message... with no need to stop and start the +program! + +Let's fix a bug live. +Our current program works great if you talk to your bot in the same +IRC channel, but what if you try to talk to them over private message? + +#+BEGIN_SRC text +IRC> /query examplebot + examplebot: hi! +#+END_SRC + +Hm, we aren't seeing any response on IRC! +Huh? What's going on? +It's time to do some debugging. +There are plenty of debugging tools in Guile, but sometimes the simplest +is the nicest, and the simplest debugging route around is good old +fashioned print debugging. + +It turns out Guile has an under-advertised feature which makes print +debugging really easy called "pk", pronounced "peek". +What pk accepts a list of arguments, prints out the whole thing, +but returns the last argument. +This makes wrapping bits of our code pretty easy to see what's +going on. +So let's peek into our program with pk. +Edit the respond section to see what channel it's really sending +things to: + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + ;; [... snip ...] + (define (respond respond-line) + (<- irc-bot (actor-id irc-bot) 'send-line (pk 'channel channel) + respond-line)) + ;; [... snip ...] + ) +#+END_SRC + +Re-evaluate. +Now let's ping our bot in both the channel and over PM. + +#+BEGIN_SRC text +;;; (channel "##botchat") + +;;; (channel "sinkbot") +#+END_SRC + +Oh okay, this makes sense. +When we're talking in a normal multi-user channel, the channel we see +the message coming from is the same one we send to. +But over PM, the channel is a username, and in this case the username +we're sending our line of text to is ourselves. +That isn't what we want. +Let's edit our code so that if we see that the channel we're sending +to looks like our own username that we respond back to the sender. +(We can remove the pk now that we know what's going on.) + +#+BEGIN_SRC scheme + (define-method (handle-line (irc-bot ) speaker channel + line emote?) + ;; [... snip ...] + (define (respond respond-line) + (<- irc-bot (actor-id irc-bot) 'send-line + (if (looks-like-me? channel) + speaker ; PM session + channel) ; normal IRC channel + respond-line)) + ;; [... snip ...] + ) +#+END_SRC + +Re-evaluate and test. + +#+BEGIN_SRC text +IRC> /query examplebot + examplebot: hi! + Oh hi foo-user! +#+END_SRC + +Horray! + +** Writing our own actors and sending messages between them + +Let's write the most basic, boring actor possible. +How about an actor that start sleeping, and keeps sleeping? + +#+BEGIN_SRC scheme + (use-modules (oop goops) + (8sync)) + + (define-class () + (actions #:allocation #:each-subclass + #:init-value (build-actions + (loop sleeper-loop)))) + + (define (sleeper-loop actor message) + (while (actor-am-i-alive? actor) + (display "Zzzzzzzz....\n") + ;; Sleep for one second + (8sleep 1))) + + (let* ((hive (make-hive)) + (sleeper (hive-create-actor hive ))) + (run-hive hive (list (bootstrap-message hive sleeper 'loop)))) +#+END_SRC + +We see some particular things in this example. +One thing is that our actor has an actions slot. +This is used to look up what the "action handler" for a message is. +We have to set the #:allocation to either #:each-subclass or #:class. +(#:class should be fine, except there is [[https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25211][a bug in Guile]] which keeps +us from using it for now.) + +In our sleeper-loop we also see a call to "8sleep". +"8sleep" is like Guile's "sleep" method, except it is non-blocking +and will always yield to the scheduler. + +Our while loop also checks "actor-am-i-alive?" to see whether or not +it is still registered. +In general, if you keep a loop in your actor that regularly yields +to the scheduler, you should check this. +(An alternate way to handle it would be to not use a while loop at all +but simply send a message to ourselves with "<-" to call the +sleeper-loop handler again. +If the actor was dead, the message simply would not be delivered and +thus the loop would stop.) + +This actor is pretty lazy though. +Time to get back to work! + +#+BEGIN_SRC scheme + (use-modules (8sync) + (oop goops)) + + (define-class () + (direct-report #:init-keyword #:direct-report + #:getter manager-direct-report) + (actions #:allocation #:each-subclass + #:init-value (build-actions + (assign-task manager-assign-task)))) + + (define (manager-assign-task manager message difficulty) + "Delegate a task to our direct report" + (display "manager> Work on this task for me!\n") + (<- manager (manager-direct-report manager) + 'work-on-this difficulty)) +#+END_SRC + +Here we're constructing a very simple manager actor. +This manager keeps track of a direct report and tells them to start +working on a task... simple delegation. +Nothing here is really new, but note that our friend "<-" (which means +"send message") is back. +There's one difference this time... the first time we saw "<-" was in +the handle-line procedure of the irc-bot, and in that case we explicitly +pulled the actor-id after the actor we were sending the message to +(ourselves), which we aren't doing here. +But that was an unusual case, because the actor was ourself. +In this case, and in general, actors don't have direct references to +other actors; instead, all they have is access to identifiers which +reference other actors. + +#+BEGIN_SRC scheme + (define-class () + (task-left #:init-keyword #:task-left + #:accessor worker-task-left) + (actions #:allocation #:each-subclass + #:init-value (build-actions + (work-on-this worker-work-on-this)))) + + (define (worker-work-on-this worker message difficulty) + "" + (set! (worker-task-left worker) difficulty) + (display "worker> Whatever you say, boss!\n") + (while (and (actor-am-i-alive? worker) + (> (worker-task-left worker) 0)) + (display "worker> *huff puff*\n") + (set! (worker-task-left worker) + (- (worker-task-left worker) 1)) + (8sync (/ 1 3))) + (display "worker> Looks like I'm done! Can I go home yet?\n")) +#+END_SRC + +The worker also contains familiar code, but we now see that we can +call 8sync with non-integer real numbers. + +Looks like there's nothing left to do but run it: + +#+BEGIN_SRC scheme + (let* ((hive (make-hive)) + (worker (hive-create-actor hive )) + (manager (hive-create-actor hive + #:direct-report worker))) + (run-hive hive (list (bootstrap-message hive manager 'assign-task 5)))) +#+END_SRC + +#+BEGIN_SRC text +manager> Work on this task for me! +worker> Whatever you say, boss! +worker> *huff puff* +worker> *huff puff* +worker> *huff puff* +worker> *huff puff* +worker> *huff puff* +worker> Looks like I'm done! Can I go home yet? +#+END_SRC + +"<-" pays no attention to what happens with the messages it has sent +off. +This is useful in many cases... we can blast off many messages and +continue along without holding anything back. + +But sometimes we want to make sure that something completes before +we do something else, or we want to send a message and get some sort +of information back. +Luckily 8sync comes with an answer to that with "<-wait", which will +suspend the caller until the callee gives some sort of response, but +which does not block the rest of the program from running. +Let's try applying that to our own code by turning our manager +into a micromanager. + +#+END_SRC +#+BEGIN_SRC scheme + ;;; Update this method + (define (manager-assign-task manager message difficulty) + "Delegate a task to our direct report" + (display "manager> Work on this task for me!\n") + (<- manager (manager-direct-report manager) + 'work-on-this difficulty) + + ;; call the micromanagement loop + (manager-micromanage-loop manager)) + + ;;; And add the following + ;;; (... Note: do not model actual employee management off this) + (define (manager-micromanage-loop manager) + "Pester direct report until they're done with their task." + (display "manager> Are you done yet???\n") + (let ((still-working + (msg-val (<-wait manager (manager-direct-report manager) + 'done-yet?)))) + (if still-working + (begin (display "manager> Harumph!\n") + (8sleep 1) + (when (actor-am-i-alive? manager) + (manager-micromanage-loop manager))) + (begin (display "manager> Oh! I guess you can go home then.\n") + (<- manager (manager-direct-report manager) 'go-home))))) +#+END_SRC + +We've appended a micromanagement loop here... but what's going on? +"<-wait", as it sounds, waits for a reply, and returns a reply +message. +In this case there's a value in the body of the message we want, +so we pull it out with msg-val. +(It's possible for a remote actor to return multiple values, in which +case we'd want to use msg-receive, but that's a bit more complicated.) + +Of course, we need to update our worker accordingly as well. + +#+BEGIN_SRC scheme + ;;; Update the worker to add the following new actions: + (define-class () + (task-left #:init-keyword #:task-left + #:accessor worker-task-left) + (actions #:allocation #:each-subclass + #:init-value (build-actions + (work-on-this worker-work-on-this) + ;; Add these: + (done-yet? worker-done-yet?) + (go-home worker-go-home)))) + + ;;; New procedures: + (define (worker-done-yet? worker message) + "Reply with whether or not we're done yet." + (<-reply worker message + (= (worker-task-left worker) 0))) + + (define (worker-go-home worker message) + "It's off of work for us!" + (display "worker> Whew! Free at last.") + (self-destruct worker)) +#+END_SRC + +"<-reply" is what actually returns the information to the actor +waiting on the reply. +It takes as an argument the actor sending the message, the message +it is in reply to, and the rest of the arguments are the "body" of +the message. +(If an actor handles a message that is being "waited on" but does not +explicitly reply to it, an auto-reply with an empty body will be +triggered so that the waiting actor is not left waiting around.) + +The last thing to note is the call to "self-destruct". +This does what you might expect: it removes the actor from the hive. +No new messages will be sent to it. +Ka-poof! ** Writing our own from scratch * API reference +* Systems reference +** IRC +** Web / HTTP +** COMMENT Websockets