X-Git-Url: https://jxself.org/git/?a=blobdiff_plain;f=md%2Fdebugging.md;h=7c457100ab161ebae55add8802f6e509691ea182;hb=74d401dab72600e0a1fec42caf940d345a893552;hp=81c1c8e3d36211933ec383f30f1a0d499787b884;hpb=5a625daa780f7e36da60b0e4a743d9b87d53a9d2;p=mudman.git diff --git a/md/debugging.md b/md/debugging.md index 81c1c8e..7c45710 100644 --- a/md/debugging.md +++ b/md/debugging.md @@ -1,28 +1,11 @@ ---- -title: 'A Dynamic Debugging System For Muddle' -author: Joel Mayer Berez -date: January 1978 -abstract: Program debugging is a time consuming process. Conventional - debugging techniques and aids typically give the user a narrow view - of the program's operation, making debugging difficult. A debugging - system that would present a clear overall picture of a program's - behavior and would be both flexible and simple to operate would be a - valuable tool. Such a system was designed and implemented in and for - Muddle, a high-level applicative programming language. This report - discusses the design alternatives considered during the debugging - system's design and implementation phases, the reasons for the - resulting design choices, and the system attributes. A major - attribute of the system (MEND) is that it does not simulate the - program being debugged but instead monitors it from another - process. This attribute results in a robust and viable debugging - system, because MEND need not be modified in order to handle each - new extension to Muddle and/or each new user-defined primitive. ---- +% A Dynamic Debugging System For Muddle +% Joel Mayer Berez +% January 1978 MIT Technical Memo 94 -Laboratory for Computer Science -Massachusetts Institute of Technology +Laboratory for Computer Science\ +Massachusetts Institute of Technology\ 545 Technology Square Cambridge, Massachusetts 02139 Copyright @@ -78,15 +61,11 @@ providing me with company during all-night console sessions; and all of the other ITS and DynaMod hackers who have built a system well worth using. -This research was supported by the Advanced Research Projects Agency -of the Department of Defense and was monitored by the Office of Naval -Research under Contract No. N00014-75-C-0661. - -Introduction -============ +I Introduction +============== -Background ----------- +I.1 Background +-------------- A time-consuming and frustrating aspect of computer programming is the debugging of faulty programs. Current debugging techniques involve @@ -222,8 +201,8 @@ execution when and if a condition was false. This is as far as MUMBLE ever progressed, and it was not in use as of the time of the proposal for the current work. -MEND ----- +I.2 MEND +-------- After the Muddle compiler became operational and many additional new software features became available, it appeared that it would be @@ -260,8 +239,8 @@ may be found in Appendix B.) the ENVIRONMENT of the program. This means the user can examine the state of the program or change it. -Possible Additional Characteristics ------------------------------------ +I.3 Possible Additional Characteristics +--------------------------------------- Certain other characteristics were seen as desirable for MEND but possibly beyond the scope of this project. If time permitted these @@ -284,8 +263,8 @@ features were to be included in the system: actually reverse execution or a simulation displaying information previously stored by the system. -Design and Implementation -------------------------- +I.4 Design and Implementation +----------------------------- MEND was designed with the intent of providing the application programmer with many options so that debugging could proceed in the @@ -341,6 +320,962 @@ quit useful for the analysis and debugging of Muddle programs. It should also serve as a good example of the type of debugging system that can be built around an applicative type language. +II Terminal Display +=================== + +II.1 IMLAC Console Program +-------------------------- + +One basic concern throughout the project was the display: how the +information made available by MEND would be presented to the user. To +a large extent the physical characteristics of Muddle, ITS[^12] +(Incompatible Timesharing System, the operating system used on the +Dynamic Modeling System computer), and the available terminals +dictated what was reasonably possible. + +The terminal most commonly used by users of the Dynamic Modeling +System is the IMLAC PDS-1. This is a minicomputer capable of having +programs loaded into it from the PDP-10 host computer. One program +written for it by Dave Lebling is MSC, a multiple-screen terminal +program. Up to four virtual screens (or pages) can be created that +will individually operate like the actual screen area of the standard +terminal program (SSV[^13]). Output may be directed to any one of the +screens and any screen may be visible or invisible at any instant of +time, at the programmer's option. Selection of screen is controllable +either by program from the host or locally by the user. + +It was originally Intended that MEND would use MSC for its normal +display. One page would constantly show MEND's representation of the +control stack of the application program. Output initiated by the +program being debugged would go to a second page. A third page would +be used for interaction with MEND and would show user typein along +with any output from MEND that one was interested in seeing. The +latter would include, by user request, full displays of both objects +printed in an abbreviated form on the page containing the control +stack of the application program and values being monitored or traced. +The user could switch back and forth among the pages at will during +the execution of the program. The application program would have a +full standard screen to write onto, and ample room would be available +for the information to be displayed by MEND. + +After a fair amount of testing, this proposal was discarded for the +following reasons. First, MSC was supposed to look just like SSV for +individual virtual screens. Unfortunately new features added to SSV +had not also been added to MSC. A primary reason for this disparity +was that the new features encroached upon the IMLAC's character space. +An SSV with all current features can only hold about one and a half +full pages of text, where a page is the amount that can be visible at +one time. The overhead required for additional screens reduces it +still further. Therefore, with all features included, four virtual +screens could each only average about one-third full. Without many of +the current features, people were reluctant to use MSC. + +The second and perhaps most devastating problem with MSC is that it is +not properly supported by ITS as is SSV. Ordinarily the operating +system will keep track of where on the page the terminal's cursor is +and will properly handle such updating as deletions even in the face +of random access performed (if done by request to the system). When +using MSC, ITS does not realize that information is being written onto +more than one screen and will therefore often move the cursor to the +wrong position. + +MEND could completely control positioning for typeout and echoing on +typein but that would add the large overhead of having to run a +non-trivial routine for each character typed out on the display. Also, +because MSC is not the standard console program, requiring the user to +load it before using MEND might discourage the use of MEND. (It takes +between about 30 seconds and a couple of minutes, depending upon +system load, to load a new console program into the IMLAC.) + +From the outset it was intended that MEND be used routinely by +programmers as debugging problems arose. Therefore it was decided that +the proper way for such a system to operate was to use, as far as +possible, the common environment so as to keep the overhead for +invoking MEND small. This philosophy, which had been seen to affect +the success of many earlier projects, decided between the alternatives +and in this case led to the final decision to use SSV instead of MSC. + +II.2 Terminal Independence +-------------------------- + +The MEND terminal handling capabilities are actually quite general and +MEND does not depend exclusively on SSV. For the purpose of dividing +the screen area into several sections, horizontal lines are sometimes +drawn (see Appendix A showing sample display). With an IMLAC and SSV +these lines could be drawn quite simply using graphics mode. However, +for purposes of generality with regard to terminals, these lines are +instead formed by using underbar characters (on a line of their own). +By using no actual graphics MEND can be used with almost any display +terminal having random access. MEND outputs display commands, such as +clearing a line, as escape codes to ITS which then translates these +into the appropriate commands for the terminal in use. ITS currently +knows about several types of display terminals in use at the +Laboratory for Computer Science, and other types of terminals located +at foreign sites on the ARPA network may be handled by interface +software that simulates a known type. Naturally MEND can handle a +large range of possible line and screen lengths. (The current version +of SSV provides four possible character sizes.) + +II.3 Control of Screen Sectioning +--------------------------------- + +There still remained the question of how to handle the +multi-sectioning of the displayed information. Originally three +sections corresponding to the three virtual screens in the aborted MSC +implementation were planned. The bottom section would hold user typein +and application program output. Three possible methods of achieving +this involved having ITS, Muddle, or MEND do various amounts of the +work with increasing overhead and decreasing speed for those three, +respectively. + +The most attractive solution utilized an ITS feature allowing the +specification of an echo area at the bottom of the screen where echoed +input would always be printed (with the echoing handled by the system, +which is the normal case) . After some experimentation this method of +handling the typein and application program output was rejected +because typeout and deletion are handled by Muddle which ignores the +echo area MEND would effectively have had to control all typeout and +monitor all typein, which would have made the echo area useless. + +Experimentation with the second solution, an indirect method, involved +monitoring of special Muddle memory locations where information +concerning horizontal and vertical page positions is stored. It was +soon discovered that Muddle becomes confused quickly, contains several +bugs with respect to this position information, and generally has a +poor idea of where it is actually printing. + +The third solution appeared to be the most painful from an +implementation and efficiency point of view. MEND would need to +control the printing of every character on output and certain +characters on input, constantly checking page positions by querying +the system. Not only did this slow output, but also MEND was forced to +constantly move the cursor to a safe position in case Muddle managed +to sneak some output past it, which it occasionally does. + +Fortuitously the problem was neatly solved when a little known feature +of ITS was discovered. It Is possible to open a channel to the +terminal in a mode that would cause all output to appear in the echo +area. By creating this echo area and reopening Muddle's normal +terminal output channel in this mode, it is possible to cause Muddle +and the application program to think that the entire screen consists +of only the echo area. All output including application program +display escape commands is automatically routed by ITS to this echo +area. MEND sends its output to a second channel opened in the normal +mode, thereby allowing it to use an area of the screen unknown to and +left untouched by the application program. The physical cursor stays +where the last used logical cursor left it, thereby eliminating most +of the unnecessary cursor movement, resulting in a more pleasing +visual effect. + +As a result of this solution, the screen is divided into only two main +sections. All typein appears in the lower section, whether it is to +the application program or to MEND. Application program output also +goes to the lower section, as does unstructured output produced by +interaction with MEND. The upper section, in general, contains only +those items that occupy one line of the display each. As will be +explained later, these items include all output automatically +displayed by MEND during execution of the application program. + +III MSTACK: MEND's Representation of the Control Stack +====================================================== + +III.1 Program Execution In Muddle +--------------------------------- + +MEND's primary role is to allow the applications programmer to +visually monitor the execution of a program. In a language like +Muddle, this is most easily accomplished by showing the programmer a +picture of the control stack. + +A Muddle program consists of the evaluation of a single object. The +object is usually structured in some manner and itself contains other +objects. The most common object is the FORM. This is a list of objects +in which the first is (or evaluates to) some function and the rest are +arguments to that function. A FORM is evaluated by applying the +function to its arguments, usually after the arguments are themselves +evaluated. This evaluation actually is initiated by the Muddle +interpreter by applying the function EVAL to the original object. EVAL +takes an object as its argument and returns the value to which it +evaluates. + +Figure 1 shows a (simplified) static representation of the evaluation +of a Muddle object. Starting with the FORM (list of objects in +angle-brackets) to be evaluated, the flow of control/evaluation may be +described as a depth-first search through the tree pictured. The +arrows represent values being returned to previous levels. At the end +of this "search" the FORM returns the value shown at the top. + + Figure 1: + + 31 + ↑ + <+ .A .B .C 5> + ↑ ↑ ↑ ↑ ↑ + + .A .B .C 5 + ↑ ↑ ↑ + 6 8 12 + +Typically the control stack will start with one object on it, the FORM +to be evaluated. This evaluation will first require that the objects +in the FORM (possibly FORMs themselves) be added to the stack and +evaluated. EVAL recursively calls itself for this purpose. The stack +builds (downward, by convention) until some object is placed on it +which ls known by EVAL to need no further evaluation. This object is +returned as its own value to the previous level and values continue to +be returned upward until a level is reached where another object must +be evaluated. In this manner, the stack grows and shrinks until the +topmost (initial) object returns a value. + +III.2 Monitoring Program Execution +---------------------------------- + +The manner in which the stack builds, the objects are evaluated, and +the values are returned illustrate most of what the program is doing. +Other factors, including side effects and complied code, will be +discussed at the end of this chapter. MEND's main display therefore +shows a representation of the stack being continuously updated as +execution/evaluation proceeds. + +There are essentially two ways that MEND could follow the flow of +control of the application program. The most direct way, as attempted +by MUMBLE[^14] and discussed in the last chapter, would be for MEND to +execute the application program by simulating the operation of Muddle. +This type of simulation has been shown to require a complex end all +too often fragile structure. The debugging program would need to be +constantly updated to match changes and additions to the Muddle +language , but more importantly it would fail to properly handle +programmer-defined primitives, several varieties of which are provided +for by Muddle. + +A far more satisfactory method is to allow Muddle to execute the +application program in more or less its normal fashion but to stand +beck somewhere and watch. Fortunately Muddle contains a mechanism +ideal for this, called multiprocessing. Basically another control +stack , or process, may be created (independent of the first) that may +be used to execute a different function with its own set of variable +bindings. One such process, in this case MEND, may place another, the +application program, into a single step mode where the latter will be +stopped before each call to EVAL and again as each call returns. The +MEND process will at these points be restarted and given information +about what the application process is doing. MEND stores this +information in a multi-level structure it creates called an MSTACK. + +Each level of an MSTACK corresponds to a level of the control stack +being monitored. Each level contains the original object (actually, a +pointer to it) being evaluated at that level and a new object, called +the "displayed object", that will or does contain the results of +evaluating each of the arguments or elements of the original. The +displayed object is initially the same as the original (in a real +sense, it is the original) but is systematically rebuilt as each +element is evaluated and replaced by what it returns. Thus MEND can +keep track of the relationship between the changing display and the +unchanging program. (Figure 2 shows the various stages that the +displayed object corresponding to the FORM being evaluated in Figure 1 +goes through. Each stage would in fact be painted over the previous +one so that all of these stages represent only one line of the screen. +The down-arrow shown in some of the stages is a place-holder that +represents an object that is actually expanded on the next line of the +screen.) A pointer is also kept showing which element is currently +above this on the stack to be evaluated unless, of course, this is the +initial element of the stack. + + Figure 2: + + <+ .A .B .C 5> + <+ ↓ .B .C 5> + <+ 6 .B .C 5> + <+ 6 ↓ .C 5> + <+ 6 8 .C 5> + <+ 6 8 ↓ 5> + <+ 6 8 12 5> + + 31 + +III.3 A Displayed Representation of Program Execution +----------------------------------------------------- + +The printed representation of the stack occupies the top section of +the screen and is the most prominent and important characteristic of +MEND. Most often, in fact, it is only necessary to watch this display +as the application program is executed to ascertain where a bug is +located or to observe exactly how the program operates. Therefore +considerable time has been spent making the operation of the MSTACK +and associated display as natural and informative as possible. + +Using the collected information described in the previous section, a +representation of the stack is displayed as a number of lines, each +corresponding to one level. Each line shows a level number and the +displayed object printed in "&-printing", named for the printing +function &[^15] created by Greg Pfister. Although strictly speaking +the stack builds upward (towards higher memory locations), it seams +more natural to display and speak of the stack as building from the +top downward, in the direction that printing normally occurs. As each +element of the bottom level (the bottom line on this section of the +screen) is placed on the stack for evaluation, a new line is added +beneath the previous bottom-level line showing that element, and the +element is replaced in the previous line by a pointer ("↓") marking +its location. When the element finally returns a value, the returned +value will replace the pointer in the displayed object. + +To avoid visual distraction, a minimum of updating is done on the +screen. In most cases random access is used to replace only those +lines that have changed. In general the complete stack will not fit in +the display area allocated for it (whose size is user adjustable) so a +scrolling procedure has been devised. The complete display area is +rewritten whenever an attempt is made to write past the bottom or +erase upward past a certain level while some lines are invisible +because they have been scrolled off. The scrolling parameters have +been selected to optimize the number of lines visible on the average +vs. the frequency of scrolling. Level numbers allow the user to see +how much is hidden "above the screen" and how deep the evaluation is +nested. + +III.4 MEND Program Steps Vs. Muddle Steps +----------------------------------------- + +To make it easier for the programmer to follow what the application +program is doing, the speed of execution of the program must be +controllable. MEND does this by inserting a constant, user adjustable +delay between program "steps." One MEND step is not precisely the same +as one Muddle step. Remember that a Muddle step is one call to or +return from EVAL. Each MSTACK level, and therefore each displayed +line, will have two Muddle steps associated with it. At the first +step, MEND will create the level and add one line to the screen. At +the second step, MEND will erase that line and put the returned value +back into the previous line. For clarity MEND adds a third step +between these two which actually occurs at the second Muddle stop. +Before substituting into the previous line, the current one is first +replaced by the returned value, the normal delay occurs, and then the +"second" step occurs as described. + +Some types of Muddle objects are self-evaluating; EVAL will simply +return the object it was given. Although nothing interesting has +happened, two steps have occurred. In this case MEND will avoid +clutter by pretending that no steps have occurred. (MEND only does +this with built-in types that MEND recognizes, and the programmer +should recognize, as being self-evaluating. Programmer-defined types +that are self-evaluating will generate the usual number of MEND +steps.) + +Another case of disparity between Muddle and MEND stops is more +complex. First some further explanations about Muddle objects are +needed Generally the interesting objects, the ones that generate MEND +steps, are linear (usually list) structures containing a number of +other Muddle objects, as are the FORMs described earlier. Normally +during evaluation of such an object the elements will be evaluated one +at a time from first to last. However this sequence is not always +followed by Muddle. MEND cannot directly determine which element is +about to be evaluated at each call to EVAL. It is only given the +object to be evaluated itself and not its position in the parent +structure. It is normally sufficient to do a comparison of this object +with the elements of the parent, starting with the first element +believed to not yet have been evaluated. Naturally, if the elements +are evaluated out of order, this procedure may fail to find the +desired match, because a Muddle object may contain the same element in +two or more positions. Thus it is possible to match the +about-to-be-evaluated object to the wrong occurrence of it. Further +complications arise because some functions can sometimes back up and +re-evaluate their arguments. + +A strong attempt was made to make MEND dependent only upon the general +characteristics of Muddle functions and not upon specific exceptions +and idiosyncrasies. It was felt to be desirable to make MEND +independent of both future changes to the language and +programmer-defined "primitives" that would not be known to MEND. +Besides, without actually simulating Muddle it is not possible to +always get the information MEND needs. It was felt that the "general +rule" approach would take care of a sufficiently large number of cases +without falling into the simulator problem. + +It was determined after some experimentation, however, that MEND could +not be made to work properly without some specific knowledge about +several important cases in Muddle. Two functions, PROG and REPEAT, +allow for branching the flow of control. They are normally +first-to-last functions as described above but at times control may +jump backwards to re-evaluate some elements. MEND was implemented so +that if it cannot locate an element in its normal search path, it will +start looking again from the beginning of the structure. If it is then +found, the displayed object will be reinitialized to be as if +evaluation had not yet proceeded past that point. Evaluation will then +continue normally. + +Another phenomenon of Muddle that we must discuss is what this author +labels the "clause" behavior. A clause is a list of objects given to a +function as a single argument. The list is not itself evaluated, but +some or all of its elements are evaluated. The most common function +illustrating this behavior is COND, a general purpose conditional +function. COND's arguments are all lists of objects. It sequences +through these lists, evaluating the first object in each, until an +evaluated object returns something considered "true." Then the rest of +the objects in that list are evaluated and COND returns what the last +object in the list returns. MEND's normal search path only looks at +top-level elements and would therefore never find the ones actually +being evaluated. + +This phenomenon seemed to be more widespread than the PROG/REPEAT one +and could not be easily attributed to certain functions. The solution +chosen here was to in all cases do a nested search in elements that +looked as if they might be clauses. (The search actually goes one +extra level deep to allow for certain special cases.) When a match is +found in a clause, MEND will for clarity generate extra steps to make +it appear to the user as if first the clause and then its appropriate +element was put on the stack for evaluation. The clause will stay on +the stack until some element that is not above it in the evaluation +tree is evaluated. At that time the clause will be removed in an +orderly manner, and the new element or clause plus element will be put +onto the stack. To do this smoothly, up to six MEND steps may have to +be generated for the one Muddle step. (See the example in Appendix A.) + +III.5 What the User Does Not See +-------------------------------- + +MEND, in its display of the MSTACK, attempts to show the user +everything of importance that is happening as the program is executed. +However certain features of Muddle cannot be captured in this sort of +representation. + +One such feature is the existence of compiled code. Although Muddle +was originally intended to be a high-level interpretive language, an +assembler was written[^16], producing machine code that executes in +the Muddle environment, to allow programmers to create "primitives" +that perform functions not otherwise available in Muddle. Once the +assembler was written, it was natural that a compiler[^17] followed. +It translates normal Muddle code into Muddle assembly code which is +then assembled. Typically Muddle programs are tested interpretively +and, when fully debugged, complied in order to obtain a major increase +in speed of execution. + +A call to a compiled (or assembled) function usually looks to the +programmer and to MEND like a call to a Muddle primitive. The +operation of the function is not seen except when it calls uncompiled +functions. This does not present a major problem to MEND since +generally only uncompiled functions are being debugged, and the +compiled ones encountered are hopefully performing known functions +properly. + +One feature of Muddle that MEND is unable to cope with is the +interrupt system. The programmer may enable a large class of +interrupts and assign handling functions wherever desired. Examples +Include interrupts for characters being typed to a certain input +channel and notification that the system is about to be brought down. +(MEND uses interrupts to catch single character commands and to catch +errors.) + +Recall that MEND monitors application program execution by placing its +process into a single-stepping mode. When Muddle passes control to an +interrupt handier, it temporarily causes the process to leave +single-stepping mode. This is necessary partially because such a +handler may be specified to run in a process other than the current +one at the time of the interrupt. It unfortunately makes the handler +invisible to MEND. it is therefore not currently possible to use MEND +to debug interrupt handlers. + +There is a whole series of side-effects, such as printing by the +application program, that are not directly seen in the MSTACK +representation but are made visible by various other features of MEND. +These will be discussed where appropriate in later sections. + +IV Issuing Commands to MEND +=========================== + +IV.1 Types of Commands +---------------------- + +There is a need in debugging systems similar to MEND for two types of +commands for controlling operation. One type is a set of short, +immediate-action commands that the user of the debugging system may +issue at any time, such as a command to completely stop all activities +of the debugging system and to exit. The other type is the set of all +commands not belonging to the first set. This includes commands that +take arguments and those that can only be given while the application +program is suspended from execution. + +IV.2 Immediate Interrupt-Level Commands +--------------------------------------- + +In MEND's normal mode, various actions occur automatically. Most +importantly, the application program is executing and the displayed +representation of the MSTACK is being updated correspondingly. During +this time the programmer may type ahead to the application program, +but, since there is only one physical cursor that is used by both +MEND's display and typein echoing that must therefore jump back and +forth between the two text entry positions, a large amount of typein +is awkward. Therefore, all immediate action commands that the user can +issue while MEND is in automatic mode are single characters. To +minimize the chance of conflict with typein that the executing +application program might read, MEND immediate action commands are +invoked by typing certain ASCII control characters. Furthermore typing +one of these control characters causes an interrupt to occur which is +handled by MEND and allows immediate response. + +Note that this arrangement requires that there be certain reserved +characters, as listed below, that cannot be used to communicate with +the application program. Normally this presents no difficulty. An +alternative method was considered that required the reservation of +only one control character. However, the user would be required to +type an additional character as an argument signifying what function +is intended. There did not seem to be a great advantage to this scheme +and it was considered after the first scheme had been implemented. For +this and other reasons stated later in this section, the alternative +scheme was discarded. + +From the user's viewpoint, the currently implemented interrupt-level +commands are as follows. (ASCII control characters are represented by +"↑" followed by the corresponding letter.) + + ---- ------------------------------------------------ + ↑L clear screen, reprint stack and input + ↑Q Quit from current MEND + ↑E End automatic mode + ↑B Begin automatic mode + ↑U Unprint (stop displaying stack) + ↑P Print (start displaying stack) + ↑O skip Over (completely evaluate) current object + ↑N Next step (used when not in automatic mode) + ---- ------------------------------------------------ + +The command invoked by typing ↑L is for housekeeping purposes. It +causes extraneous information (e.g., old program output) to be cleared +from the screen and all MEND-related constantly-displayed information +to be reprinted. Unread input is also reprinted. + +The command invoked by typing ↑Q is used to exit from an invocation of +MEND. It has the effect of stopping all actions related to monitoring +the application program and allowing the program to continue normally. +In this way a programmer may discard MEND and continue running the +application program without the need for restarting it at the +beginning. + +The two commands invoked by typing ↑E and ↑B switch MEND between +automatic mode, the only one described thus far, and non-automatic +(command) mode. Command mode will be described in the next section. + +Sometimes it is desirable for the sake of saving both computer time +and the application programmer's time to turn off printing of the +MSTACK. MEND will continue to monitor program execution and store the +appropriate information but it does not display it. Nor does it pause +between steps in the normal manner. This is mainly useful when +breakpoints are present to turn on printing or switch to command mode. +At that time MEND will know how it got to the current level and will +be able to display the MSTACK as usual. Two commands invoked by typing +↑U and ↑P toggle the state of printing. + +Whenever the programmer is satisfied that a particular call of a +function is working properly or for some reason he or she is not +interested in seeing the details of evaluation of some object, typing +↑O just before the object's evaluation invokes a command that causes +MEND to skip over that evaluation and continue normal monitoring and +display immediately after a value is returned. Unlike turning off the +printing, no monitoring is performed here at all so the evaluation +proceeds as fast as it would without MEND. This is actually handled by +causing Muddle to leave single-stepping mode during the evaluation of +the object. + +The last command, invoked by typing ↑N, is special in that it is +ignored in automatic mode. Its actions in command mode will be +described in the next section. It is an interrupt-level command mainly +for convenience and compatibility with DEBUGR. + +Bruce Daniels' DEBUGR is the prototype Muddle multiprocessing +debugging system. It provides a simple user interface to Muddle's +single-stepping functions that makes single-stepping through a Muddle +program appear similar to single-stepping through an assembly-language +program with DDT[^18][^19]. The choices of characters for invoking +many of the MEND functions were based upon the characters used to +invoke similar functions in DEBUGR. Many of DEBUGR's command +invocation characters were in turn based upon those in DDT. The object +of all of this is to build character/command associations in the +user's mind that may be carried over from one system to the next. + +Several other control-character commands are handled by MEND +invisibly. That is, the user may not be aware that they are being +handled. Most of these are commands that are already being handled by +other subsystems but must be intercepted by MEND to maintain +consistency in its environment. Two of this type, invoked by typing ↑S +and ↑G, are already set up in an initial Muddle and a third, invoked +by typing ↑F , is set up by the subsystem EDIT, which is called by +MEND as described in the next section. All three of these are used to +escape to various command levels. MEND has its own command level (see +next section) and must in many cases escape to that one instead to +maintain control. Until a method was discovered for having ITS section +the screen, it was necessary to also interrupt on carriage-returns +(↑M, the new-line character) on input to insure that typein was kept +in the proper section. + +IV.3 MEND's Command Level +------------------------- + +Sometimes the user may wish to stop the application program at some +point to examine it in more detail or to alter it or its environment +in some way. Alternatively the user may want to issue commands to MEND +that require arguments. A command level is provided for both of these +activities. + +The command level differs from the automatic mode described in the +last section in two ways. First, the application program is not +continually executing. It runs one step at a time under the direct +control of the user rather than automatically. Second, in +non-automatic mode typein is normally passed to MEND instead of the +application program, even that class of typein which does not produce +immediate action. + +When the application program is stopped, the user may either request a +service from the debugging system or examine and/or modify the program +and/or its environment. Since editing functions form a large part of +the latter class of actions, it was decided that instead of requiring +the user to "import" an editor, MEND should provide an editor by +making one available at command level. + +It is generally believed and empirical evidence indicates, that +creating a new editor is "the kiss of death" for a Muddle subsystem. A +number of Muddle editors have been tried over the years and the only +one that finally became generally accepted (and is now pre-loaded in +Muddle) is EDIT[^20] . Members of the Muddle community will tolerate +minor changes to EDIT, but they will not accept a new editing system. +The situation is analogous to one of the major obstacles blocking the +acceptance of ESP. Programmers were accustomed to using DDT to examine +and modify their machine-language programs. ESP was not compatible +with DDT and therefore did not provide the familiar interface desired. + +In view of the situation, it seemed desirable to incorporate EDIT into +the command level, essentially unaltered. EDIT uses a special reader +that either interprets input as a command to it or passes input on to +the normal Muddle reader. At first the task of superimposing a MEND +reader on top of both of these appeared difficult. After much +discussion with the current maintainer of EDIT, a very satisfactory +solution was arrived at. + +A general capability was added to EDIT allowing the specification at +runtime of a table of EDIT-like commands to be handled by +programmer-defined functions. This table is searched before EDIT's +command table and thereby provides a capability to override standard +EDIT commands. MEND basically uses an invocation of EDIT as its +command level with all MEND commands included in a table as described. +EDIT-format commands are all one or two letters and may take suffix +arguments. Currently the MEND table includes the following commands. +(FIX here means a Muddle object of type FIX, i.e., a fixed-point +number. FLOAT means a floating-point number. PPRINT[^21] is a function +that prints a Muddle object in a format which indicates the positions +of its elements and sub-elements in the tree hierarchy.) + + NAME ARG TYPE MEANING + ------ ---------- -------------------------------------------------- + ? none type out short summary of MEND commands + ?? none type out complete summary of all commands + O any Open object or MCSTACK level n (if arg n is FIX) + V none toggle Verbosity + N FIX do Next n steps (like ↑N) + OV none skip OVer current object (like ↑O) + Q none Quit EDIT & return to automatic mode (like ↑B) + QM none Quit MEND (like ↑Q) + SN FIX Set number of lines used for stack display + SV FIX Set leVel below which MEND will not 1STEP + SD number Set Delay time (FIX or FLOAT) between steps + PD FIX Pprint Displayed object in level n + PO FIX Pprint Original (actual) object in level n + AI any Add an Item to the list being monitored + DI FIX Delete Item number n from the list + PI FIX Pprint Item number n + +What the user types to the command level is inspected by EDIT. If it +looks like an EDIT command, it is looked-up in the MEND command table +or the EDIT one and handled as appropriate. Otherwise it is passed to +the Muddle reader. In general any interesting (useful) input that +Muddle should read and evaluate will not look like an EDIT command, so +there is no confusion. + +As can be seen from the table, the MEND commands include general +information retrieval ones (? and ??), system tailoring ones (V, SN, +SV, and SD), and commends that allow objects to be accessed in special +MEND locations (O, PD, PO, AI, DI, and PI. The O command is special in +that it allows an object known only by its location in the MSTACK to +be opened for examination and alteration by the normal EDIT commands. +The remaining commands duplicate control-character commands except +that they are read at normal input level rather than at interrupt +level. + +The most powerful feature of this command level is seen when Muddle +objects are evaluated. This level, and therefore the evaluation, uses +stack space below the application program and in the same process. All +bindings and pending evaluations of the program are above the command +level where they may be examined and modified at will. Most of what +constitutes MEND is in another process safely removed and hidden from +the user. Even the small amount of overhead constituting the command +level function is hidden from functions such as FRAMES[^22], which may +be used to view the levels of the application program's current +control stack. The effect is much the same as that of Muddle's +listening loop which is invoked at the current bottom of the stack in +case of error, to allow the programmer to run any program (evaluate +any object) to find and correct the problem. + +Note that for critical examination of the program, the user may +request that it be continued for a limited number of steps (usually +one) with control then being returned to him or her at the command +level. One normally would employ such a feature when the program is +executing statements near the area of suspected trouble but it is not +entirely clear how or why the program is misbehaving. + +IV.4 Breakpoints +---------------- + +Normally MEND operates in automatic mode where the application program +is running continuously. If one is aware of a particular area of +concern, one cannot and should not have to see that area as it is +reached and quickly type ↑E in order to stop the program. One may have +the program running with a very small delay time or have the printing +turned off. + +In their simplest form, MEND breakpoints stop the program when +evaluated, putting the user in command mode if she or he was not +already in it. This is the Muddle equivalent of DDT breakpoints, which +stop a program at a certain instruction. The breakpoints can also be +conditional, where the evaluation of an arbitrary object determines +whether or not to actually break at that point. + +The breakpoints as so far described are very much like the breakpoints +that can be set by EDIT. As a matter of fact, EDIT is used to set and +clear these breakpoints just as it would be in the absence of MEND. +The only real difference is that, when MEND is present, a MEND +function is called to handle the break instead of an EDIT function. + +MEND breakpoints would be useful even if they only did what was just +described. However they are more powerful. A standard EDIT-style +breakpoint is a call to the break function with a number of arguments. +The first of these is the object at which the breakpoint is set, which +will be evaluated when the user returns from the breakpoint and whose +value will be returned by the break function to the application +program. The next is the conditional object and the rest are objects +to be evaluated and printed at each break. MEND's break function +handles these as EDIT's does but also looks for a recognized ATOM +(variable name), which may come after or replace the conditional. If +such an ATOM is found, it will cause a special action to occur instead +of stopping the program: + + ------- --------------------------------------- + ON printing will be turned on (like ↑P) + OFF printing will be turned off (like ↑U) + GO free-run the object (like ↑O) + PRINT just print arguments & continue + ------- --------------------------------------- + +Other arguments will still be printed and all actions are still +predicated upon the value of the conditional. + +The first two special ATOMs (ON and OFF), perhaps coupled with manual +control, are used to allow the MSTACK to be viewed only during areas +of interest while maintaining maximum speed in other places. The next +one (GO) is used to avoid wasting time examining the details of a +functional call that is known to be working and is probably evaluated +repeatedly. When the break function returns, single-stepping resumes +as before the break. The last special ATOM (PRINT) is used to give +information at key places that might not otherwise be seen. + +One thing that was considered extremely important was to make MEND's +breakpoints compatible with those of EDIT. The problem is that a +breakpoint inserted outside of MEND might still be present when MEND +is started and vice versa. As has been stated EDIT breakpoints work +almost exactly as usual when handled by MEND's break function. The +converse is not quite true. EDITs break function will ignore the +significance of a special ATOM end simply print it as a normal +argument. Fortunately, an ATOM evaluates to itself. If it is in the +conditional position it will always be considered "true" and no harm +will be done. + +It would be easy to extend this arrangement to other special ATOMs if +other functions were considered useful. The above seem to form a basic +set in this system. At least they are useful and no one has yet +suggested another type of breakpoint that would also prove useful. + +V Final Thoughts about MEND +=========================== + +V.1 Monitoring of Values +------------------------ + +A feature that was originally considered to be important, and is +present in ESP[^4][^5], is the constant monitoring of values. This +feature has not been too happily received in MEND, but that may be due +more to the current implementation than to an inherent lack of +utility. + +Ideally one should not have to see the object being monitored or its +value if one does not want to. The debugging system should be capable +of performing certain actions upon, for example, a change in value. +This would be analogous to conditional breakpoints except that the +test would be performed after each step of execution of the +application program rather than at certain arbitrary times. + +One problem with the above scheme is the difficulty of testing for a +change in value (the basic test). For example, the value of an object +being monitored may be some sort of structure. During execution the +application program may not explicitly change the value of the object +but may change an element of the structure that the object evaluates +to. Assuming MEND had saved a pointer to the structure, if it now +evaluated the object being monitored, it would again find that +structure and assume that the value had not changed. However since an +element of the structure had changed, the programmer would probably +want to be notified. + +The only solution to the above dilemma is to at each step save a +complete representation to all levels of the value of the object in +some form that cannot be affected by the application program. This can +be done by, at each step, unparsing the value to be saved for future +tests into its printed (string) representation. Then the comparison of +current and previous values will simply be a string comparison, not +subject to sharing by the values. Unfortunately this may create +incredible amounts of "garbage." In fact, a circular list (quite +legal) will produce a string of infinite length! + +The solution finally chosen was to print the values of the objects +being monitored at each step and to let the user visually compare +versions. The values are &-printed as usual but may be examined in +full at will. + +When tested it became immediately obvious that the constant reprinting +wasted time and was distracting. A feature was added to cause printing +to occur only at controllable intervals, but that produced other +problems. Primarily the user might not see a change that had occurred +at one step until some later step when valuable information had +already been lost. One would also not have the opportunity to take +immediate action. + +What needs to be done at this point is to reduce the emphasis on such +pathological cases as described above. Printing should only occur at +well defined moments (i.e., when requested or when some recognizable +condition occurs). Only gross and easily testable changes in value +should be watched for (i.e., the value is a new object, entirely). +Conditional monitoring should be installed analogous to the conditions +of breakpoints. With all of this testing, a variety of indications +should be available as with breakpoints. + +V.2 Control Stack Display +------------------------- + +MEND's display of the application program's control stack has proven +to be by itself a highly useful debugging aid. This display can be and +often is used without any of MEND's other features to debug a faulty +program. It should also be emphasized that a valid use of this display +is to monitor a working program in order to understand its operation. +In this case, where the user of MEND is completely unfamiliar with the +operation of the application program, MEND's step-by-step display of +program execution is even more useful than in the normal debugging +situation. + +One of the unique features of MEND's "trace" of the application +program is that MEND shows both the code being executed and the +results of that execution. Since Muddle is an applicative language, +most of the results consist of values returned by evaluated objects. +By reflecting the returned values back into the original code, MEND +shows intermediate results in an intuitive manner for the user. + +Other programming languages could also benefit from this sort of +display. Too often a trace routine provided for debugging in a +language shows only results of execution, such as value assignments, +without showing the corresponding program steps, or it shows program +steps but leaves it to the programmer to insert instructions to show +intermediate results. What the programmer really needs to see is a +well-integrated display of both program instructions and results, such +as MEND provides. + +Muddle is an exceptional language in that the MEND display was +implemented fairly cleanly with just a package of Muddle functions and +without changes to the interpreter itself. With suitable language +enhancements, however, a similar display could be provided for LISP or +any other LISP-like language. Although major compiler modifications +would be required, a stack-oriented display might also be suitable for +a block structured language like ALGOL. + +Especially in a block structured language, where instructions might be +treated in groups rather than individually, but also in applicative +languages, it would be useful to be shown variable bindings as they +occur. Muddle does not provide any information about the occurrence of +bindings, so the programmer is not informed of such occurrences by +MEND. However when major modifications are made in a language to +provide a display similar to MEND's, it would probably be a mistake +not to include a provision for notifying the programmer of variable +bindings and unbindings as they occur. + +V.3 Simulation Vs. Multiprocessing +---------------------------------- + +Besides its control stack display, MEND's other major feature, and its +major difference from debugging systems like EEP or the first version +of MUMBLE, is its use of multiprocessing rather than simulation to +follow application program execution. (Even debugging aids that use +multiprocessing often do not work the way MEND does. While MEND is a +program written in the same language and running concurrently in the +same interpreter as the application program, some debuggers , such a +DDT, are machine language programs running near the application +program, and others are simply complied into the language itself.) +This feature is perhaps the key to MEND's success as a debugger. The +choice to avoid simulation certainly reduced the size and complexity +of MEND by two or three times and similarly affected run time. What +remains to be discussed are the tradeoffs involved in the choice. The +advantages just mentioned heavily favor the multiprocessing solution +but there are other considerations. + +If MEND had been implemented using simulation, it would for the +evaluation of each object determine which elements need evaluation; +evaluate those elements by recursively calling MEND's own evaluation +simulator; determine whether some function should be applied; +determine the effects of the required functional application, if any; +and cause those effects while displaying some representation of them +for the programmer. In short, MEND would duplicate the actions of +Muddle while maintaining and updating its own information about those +actions. Therefore MEND would at all times know exactly what the +application program was doing and could never miss some important +effects of execution or be fooled by an unusual program. In the +previous section it was stated that it is desirable to show the +occurrence of variable bindings to the programmer. This feature would +be easy to implement in a debugging system that simulated execution of +the application program. + +But what about a debugging system using multiprocessing? A +multiprocessing debugger, such as MEND, allow s the application +program to execute more-or-less freely in its own process while +monitoring it from another process. Since MEND id not in direct +control of the application program, it must rely for its information +on whatever data it is given by the multiprocessing mechanism built +into Muddle and on inferences based on examination of the application +program's environment, its knowledge of the general behavior of Muddle +programs, and some specific knowledge of special cases. The occurrence +of variable bindings is not decipherable from the information that +MEND can gather. Fortunately, as has been previously shown, MEND does +have the information essential to the creation of its display and the +display is a sufficient debugging tool. + +Monitoring execution of the application program from another process +provides sufficient information for a useful debugging system while +simulation of the application program provides enough information for +a more comprehensive system. Therefore if speed and size of the +debugger were not important factors, then, although not a necessary +choice, simulation would be the favored choice for a debugging system +except for one other factor. + +The simulator needs much more knowledge of specific details of the +behavior of program functions than does the monitor. In fact the +simulator must know exactly the effects of each function that might be +used in the application program. Uncompiled user-defined functions are +no problem. They consist merely of one or more applications of other +functions to given arguments. Complied user-defined functions cannot +be dealt with at all without actually creating a simulator for machine +language programs, a project comparable in magnitude to the design and +implementation of all of the other parts of the simulator combined! +The remaining functions are Muddle's built-in primitives. They are +well defined but frequently change (usually upward compatibly). +Furthermore new primitives are constantly being added. A simulator for +Muddle programs would therefore quickly become outdated. + +Given the problems of a simulator and the relatively few drawbacks of +a monitor, it is clear now why MEND was implemented as the latter. The +same arguments apply to debugging systems for other languages. +Whenever the appropriate multiprocessing features exist, with +sufficient information available about execution in another process, a +debugging system based upon simulation of the application program +should be rejected in favor of a monitor-type system. MEND has proven +to be an effective debugging aid and should serve as a good example of +the latter type of system. + [^1]: G.A. Mann, "A Survey of Debug Systems", Honeywell Computer Journal 1973, volume 7, number 3, pages 182-198 @@ -380,3 +1315,23 @@ that can be built around an applicative type language. [^11]: N. Ryan, EDIT: The Muddle Editor, SYS.11.14, Programming Technology Division Document, Laboratory for Computer Science, M.I.T., August, 1974 + +[^12]: D. Eastlake, R. Greenblatt, J. Holloway, T. Knight, and S. + Nelson, ITS 1.5 Reference Manual, Memo No. 161A, Artificial + Intelligence Laboratory, M.I.T., July 1969 + +[^13]: P.D. Lebling, SSV User's Manual, SYS.52.07, Programming + Technology Division Document, Laboratory for Computer Science, + M.I.T., (in preparation) + +[^14]: S.W. Galley, Pre-loaded Pure MDL RSUBRs, SYS.11.28, Programming + Technology Division Document, Laboratory for Computer Science, + M.I.T., November 1975 + +[^15]: B. Daniels, The MDL Assembler, SYS.11.07, Programming + Technology Division Document, Laboratory for Computer Science, + M.I.T., (in preparation) + +[^16]: C. Reeve, The MDL Compiler, SYS.11.25, Programming Technology + Division Document, Laboratory for Computer Science, M.I.T., (in + preparation)