----
-title: 'A Dynamic Debugging System For Muddle'
-author: Joel Mayer Berez
-date: January 1978
-abstract: Program debugging is a time consuming process. Conventional
- debugging techniques and aids typically give the user a narrow view
- of the program's operation, making debugging difficult. A debugging
- system that would present a clear overall picture of a program's
- behavior and would be both flexible and simple to operate would be a
- valuable tool. Such a system was designed and implemented in and for
- Muddle, a high-level applicative programming language. This report
- discusses the design alternatives considered during the debugging
- system's design and implementation phases, the reasons for the
- resulting design choices, and the system attributes. A major
- attribute of the system (MEND) is that it does not simulate the
- program being debugged but instead monitors it from another
- process. This attribute results in a robust and viable debugging
- system, because MEND need not be modified in order to handle each
- new extension to Muddle and/or each new user-defined primitive.
----
+% A Dynamic Debugging System For Muddle
+% Joel Mayer Berez
+% January 1978
MIT Technical Memo 94
-Laboratory for Computer Science
-Massachusetts Institute of Technology
+Laboratory for Computer Science\
+Massachusetts Institute of Technology\
545 Technology Square Cambridge, Massachusetts 02139
Copyright
of the other ITS and DynaMod hackers who have built a system well
worth using.
-This research was supported by the Advanced Research Projects Agency
-of the Department of Defense and was monitored by the Office of Naval
-Research under Contract No. N00014-75-C-0661.
-
-Introduction
-============
+I Introduction
+==============
-Background
-----------
+I.1 Background
+--------------
A time-consuming and frustrating aspect of computer programming is the
debugging of faulty programs. Current debugging techniques involve
ever progressed, and it was not in use as of the time of the proposal
for the current work.
-MEND
-----
+I.2 MEND
+--------
After the Muddle compiler became operational and many additional new
software features became available, it appeared that it would be
the ENVIRONMENT of the program. This means the user can examine
the state of the program or change it.
-Possible Additional Characteristics
------------------------------------
+I.3 Possible Additional Characteristics
+---------------------------------------
Certain other characteristics were seen as desirable for MEND but
possibly beyond the scope of this project. If time permitted these
actually reverse execution or a simulation displaying information
previously stored by the system.
-Design and Implementation
--------------------------
+I.4 Design and Implementation
+-----------------------------
MEND was designed with the intent of providing the application
programmer with many options so that debugging could proceed in the
should also serve as a good example of the type of debugging system
that can be built around an applicative type language.
+II Terminal Display
+===================
+
+II.1 IMLAC Console Program
+--------------------------
+
+One basic concern throughout the project was the display: how the
+information made available by MEND would be presented to the user. To
+a large extent the physical characteristics of Muddle, ITS[^12]
+(Incompatible Timesharing System, the operating system used on the
+Dynamic Modeling System computer), and the available terminals
+dictated what was reasonably possible.
+
+The terminal most commonly used by users of the Dynamic Modeling
+System is the IMLAC PDS-1. This is a minicomputer capable of having
+programs loaded into it from the PDP-10 host computer. One program
+written for it by Dave Lebling is MSC, a multiple-screen terminal
+program. Up to four virtual screens (or pages) can be created that
+will individually operate like the actual screen area of the standard
+terminal program (SSV[^13]). Output may be directed to any one of the
+screens and any screen may be visible or invisible at any instant of
+time, at the programmer's option. Selection of screen is controllable
+either by program from the host or locally by the user.
+
+It was originally Intended that MEND would use MSC for its normal
+display. One page would constantly show MEND's representation of the
+control stack of the application program. Output initiated by the
+program being debugged would go to a second page. A third page would
+be used for interaction with MEND and would show user typein along
+with any output from MEND that one was interested in seeing. The
+latter would include, by user request, full displays of both objects
+printed in an abbreviated form on the page containing the control
+stack of the application program and values being monitored or traced.
+The user could switch back and forth among the pages at will during
+the execution of the program. The application program would have a
+full standard screen to write onto, and ample room would be available
+for the information to be displayed by MEND.
+
+After a fair amount of testing, this proposal was discarded for the
+following reasons. First, MSC was supposed to look just like SSV for
+individual virtual screens. Unfortunately new features added to SSV
+had not also been added to MSC. A primary reason for this disparity
+was that the new features encroached upon the IMLAC's character space.
+An SSV with all current features can only hold about one and a half
+full pages of text, where a page is the amount that can be visible at
+one time. The overhead required for additional screens reduces it
+still further. Therefore, with all features included, four virtual
+screens could each only average about one-third full. Without many of
+the current features, people were reluctant to use MSC.
+
+The second and perhaps most devastating problem with MSC is that it is
+not properly supported by ITS as is SSV. Ordinarily the operating
+system will keep track of where on the page the terminal's cursor is
+and will properly handle such updating as deletions even in the face
+of random access performed (if done by request to the system). When
+using MSC, ITS does not realize that information is being written onto
+more than one screen and will therefore often move the cursor to the
+wrong position.
+
+MEND could completely control positioning for typeout and echoing on
+typein but that would add the large overhead of having to run a
+non-trivial routine for each character typed out on the display. Also,
+because MSC is not the standard console program, requiring the user to
+load it before using MEND might discourage the use of MEND. (It takes
+between about 30 seconds and a couple of minutes, depending upon
+system load, to load a new console program into the IMLAC.)
+
+From the outset it was intended that MEND be used routinely by
+programmers as debugging problems arose. Therefore it was decided that
+the proper way for such a system to operate was to use, as far as
+possible, the common environment so as to keep the overhead for
+invoking MEND small. This philosophy, which had been seen to affect
+the success of many earlier projects, decided between the alternatives
+and in this case led to the final decision to use SSV instead of MSC.
+
+II.2 Terminal Independence
+--------------------------
+
+The MEND terminal handling capabilities are actually quite general and
+MEND does not depend exclusively on SSV. For the purpose of dividing
+the screen area into several sections, horizontal lines are sometimes
+drawn (see Appendix A showing sample display). With an IMLAC and SSV
+these lines could be drawn quite simply using graphics mode. However,
+for purposes of generality with regard to terminals, these lines are
+instead formed by using underbar characters (on a line of their own).
+By using no actual graphics MEND can be used with almost any display
+terminal having random access. MEND outputs display commands, such as
+clearing a line, as escape codes to ITS which then translates these
+into the appropriate commands for the terminal in use. ITS currently
+knows about several types of display terminals in use at the
+Laboratory for Computer Science, and other types of terminals located
+at foreign sites on the ARPA network may be handled by interface
+software that simulates a known type. Naturally MEND can handle a
+large range of possible line and screen lengths. (The current version
+of SSV provides four possible character sizes.)
+
+II.3 Control of Screen Sectioning
+---------------------------------
+
+There still remained the question of how to handle the
+multi-sectioning of the displayed information. Originally three
+sections corresponding to the three virtual screens in the aborted MSC
+implementation were planned. The bottom section would hold user typein
+and application program output. Three possible methods of achieving
+this involved having ITS, Muddle, or MEND do various amounts of the
+work with increasing overhead and decreasing speed for those three,
+respectively.
+
+The most attractive solution utilized an ITS feature allowing the
+specification of an echo area at the bottom of the screen where echoed
+input would always be printed (with the echoing handled by the system,
+which is the normal case) . After some experimentation this method of
+handling the typein and application program output was rejected
+because typeout and deletion are handled by Muddle which ignores the
+echo area MEND would effectively have had to control all typeout and
+monitor all typein, which would have made the echo area useless.
+
+Experimentation with the second solution, an indirect method, involved
+monitoring of special Muddle memory locations where information
+concerning horizontal and vertical page positions is stored. It was
+soon discovered that Muddle becomes confused quickly, contains several
+bugs with respect to this position information, and generally has a
+poor idea of where it is actually printing.
+
+The third solution appeared to be the most painful from an
+implementation and efficiency point of view. MEND would need to
+control the printing of every character on output and certain
+characters on input, constantly checking page positions by querying
+the system. Not only did this slow output, but also MEND was forced to
+constantly move the cursor to a safe position in case Muddle managed
+to sneak some output past it, which it occasionally does.
+
+Fortuitously the problem was neatly solved when a little known feature
+of ITS was discovered. It Is possible to open a channel to the
+terminal in a mode that would cause all output to appear in the echo
+area. By creating this echo area and reopening Muddle's normal
+terminal output channel in this mode, it is possible to cause Muddle
+and the application program to think that the entire screen consists
+of only the echo area. All output including application program
+display escape commands is automatically routed by ITS to this echo
+area. MEND sends its output to a second channel opened in the normal
+mode, thereby allowing it to use an area of the screen unknown to and
+left untouched by the application program. The physical cursor stays
+where the last used logical cursor left it, thereby eliminating most
+of the unnecessary cursor movement, resulting in a more pleasing
+visual effect.
+
+As a result of this solution, the screen is divided into only two main
+sections. All typein appears in the lower section, whether it is to
+the application program or to MEND. Application program output also
+goes to the lower section, as does unstructured output produced by
+interaction with MEND. The upper section, in general, contains only
+those items that occupy one line of the display each. As will be
+explained later, these items include all output automatically
+displayed by MEND during execution of the application program.
+
+III MSTACK: MEND's Representation of the Control Stack
+======================================================
+
+III.1 Program Execution In Muddle
+---------------------------------
+
+MEND's primary role is to allow the applications programmer to
+visually monitor the execution of a program. In a language like
+Muddle, this is most easily accomplished by showing the programmer a
+picture of the control stack.
+
+A Muddle program consists of the evaluation of a single object. The
+object is usually structured in some manner and itself contains other
+objects. The most common object is the FORM. This is a list of objects
+in which the first is (or evaluates to) some function and the rest are
+arguments to that function. A FORM is evaluated by applying the
+function to its arguments, usually after the arguments are themselves
+evaluated. This evaluation actually is initiated by the Muddle
+interpreter by applying the function EVAL to the original object. EVAL
+takes an object as its argument and returns the value to which it
+evaluates.
+
+Figure 1 shows a (simplified) static representation of the evaluation
+of a Muddle object. Starting with the FORM (list of objects in
+angle-brackets) to be evaluated, the flow of control/evaluation may be
+described as a depth-first search through the tree pictured. The
+arrows represent values being returned to previous levels. At the end
+of this "search" the FORM returns the value shown at the top.
+
+ Figure 1:
+
+ 31
+ ↑
+ <+ .A .B .C 5>
+ ↑ ↑ ↑ ↑ ↑
+ + .A .B .C 5
+ ↑ ↑ ↑
+ 6 8 12
+
+Typically the control stack will start with one object on it, the FORM
+to be evaluated. This evaluation will first require that the objects
+in the FORM (possibly FORMs themselves) be added to the stack and
+evaluated. EVAL recursively calls itself for this purpose. The stack
+builds (downward, by convention) until some object is placed on it
+which ls known by EVAL to need no further evaluation. This object is
+returned as its own value to the previous level and values continue to
+be returned upward until a level is reached where another object must
+be evaluated. In this manner, the stack grows and shrinks until the
+topmost (initial) object returns a value.
+
+III.2 Monitoring Program Execution
+----------------------------------
+
+The manner in which the stack builds, the objects are evaluated, and
+the values are returned illustrate most of what the program is doing.
+Other factors, including side effects and complied code, will be
+discussed at the end of this chapter. MEND's main display therefore
+shows a representation of the stack being continuously updated as
+execution/evaluation proceeds.
+
+There are essentially two ways that MEND could follow the flow of
+control of the application program. The most direct way, as attempted
+by MUMBLE[^14] and discussed in the last chapter, would be for MEND to
+execute the application program by simulating the operation of Muddle.
+This type of simulation has been shown to require a complex end all
+too often fragile structure. The debugging program would need to be
+constantly updated to match changes and additions to the Muddle
+language , but more importantly it would fail to properly handle
+programmer-defined primitives, several varieties of which are provided
+for by Muddle.
+
+A far more satisfactory method is to allow Muddle to execute the
+application program in more or less its normal fashion but to stand
+beck somewhere and watch. Fortunately Muddle contains a mechanism
+ideal for this, called multiprocessing. Basically another control
+stack , or process, may be created (independent of the first) that may
+be used to execute a different function with its own set of variable
+bindings. One such process, in this case MEND, may place another, the
+application program, into a single step mode where the latter will be
+stopped before each call to EVAL and again as each call returns. The
+MEND process will at these points be restarted and given information
+about what the application process is doing. MEND stores this
+information in a multi-level structure it creates called an MSTACK.
+
+Each level of an MSTACK corresponds to a level of the control stack
+being monitored. Each level contains the original object (actually, a
+pointer to it) being evaluated at that level and a new object, called
+the "displayed object", that will or does contain the results of
+evaluating each of the arguments or elements of the original. The
+displayed object is initially the same as the original (in a real
+sense, it is the original) but is systematically rebuilt as each
+element is evaluated and replaced by what it returns. Thus MEND can
+keep track of the relationship between the changing display and the
+unchanging program. (Figure 2 shows the various stages that the
+displayed object corresponding to the FORM being evaluated in Figure 1
+goes through. Each stage would in fact be painted over the previous
+one so that all of these stages represent only one line of the screen.
+The down-arrow shown in some of the stages is a place-holder that
+represents an object that is actually expanded on the next line of the
+screen.) A pointer is also kept showing which element is currently
+above this on the stack to be evaluated unless, of course, this is the
+initial element of the stack.
+
+ Figure 2:
+
+ <+ .A .B .C 5>
+ <+ ↓ .B .C 5>
+ <+ 6 .B .C 5>
+ <+ 6 ↓ .C 5>
+ <+ 6 8 .C 5>
+ <+ 6 8 ↓ 5>
+ <+ 6 8 12 5>
+
+ 31
+
+III.3 A Displayed Representation of Program Execution
+-----------------------------------------------------
+
+The printed representation of the stack occupies the top section of
+the screen and is the most prominent and important characteristic of
+MEND. Most often, in fact, it is only necessary to watch this display
+as the application program is executed to ascertain where a bug is
+located or to observe exactly how the program operates. Therefore
+considerable time has been spent making the operation of the MSTACK
+and associated display as natural and informative as possible.
+
+Using the collected information described in the previous section, a
+representation of the stack is displayed as a number of lines, each
+corresponding to one level. Each line shows a level number and the
+displayed object printed in "&-printing", named for the printing
+function &[^15] created by Greg Pfister. Although strictly speaking
+the stack builds upward (towards higher memory locations), it seams
+more natural to display and speak of the stack as building from the
+top downward, in the direction that printing normally occurs. As each
+element of the bottom level (the bottom line on this section of the
+screen) is placed on the stack for evaluation, a new line is added
+beneath the previous bottom-level line showing that element, and the
+element is replaced in the previous line by a pointer ("↓") marking
+its location. When the element finally returns a value, the returned
+value will replace the pointer in the displayed object.
+
+To avoid visual distraction, a minimum of updating is done on the
+screen. In most cases random access is used to replace only those
+lines that have changed. In general the complete stack will not fit in
+the display area allocated for it (whose size is user adjustable) so a
+scrolling procedure has been devised. The complete display area is
+rewritten whenever an attempt is made to write past the bottom or
+erase upward past a certain level while some lines are invisible
+because they have been scrolled off. The scrolling parameters have
+been selected to optimize the number of lines visible on the average
+vs. the frequency of scrolling. Level numbers allow the user to see
+how much is hidden "above the screen" and how deep the evaluation is
+nested.
+
+III.4 MEND Program Steps Vs. Muddle Steps
+-----------------------------------------
+
+To make it easier for the programmer to follow what the application
+program is doing, the speed of execution of the program must be
+controllable. MEND does this by inserting a constant, user adjustable
+delay between program "steps." One MEND step is not precisely the same
+as one Muddle step. Remember that a Muddle step is one call to or
+return from EVAL. Each MSTACK level, and therefore each displayed
+line, will have two Muddle steps associated with it. At the first
+step, MEND will create the level and add one line to the screen. At
+the second step, MEND will erase that line and put the returned value
+back into the previous line. For clarity MEND adds a third step
+between these two which actually occurs at the second Muddle stop.
+Before substituting into the previous line, the current one is first
+replaced by the returned value, the normal delay occurs, and then the
+"second" step occurs as described.
+
+Some types of Muddle objects are self-evaluating; EVAL will simply
+return the object it was given. Although nothing interesting has
+happened, two steps have occurred. In this case MEND will avoid
+clutter by pretending that no steps have occurred. (MEND only does
+this with built-in types that MEND recognizes, and the programmer
+should recognize, as being self-evaluating. Programmer-defined types
+that are self-evaluating will generate the usual number of MEND
+steps.)
+
+Another case of disparity between Muddle and MEND stops is more
+complex. First some further explanations about Muddle objects are
+needed Generally the interesting objects, the ones that generate MEND
+steps, are linear (usually list) structures containing a number of
+other Muddle objects, as are the FORMs described earlier. Normally
+during evaluation of such an object the elements will be evaluated one
+at a time from first to last. However this sequence is not always
+followed by Muddle. MEND cannot directly determine which element is
+about to be evaluated at each call to EVAL. It is only given the
+object to be evaluated itself and not its position in the parent
+structure. It is normally sufficient to do a comparison of this object
+with the elements of the parent, starting with the first element
+believed to not yet have been evaluated. Naturally, if the elements
+are evaluated out of order, this procedure may fail to find the
+desired match, because a Muddle object may contain the same element in
+two or more positions. Thus it is possible to match the
+about-to-be-evaluated object to the wrong occurrence of it. Further
+complications arise because some functions can sometimes back up and
+re-evaluate their arguments.
+
+A strong attempt was made to make MEND dependent only upon the general
+characteristics of Muddle functions and not upon specific exceptions
+and idiosyncrasies. It was felt to be desirable to make MEND
+independent of both future changes to the language and
+programmer-defined "primitives" that would not be known to MEND.
+Besides, without actually simulating Muddle it is not possible to
+always get the information MEND needs. It was felt that the "general
+rule" approach would take care of a sufficiently large number of cases
+without falling into the simulator problem.
+
+It was determined after some experimentation, however, that MEND could
+not be made to work properly without some specific knowledge about
+several important cases in Muddle. Two functions, PROG and REPEAT,
+allow for branching the flow of control. They are normally
+first-to-last functions as described above but at times control may
+jump backwards to re-evaluate some elements. MEND was implemented so
+that if it cannot locate an element in its normal search path, it will
+start looking again from the beginning of the structure. If it is then
+found, the displayed object will be reinitialized to be as if
+evaluation had not yet proceeded past that point. Evaluation will then
+continue normally.
+
+Another phenomenon of Muddle that we must discuss is what this author
+labels the "clause" behavior. A clause is a list of objects given to a
+function as a single argument. The list is not itself evaluated, but
+some or all of its elements are evaluated. The most common function
+illustrating this behavior is COND, a general purpose conditional
+function. COND's arguments are all lists of objects. It sequences
+through these lists, evaluating the first object in each, until an
+evaluated object returns something considered "true." Then the rest of
+the objects in that list are evaluated and COND returns what the last
+object in the list returns. MEND's normal search path only looks at
+top-level elements and would therefore never find the ones actually
+being evaluated.
+
+This phenomenon seemed to be more widespread than the PROG/REPEAT one
+and could not be easily attributed to certain functions. The solution
+chosen here was to in all cases do a nested search in elements that
+looked as if they might be clauses. (The search actually goes one
+extra level deep to allow for certain special cases.) When a match is
+found in a clause, MEND will for clarity generate extra steps to make
+it appear to the user as if first the clause and then its appropriate
+element was put on the stack for evaluation. The clause will stay on
+the stack until some element that is not above it in the evaluation
+tree is evaluated. At that time the clause will be removed in an
+orderly manner, and the new element or clause plus element will be put
+onto the stack. To do this smoothly, up to six MEND steps may have to
+be generated for the one Muddle step. (See the example in Appendix A.)
+
+III.5 What the User Does Not See
+--------------------------------
+
+MEND, in its display of the MSTACK, attempts to show the user
+everything of importance that is happening as the program is executed.
+However certain features of Muddle cannot be captured in this sort of
+representation.
+
+One such feature is the existence of compiled code. Although Muddle
+was originally intended to be a high-level interpretive language, an
+assembler was written[^16], producing machine code that executes in
+the Muddle environment, to allow programmers to create "primitives"
+that perform functions not otherwise available in Muddle. Once the
+assembler was written, it was natural that a compiler[^17] followed.
+It translates normal Muddle code into Muddle assembly code which is
+then assembled. Typically Muddle programs are tested interpretively
+and, when fully debugged, complied in order to obtain a major increase
+in speed of execution.
+
+A call to a compiled (or assembled) function usually looks to the
+programmer and to MEND like a call to a Muddle primitive. The
+operation of the function is not seen except when it calls uncompiled
+functions. This does not present a major problem to MEND since
+generally only uncompiled functions are being debugged, and the
+compiled ones encountered are hopefully performing known functions
+properly.
+
+One feature of Muddle that MEND is unable to cope with is the
+interrupt system. The programmer may enable a large class of
+interrupts and assign handling functions wherever desired. Examples
+Include interrupts for characters being typed to a certain input
+channel and notification that the system is about to be brought down.
+(MEND uses interrupts to catch single character commands and to catch
+errors.)
+
+Recall that MEND monitors application program execution by placing its
+process into a single-stepping mode. When Muddle passes control to an
+interrupt handier, it temporarily causes the process to leave
+single-stepping mode. This is necessary partially because such a
+handler may be specified to run in a process other than the current
+one at the time of the interrupt. It unfortunately makes the handler
+invisible to MEND. it is therefore not currently possible to use MEND
+to debug interrupt handlers.
+
+There is a whole series of side-effects, such as printing by the
+application program, that are not directly seen in the MSTACK
+representation but are made visible by various other features of MEND.
+These will be discussed where appropriate in later sections.
+
+IV Issuing Commands to MEND
+===========================
+
+IV.1 Types of Commands
+----------------------
+
+There is a need in debugging systems similar to MEND for two types of
+commands for controlling operation. One type is a set of short,
+immediate-action commands that the user of the debugging system may
+issue at any time, such as a command to completely stop all activities
+of the debugging system and to exit. The other type is the set of all
+commands not belonging to the first set. This includes commands that
+take arguments and those that can only be given while the application
+program is suspended from execution.
+
+IV.2 Immediate Interrupt-Level Commands
+---------------------------------------
+
+In MEND's normal mode, various actions occur automatically. Most
+importantly, the application program is executing and the displayed
+representation of the MSTACK is being updated correspondingly. During
+this time the programmer may type ahead to the application program,
+but, since there is only one physical cursor that is used by both
+MEND's display and typein echoing that must therefore jump back and
+forth between the two text entry positions, a large amount of typein
+is awkward. Therefore, all immediate action commands that the user can
+issue while MEND is in automatic mode are single characters. To
+minimize the chance of conflict with typein that the executing
+application program might read, MEND immediate action commands are
+invoked by typing certain ASCII control characters. Furthermore typing
+one of these control characters causes an interrupt to occur which is
+handled by MEND and allows immediate response.
+
+Note that this arrangement requires that there be certain reserved
+characters, as listed below, that cannot be used to communicate with
+the application program. Normally this presents no difficulty. An
+alternative method was considered that required the reservation of
+only one control character. However, the user would be required to
+type an additional character as an argument signifying what function
+is intended. There did not seem to be a great advantage to this scheme
+and it was considered after the first scheme had been implemented. For
+this and other reasons stated later in this section, the alternative
+scheme was discarded.
+
+From the user's viewpoint, the currently implemented interrupt-level
+commands are as follows. (ASCII control characters are represented by
+"↑" followed by the corresponding letter.)
+
+ ---- ------------------------------------------------
+ ↑L clear screen, reprint stack and input
+ ↑Q Quit from current MEND
+ ↑E End automatic mode
+ ↑B Begin automatic mode
+ ↑U Unprint (stop displaying stack)
+ ↑P Print (start displaying stack)
+ ↑O skip Over (completely evaluate) current object
+ ↑N Next step (used when not in automatic mode)
+ ---- ------------------------------------------------
+
+The command invoked by typing ↑L is for housekeeping purposes. It
+causes extraneous information (e.g., old program output) to be cleared
+from the screen and all MEND-related constantly-displayed information
+to be reprinted. Unread input is also reprinted.
+
+The command invoked by typing ↑Q is used to exit from an invocation of
+MEND. It has the effect of stopping all actions related to monitoring
+the application program and allowing the program to continue normally.
+In this way a programmer may discard MEND and continue running the
+application program without the need for restarting it at the
+beginning.
+
+The two commands invoked by typing ↑E and ↑B switch MEND between
+automatic mode, the only one described thus far, and non-automatic
+(command) mode. Command mode will be described in the next section.
+
+Sometimes it is desirable for the sake of saving both computer time
+and the application programmer's time to turn off printing of the
+MSTACK. MEND will continue to monitor program execution and store the
+appropriate information but it does not display it. Nor does it pause
+between steps in the normal manner. This is mainly useful when
+breakpoints are present to turn on printing or switch to command mode.
+At that time MEND will know how it got to the current level and will
+be able to display the MSTACK as usual. Two commands invoked by typing
+↑U and ↑P toggle the state of printing.
+
+Whenever the programmer is satisfied that a particular call of a
+function is working properly or for some reason he or she is not
+interested in seeing the details of evaluation of some object, typing
+↑O just before the object's evaluation invokes a command that causes
+MEND to skip over that evaluation and continue normal monitoring and
+display immediately after a value is returned. Unlike turning off the
+printing, no monitoring is performed here at all so the evaluation
+proceeds as fast as it would without MEND. This is actually handled by
+causing Muddle to leave single-stepping mode during the evaluation of
+the object.
+
+The last command, invoked by typing ↑N, is special in that it is
+ignored in automatic mode. Its actions in command mode will be
+described in the next section. It is an interrupt-level command mainly
+for convenience and compatibility with DEBUGR.
+
+Bruce Daniels' DEBUGR is the prototype Muddle multiprocessing
+debugging system. It provides a simple user interface to Muddle's
+single-stepping functions that makes single-stepping through a Muddle
+program appear similar to single-stepping through an assembly-language
+program with DDT[^18][^19]. The choices of characters for invoking
+many of the MEND functions were based upon the characters used to
+invoke similar functions in DEBUGR. Many of DEBUGR's command
+invocation characters were in turn based upon those in DDT. The object
+of all of this is to build character/command associations in the
+user's mind that may be carried over from one system to the next.
+
+Several other control-character commands are handled by MEND
+invisibly. That is, the user may not be aware that they are being
+handled. Most of these are commands that are already being handled by
+other subsystems but must be intercepted by MEND to maintain
+consistency in its environment. Two of this type, invoked by typing ↑S
+and ↑G, are already set up in an initial Muddle and a third, invoked
+by typing ↑F , is set up by the subsystem EDIT, which is called by
+MEND as described in the next section. All three of these are used to
+escape to various command levels. MEND has its own command level (see
+next section) and must in many cases escape to that one instead to
+maintain control. Until a method was discovered for having ITS section
+the screen, it was necessary to also interrupt on carriage-returns
+(↑M, the new-line character) on input to insure that typein was kept
+in the proper section.
+
+IV.3 MEND's Command Level
+-------------------------
+
+Sometimes the user may wish to stop the application program at some
+point to examine it in more detail or to alter it or its environment
+in some way. Alternatively the user may want to issue commands to MEND
+that require arguments. A command level is provided for both of these
+activities.
+
+The command level differs from the automatic mode described in the
+last section in two ways. First, the application program is not
+continually executing. It runs one step at a time under the direct
+control of the user rather than automatically. Second, in
+non-automatic mode typein is normally passed to MEND instead of the
+application program, even that class of typein which does not produce
+immediate action.
+
+When the application program is stopped, the user may either request a
+service from the debugging system or examine and/or modify the program
+and/or its environment. Since editing functions form a large part of
+the latter class of actions, it was decided that instead of requiring
+the user to "import" an editor, MEND should provide an editor by
+making one available at command level.
+
+It is generally believed and empirical evidence indicates, that
+creating a new editor is "the kiss of death" for a Muddle subsystem. A
+number of Muddle editors have been tried over the years and the only
+one that finally became generally accepted (and is now pre-loaded in
+Muddle) is EDIT[^20] . Members of the Muddle community will tolerate
+minor changes to EDIT, but they will not accept a new editing system.
+The situation is analogous to one of the major obstacles blocking the
+acceptance of ESP. Programmers were accustomed to using DDT to examine
+and modify their machine-language programs. ESP was not compatible
+with DDT and therefore did not provide the familiar interface desired.
+
+In view of the situation, it seemed desirable to incorporate EDIT into
+the command level, essentially unaltered. EDIT uses a special reader
+that either interprets input as a command to it or passes input on to
+the normal Muddle reader. At first the task of superimposing a MEND
+reader on top of both of these appeared difficult. After much
+discussion with the current maintainer of EDIT, a very satisfactory
+solution was arrived at.
+
+A general capability was added to EDIT allowing the specification at
+runtime of a table of EDIT-like commands to be handled by
+programmer-defined functions. This table is searched before EDIT's
+command table and thereby provides a capability to override standard
+EDIT commands. MEND basically uses an invocation of EDIT as its
+command level with all MEND commands included in a table as described.
+EDIT-format commands are all one or two letters and may take suffix
+arguments. Currently the MEND table includes the following commands.
+(FIX here means a Muddle object of type FIX, i.e., a fixed-point
+number. FLOAT means a floating-point number. PPRINT[^21] is a function
+that prints a Muddle object in a format which indicates the positions
+of its elements and sub-elements in the tree hierarchy.)
+
+ NAME ARG TYPE MEANING
+ ------ ---------- --------------------------------------------------
+ ? none type out short summary of MEND commands
+ ?? none type out complete summary of all commands
+ O any Open object or MCSTACK level n (if arg n is FIX)
+ V none toggle Verbosity
+ N FIX do Next n steps (like ↑N)
+ OV none skip OVer current object (like ↑O)
+ Q none Quit EDIT & return to automatic mode (like ↑B)
+ QM none Quit MEND (like ↑Q)
+ SN FIX Set number of lines used for stack display
+ SV FIX Set leVel below which MEND will not 1STEP
+ SD number Set Delay time (FIX or FLOAT) between steps
+ PD FIX Pprint Displayed object in level n
+ PO FIX Pprint Original (actual) object in level n
+ AI any Add an Item to the list being monitored
+ DI FIX Delete Item number n from the list
+ PI FIX Pprint Item number n
+
+What the user types to the command level is inspected by EDIT. If it
+looks like an EDIT command, it is looked-up in the MEND command table
+or the EDIT one and handled as appropriate. Otherwise it is passed to
+the Muddle reader. In general any interesting (useful) input that
+Muddle should read and evaluate will not look like an EDIT command, so
+there is no confusion.
+
+As can be seen from the table, the MEND commands include general
+information retrieval ones (? and ??), system tailoring ones (V, SN,
+SV, and SD), and commends that allow objects to be accessed in special
+MEND locations (O, PD, PO, AI, DI, and PI. The O command is special in
+that it allows an object known only by its location in the MSTACK to
+be opened for examination and alteration by the normal EDIT commands.
+The remaining commands duplicate control-character commands except
+that they are read at normal input level rather than at interrupt
+level.
+
+The most powerful feature of this command level is seen when Muddle
+objects are evaluated. This level, and therefore the evaluation, uses
+stack space below the application program and in the same process. All
+bindings and pending evaluations of the program are above the command
+level where they may be examined and modified at will. Most of what
+constitutes MEND is in another process safely removed and hidden from
+the user. Even the small amount of overhead constituting the command
+level function is hidden from functions such as FRAMES[^22], which may
+be used to view the levels of the application program's current
+control stack. The effect is much the same as that of Muddle's
+listening loop which is invoked at the current bottom of the stack in
+case of error, to allow the programmer to run any program (evaluate
+any object) to find and correct the problem.
+
+Note that for critical examination of the program, the user may
+request that it be continued for a limited number of steps (usually
+one) with control then being returned to him or her at the command
+level. One normally would employ such a feature when the program is
+executing statements near the area of suspected trouble but it is not
+entirely clear how or why the program is misbehaving.
+
+IV.4 Breakpoints
+----------------
+
+Normally MEND operates in automatic mode where the application program
+is running continuously. If one is aware of a particular area of
+concern, one cannot and should not have to see that area as it is
+reached and quickly type ↑E in order to stop the program. One may have
+the program running with a very small delay time or have the printing
+turned off.
+
+In their simplest form, MEND breakpoints stop the program when
+evaluated, putting the user in command mode if she or he was not
+already in it. This is the Muddle equivalent of DDT breakpoints, which
+stop a program at a certain instruction. The breakpoints can also be
+conditional, where the evaluation of an arbitrary object determines
+whether or not to actually break at that point.
+
+The breakpoints as so far described are very much like the breakpoints
+that can be set by EDIT. As a matter of fact, EDIT is used to set and
+clear these breakpoints just as it would be in the absence of MEND.
+The only real difference is that, when MEND is present, a MEND
+function is called to handle the break instead of an EDIT function.
+
+MEND breakpoints would be useful even if they only did what was just
+described. However they are more powerful. A standard EDIT-style
+breakpoint is a call to the break function with a number of arguments.
+The first of these is the object at which the breakpoint is set, which
+will be evaluated when the user returns from the breakpoint and whose
+value will be returned by the break function to the application
+program. The next is the conditional object and the rest are objects
+to be evaluated and printed at each break. MEND's break function
+handles these as EDIT's does but also looks for a recognized ATOM
+(variable name), which may come after or replace the conditional. If
+such an ATOM is found, it will cause a special action to occur instead
+of stopping the program:
+
+ ------- ---------------------------------------
+ ON printing will be turned on (like ↑P)
+ OFF printing will be turned off (like ↑U)
+ GO free-run the object (like ↑O)
+ PRINT just print arguments & continue
+ ------- ---------------------------------------
+
+Other arguments will still be printed and all actions are still
+predicated upon the value of the conditional.
+
+The first two special ATOMs (ON and OFF), perhaps coupled with manual
+control, are used to allow the MSTACK to be viewed only during areas
+of interest while maintaining maximum speed in other places. The next
+one (GO) is used to avoid wasting time examining the details of a
+functional call that is known to be working and is probably evaluated
+repeatedly. When the break function returns, single-stepping resumes
+as before the break. The last special ATOM (PRINT) is used to give
+information at key places that might not otherwise be seen.
+
+One thing that was considered extremely important was to make MEND's
+breakpoints compatible with those of EDIT. The problem is that a
+breakpoint inserted outside of MEND might still be present when MEND
+is started and vice versa. As has been stated EDIT breakpoints work
+almost exactly as usual when handled by MEND's break function. The
+converse is not quite true. EDITs break function will ignore the
+significance of a special ATOM end simply print it as a normal
+argument. Fortunately, an ATOM evaluates to itself. If it is in the
+conditional position it will always be considered "true" and no harm
+will be done.
+
+It would be easy to extend this arrangement to other special ATOMs if
+other functions were considered useful. The above seem to form a basic
+set in this system. At least they are useful and no one has yet
+suggested another type of breakpoint that would also prove useful.
+
+V Final Thoughts about MEND
+===========================
+
+V.1 Monitoring of Values
+------------------------
+
+A feature that was originally considered to be important, and is
+present in ESP[^4][^5], is the constant monitoring of values. This
+feature has not been too happily received in MEND, but that may be due
+more to the current implementation than to an inherent lack of
+utility.
+
+Ideally one should not have to see the object being monitored or its
+value if one does not want to. The debugging system should be capable
+of performing certain actions upon, for example, a change in value.
+This would be analogous to conditional breakpoints except that the
+test would be performed after each step of execution of the
+application program rather than at certain arbitrary times.
+
+One problem with the above scheme is the difficulty of testing for a
+change in value (the basic test). For example, the value of an object
+being monitored may be some sort of structure. During execution the
+application program may not explicitly change the value of the object
+but may change an element of the structure that the object evaluates
+to. Assuming MEND had saved a pointer to the structure, if it now
+evaluated the object being monitored, it would again find that
+structure and assume that the value had not changed. However since an
+element of the structure had changed, the programmer would probably
+want to be notified.
+
+The only solution to the above dilemma is to at each step save a
+complete representation to all levels of the value of the object in
+some form that cannot be affected by the application program. This can
+be done by, at each step, unparsing the value to be saved for future
+tests into its printed (string) representation. Then the comparison of
+current and previous values will simply be a string comparison, not
+subject to sharing by the values. Unfortunately this may create
+incredible amounts of "garbage." In fact, a circular list (quite
+legal) will produce a string of infinite length!
+
+The solution finally chosen was to print the values of the objects
+being monitored at each step and to let the user visually compare
+versions. The values are &-printed as usual but may be examined in
+full at will.
+
+When tested it became immediately obvious that the constant reprinting
+wasted time and was distracting. A feature was added to cause printing
+to occur only at controllable intervals, but that produced other
+problems. Primarily the user might not see a change that had occurred
+at one step until some later step when valuable information had
+already been lost. One would also not have the opportunity to take
+immediate action.
+
+What needs to be done at this point is to reduce the emphasis on such
+pathological cases as described above. Printing should only occur at
+well defined moments (i.e., when requested or when some recognizable
+condition occurs). Only gross and easily testable changes in value
+should be watched for (i.e., the value is a new object, entirely).
+Conditional monitoring should be installed analogous to the conditions
+of breakpoints. With all of this testing, a variety of indications
+should be available as with breakpoints.
+
+V.2 Control Stack Display
+-------------------------
+
+MEND's display of the application program's control stack has proven
+to be by itself a highly useful debugging aid. This display can be and
+often is used without any of MEND's other features to debug a faulty
+program. It should also be emphasized that a valid use of this display
+is to monitor a working program in order to understand its operation.
+In this case, where the user of MEND is completely unfamiliar with the
+operation of the application program, MEND's step-by-step display of
+program execution is even more useful than in the normal debugging
+situation.
+
+One of the unique features of MEND's "trace" of the application
+program is that MEND shows both the code being executed and the
+results of that execution. Since Muddle is an applicative language,
+most of the results consist of values returned by evaluated objects.
+By reflecting the returned values back into the original code, MEND
+shows intermediate results in an intuitive manner for the user.
+
+Other programming languages could also benefit from this sort of
+display. Too often a trace routine provided for debugging in a
+language shows only results of execution, such as value assignments,
+without showing the corresponding program steps, or it shows program
+steps but leaves it to the programmer to insert instructions to show
+intermediate results. What the programmer really needs to see is a
+well-integrated display of both program instructions and results, such
+as MEND provides.
+
+Muddle is an exceptional language in that the MEND display was
+implemented fairly cleanly with just a package of Muddle functions and
+without changes to the interpreter itself. With suitable language
+enhancements, however, a similar display could be provided for LISP or
+any other LISP-like language. Although major compiler modifications
+would be required, a stack-oriented display might also be suitable for
+a block structured language like ALGOL.
+
+Especially in a block structured language, where instructions might be
+treated in groups rather than individually, but also in applicative
+languages, it would be useful to be shown variable bindings as they
+occur. Muddle does not provide any information about the occurrence of
+bindings, so the programmer is not informed of such occurrences by
+MEND. However when major modifications are made in a language to
+provide a display similar to MEND's, it would probably be a mistake
+not to include a provision for notifying the programmer of variable
+bindings and unbindings as they occur.
+
+V.3 Simulation Vs. Multiprocessing
+----------------------------------
+
+Besides its control stack display, MEND's other major feature, and its
+major difference from debugging systems like EEP or the first version
+of MUMBLE, is its use of multiprocessing rather than simulation to
+follow application program execution. (Even debugging aids that use
+multiprocessing often do not work the way MEND does. While MEND is a
+program written in the same language and running concurrently in the
+same interpreter as the application program, some debuggers , such a
+DDT, are machine language programs running near the application
+program, and others are simply complied into the language itself.)
+This feature is perhaps the key to MEND's success as a debugger. The
+choice to avoid simulation certainly reduced the size and complexity
+of MEND by two or three times and similarly affected run time. What
+remains to be discussed are the tradeoffs involved in the choice. The
+advantages just mentioned heavily favor the multiprocessing solution
+but there are other considerations.
+
+If MEND had been implemented using simulation, it would for the
+evaluation of each object determine which elements need evaluation;
+evaluate those elements by recursively calling MEND's own evaluation
+simulator; determine whether some function should be applied;
+determine the effects of the required functional application, if any;
+and cause those effects while displaying some representation of them
+for the programmer. In short, MEND would duplicate the actions of
+Muddle while maintaining and updating its own information about those
+actions. Therefore MEND would at all times know exactly what the
+application program was doing and could never miss some important
+effects of execution or be fooled by an unusual program. In the
+previous section it was stated that it is desirable to show the
+occurrence of variable bindings to the programmer. This feature would
+be easy to implement in a debugging system that simulated execution of
+the application program.
+
+But what about a debugging system using multiprocessing? A
+multiprocessing debugger, such as MEND, allow s the application
+program to execute more-or-less freely in its own process while
+monitoring it from another process. Since MEND id not in direct
+control of the application program, it must rely for its information
+on whatever data it is given by the multiprocessing mechanism built
+into Muddle and on inferences based on examination of the application
+program's environment, its knowledge of the general behavior of Muddle
+programs, and some specific knowledge of special cases. The occurrence
+of variable bindings is not decipherable from the information that
+MEND can gather. Fortunately, as has been previously shown, MEND does
+have the information essential to the creation of its display and the
+display is a sufficient debugging tool.
+
+Monitoring execution of the application program from another process
+provides sufficient information for a useful debugging system while
+simulation of the application program provides enough information for
+a more comprehensive system. Therefore if speed and size of the
+debugger were not important factors, then, although not a necessary
+choice, simulation would be the favored choice for a debugging system
+except for one other factor.
+
+The simulator needs much more knowledge of specific details of the
+behavior of program functions than does the monitor. In fact the
+simulator must know exactly the effects of each function that might be
+used in the application program. Uncompiled user-defined functions are
+no problem. They consist merely of one or more applications of other
+functions to given arguments. Complied user-defined functions cannot
+be dealt with at all without actually creating a simulator for machine
+language programs, a project comparable in magnitude to the design and
+implementation of all of the other parts of the simulator combined!
+The remaining functions are Muddle's built-in primitives. They are
+well defined but frequently change (usually upward compatibly).
+Furthermore new primitives are constantly being added. A simulator for
+Muddle programs would therefore quickly become outdated.
+
+Given the problems of a simulator and the relatively few drawbacks of
+a monitor, it is clear now why MEND was implemented as the latter. The
+same arguments apply to debugging systems for other languages.
+Whenever the appropriate multiprocessing features exist, with
+sufficient information available about execution in another process, a
+debugging system based upon simulation of the application program
+should be rejected in favor of a monitor-type system. MEND has proven
+to be an effective debugging aid and should serve as a good example of
+the latter type of system.
+
[^1]: G.A. Mann, "A Survey of Debug Systems", Honeywell Computer
Journal 1973, volume 7, number 3, pages 182-198
[^11]: N. Ryan, EDIT: The Muddle Editor, SYS.11.14, Programming
Technology Division Document, Laboratory for Computer Science,
M.I.T., August, 1974
+
+[^12]: D. Eastlake, R. Greenblatt, J. Holloway, T. Knight, and S.
+ Nelson, ITS 1.5 Reference Manual, Memo No. 161A, Artificial
+ Intelligence Laboratory, M.I.T., July 1969
+
+[^13]: P.D. Lebling, SSV User's Manual, SYS.52.07, Programming
+ Technology Division Document, Laboratory for Computer Science,
+ M.I.T., (in preparation)
+
+[^14]: S.W. Galley, Pre-loaded Pure MDL RSUBRs, SYS.11.28, Programming
+ Technology Division Document, Laboratory for Computer Science,
+ M.I.T., November 1975
+
+[^15]: B. Daniels, The MDL Assembler, SYS.11.07, Programming
+ Technology Division Document, Laboratory for Computer Science,
+ M.I.T., (in preparation)
+
+[^16]: C. Reeve, The MDL Compiler, SYS.11.25, Programming Technology
+ Division Document, Laboratory for Computer Science, M.I.T., (in
+ preparation)
projects / mudman.git / blobdiff