Copyright (c) 1999 Massachusetts Institute of Technology This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. ------------------------------ PROGRAMMING TECHNOLOGY DIVISION DOCUMENT SYS.xx.yy IDENTIFICATION The JOB/BOJ Device: A Mechanism for Implementing Non-standard Devices Marc S. Seriff, Jack Haverty, Richard Stallman (MIT-AI) September 18, 1974 INTRODUCTION It is occasionally desirable to extend the ITS system to allow for the use of non-standard I/O devices. A very desirable feature would be to allow user-level programs to make use of the newly defined devices with out modification of the user-level programs. For this reason the JOB/BOJ device was implemented. When a job (call it the user) attempts an OPEN for a file whose name is "JOB: ; ", the system creates a new job, called the BOJ device handler, or BDH (this is a terrible name; it USES the BOJ device and HANDLES the JOB device), and attempts to load the file "DSK: ; " into that job. If the load fails, the OPEN will fail. If this load succeeds, the OPEN, as well as all further I/O operations on the user's JOB channel will be referred to the BDH for execution. At that point, it will be up to the BDH to decide whether or not the open should fail. In order to get a handle on information about the user's open, The BDH opens one or more channels to the "BOJ" device. The system will insure that the BDH's BOJ channel is logically linked to the user's JOB channel. The following diagram illustrates what happens: __________ _________ | | | | | User |-----v______+---->| BDH | |__________| | | V--X---^ | | |_________| | | System | | ____| |____|_____| |____ | | | The JOB channel | The BOJ channel | V System mediates Once the BDH has successfully opened the BOJ channel, it will be notified (via an I/O channel interrupt) every time the user attempts an I/O operation on the JOB channel. The BDH may then examine the data that is made available to it about the I/O operation and may simulate the operation for the user. In this way, BDHs can be written to give anything the appearance of a disk, tape or other specific type of I/O device. USING THE JOB DEVICE Programs that use the JOB device should not be required to make special provisions. A BDH should, in general, be written so that the non-standard device will behave exactly like some standard device. In other words, a user program may open a JOB device using a standard open and may then execute any legal I/O operation. A file name with something other than "JOB" for a device code can be made to act like a JOB device in one of two ways. The first way should be used for devices that are not generally applicable or are expected to be used only for a short time. To use this method simply use translates (using MONIT, DDT or directly via system calls) to inform the system which file should be used as a BDH. For instance, if we translate all file names of the form "XXX:" into the file name "JOB:ABC;TS RUNXXX", then the system will use the file "DSK:ABC;TS RUNXXX" as the BDH for any file name whose device code is "XXX". The other method is much simpler and is designed for non-standard devices that are to be made available to the entire user community. When an OPEN is attempted on a device whose name the system does not recognize, the system will examine the "DEVICE;" directory for a file whose first name is "JOBDEV" and whose second name is the unrecognized device name. If such a file exists, then the contents of that file will be loaded by the system and used as the BDH for the user's OPEN. USING THE BOJ DEVICE - THE BOJ DEVICE HANDLER The BDH is the workhorse in the JOB/BOJ device scheme. It is required to interpret all of the user's I/O operations. It must do everything to make the non-standard device that it represents appear standard. The system provides five .CALLs to give the BDH some of the required communcation with the user. The first thing that the BDH should do when he is loaded and started by the system is to execute an OPEN on the BOJ device. This establishs a logical link between the user and the BDH. The BDH may enable a channel interrupt on the BOJ channel, in order to be informed whenever the user attempts an I/O operation. It should be noted that the ITS system allows for the possibility of a full duplex connection between the user and the BDH (i.e. a channel in each direction). If a user attempts an open for reading and an open for writing on the same device, then the system will present both opens to the same activation of the BDH. Should this be undesirable for a particular BDH, that BDH should use an open mode with the 3.4 bit set (10 in the left half) each time it attempts to open the BOJ device. JOBCAL SYSTEM CALL The first .CALL that will be discussed is the JOBCAL. This call enables the BDH to find out what I/O operation the user has attempted to execute. It has the following calling sequence: .CALL JOBCAL ... ; Error return ... ; Success return JOBCAL: SETZ SIXBIT/JOBCAL/ [] 2000,, SETZ [-,,] This .CALL, as well as the other three discussed below, will fail to skip if any of the following conditions hold: 1. is an illegal channel number. 2. is not open on the BOJ device. 3. the user job does not exist or as has closed his channel. 4. the user job is no longer attempting to execute the I/O operation. should be an address into which information about the I/O operation requested will be placed. This word contains three sections. The right half of the word contains an integer indicating which operation was attempted. The following table indicates what the various integers mean: CODE OPERATION 0 OPEN 1 IOT (transfer) 2 STATUS (currently unused) 3 RESET 4 RCHST (complete status) 5 ACCESS 6 FDELE (for delete or rename while not open) 7 FDELE (for rename while open) 8 CALL (see below) It is up to the BDH to insure that these I/O operations perform as they would for standard ITS devices. The top three bits (4.9-4.7) of the op-code word contain the mode in which user is attempting to open his channel. This is only meaningful for op-code 0, OPEN. If either of the next two bits (4.6 and 4.5) are non-zero, then the user is requesting that a channel be closed. As was mentioned above, the system allows the connection between user and BDH to be full duplex. If the 4.6 bit of the opcode is set, then the user is requesting that his input channel be closed, while the 4.5 bit requests a close of the user's output channel. It is possible to get an opcode in which both bits are set. In the case the BDH should perform the obvious task of closing both channels. Most I/O operations require more information than just the name of the operation. The third argument to JOBCAL tells the system where any additional information that is available should be placed. is the largest number of words that the BDH is willing to accept (it should normally be 12). is the address where the first word of the information should be placed. What information is placed in this buffer is determined by the I/O operation being attempted. If the op-code (in the right half of the op-code word) is less than 8, then five words will be written into the buffer (beginning at ) in response to the JOBCAL. The meanings of the words are given in the following table: WORD OPERATIONS MEANING 1 IOT (1) Meaningful only for block IOTs. This word will contain the user's IOT pointer. The left half of this word will be the negative of the number of words that the user wants. 1 ACCESS (5) The address within the virtual file that is open from (or to) which the next IOT should be done. The first word of the virtual file is word zero. 1 FDELE (6 or 7) Zero implies delete requested. Non-zero means that a rename is requested. This word is the name (in sixbit) to be used as the new first file name. 2 OPEN (0) or FDELE (6) First name of the file that the user is attempting to open, rename or delete. 3 OPEN (0) or FDELE (6) Second name of the file that the user is attempting to open, rename or delete. 4 OPEN (0) or FDELE (6) Name of directory to be used for open, rename or delete. 5 OPEN (0) or FDELE (6) Name of device to be used for open, rename or delete. 6 OPEN (0) The full 18 bit open mode (right justified). 6 FDELE (6 or 7) Zero implies delete. Non-zero means that a rename is requested. This word is the name (in sixbit) to be used as the new second file name. If the op-code returned by JOBCAL is 8, then the user has attempted some I/O operation not recognized by the systems JOB/BOJ device code (this does not mean that the operation is illegal). In this case, more information is written into the BDH's buffer (up to twelve words). The first of the data words will be the SIXBIT name of the operation being performed. (This will normally be the name of the .CALL executed by the user - see Appendix 1 for a list of possiblities.) The second word will contain all of the flags that the user has set using the "flag" feature of the .CALL UUO. The third word will be an integer indicating how many input (to the system) arguments the user supplied in his .CALL. It should be noted that if this integer is larger than +3, then some information will be lost. The values of the input arguments will appear in the remainder of the words in the buffer. Remember that, in almost all cases, the first of the input arguments will be the user's channel number. JOBRET SYSTEM CALL Once the BDH has interpreted the user's request for execution of an I/O operation, the BDH must have some way of responding to the user. This is provided with the JOBRET .CALL. This .CALL serves three purposes: 1. to unblock the user who is waiting for completion of his I/O request (possibly causing the request to skip) 2. to set lossage codes in the user's status words (e.g. for failed opens). 3. to return information requested by the user's I/O operation. The format of this CALL is as follows: .CALL JOBRET ... ; error return ... ; success return JOBRET: SETZ SIXBIT /JOBRET/ [] [] SETZ [-,,] where satisfies requirements 1 and 2 above. If is zero, then the user's I/O call will not skip or set status bits. If is of the form ,, , then will be placed in the "open-loss" field of the status word for the user's channel and the user's I/O call will skip times. A list of all currently recognized open loss codes can be found in Appendix 2. If the user's I/O call requested data, then the BDH can supply this data by supplying the third argument to the JOBRET call. The system will use the words beginning at location as the values for the return arguments in the user's call. SETIOC AND JOBINT -- INTERRUPT SYSTEM CALLS There are two interrupt oriented functions that the BDH must perform and there are CALLs available to perform them. The first is to notify the user when he has done something catastrophic. The user should be notified of his error by causing the system to awaken him with an I/O channel error. This done with the SETIOC call, as follows: .CALL SETIOC ... ; error return (standard reasons + illegal IOCERR code) ... ; success return SETIOC: SETZ SIXBIT/SETIOC/ [] SETZ [] where is an integer identifying the reason for the error. This integer should be chosen from the list that can be found in Appendix 2. Since BDHs are written to handle non-standard devices, these error codes will quite often not apply. Simply choose the code that comes closest. Occassionally, the BDH must notify the user of a non-catastrophic situation (e.g. the arrival of data). The JOBINT call allows the BDH to give the user a second-word I/O channel interrupt, as follows: .CALL JOBINT ... ; error return ... ; success return JOBINT: SETZ SIXBIT /JOBINT/ SETZ [] JOBSTS SYSTEM CALL Finally, the BDH must have some way to notify the system what the status of the pseudo-device is. For this purpose, the JOBSTS call is available. Its calling sequence is as follows: .CALL JOBSTS ... ; error return ... ; success return JOBSTS: SETZ SIXBIT /JOBSTS/ [] SETZ [] The right half of will be given to anyone requesting the status of the user's channel until another JOBSTS is done. It should be remembered that the low order six bits (1.1-1.6) should contain the device code of the pseudo-device. Unless, you know what you are doing, this should always be 22 octal. DATA TRANSFERS - HOW TO IOT Since the main purposes of I/O devices is transfer of information, let us look at how the BDH manages to respond to the user's IOTs. The method for doing this is very simply. If we think back to the diagram presented early in this memo, we will remember that the BOJ and JOB channels (i.e. the channels belonging to the BDH and the user, respectively) are logically connected. The full meaning of this now becomes clear. If the JOB channel is open for input and the BOJ channel is open for output, then anything that BDH outputs on the BOJ channel will be available to the user as input on the JOB channel. (For this reason, the BDH must insure that he opens the BOJ channel in the opposite direction from the user's JOB channel even if this requires opening the BOJ channel a second time in the correct mode.) There are two features of which the BDH implementer should be aware when writing the I/O sections of the BDH. Let us assume for the discussion here, that the user has the JOB channel open for reading and that the BDH has the BOJ channel open for writing. Let us also assume that the user is currently hung attempting to read 100 words on his JOB channel. First it should be noted that the BDH does not have to respond to this request with a single transfer of 100 words. The system will act as an mediator between different transfer sizes. The BDH can respond to the user's request for 100 words in several ways: 1. He can send all 100 words in small pieces (e.g. 10 transfers of 10 words each. 2. He can send all 100 words in a single transfer. 3. He can send more than 100 words. In this case, the BDH will remain hung in his IOT until the user has read all of the data that the BDH is attempting to send. (If this is undesirable, set the 3.5 in the BOJ open mode. This will cause BOJ IOTs to unhang whenever the user's IOT is satisfied. When that happens, the BOJ IOT pointer will have been counted out only partially; the RH will point to the first word not transfered) 4. He may send less than 100 words. In this case, the BDH must manually awaken the user as described below. The user, who is hung awaiting his 100 words, will stay hung until he has received all 100 words. Suppose, however, that the BDH only wishes to send 50 words (e.g. the last 50 words of the virtual file). To do this, he can send the 50 words normally, but must then use the JOBRET call (described above) to awaken the user. JOBRET should be called with the BOJ channel number as the first argument, zero as the second argument and no third argument. Secondly, the BDH implementer must be aware that the ITS system guarantees that IOTs to a channel open in block mode, will never generate an I/O channel error. In other words, the following algorithm should be followed: 1. If the user requests n words and there are n words or more left in the "file", give him n words. 2. If the user requests n words and there are only m words (n>m), give him m words and manually awaken him (using JOBRET). 3. If the user requests n words and there are no words left, then give him nothing and manually awaken him (using JOBRET). HINTS TO BDH IMPLEMENTERS The following hints should ease the task of BDH implementers a little. It is hoped that anyone who attempts to a BDH implementation will add his harshly acquired knowledge to this section of this memo. 1. Remember that the BDH is essentially a disowned job and should attempt to log out after a close has been requested. 2. Be very leery of logging out for any reason other than a requested CLOSE. In particular: a. if a JOBCAL fails, only log out if a request for OPEN has not yet been received. b. when you generate an I/O channel error for the user, only log out if the error is irrecoverable. Remember that he can correct an access beyond end-of-file by doing an access before attempting another IOT. c. if he requests an I/O operation that you do not recognize, generate a "mode not available" error (via JOBRET) and wait for his next request. d. if you decide, for your own reasons, to make the initial open fail, you should log out. THE OJB DEVICE -- AN AID FOR DEBUGGING THE BDH One of the problems with debugging BDH programs is that when the JOB device is used, it is loaded into a newly created job. That job is not inferior to any DDT, and there is no way to put breakpoints in it before it starts. The OJB device makes it possible to run the BDH program under DDT. The OJB device acts just like the JOB device except during the initial open. When the JOB device would be creating and loading a new job, the OJB device is looking for an existing job whose UNAME and JNAME are the same as the filenames specified in the open. If such a job is found, it is connected to the job opening the OJB device through a standard JOB-BOJ pipeline. If such a job does not exist, the open of the OJB device fails. In order to protect innocent jobs from being hacked in this matter, the job opened on the OJB device is required to have set its OPTOJB bit (this is bit 4.2 in the .OPTION varisble, settable with .SUSET). Also, it must not already be a BDH, for the system cannot consider one job to be a BDH through two connections at once. If either of those conditions is not met, the OJB open WAITS until they are. To remove a possible timing screw, a BOJ device open by a job that is not a BDH, which usually fails, will wait instead for the job to become a BDH if the job's OPTOJB bit is set. The procedure for using the OJB device for debugging is: 1. create a job to use for the BDH (call it J, in this example). 2. run (in another job) the program which would normally open the JOB device, but tell it (perhaps by means of a translation) to use the OJB device instead. This program's function is to issue system calls so that the BDH's responses to them can be tested. Since the OPTOJB bit of job J is now 0, when this program reaches the OJB open it will hang. 3. ^Z that program, and ^P it. it will go back to hanging in the open of the OJB device. 4. switch to job J, load in the BDH program. This RESET's the job so that if it had been a BDH before, it will not be one now. 5. turn on the OPTOJB bit by depositing from DDT in .OPTION. The conditions for a successful OJB open by job JJ have now been met, so that job will make some headway, turning job J into a BDH and waiting for job J to issue a JOBRET. 6. it is now possible to start running job J, with breakpoints as desired, to step through the code for handling the initial open. It is unlikely, but theoretically possible, for J to execute its BOJ open before JJ gets around to turning J into a BDH. In this case, J will wait for JJ to do so. 7. if it becomes necessary to try again after discovering a bug, go back to step 2. The OJB open will hang up this time not because the OPTOJB bit is off (since it is still 1) but because J is already a BDH. Loading J in step 4 will make J cease to be a BDH but also turn off the OPTOJB bit, so JJ will still be waiting. 8. it is possible for job JJ to be "PCLSR'ed" or backed up out of its open while job J is being traced through the handling of the initial open. For example, job JJ might receive a real-time interrupt. If that happens, J will cease to be a BDH. This will not interfere with the tracing of J until the next BOJ device system call is executed; that call will probably fail. It is impossible to continue after such an occurrence, so the jobs must be restarted by returning to step 2. Of course, if there are no breakpoints before the first JOBRET, this is very unlikely to happen. 9. if the initial open has been handled successfully, to go on to debug handling of other system calls, simply tell JJ to execute them with J stopped while JJ is being told, then ^Z^P JJ and go back to J. 10. the usual .LOGOUT in the BDH program will be a no-op when the program is run as an inferior in this manner, so it should be followed by a .VALUE.