Consolidate license copies

[its.git] / sysdoc / jobonl.100

Copyright (c) 1999 Massachusetts Institute of Technology

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or (at
your option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
------------------------------

PROGRAMMING TECHNOLOGY DIVISION DOCUMENT                     SYS.xx.yy

IDENTIFICATION
     The JOB/BOJ Device:  A Mechanism for Implementing
                          Non-standard Devices
     Marc S. Seriff, Jack Haverty, Richard Stallman (MIT-AI)
     September 18, 1974

INTRODUCTION
     It is occasionally desirable to extend the ITS system to allow
for the use of non-standard I/O devices.  A very desirable feature
would be to allow user-level programs to make use of the newly defined
devices with out modification of the user-level programs.  For this
reason the JOB/BOJ device was implemented.
     When a job (call it the user) attempts an OPEN for a file whose
name is "JOB: <dir>; <fn1> <fn2>", the system creates a
new job, called the BOJ device handler, or BDH (this is a terrible
name; it USES the BOJ device and HANDLES the JOB device), and attempts
to load the file "DSK: <dir>; <fn1> <fn2>" into that job.
If the load fails, the OPEN will fail.  If this load succeeds, the
OPEN, as well as all further I/O operations on the user's JOB channel
will be referred to the BDH for execution.  At that point, it will be
up to the BDH to decide whether or not the open should fail.  In order
to get a handle on information about the user's open, The BDH opens
one or more channels to the "BOJ" device.  The system will insure that
the BDH's BOJ channel is logically linked to the user's JOB channel.

The following diagram illustrates what happens:

        __________                    _________
       |          |                  |         |
       |   User   |-----v______+---->|   BDH   |
       |__________| | | V--X---^ | | |_________|
                    | |  System  | |
                ____| |____|_____| |____
               |           |            |
        The JOB channel    |     The BOJ channel
                           |
                           V
                     System mediates

     Once the BDH has successfully opened the BOJ channel, it will be
notified (via an I/O channel interrupt) every time the user attempts
an I/O operation on the JOB channel.  The BDH may then examine the
data that is made available to it about the I/O operation and may
simulate the operation for the user.  In this way, BDHs can be written
to give anything the appearance of a disk, tape or other specific type
of I/O device.

USING THE JOB DEVICE
     Programs that use the JOB device should not be required to make
special provisions.  A BDH should, in general, be written so that the
non-standard device will behave exactly like some standard device.  In
other words, a user program may open a JOB device using a standard
open and may then execute any legal I/O operation.

     A file name with something other than "JOB" for a device code can
be made to act like a JOB device in one of two ways.  The first way
should be used for devices that are not generally applicable or are
expected to be used only for a short time.  To use this method simply
use translates (using MONIT, DDT or directly via system calls) to
inform the system which file should be used as a BDH.  For instance,
if we translate all file names of the form "XXX:" into the file name
"JOB:ABC;TS RUNXXX", then the system will use the file "DSK:ABC;TS
RUNXXX" as the BDH for any file name whose device code is "XXX".

     The other method is much simpler and is designed for non-standard
devices that are to be made available to the entire user community.
When an OPEN is attempted on a device whose name the system does not
recognize, the system will examine the "DEVICE;" directory for a file
whose first name is "JOBDEV" and whose second name is the unrecognized
device name.  If such a file exists, then the contents of that file
will be loaded by the system and used as the BDH for the user's OPEN.

USING THE BOJ DEVICE - THE BOJ DEVICE HANDLER

     The BDH is the workhorse in the JOB/BOJ device scheme.  It is
required to interpret all of the user's I/O operations.  It must do
everything to make the non-standard device that it represents appear
standard.  The system provides five .CALLs to give the BDH some of the
required communcation with the user.

     The first thing that the BDH should do when he is loaded and
started by the system is to execute an OPEN on the BOJ device.  This
establishs a logical link between the user and the BDH.  The BDH may
enable a channel interrupt on the BOJ channel, in order to be informed
whenever the user attempts an I/O operation.

     It should be noted that the ITS system allows for the possibility
of a full duplex connection between the user and the BDH (i.e. a
channel in each direction).  If a user attempts an open for reading
and an open for writing on the same device, then the system will
present both opens to the same activation of the BDH.  Should this be
undesirable for a particular BDH, that BDH should use an open mode
with the 3.4 bit set (10 in the left half) each time it attempts to
open the BOJ device.

JOBCAL SYSTEM CALL

     The first .CALL that will be discussed is the JOBCAL.  This call
enables the BDH to find out what I/O operation the user has attempted
to execute.  It has the following calling sequence:

        .CALL   JOBCAL
        ...             ; Error return
        ...             ; Success return

JOBCAL: SETZ
        SIXBIT/JOBCAL/
        [<BOJ channel number>]
        2000,,<opcode>
        SETZ [-<n>,,<data>]

This .CALL, as well as the other three discussed below, will fail to
skip if any of the following conditions hold:

        1.  <BOJ channel number> is an illegal channel number.
        2.  <BOJ channel number> is not open on the BOJ device.
        3.  the user job does not exist or as has closed his channel.
        4.  the user job is no longer attempting to execute the I/O
            operation.

     <opcode> should be an address into which information about
the I/O operation requested will be placed.  This word contains three
sections.  The right half of the word contains an integer indicating
which operation was attempted.  The following table indicates what the
various integers mean:

       CODE  OPERATION

        0    OPEN
        1    IOT (transfer)
        2    STATUS (currently unused)
        3    RESET
        4    RCHST (complete status)
        5    ACCESS
        6    FDELE (for delete or rename while not open)
        7    FDELE (for rename while open)
        8    CALL (see below)

It is up to the BDH to insure that these I/O operations perform as
they would for standard ITS devices.

     The top three bits (4.9-4.7) of the op-code word contain the mode
in which user is attempting to open his channel.  This is only
meaningful for op-code 0, OPEN.  If either of the next two bits (4.6
and 4.5) are non-zero, then the user is requesting that a channel be
closed.  As was mentioned above, the system allows the connection
between user and BDH to be full duplex.  If the 4.6 bit of the opcode
is set, then the user is requesting that his input channel be closed,
while the 4.5 bit requests a close of the user's output channel.  It
is possible to get an opcode in which both bits are set.  In the case
the BDH should perform the obvious task of closing both channels.

     Most I/O operations require more information than just the name
of the operation.  The third argument to JOBCAL tells the system where
any additional information that is available should be placed.
<n> is the largest number of words that the BDH is willing to

accept (it should normally be 12).  <data> is the address where
the first word of the information should be placed.  What information
is placed in this buffer is determined by the I/O operation being
attempted.

     If the op-code (in the right half of the op-code word) is less
than 8, then five words will be written into the buffer (beginning at
<data>) in response to the JOBCAL.  The meanings of the words are
given in the following table:

WORD          OPERATIONS               MEANING

  1      IOT (1)                  Meaningful only for block IOTs.
                                  This word will contain the user's
                                  IOT pointer.  The left half of this
                                  word will be the negative of the
                                  number of words that the user wants.

  1      ACCESS (5)               The address within the virtual file
                                  that is open from (or to) which the
                                  next IOT should be done.  The first
                                  word of the virtual file is word
                                  zero.

  1      FDELE (6 or 7)           Zero implies delete requested.
                                  Non-zero means that a rename is
                                  requested.  This word is the name
                                  (in sixbit) to be used as the new
                                  first file name.

  2      OPEN (0) or FDELE (6)    First name of the file that the user
                                  is attempting to open, rename or
                                  delete.

  3      OPEN (0) or FDELE (6)    Second name of the file that the
                                  user is attempting to open, rename
                                  or delete.

  4      OPEN (0) or FDELE (6)    Name of directory to be used for
                                  open, rename or delete.

  5      OPEN (0) or FDELE (6)    Name of device to be used for open,
                                  rename or delete.

  6      OPEN (0)                 The full 18 bit open mode (right
                                  justified).

  6      FDELE (6 or 7)           Zero implies delete.  Non-zero means
                                  that a rename is requested.  This
                                  word is the name (in sixbit) to be
                                  used as the new second file name.

     If the op-code returned by JOBCAL is 8, then the user has
attempted some I/O operation not recognized by the systems JOB/BOJ
device code (this does not mean that the operation is illegal).  In
this case, more information is written into the BDH's buffer (up to
twelve words).  The first of the data words will be the SIXBIT name of
the operation being performed.  (This will normally be the name of the
.CALL executed by the user - see Appendix 1 for a list of
possiblities.) The second word will contain all of the flags that the
user has set using the "flag" feature of the .CALL UUO.  The third
word will be an integer indicating how many input (to the system)
arguments the user supplied in his .CALL.  It should be noted that if
this integer is larger than <n>+3, then some information will be
lost.  The values of the input arguments will appear in the remainder
of the words in the buffer.  Remember that, in almost all cases, the
first of the input arguments will be the user's channel number.

JOBRET SYSTEM CALL

     Once the BDH has interpreted the user's request for execution of
an I/O operation, the BDH must have some way of responding to the
user.  This is provided with the JOBRET .CALL.  This .CALL serves
three purposes:

        1.  to unblock the user who is waiting for completion of his
            I/O request (possibly causing the request to skip)

        2.  to set lossage codes in the user's status words (e.g. for
            failed opens).

        3.  to return information requested by the user's I/O
            operation.

The format of this CALL is as follows:

        .CALL   JOBRET
        ...             ; error return
        ...             ; success return

JOBRET: SETZ
        SIXBIT /JOBRET/
        [<BOJ channel number>]
        [<return>]
        SETZ [-<n>,,<data>]

where <return> satisfies requirements 1 and 2 above.  If
<return> is zero, then the user's I/O call will not skip or set
status bits.  If <return> is of the form <i>,,<j> , then
<i> will be placed in the "open-loss" field of the status word for
the user's channel and the user's I/O call will skip <j> times.  A
list of all currently recognized open loss codes can be found in

Appendix 2.

     If the user's I/O call requested data, then the BDH can supply
this data by supplying the third argument to the JOBRET call.  The
system will use the <n> words beginning at location <data> as
the values for the return arguments in the user's call.

SETIOC AND JOBINT -- INTERRUPT SYSTEM CALLS

     There are two interrupt oriented functions that the BDH must
perform and there are CALLs available to perform them.  The first is
to notify the user when he has done something catastrophic.  The user
should be notified of his error by causing the system to awaken him
with an I/O channel error.  This done with the SETIOC call, as
follows:

        .CALL   SETIOC
        ...             ; error return (standard reasons
                                + illegal IOCERR code)
        ...             ; success return

SETIOC: SETZ
        SIXBIT/SETIOC/
        [<BOJ channel number>]
        SETZ [<IOCERR code>]

where <IOCERR code> is an integer identifying the reason for the
error.  This integer should be chosen from the list that can be found
in Appendix 2.  Since BDHs are written to handle non-standard devices,
these error codes will quite often not apply.  Simply choose the code
that comes closest.

     Occassionally, the BDH must notify the user of a non-catastrophic
situation (e.g. the arrival of data).  The JOBINT call allows the BDH
to give the user a second-word I/O channel interrupt, as follows:

        .CALL   JOBINT
        ...             ; error return
        ...             ; success return

JOBINT: SETZ
        SIXBIT /JOBINT/
        SETZ [<BOJ channel number>]

JOBSTS SYSTEM CALL

     Finally, the BDH must have some way to notify the system what the
status of the pseudo-device is.  For this purpose, the JOBSTS call is
available.  Its calling sequence is as follows:

        .CALL   JOBSTS
        ...             ; error return
        ...             ; success return

JOBSTS: SETZ
        SIXBIT /JOBSTS/
        [<BOJ channel number>]
        SETZ [<new status>]

The right half of <new status> will be given to anyone requesting
the status of the user's channel until another JOBSTS is done.  It
should be remembered that the low order six bits (1.1-1.6) should
contain the device code of the pseudo-device.  Unless, you know what
you are doing, this should always be 22 octal.

DATA TRANSFERS - HOW TO IOT

     Since the main purposes of I/O devices is transfer of
information, let us look at how the BDH manages to respond to the
user's IOTs.  The method for doing this is very simply.  If we think
back to the diagram presented early in this memo, we will remember
that the BOJ and JOB channels (i.e. the channels belonging to the BDH
and the user, respectively) are logically connected.  The full meaning
of this now becomes clear.  If the JOB channel is open for input and
the BOJ channel is open for output, then anything that BDH outputs on
the BOJ channel will be available to the user as input on the JOB
channel.  (For this reason, the BDH must insure that he opens the BOJ
channel in the opposite direction from the user's JOB channel even if
this requires opening the BOJ channel a second time in the correct
mode.)

     There are two features of which the BDH implementer should be
aware when writing the I/O sections of the BDH.  Let us assume for the
discussion here, that the user has the JOB channel open for reading
and that the BDH has the BOJ channel open for writing.  Let us also
assume that the user is currently hung attempting to read 100 words on
his JOB channel.

     First it should be noted that the BDH does not have to respond to
this request with a single transfer of 100 words.  The system will act
as an mediator between different transfer sizes.  The BDH can respond
to the user's request for 100 words in several ways:

        1.  He can send all 100 words in small pieces (e.g. 10
            transfers of 10 words each.
        2.  He can send all 100 words in a single transfer.
        3.  He can send more than 100 words.  In this case, the BDH
            will remain hung in his IOT until the user has read all of
            the data that the BDH is attempting to send.  (If this is
            undesirable, set the 3.5 in the BOJ open mode.  This will
            cause BOJ IOTs to unhang whenever the user's IOT is
            satisfied. When that happens, the BOJ IOT pointer will
            have been counted out only partially; the RH will point to
            the first word not transfered)
        4.  He may send less than 100 words.  In this case, the BDH
            must manually awaken the user as described below.

The user, who is hung awaiting his 100 words, will stay hung until he
has received all 100 words.  Suppose, however, that the BDH only
wishes to send 50 words (e.g. the last 50 words of the virtual file).
To do this, he can send the 50 words normally, but must then use the
JOBRET call (described above) to awaken the user.  JOBRET should be
called with the BOJ channel number as the first argument, zero as the
second argument and no third argument.  Secondly, the BDH implementer
must be aware that the ITS system guarantees that IOTs to a channel
open in block mode, will never generate an I/O channel error.  In
other words, the following algorithm should be followed:

        1.  If the user requests n words and there are n words or more
            left in the "file", give him n words.
        2.  If the user requests n words and there are only m words
            (n>m), give him m words and manually awaken him (using
            JOBRET).
        3.  If the user requests n words and there are no words left,
            then give him nothing and manually awaken him (using
            JOBRET).

HINTS TO BDH IMPLEMENTERS

     The following hints should ease the task of BDH implementers a
little.  It is hoped that anyone who attempts to a BDH implementation
will add his harshly acquired knowledge to this section of this memo.

1.  Remember that the BDH is essentially a disowned job and should
    attempt to log out after a close has been requested.

2.  Be very leery of logging out for any reason other than a requested
    CLOSE.  In particular:

    a.  if a JOBCAL fails, only log out if a request for OPEN has not
        yet been received.
    b.  when you generate an I/O channel error for the user, only log
        out if the error is irrecoverable.  Remember that he can
        correct an access beyond end-of-file by doing an access before
        attempting another IOT.
    c.  if he requests an I/O operation that you do not recognize,
        generate a "mode not available" error (via JOBRET) and wait
        for his next request.
    d.  if you decide, for your own reasons, to make the initial open
        fail, you should log out.

THE OJB DEVICE -- AN AID FOR DEBUGGING THE BDH

     One of the problems with debugging BDH programs is that when the
JOB device is used, it is loaded into a newly created job. That job is
not inferior to any DDT, and there is no way to put breakpoints in it
before it starts. The OJB device makes it possible to run the BDH
program under DDT.

     The OJB device acts just like the JOB device except during the
initial open. When the JOB device would be creating and loading a new
job, the OJB device is looking for an existing job whose UNAME and
JNAME are the same as the filenames specified in the open.  If such a
job is found, it is connected to the job opening the OJB device
through a standard JOB-BOJ pipeline.  If such a job does not exist,
the open of the OJB device fails.  In order to protect innocent jobs
from being hacked in this matter, the job opened on the OJB device is
required to have set its OPTOJB bit (this is bit 4.2 in the .OPTION
varisble, settable with .SUSET).  Also, it must not already be a BDH,
for the system cannot consider one job to be a BDH through two
connections at once.  If either of those conditions is not met, the
OJB open WAITS until they are.

     To remove a possible timing screw, a BOJ device open by a job
that is not a BDH, which usually fails, will wait instead for the job
to become a BDH if the job's OPTOJB bit is set.

     The procedure for using the OJB device for debugging is:

     1.  create a job to use for the BDH (call it J, in this example).
     2.  run (in another job) the program which would normally open
         the JOB device, but tell it (perhaps by means of a translation)
         to use the OJB device instead.  This program's function is to
         issue system calls so that the BDH's responses to them can be
         tested.

         Since the OPTOJB bit of job J is now 0, when this program reaches
         the OJB open it will hang.

     3.  ^Z that program, and ^P it. it will go back to hanging in the
         open of the OJB device.
     4.  switch to job J, load in the BDH program.  This RESET's the
         job so that if it had been a BDH before, it will not be one now.
     5.  turn on the OPTOJB bit by depositing from DDT in .OPTION.

The conditions for a successful OJB open by job JJ have now been met,
so that job will make some headway, turning job J into a BDH and
waiting for job J to issue a JOBRET.

     6.  it is now possible to start running job J, with breakpoints
as desired, to step through the code for handling the initial open.
It is unlikely, but theoretically possible, for J to execute its BOJ
open before JJ gets around to turning J into a BDH.  In this case, J
will wait for JJ to do so.

     7.  if it becomes necessary to try again after discovering a bug,
go back to step 2.  The OJB open will hang up this time not because
the OPTOJB bit is off (since it is still 1) but because J is already a
BDH.  Loading J in step 4 will make J cease to be a BDH but also turn
off the OPTOJB bit, so JJ will still be waiting.

     8.  it is possible for job JJ to be "PCLSR'ed" or backed up out
of its open while job J is being traced through the handling of the
initial open.  For example, job JJ might receive a real-time
interrupt.  If that happens, J will cease to be a BDH.  This will not
interfere with the tracing of J until the next BOJ device system call
is executed; that call will probably fail.  It is impossible to
continue after such an occurrence, so the jobs must be restarted by
returning to step 2.  Of course, if there are no breakpoints before
the first JOBRET, this is very unlikely to happen.

     9.  if the initial open has been handled successfully, to go on
to debug handling of other system calls, simply tell JJ to execute
them with J stopped while JJ is being told, then ^Z^P JJ and go back
to J.

     10. the usual .LOGOUT in the BDH program will be a no-op when the
program is run as an inferior in this manner, so it should be followed
by a .VALUE.