This is the 5th excerpt from the second book in the Defen series: BIT: Business Information Technology: Foundations, Infrastructure, and Culture
Note that the section this is taken from, on the evolution of the data processing culture, includes numerous illustrations and note tables omitted here.
COBOL was first codified in 1959; five years later, in 1964, IBM released the matching System 360 computer family and completely revolutionized the systems industry.
The 360 reflected in hardware the fundamental COBOL operations which, themselves, derived from the card sorting requirements of pre-digital data processing. Even then IBM was still hedging its bet on commercial data processing so the 360 was designed to be an "all round" processor which could be configured to excel at either scientific or commercial processing depending on customer needs, while maintaining backward instruction set compatibility with two previous, and mutually incompatible, IBM computer products:
The System 360 was revolutionary in four main ways:
Because the 360 was designed to be compatible with the COBOL transactions model it had I/O capacities that were well in advance of those available on competing gear and out of balance with its memory and CPU capacity. Thus a single 360 could run several tape drives at the same time and work with, or without, a disk pack to act as a fast buffer for data being moved from tape to memory and back again, but could not compete with a four year old CDC machine on floating point precision or performance.
In addition, it was the first system to codify the 8 bit byte and is often said, incorrectly, to have been the first to incorporate a resident operating system - software which manages system resources and makes them available to user applications.
Prior to the development of the resident operating system people had generally worked with the "bare metal" - meaning that each job run had to first identify the resources it would use and then clear them on completion.
| From a design perspective: APL is to Unix as COBOL is to the 360 |
|
One of the many revolutionary ideas that found a home in the 360 design effort was its use of A Programming
Language or APL. Also known as Iverson's Better Math, APL by-passed the need for formula translators like Fortran
by directly employing a mathematical symbology as a computer language.
APL was an enormous technical success, presaging today's object oriented environments by nearly fifty years. Although APL never became a staple in the Unix community, several of the creators of Unix were APL users and APL ideas significantly affected the overall Unix functional design. In particular the Unix focus on fully encapsulated, user accessible, interactive processing elements that can be easily strung together to build complex ad hoc applications reflects their acceptance of fundamental APL ideas. As early as 1974, Toronto's I.P. Sharp Associates ran a (western) world wide on-line network providing time share access to APL-Plus on a 370/168. This service included both email and instant messaging for its users, thereby allowing extensive international collaboration on projects. Dr. Iverson wrote several introductory texts in both Calculus and Algebra aimed at standardizing terminology and proving the case for integrating math education with computer education. Unfortunately APL is simply too difficult and abstract for most data processing professionals to understand, and has been seriously marginalized as a result of their consequent refusal to accept it. |
Because early machines did not have resident operating systems, programmers had to load a basic operating system with each job. JCL, job control language, evolved to meet that need. The System 360 didn't need this, but kept it for backwards compatibility reasons - and it is still used today.
Here is a simple compile job for a batch program. Except for the fact that for a DB2 program there is an additional precompile step, this compile JCL also applies to compiling a DB2 program, including a DB2 Stored Procedure.
//CONZETTC JOB (999,POK),NOTIFY=CONZETT,
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),TIME=1440
//COB EXEC PGM=IGYCRCTL,
// PARM='QUOTE,LIB,OBJECT,XREF,RENT,TEST'
//STEPLIB DD DSNAME=IGY.V2R2M0.SIGYCOMP,DISP=SHR
//SYSLIB DD DSNAME=CONZETT.AJC.COPY,DISP=SHR
//SYSIN DD DSNAME=CONZETT.AJC.COBOL(HELLO),DISP=SHR
//SYSPRINT DD DSNAME=CONZETT.AJC.LISTING(HELLO),DISP=SHR
//SYSLIN DD DSNAME=&&LOADSET,UNIT=SYSDA,
// DISP=(MOD,PASS),SPACE=(TRK,(3,3)),
// DCB=(BLKSIZE=3200)
//SYSUT1 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT2 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT3 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT4 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT5 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT6 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//SYSUT7 DD UNIT=SYSDA,SPACE=(CYL,(1,1))
//LKED EXEC PGM=HEWL,COND=(8,LT,COB),REGION=4096K
//SYSLIB DD DSNAME=CEE.SCEELKED,DISP=SHR
// DD DSNAME=CONZETT.AJC.LOAD,DISP=SHR
// DD DSNAME=SYS1.LINKLIB,DISP=SHR
//SYSPRINT DD SYSOUT=*
//SYSLIN DD DSNAME=&&LOADSET,DISP=(OLD,DELETE)
// DD DDNAME=SYSIN
//SYSLMOD DD DSNAME=CONZETT.AJC.LOAD(HELLO),DISP=SHR
//SYSUT1 DD UNIT=SYSDA,SPACE=(TRK,(10,10))
//LKED.SYSIN DD *
ENTRY HELLO
NAME HELLO(R)/*
(This material is from a February 2002 IBM Redbook by Conzetti et al on using IBM VisualAge Cobol with CICS and DB2 on the then newest z900 mainframes.)
Most computer companies of the day developed tools of their own to help with this, thereby enabling users to avoid the time consuming and frustrating task of building and debugging their own. For example, the CDC 1600, introduced in 1960, would load a standard set of resource definitions at start-up or "initial program load" (IPL) and programmers could thus be assured that a print statement would cause connection to an actual printer just as a READ FROM statement accessed a known physical disk device.
This was "new fangled" and quite a lot of people had invested years in doing things the old way - and thus had boxes of cards embedding initial program load (and eventual unload) instructions that constituted part of their personal competitive advantage. Most computer companies therefore built emulators or translators into their resident operating systems to enable their machines to continue reading and using such card decks even though they were no longer either necessary or useful.
Today straight through processing, or STP, is a significant issue in financial management systems focussed on the stock market where the acronym refers to having all of the accounting entries on a trade done within a single transaction cycle.
In 1960, however, "straight through processing" described single user, single tasking, computer environments that loaded from source, processed straight through to the end, and then released the machine's resources for other jobs. Since it is obviously wasteful to have a million dollar central processor or CPU (called an instruction set processor in the 360 environment) sitting idle waiting for data to come from a disk or tape, there was a lot of pressure from about 1956 onwards to develop ways to share the CPU and memory (called storage in the 360 environment) between multiple concurrent programs.
This is another very difficult problem to which multiple answers are possible. Most of today's computers use a scheduler, a piece of systems software which allocates run-time and memory resources on some basis such as a round-robin tournament or some administratively defined prioritization scheme.
In this type of environment a running program has full access to available memory, CPU cycles, and I/O devices during its time slice -or active period. On a modern machine using this approach time slices are short, usually well under 1/100th of a second, but the CPU cost of doing a context switch is small enough that all programs appear to run continuously.
This is not, however, the only way to achieve the goal of using one computer to run multiple applications at, from the human perspective, the same time.
| Hipervisor? what's that? |
|
You'd expect the etiology of a word like "Hipervisor" to be simple: since supervisors are good, hypervisors must
be better; but that isn't what happened.
The System 360's 24bit addressing imposed a fundamental limitation on memory management and application access to memory. With 24bits, machines could not exceed 16MB. By 1979 RAM costs were reduced to about $100,000 per MB and many database applications needed to access far more. The follow-ons to the System 370 therefore included an eight bit latch, that could be used to extend a 24bit address space to 32bits and thus expand system capacity to 2.1GB and application addressible memory to 512MB. Since 512MB wasn't affordable (or physically possible for a 308X machine) most of that memory was virtual - meaning on disk but managed by the paging system instead of the channel I/O controllers. That made it very fast, so MVS/ESA included the ability to use real or virtual memory outside the addressible range for temporary storage - and this became known as "high performance space" or hiperspace ultimately to be managed by, of course, a high performance supervisor or hipervisor. Ten years later Intel repeated both the latch idea and the hiperspace idea with the i80386, thereby enabling PC-DOS programs, which were limited to 1MB, to use storage above the 1MB line - a solution they called "expanded memory" rather than hiperspaces. |
The 370 and its successors, for example, use memory partitioning and CPU prioritization to achieve something similar. Thus a 370 manager could allocate 16K of memory, 30% of his CPU cycles, and two tape drives to one partition knowing that all programs run in that partition would then get those resources.
The earlier system 360 disk operating system, written mainly in assembler and generated separately for each machine so memory size, peripherals, instruction set, and available tools could be hardwired in, was centered around a supervisor (now called a hipervisor) program which managed memory while acting as a kind of pseudo scheduler within the constraints set by JCL or external resource allocations. It handled interrupts (process changes forced by things like having to wait for I/O) and dynamically allocated cycles between jobs, but managed memory only within preset partitions defined in the JCL used to load and start each job.
The 360 operating system came in three main variations all of which used software based memory partitioning to create what amounted to single user environments for each running job:
The latter eventually became MVS (Multiple Virtual Spaces), then MVS/XA, and is now offered as zOS while the former ultimately migrated part way into microcode (also known as firmware, microcode is not user accessible and usually very efficient because functionally built into the hardware), to become the CP (command processor) layer on top of the PR/SP (Partition and license manager, hardware identification and operations) primary boot layer called at the hardware initial program load, or IPL, stage of system start-up.
| Acronyms |
|
The reason I'm citing all these acronyms is that the use of
these is a component of professionalism among data processing
people.
I once had to program a 3274 controller - a sampling of its documentation suggested that, on average, each of the 200+ pages probably contained 32 acronyms. |
Initially, therefore, memory partitioning was done in software when the JCL controlling each job loaded, but this approach soon proved vulnerable to poor programming practices. On the Assembler side, for example, programmers could address memory outside the blocks assigned to the batch in the JCL and thus affect memory used by other jobs; often causing them to suffer mysterious errors or fail outright. On the COBOL side, COBOL's failure to explicitly clear memory before use meant that an application could read different values from temporary storage than it wrote.
If, for example, a programmer wrote a four character value like "5223" into the rightmost slots of an eight character temporary data store, and then retrieved the value using a named variable, the system would return all eight characters. If, therefore, the previous value had been "11111111," then someone's $52.23 phone bill would get printed as $111,152.23.
To address the first kind of problem IBM strengthened the memory partitioning process, first in the JCL and later by imposing firmware barriers to break available storage into small chunks that could then be allocated within JCL. Since this approach extended the independence of multiple batch jobs running on the same machine, it also offered a way to share the hardware among several users because each partition looked like a real computer to the programmer.
| Other mainframe Operating Systems |
|
IBM itself has two major operating systems for 360 and successor products: the batch oriented MVS variants now called zOS and the VM (virtual machine) variants now marketed as zVM. IBM ported an early version of Unix to the 370 as IX/370 in about 1976 and there are still dozens of sites running Amdahl's mainframe Unix (UTS) where the benefits from continuity and the redundancies built into the System 360 architecture are thought to outweigh the combination of high cost and relatively poor performance. Subsequently IBM built Unix services into MVS/XA and upgraded IX/370 to a significant stand-alone product line on dedicated hardware, renaming it AIX for the purpose. Today most S/390 and later gear can also run Linux either natively or as a VM guest operating system replacing the CMS shell. Systems like MTS (Michigan Terminal System) have OS like characteristics but are actually tasks running as a virtual machine under PR/SP and many older IBM operating systems like DOS/VSE or MVS/SP merely represent different ways of implementing the same basic view of processing. As such they are primarily important, not as technology differentiators, but as software licensing and related cost differentiators. Other companies, notably Sperry and Honeywell in the US, built machines classified as mainframes for marketing reasons but which did not run any of the major IBM operating systems. For example, the Sperry OS/500 machines (later Unisys) were direct descendents of the UNIVAC machines built for Navy logistics processing and allowed the company to retain its Navy (and related municipal) accounts well into the nineties. The Honeywell machine, last sold as the Bull 9000 in the early nineties, mainly ran GCOS (General Comprehensive Operating System). This had originated as GECOS but its subsequent evolution was almost completely taken over by its use at MIT and Bell Labs as the Multics development environment - the project whose collapse led indirectly to Unix. |
This approach to managing the conflicting demands multiple users put on a machine first became known as virtualization and eventually as partitioning. In its virtualization guise it formed the foundation for IBM's attempt to build a multi-user operating system on top of CP and PR/SP that became known as VM; in its partitioning guise it became the foundation for a critical mainframe "best practice" - hardware partitioning.
Hardware partitioning attacked the problem of the million dollar phone bill by enforcing memory partitioning at the hardware microcode (firmware) level. Equally importantly, however, it met the data center manager's need to isolate test and development work from production work without forcing him to buy another multi-million dollar machine.
Today VM, a virtual machine environment usually layered on top of hardware partitioning, is IBM's major interactive user environment for the mainframe. In VM, individual users are given software managed private partitions within a hardware partition or dedicated machine. Thus data center management can enforce process resource utilization priorities and control within the VM user configuration and each user gets access to a virtual mainframe with which he, or she, can do just about anything that could be done with an actual private machine - including loading a "guest" operating system such as VM itself, Linux, or MVS.
Note, however, that Linux is a Unix variant and, as such, fundamentally oriented toward providing multi-user support and enabling many concurrent users to collaborate. Thus using Linux as a single user shell to run "server" applications like DB2 in lieu of running them directly under the VM shell (CMS) doesn't make technical sense. It often makes practical sense, however, for users who don't pay for the hardware, in part because there is far more software available for Linux than for CMS and in part because Linux software tends to be free while comparable mainframe software, if available, tends to be extremely expensive.
It is this virtual machine idea, coupled with the fact that the System 360 had an "instruction set processor" instead of a CPU, that leads to IBM's usage of "engine" to mean the combination of a CPU, some memory, and one or more I/O channels. Thus the 2004 zSeries 990 mainframes (the latest 360s) can have up to 32 "engines" -technically meaning partitionable CPU, memory, and I/O component combinations on the main board.
As a commercial machine the 360 was an enormous success and continued IBM's almost complete dominance of the electro-mechanical data processing market into the automated data processing era.
---
Some notes:
Notice that getting the facts right is particularly important for BIT - and that the length of the thing plus the complexity of the terminology and ideas introduced suggest that any explanatory anecdotes anyone may want to contribute could be valuable.