This is the 24th excerpt from the second book in the Defen series: BIT: Business Information Technology: Foundations, Infrastructure, and Culture
Note that the case study used here was orginally written for Linuxworld.com
Making the case for Unix: Systems Decisions at Cutter Mills
The "I" in this story belongs to a systems consultant brought in to advise the executive committee on the choices they face with respect to Systems operations and an internally generated systems change proposal.
Martin Cutter Mills Inc. of St. Paul, Minnesota, its staff, structure, and plans, are fabrications but the situation presented, and the remedies offered, reflect the author's experience with real-world organizations facing broadly similar situations. Cutter Mills is imaginary, but the conditions, decisions, and outcomes described are broadly based on real events.
This story is primarily about governance; about what happens when senior management abdicates its responsibility for information systems to "technical people." Remember high school? the pretty party people acted as if nerds belonged to a lower social class - and carried a contagious disease that lowered the social standing of those seen with them. When senior management acts like that with respect to Systems, they put the company at risk and make attitudinal change a much bigger issue for an incoming CIO than anything directly related to systems or technology.
Case Background
Martin Cutter Mills has about 1,800 employees and makes and packages milled grain products, predominantly breakfast foods, for both relabeled and own label distribution. Cutter's 5.5% market share makes them the fifth largest producer in a highly concentrated market in which the four biggest companies control 84% of North American production. Plant gate margins are good at about 65%, but sales and administrative expenses eat this up to leave them with only about a 2.7% return on sales of nearly $800 million in fiscal 2000.
Systems use at Cutter started in the seventies with a warehouse management suite built on the IBM System 36 with core manufacturing and order management capabilities added in the OS/400 environment during the late eighties and early nineties. From about 1995 through to mid 1999 substantial effort had been made to rebuild this functionality in a Windows environment but production processing remained on the AS/400. In late 1999 an audit letter led to the decision to stop this development effort and seek a commercial package solution instead.
The project manager for the development effort was given the job
of organizing the software selection and subsequent
implementation. In that role he reports directly to the systems
steering committee and has now asked them to approve an "action
plan". I have been brought in to review this plan and make any
appropriate recommendations for change.
When I look at the plan, four things are immediately apparent:
As an outsider you can never really be sure whether or not a
software vendor is financially at risk. In my opinion, however, a
vendor which exhibits one or more of the following
characteristics:
|
After a day or two of wandering around talking to people it seems that:
I get no actual answers from either the project manager or the nominal Systems Director, but the sales and production people turn over their files and so I get the answers they got. From these it seems that:
All of these are true lies. This vendor doesn't offer an AS/400 product - and may well have been picked for that reason; the other companies seem to have offered very different product packages - and may have been misled on what to propose; the steering committee delays were foreseeable - and are probably being cited to offload the project manager's responsibility for the cost consequences of his earlier decisions.
You would expect the current plan to fit into a wider corporate IT vision; but it does not; there simply is no global plan or vision to be found. Instead, each new action is treated as a project undertaken for, and funded by, a divisional or other operational group; creating a system which is locally satisficing but globally irresponsible. There are many examples of this syndrome's effects, but one, their existing supply chain solution, is particularly illustrative of the dismal consequences this produces.
A supply chain system links a company's ERP system with those maintained by its suppliers and so enables it to cost and schedule orders for which the required resources must be sourced from those suppliers.
Cutter has customers who have supply chain software and so insist that their suppliers, including Cutter, must accommodate it. Cutter, however, does not have a functioning ERP system for them to connect to and so the first customer to make supply chain functionality a requirement for Cutter to continue doing business with them caused a major kerfuffle within Cutter's management and IT ranks.
The solution they arrived at perfectly illustrates IT management's abdication of their corporate responsibilities in favor of local solutions. Since this was seen as a marketing problem they set up dedicated PCs in the marketing offices - one for each customer requesting supply chain access. Each PC was the equipped with either a traditional telecom or network connection and the appropriate EDI software for that customer. These machines automatically issue acceptance confirmations on all volume and ship date requests received and then print the orders.
When marketing gets the paper order, their people work the phones to ensure that it gets produced and shipped on time.
Marketing is quite proud of this solution because it has allowed them to present Cutter Mills as a flexible out-source supplier to various national brand marketing organizations. To Marketing, this looks like an efficient, low cost, solution; but the actual cost is orders of magnitude greater than what they see being spent on a few PCs and daily phone calls for production liaison. Those costs hardly amount to nickels and dimes; the thousand dollar bills are in manufacturing accommodations to cope with the resulting unpredictability in both product volume and product mix.
By simply accepting all such orders without back-checking availability-to-promise against production resources, the company forces itself to keep a larger finished goods inventory as a kind of surge protector and, more subtly, to make its raw products -the ready to package stuff coming off the conveyor belts- as interchangeable as possible.
The retail product name is on the boxes the stuff coming off the conveyor belt goes into. Since you can't store breakfast flakes in bulk (the edges crumble to produce a fine powder no one will buy) the answer is to put the same stuff into all the boxes so that production planning is unaffected by variations in the quantities boxed under each brand name produced. After all, as one sales guy expressed it to me, "corn flakes, is corn flakes"
Unfortunately this isn't true, and attempts to make it so are expensive. For example, the Winnipeg plant makes a multi-grain and raisin mix that's sold under various labels including a house brand for a Canadian chain of grocery super stores and that of a nationally advertised brand in the US. The American company, which presents the product as a premium brand, is mainly concerned about "mouthfeel" and requires all of its suppliers to meet various technical standards designed to ensure that the product meets consumer expectations.
During packaging the product is placed in sealed plastic bags that are then put in pre-printed pressed paper boxes. For the Canadian super stores these are shipped directly to the stores and have an average shelf life there of about three days with most customers buying for weekly replenishment to produce an average box life in the 6 to 8 day range. The American brand, in contrast, is shipped to a wide variety of retailers including many smaller stores that extend the period between production and final consumption to forty days or more.
As soon as the plastic bag is sealed in the packaging process water starts to migrate from the raisins (15.3% water initially) to some of the grain flakes (which average 3.9% water initially) and this, of course, affects crunchiness or "mouthfeel" - thereby limiting shelf life. As a result Cutter employs a variety of production and storage techniques -including using additional humectants (which osmotically transfer water out of, and sugars into, grain flakes), allowing for longer pre-production drying times, and refrigerating finished product storage- to slow this process enough to meet the US customer's standards.
Those actions raise fully allocated production costs by about 4% relative to the minimum needed to meet the Superstore's standards. Since the US product commands a premium this wouldn't be an issue except that the need to position Cutter as an instant response outsourcer forces the plant to make all of the product to the same standard -and so to incur these costs on shipments to the superstores. Since those shipments account for about 75% of the demand for this product, the net result is an average 3% in unnecessary and uncompensated production cost.
Across the company both the numbers and the accommodations made vary. In one area, product wastage is double what it should be, in another a customer responsible for about 30% of production requires that a relatively minor input be purchased from a company it owns - at well above market prices. Storage cost for both raw and finished goods exceed industry averages for almost everything Cutter touches - and turns are several points worse than industry averages too. Guessing the average cost of these adaptations at about 4% of revenue suggests that this this marketing kludge is probably costing the company about $32 million a year -a third more than the bottom line.
The Report
In the meeting called to discuss findings they don't want to hear about governance or controls. Instead they signal their eagerness not to face up to their responsibilities by talking about "where we are at this point in time" and about "moving forward from here." One even talks about "progressing the plan."
I was hired to provide a review of the current plan, not a new IT strategy; but two of the more aggressive vice presidents have talked to me before hand so I'm ready to look surprised - and sketch out their more obvious options. I tell them we'll spend a few minutes looking at where they are and then talk about the options for the future.
Where they are is easy:
The bottom line on this project is that pursuing it will place the company at serious risk of short term operational failure.
As a group, they don't want to talk about reporting and control relationships and most assuredly don't want to know that the problems are natural and predictable responses to their organizational negligence. What they want is a set of action recommendations that get them off the hook; it's not that they didn't already know that this project needs to be cancelled, it's that the executive is deeply fractured along generational lines and they've all been trying to wield Systems as a weapon against the other guy but now push has come to shove and they need some way to bail out of the mess they've made.
What's needed here, I tell them, is exactly what the plan promises but has no obvious intention of delivering: an across the board Systems re-engineering effort aimed at aligning systems operations with business needs. It takes time and effort to develop a real plan to do this, I certainly can't do it based on a few interviews focussed on evaluating the existing project plan. What I can do, however, is talk about what the choices are and what to look for in evaluating them.
To re-engineer Systems they must first decide what kind of systems culture they want. To discuss that issue in any meaningful way we have to compare two alternatives. Since, I tell them, their project manager has picked Windows and I'm a Unix bigot, those are the two cultures we'll compare, but they are not the only choices.
Staying with the AS/400 would provide a nice middle-of-the-road choice from a cost and security perspective. Many of the facilities, particularly the use of smart displays, network inter-operability, and scalability within the limits of this business, are available in both environments; the big differences are in management approach and Systems culture.
In the AS/400 world control has to be heirachial with the relationships between systems people and users mediated by one or more layers of resource managers who control what gets done, when it gets done, and by whom it gets done. As in Windows, the default decision in terms of adding or changing a service is "no". In the Unix world control is distributed, users interact directly with systems staff, decisions to do work are mostly made by the people who do the work, and the default decision in terms of adding or changing a service is "yes".
This company has always been intensely heirachial and the only things wrong with its present AS/400 set-up are that management is distracted, the software is hopelessly obsolete, the current machine is underpowered, and costs are out of control because the people in charge of IT starved the AS/400 operation for resources while focusing on growing the Windows side of their business. But, get a good systems management team in here with the right experience and attitudes; pick something like the J.D. Edwards OneWorld package to run on it, and they should be able to transform the existing systems organization into something that delivers first class results while fitting well with the current corporate culture.
The other choice is to remake Systems into something that drives corporate innovation by lowering costs and dispersing decision making to those most qualified to make them. I believe that this is a better long run bet than extending the AS/400 investment because it will give them a serious competitive advantage relative to their four biggest competitors - one of which has OneWorld on a new iSeries, one of which is in the late stages of being SAP'd, and two of which are said to have MVS based data processing operations that I know nothing about.
One of the things they have to think about in this context, I tell them, is that you don't get competitive advantage or even competitive parity by adopting your competitor's software; you always end up at a relative disadvantage because his people are ahead of yours on the learning and acceptance curve.
"So you're saying", one of them asks, "that we should not go with Edwards?" but I'm not saying that at all. With proper management Edwards would be a lot better than what they have, or the stuff this project manager wants to buy or build. I tell them again that I'm not actually recommending a specific strategy here; I'm pointing out differences so they can see what decisions have to be made and how those decisions fit together.
We'll start, I tell them, by specifying what Cutter needs to achieve and then look at how to get there using each of the two options. After that, we'll look at what each one means in terms of things like cost, productivity, and available software.
Getting people to articulate systems goals is difficult because people tend not to look beyond work processes to the purposes of those processes. In talking about goals people tend to talk about what they do, not why it's done. Here, people mentioned work processes like matching accounts receivable against orders, monitoring grain inventories at plant sites, and confirming the customer's payment record for order acceptance. All of these are important but none are relevant. The key to business process analysis, and so to strategic goal setting, is to focus on what has to be done, not on the processes implemented to do those things.
|
In the meeting I use a whiteboard to put up the key points; here I'll be replicating that using these stand-out boxes. This is the first one and reflects our discussion of what systems is supposed to do for this company.
|
Finally they grudgingly agree: a great systems solution would do two things:
A few people in the room think that this requires supply chain software - but this is a serious mistake. The company functions as part of the supply chain for its bigger customers; these are the guys who want automated ordering and immediate access to availability and delivered cost information. To them this is immensely valuable because Cutter Mills is one of several intermediate producers from which they can source a range of products. It is the supply flexibility they get from this that lets them maximize product turns while minimizing transaction costs and out-of-stock risk.
The right answer for Cutter is to adopt an integrated ERP/Financials system that lets Cutter participate fully in their customer's supply chains without racking up extra production or inventory expenses to do it. With an ERP system in place a customer's order would become part of a production plan before being approved for acceptance. From Cutter's perspective a fully functioning system that meshed with customer supply chain software would cut inventory costs, cut finished goods production costs, and improve revenue predictability.
The abstraction that makes this possible is the treatment of each order as a job-lot to be independently allocated among competing production facilities. This makes it possible to maximize use of the most efficient resources while minimizing overall idle time and avoiding wasteful production or inventory decisions. This would work for Cutter because its size, diversity, and customer relationships allow it to assume that there will always be orders in the pipeline and that these orders, although highly variable individually, will exhibit statistical stability.
This abstraction does not apply to the vast majority, whether measured by dollars or delivery criticality, of Cutter's suppliers. On the contrary, Cutter deals mainly with primary producers who incur high costs when faced with short term change in the type, quantity, or quality of deliveries requested. Since these guys don't have the size and diversity needed for averaging effects across many orders to compensate for change in specific orders, any gains made by Cutter through use of automated supply chain software would come directly at their expense.
That would create a trading imbalance, inviting producers to counter with things like hedging and marketing co-operatives; thereby raising Cutter's input costs. Since these counter-measures have overheads paid to third parties, everybody concerned -except the third parties- would be worse off if Cutter tried to enforce use of automated supply chain logic.
The bottom line on this is that Cutter works at the end of the retailer's supply chain and should not try to add another link. Basically, ERP doesn't make sense at the producer level and so supply chain software to link to it doesn't make sense for Cutter. What Cutter needs is the classic fully integrated ERP stuff and nothing more.
Governance and Control
|
All of the major vendors, I tell them, including SAP, PeopleSoft, and Oracle offer exactly what they need in both Windows based and Unix based versions. These are rapid implementation packages, in the $400-800K range for software and services, that offer little up front customization but deliver company wide working systems quickly and cheaply.
At this point the Finance VP, who has been getting increasingly worried about my presentation, makes the strategic mistake that ties him so firmly to the existing Systems group that he loses any chance of prevailing in next week's executive committee meeting. "We rejected bids from those guys," he says, with both anger and triumph obvious in his voice, "because they were priced way out of our league. Millions, not hundreds of thousands."
He hoped to discredit the entire exercise, but he had been setup by the project manager. What I think happened was that this guy briefed the vendors he didn't want on the importance of the customization work required to fit their products to the stuff he wants to carry forward from his own development effort. I don't know this for sure, vendor complaints don't amount to evidence and proposals not short listed were ; but, the proposal he selected doesn't meet the detailed terms of reference set forth in the RFP and there seem to be no available records on what was said at various vendor briefings.
To answer the VP's charge, I point out that the customization requested in the RFP is not in the winning bid - having been moved off-budget, and in-house. To support that I get them to compare several pages from the RFP to the modules and work listed in the plan. Heads nod, Finance is isolated and knows it; I can see that the Marketing and Production VPs are already planning the motion they'll be putting to the executive. Good, to the extent that I can take sides, I'm with them on this.
Once they agree that they need ERP with integrated financials and HR elements we move to the discussion about getting that in place.
Governance issues top the list, but governance is a topic they don't want to discuss. Going through the CoBiT framework would, I'm sure, make them all extremely uncomfortable to the point of rebellion, but we have to discuss some high level management issues whether they want to or not.
So I give them my Guide to Defenestration book and tell them that Systems isn't any different from any other part of the organization. It has its internal incentives to growth and it has a job to do. The structure provided for it influences how the people hired to work in it behave in pursuit of those goals and it also influences how they are seen by people outside Systems. Right now, Systems reports to Finance and one reason Cutter has had so much trouble implementing even simple new applications is that they're seen as part of Finance; as overhead; as something to be worked around -a necessary but resented imposition that gets in the way of getting the job done.
The new CIO has to report to the CEO and the executive committee - not because the person holding the Finance role can't handle the job, but because perception gets in the way of reality. They need production applications that are enthusiastically accepted by production users. Having Systems report to Finance creates an enormous, and unnecessary, implementation barrier that can be easily demolished, right now; before they try to hire a new CIO. Don't even bother, I tell them, to interview someone who doesn't make that a condition for considering the job.
This gets a hostile, silent, reaction so I move quickly to the most basic meat and potatoes issue: organizational posture. Do they want a rigidly controlled, hierarchical, organization that focuses on managing the resource? or a leadership oriented organization that delegates decision making to the lowest level possible?
This decision is critical to the choice of technology. It is possible, I tell them, to manage Unix in the traditional heirarchical way but the results will include user dissatisfaction and high cost, inefficient operation. That happens, I tell them, because Unix embeds very basic concepts about how systems work and provides for user freedom to do whatever they want to with the system. Using this technology in a tightly controlled way is unfortunately common but really means paying for a lot of people whose job is to stomp out the use of the capabilities that come with the gear.
What Unix calls for is leadership - the process of getting more brains focussed on a job that changes as you do it - rather than management - the process of getting more hands on a well defined job. Posture the systems organization as intensely heirarchical and you will get Windows like systems services whether you use Unix or not. You can, I tell them, drive a 2 inch nail with a sledgehammer if you want to - but it doesn't work the other way. Try to apply a leadership based approach in an all Windows environment and you'll find yourself doing the equivalent of trying to drive railway spikes with an 8 ounce hammer.
It's a difficult concept and acceptance isn't helped by the fact that most places they've seen Unix used have been disasters - usually precisely because management tried to treat it as just another constrained resource to be managed. They don't understand, of course, but I ask them to read Chapter Three of the Guide and a few promise to do it, so I let it go.
That brings us to infrastructure deployment options and costs. For this, we're going to compare the effect a Windows client-server approach with normal heirarchical systems management would have on the company to the effect of going with Unix, smart displays, and a leadership based approach to management.
|
Somewhere in Paul Strassman's book: The Politics of information Management (New Information Economics Press, New Canaan, Conn, 1995) he makes a throwaway comment to the effect that IBM lost the competition with Microsoft in part because they believed that the client-server revolution would require massive servers and so ramped up mainframe production instead of focusing on PCs and developing OS/2. IBM was right of course --and they would have known, having abandoned client-server as unworkable with the Future Systems project of 1967-72. Despite the Windows myth of personal desktop device control, Cutter will have to implement very tight, centralized, control of desktop PCs to make this work - thus leaving users with responsibility for, but no control over, desktop gear. |
They know what PCs are and have some idea that the PC client-server architecture involves PC desktops and racks of PC servers but they don't know what the Unix architecture is. So I tell them there are three big differences:
With this background we start by looking at what the Windows decision would probably mean, including:
The Unix solution, in contrast, would have:
The biggest operational differences are in how people work with the systems, not the systems themselves. Both are based on centralized processing, but with Windows you also centralize control and responsibility, thus creating both opportunities and incentives for people to behave in counter productive ways.
One of the issues here is depersonalization. You hear a lot about people not wanting to work with applications they don't perceive as theirs, and it's true; this does have an enormous negative impact. Underlying it, however, is something more basic yet: by tying user perception of the application to a group, whether that's Systems or Finance, you depersonalize it. That makes it easier for people to rationalize actions that sabotage acceptance.
In a well run Unix environment users work directly with systems staff to make real decisions. Not only does this give users an ownership stake in the application, but it ties their perception of Systems to particular individuals. When you personalize the relationship in this way, you make it harder for people to rationalize actions that sabotage acceptance.
With Unix you centralize processing but try to decentralize control - giving users what they otherwise try to take by stealth. This is another hard issue for them to get their heads around because they think of the PC as distributed - after all the computer is on the user's desktop- and Unix as centralized because the computer is in the data center. In reality this confuses presence with control. It is an objective of Unix administration to decentralize control - to create knowledgeable users who directly control their relationships with Systems. That's one reason I usually recommend that PCs replaced by smart displays be given to their former users as take-home Linux gear -provided they attend a Linux or BSD course first. (The other reason is that having IT people give those introductory Unix courses strengthens both their relationships with users and their knowledge of Unix).
A big part of Windows systems management, I tell them, is the exact opposite. You have to work hard at ensuring that you have control over those desktops and what people do with them. Otherwise the whole network becomes utterly unmanageable; but the result, of course, is that people will fight you for that control and so part of the company's energy gets wasted in the conflict.
Cutter provides a perfect example. You think, I tell them, that you currently have 32 IT positions filled out of 35 FTEs approved - but this doesn't reflect your real staffing costs at all. Richard, who works for Finance in Accounts Payable, is really a full time Windows support person and the only one there who can debug the completely audit-proof spreadsheets he wrote and got other people to use. Brenda, in HR, is another full time Windows support person who has developed some reporting scripts for Microsoft Access that no-one else understands or is allowed near. There's a quality control lab in every plant -and they all have IT people who don't report to Systems. Telecom isn't handled by Systems - and there are part time people looking after that all over the company. All in, I'm guessing Cutter pays for upwards of 60 IT people - almost all of whom spend a big part of their time just fighting each other.
Go all Windows and that situation gets worse. That AS/400 they've got is a solid piece of gear; replace it with applications that run on Windows servers and they can expect another half dozen PC babysitters to emerge, -but it gets worse than that too; people like Brenda will be unable to maintain the support level they provide now because they'll have a complicated PC client to deal with instead of the dumb but simple TN5250 emulation used now. That will inevitably mean that more people will stop doing their real jobs to provide stealthy Windows support instead.
Go Unix and things get a lot better. Richard and Brenda will have to leave or go back to their real jobs -there are no support requirements with smart displays -none, zero, nothing- the new software will obsolete the stuff they're so committed to, and, most importantly, the devolution of control to users will cut the ground out from under them.
Rough Comparisons: Unix vs. Windows at Cutter Mills
|
Projected Costs with ERP | ||
|---|---|---|---|
| With Windows | With Unix | ||
| Impact on Users | Average desktop application crashes per week1 | 120 | 0 |
| Average server failures per week2 | 4 | 0 | |
| Performance (Load 1000 email messages) | >2 minutes | 1-3 seconds | |
| Security (e.g. for employees authorized to access/change quality and health related records) | Login and Password within Advanced Directory; second login/password at client start-up | Smart Card plus login/Password at database level with full audit trail | |
| Security (from email and related attacks) | Entirely reactive; every attack costs something | Entirely pro-active; most attacks irrelevant | |
| Employee Support | Payroll based PC purchase program; remote access is by dial-in with PC-Anywhere | Older PCs given to those who take Linux training provided by IT staff; access is via dedicated VPN | |
| Impact on Costs (in thousands) | Staffing Costs (fully burdened)3 | 2,200 | 1,430 | Approximate cost of ERP/Financials/HR software & uncustomized implementation | 600 | 600 |
| Systems hardware and software costs | 1,3804 | 1,8214 | |
| Month 36 HW/SW replacement | 1,3805 | 2005 | |
| Total Five Year Cost | 13,760 | 9,171 | |
|
1Failure rates for 560 PCs running 8 x 5 are based on the average time between failures (188 hours) for all Microsoft Windows branded operating systems as computed from numbers reported by bugtoaster.com.
2Failure rates for 65 servers running 24 x 7 are based on the average time between failures (2983 hours) claimed by Microsoft for Windows 2000 Server. Bugtoaster appears to focus on desktops, the comparable number for Windows 2000 there is about 240 hours or about 45 failures per week. 3Unix salaries estimated at 65K fully burdened; Windows salaries at 55K. 4Cost information is from the Sun, Worldcom, Microsoft, Dell, Compaq, and Lucent websites as of Nov 18/01. 5Estimates are based on a single HW/SW refresh cycle for the Windows option. Actuals are likely to be higher. See the TCO article earlier in this series for a discussion of this issue. | |||
The big issue here, I tell them, pointing at the bottom lines in both columns, isn't this cost difference. The cost difference is important and typically increases as a percentage of total cost as the overall system increases in size, but cost is not the critical issue. Cutter doesn't make great returns, but a few million more or less on systems isn't going to make or break the company.
Embarking on a suicidal attempt to implement supply chain without considering the nature of your suppliers, and without having an ERP system in place - that can kill your company. The alternatives I've suggested look and sound like they're technology based - but they're really not. They're based on first choosing how you want to posture systems in your company and then picking the technology to make that work.
Choose to position Systems as an asset and you'll need Unix technology and leadership oriented
control. Choose to position Systems as an infrastructure item, something you treat as a cost of
doing business, and you'll need traditional systems management and software like the Edwards
OneWorld package on an Iseries. Don't make a decision and you'll get more of what you've got; uncontrolled
cost growth, escalating failures, and the dominance of personal agendas over corporate responsibility.
Some notes:
Notice that getting the facts right is particularly important for BIT - and that the length of the thing plus the complexity of the terminology and ideas introduced suggest that any explanatory anecdotes anyone may want to contribute could be valuable.