Why 'Enterprise' Development is Hard
I’m an ‘enterprise’ developer, this means I work for a company which spends an obscene amount of money on its hardware and software. For people working outside the ‘enterprise’ world this may seem an interesting question, why exactly is so much money spent on development for these big companies?
Some may think it’s the ‘scale’ of the problem, but most of that is very well understood. Just add a little caching here, a few indexes there and you can meet your ‘non-functional’ requirements. Besides, it’s mostly an operational cost anyway, not really something which adds significant development overhead. Some vendors are one part of the problem, from their perspective, the more complex a solution, the better. Remember, IBM didn’t make $46,213,000,000.00 in consulting revenue by selling software that’s easy to install.
But besides the scale and the bad vendors, what is it that makes enterprise development so expensive?
I’ve come to the realisation that a major source of the pain is the crazy requirements, not only from ‘the business’ but also those IT imposes on itself. Take a nice, simple application like Tadalist. Simple, sharable todo lists. Even if you don’t like the application you can at least realise that it’s a simple but functional appliation. I present:
Tadalist Enterprise Edition
“We want an application to maintain and share todo lists amongst our users”
However, enterprise applications never stop there. You have to consult with every business unit in the company, gather their requirements and design a system which solves them all.
Visiting Finance.
“Financial lists need to be clicked twice in order to be marked ‘complete’, and one user can’t click more than 40 items per day”
Great, instead of a boolean checked attribute, we now have to store a collection of Approvals and the User who approved them. Also, when adding a todo list we need to ask if they’re creating a list about finance. Finance lists need some kind of special behaviour for completion.
Visiting Risk and Compliance
“After you complete a Compliance item it must automatically get copied to the Risk Review list. Only people from Risk & compliance can complete items on the Risk Review list.”
Ok, so instead of having an financeRelated attribute or using polymorphism, we’re now assigning some kind of ListType to our TodoLists and driving the custom behaviour off that. We’re also keeping track of the ‘type’ of user we’re dealing with, perhaps we could use roles or maybe a worksForRisk attribute… Just as we’re about to leave the room, the compliance manager says:
“Regular employees can only finish one compliance item each day.”
Roles it is.
Back to IT
Next we go and speak with the ‘Lead Enterprise Data Custodian’, this guy’s one of us. He’ll have a nice simple set of requirements
“We already have a table in DB2/390 (XY_CRAZYSHIT) for storing the values an ‘event’s ‘status’ can take. We use this table throughout the company, you must use it too. Your values are TODOLIST_NONCMPLET and TODOLIST_NTINCMPLETE.”
Ok, that sucks.
“We have another table on the oracle servers (KJ_USRSVCS) which stores the different roles a User can have, you’ll have to use that”
Ok, two different databases. That sucks even more. This is looking messy, perhaps there is some kind of simple service available to list these roles so we can just plug in to it. Off to see the ‘Principle Enterprise Technologist’
“We don’t have any shared services for accessing the user services”
Bummer, two database connections it is.
“However, applications aren’t allowed to connect to two databases, so you’ll need to build the shared service as part of your project. WS-I compliant SOAP services are our standard.”
OK, so our simple shared todo lists project now includes EAI development, installing and maintaining web services infrastructure.
Conclusion
Our simple, sharable todolists have grown into a hugely complicated system of rules, policies and IT ‘standards’. What was going to be a few months work is now a multi million dollar project, thanks to the business units and IT. That’s where the money goes.
All this leads me to the conclusion that “Enterprise Software” is a polite way of saying shitty legacy systems and overly complex requirements. The resulting systems are horrible to use cost too much to maintain.
Alan May 22nd, 2005 @ 04:14 AM
I am so totally hearing this.
I’m the “business owner” of one of these large apps at my place of work… and I know how hard it is to explain to senior business management that new feature X, widely seen as dead simple (and it is, at least for the end user), is in fact quite complex to implement and will (say)take 12 months elapsed time and a million bucks.
How did we get to this? Sometimes I wonder if IT is a victim of its own success – and that “success” is becoming harder and more expensive. Perhaps it’ll become easier as soon as we have AIs writing the software. And maybe AIs running the business too. :-)
Koz May 22nd, 2005 @ 04:29 AM
I’ll probably wait till a future posting, but I think IT is a pretty big part of the problem here. Always looking for ‘interesting’ problems rather than the best solution
Geert Bevin May 22nd, 2005 @ 09:17 AM
I posted an entry about this here: http://rifers.org/blogs/gbevin/2005/5/21/enterprise_development_requires
Dee Zsombor May 24th, 2005 @ 09:32 AM
Another problem is that communication slowly becomes more and more formalized and hidden behind tool that enforce certain patterns. You slowly end up with permissions, roles and so one when simple communication would suffice.
For the `enterprise` it’s about displaying power. That is why you have so many rules and enforced uniformity. And there is no need for this at all. Even the larger organizations can structure themselves without loosing diversity need for true agility.
Koz May 24th, 2005 @ 07:44 PM
Rev.
All I’m pointing out is why IT projects cost many millions of dollars. To someone who is outside the industry it can seem strange, why can one company spend ~100k on a task tracking system and another have to spend several million.
It is absolutely my job to navigate all these pitfalls and I do it every day.
I’ve edited your comment, if you object to the stuff I removed, let me know and I’ll delete the whole thing.
stephen May 24th, 2005 @ 11:53 PM
There are reasons behind the reasons.
The longevity of the enterprise.
We are always predisposed to extend existing systems, because their behaviour is known; if they are not bug-free, at least we understand their problems. Thus we build up ever-more complex heaps of cruft talking to cruft.
The tadalist is simple because it is a “greenfields” development. Where I am now, nothing, absolutely nothing is greenfields. Everything must take account of the other things.
The compartmentalised nature of projects and funding in the enterprise.
Suppose we can see a way to unify/simplify/extend some enterprise services that will promote reuse, ease maintenance, whatever. In an organisation where capex is distributed to projects and cost centres, I can guarantee that such a project will not be funded, because no single cost centre will agree to bear the cost of work that doesn’t benefit their project. The stovepipe antipattern emerges inevitably.
We can come up with others all day :-)
Bill Seitz May 26th, 2005 @ 06:53 PM
This is like the history of Content Management Systems.
Ian Bicking May 26th, 2005 @ 10:57 PM
All too often people want to resolve social and organization problems in software instead of in the people. Much of the eclectic workflow and permissions are an attempt to prevent problems, even though the problems are usually much, much easier to resolve after they occur compared to the difficulty of maintaining (and working around) an infrastructure that avoids these problems.
Better architecture would emphasize safe and undoable operations, auditing, logging, and transparency. I think this might be generally achievable with a consistent and confident message. Ask questions that lead in this direction, and when people get into restrictive mindsets steer them away and offer up constructive alternatives. Emphasizing results and motivations, and avoiding implementation.
Unfortunately when the message isn’t consistent across the team and all the people interacting with the users, things start to degrade into these old, broken patterns. And then people start saying “that’s what the users want”, and the person suggesting the sensible and conservative transparent implementation has to defend something that’s far more user-friendly against an institution’s disfunctions presented as user requirements.
Gary Beri May 27th, 2005 @ 04:44 AM
As noted by others, this is caused by fragmentation over time. If you didn’t have multiple disparate systems you would be much better off.
Large shops that have maintained a strict mainframe-centric approach have done very well over the years, primarily because all data is in a single place and accessed in a standard manner. They have difficulty attracting younger workers, but no difficulty adding features to their software.
I would advise any company to settle on a hardware platform and a database and to move everything to that platform and database.
Of course that’s a compromise from a security standpoint, but that’s my perspective.
stephen judd May 27th, 2005 @ 09:29 AM
”...an institution’s disfunctions presented as user requirements.”
hollow laugh
That would be my current job. Thank god I’m a contractor.
MrChucho May 27th, 2005 @ 02:48 PM
You forgot to mention all of the useless paperwork: scope documents that are meaningless, requirements documents that are ignored and the endless Excel spreadsheets used for everything from Design Documents to project tracking.
I can’t tell you how many projects are ruined by Project Management. Or how many otherwise interesting projects I’ve passed up because of the red-tape. Why do business people still expect developers to ALSO be Project Managers? We don’t expect them to write code, do we?
Rik May 29th, 2005 @ 03:47 PM
MrChucho wrote: “Why do business people still expect developers to ALSO be Project Managers?”
Because if the job title matches /developer/, you don’t have to pay them as much as if it matches /manager/.
Rik
S Page June 14th, 2005 @ 08:12 PM
What I’ve noticed is that when business owners come in with “eclectic workflow and permissions” for roles, notifications, approvals, etc., it not only takes forever to build the system, but worse they’re rarely happen with it, and often they never adopt the system and they stick to e-mail with Excel attachments.
Whenever business owners ask for such things, the first thing you should do is set them up with a Bugzilla. Bugzilla lets you do a lot of stuff ad hoc using dependencies, see also links, flags, status whiteboard, etc. The bug assignee gives you workflow, the comments system provides a visible audit trail, and the Cc e-mails remind everyone to visit the system.
After a month with Bugzilla end-users will come up with a workable approach to all their workflow requirements using its facilities. They may still need custom development of some core feature of the original project, but you may be able to keep some of the organizational stuff in Bugzilla.
Bugzilla is the Lotus Notes of the ‘00s for ad hoc corporate systems in small to mid-size companies.
Camcrush August 6th, 2005 @ 02:45 PM
All too often people want to resolve social and organization problems in software instead of in the people. Much of the eclectic workflow and permissions are an attempt to prevent problems, even though the problems are usually much, much easier to resolve after they occur compared to the difficulty of maintaining (and working around) an infrastructure that avoids these problems.
Better architecture would emphasize safe and undoable operations, auditing, logging, and transparency. I think this might be generally achievable with a consistent and confident message. Ask questions that lead in this direction, and when people get into restrictive mindsets steer them away and offer up constructive alternatives. Emphasizing results and motivations, and avoiding implementation.
Unfortunately when the message isn’t consistent across the team and all the people interacting with the users, things start to degrade into these old, broken patterns. And then people start saying “that’s what the users want”, and the person suggesting the sensible and conservative transparent implementation has to defend something that’s far more user-friendly against an institution’s disfunctions presented as user requirements.
ann August 31st, 2005 @ 01:54 PM
hi !! i am from poland .. do you know who is Koziarki ?Mieczyslaw… it is Probably your family..his children live in canada..give me reply this miecz.koziarski is my grandpa
ann September 17th, 2005 @ 07:19 AM
SO HI~~!!! I WOULD LIKE TO HAVE CONTACT IF IT POSSIBLE..COULD BE GREAT IF COULD FIND OUT SOMETHING MORE ABOUT YOUR OR MAYBE OUR FAMILY. MAYBE WE ARE RALATIVES..WHO KNOWS ABOUT IT?? cAN U WRITE ME MORE ABOUT YOU AND YOUR FAMILY ?? DO YOU HAVE ANY IN CANADA. I AM 18-YEAR-OLD GIRL FROM poland bybeye
I am looking forward to hearing from u :):)
Best Wishes
ann September 21st, 2005 @ 04:11 PM
huhuhuhu any reply :/
Anonymous May 7th, 2006 @ 03:16 PM
Have found everyting I needed on your site. And already put in my bookmarks :)