Why Is UML So Hard?

I changed careers back in 1995. I jumped from mechanical engineering to software development. I've worked hard to try and learn what object-oriented programming is all about, what advantages it brings to helping to solve problems in software design and implementation.

First I learned C++, the great new language that Bjarne Stroustrup gave us. I thought that figuring out pointers when I moved from FORTRAN to C was hard; wrapping my brain around objects was much more difficult.

Then Java came along. I took a one-week class, but I didn't really get it.

Then I moved along to a company that wrote client-server accounting software using Visual C++. One day the CTO asked if I was willing to tackle an assignment that required Java. "Oh sure, I know that language," I said. I really had no business taking on that problem, but I muddled my way through it well enough to deliver something.

That company was struggling with the transition from client-server to a distributed, three-tier architecture. They had a long history with the Microsoft platform, but they liked Java's "write once, run anywhere" promise. Their clients were banks and businesses, not all of which ran on Windows. They also wanted to get away from the tight coupling between their user interface and the database tier. They had all their business logic tied up in stored procedures. This meant that they had to support Oracle, DB2, Microsoft SQL Server, Informix, Sybase - any flavor of stored procedure language that a client wished to run. They had a "can do" cowboy attitude that said hacking stored procedure code on site for a new customer was just good business, even if it meant that every installation was a custom. Why let an out-of-synch source code repository stop you from saying "Yes, sir!" to the customer?

The CTO brought in a bunch of folks to try and help them move to a more object-oriented approach. He bought several licenses to the most well-known UML tool of the day. He hired a consulting firm from the Washington DC area to come up and give us a week's intensive training in the use of this UML tool. When the pressures of keeping the production version rolling out the door subsided, he took us all to a hotel conference room, away from the office, and had us spend two weeks locked away with our UML tool, flip charts, and markers. When we were done, we'd have an awe-inspiring object-oriented design for the 21st century accounting system.

As you can guess, the two weeks were a disaster. No object-oriented design came out of those sessions. The company didn't get their distributed accounting system.

What went wrong?

We lacked a strong leader with experience at object-oriented design. We were still learning the tools. Domain knowledge in accounting and experience with the product varied among the participants.

Each session would start with something like "Let's do one use case." We'd draw stuff on flip charts and quickly veer off the road. Every discussion would descend into a dizzying argument that was a roller coaster ride from the heights of abstraction to the stomach-churning drop into implementation details. I was trying to persuade them to list the steps for accounts payable when one old hand smirked and said "I can tell you what accounts payable is! Pay me!", holding out his hand with palm facing up.

The developers would scowl and listen quietly until one of them would stomp out of the room, tossing something like "If you don't make up your mind soon, I'm just going to start coding what I want" over their shoulder as they headed towards the soda machine.

We couldn't agree on what words meant. We'd have bike shed arguments for hours about what "customer" meant. We couldn't agree on how to drive from a meaningful description of the problem at hand to artifacts that a developer could use to produce working code. It's as if we'd get bored or frustrated doing that very hard work and give up before the payoff.

I left the company soon after those sessions ended. There was a layoff within six months. The CTO was forced out in a power struggle with the other two founding partners.

Fast forward eleven years. I'm working for another large company that is struggling with a transition from an older platform to a more modern one. UML has been championed as the cure for what ails us. Licenses to another UML tool have been procured. Training will commence. A large cross-disciplinary team has been convened to go through the UML design process. Consultants have been hired to shepherd us along the path of righteousness.

The funny thing is that it feels just like those sessions I sat through eleven years ago. Every discussion descends into a dizzying argument that's a roller coaster ride from the heights of abstraction to the stomach-churning drop into implementation details. We can't agree on what words mean. We have bike shed arguments for hours about design minutia. We can't agree on how to drive from a meaningful description of the problem at hand to artifacts that a developer can use to produce working code.

We'll see if we get bored or frustrated doing that very hard work and give up before any payoff comes through.

This might be the growing pains of a new team. But what if it's something wrong at the heart of UML? This object-oriented notation for rendering design decisions, codified and maintained by the Object Management Group, was born out of years of notation wars among the Three Amigos - Booch, Jacobsen, and Rumbaugh. They created a notation (UML), a company (Rational), a software tool (Rational Rose), and a software development methodology (Rational Unified Process) before selling out to IBM.

Agile approaches have largely discredited heavy approaches like a full-blown UML design.

Maybe somebody has found that this is a good way to develop large software systems in a team setting, but I haven't seen it yet. Things don't seem to have improved a bit in the past eleven years.

Where Have All The Data Modelers Gone?

I noticed a curious thing at work recently. We have hundreds of instances of Oracle and SQL Server deployed on servers in our data center. Data warehouses that dwarf Home Depot archive many terabytes of information. There's an army of administrators keeping watch, ensuring that no SELECT goes unfulfilled. You can't throw a rock without hitting a Java or .NET developer, well-versed in SQL, chomping at the bit to write an application to deliver that information via the web to any and all clients. Everyone agrees that data are the crown jewels, the lifeblood of any modern business.

If this is true, where have all the data modelers gone?

I see very few people who are comfortable and conversant with concepts like normalization and dimensional modeling. Application developers who possess encyclopedic knowledge of SQL syntax, but lack sound relational design principles and best practices, are given free rein to create whatever tables satisfy the immediate needs of their current project. Extracts are taken, data is duplicated, modifications are made without communicating them back to the source system from whence they came. The number of servers and schemas multiplies. The disk storage required grows exponentially at an ever-increasing rate. Terabytes are maintained, but the unique data might fit on a modest USB key if it could be normalized and extracted. The database administrators don't have any passion for the discipline. Data warehouse designers, ignorant or dismissive of dimensional modeling ideas, create duplicates of source data that are little more than staging areas. There's no cleansing or de-duplication going on. Reports are generated from these staging tables, in spite of better advice.

How did we get here? Where did all that knowledge go?

When the COBOL dinosaurs walked the earth, and mainframes were the kings of business, developers were of one mind. The person who wrote the logic also handled the flat files for persistence and the green screens for the user interface. They were expected to know everything about the system.

Relational databases came along in the 1970s after Ted Codd published his seminal paper on the relational model. There was a land rush to create working implementations of the model. Where there once was a single dominant language for business logic and persistence, COBOL, now there were two: client and SQL. Client/server was all the rage.

Personal computing took processing power out of the data centers, with their raised floors and frigid temperatures, and put it out on desktops. Networks knitted those islands together, first using LANs inside companies and then the Internet over the whole world. Client/server fell out of favor. Fat clients gave way to thin. Two tiers became three; three became four or more. Object-oriented programming took over both the desktop and middle layers, relegating relational databases to the back room. First C++, then Java, and now C# became the brokers between the persistent data and the end users. Object-relational impedance mismatch became the order of the day.

I don't know if increased specialization and layering has put relational practitioners in a funk, but it looks to me like true expertise in this area is dying. I've been fortunate to work with someone lately who is both a well-read devotee of Ralph Kimball and experienced at standing up several successful data warehouses that use dimensional techniques. But he's a singleton; I don't know of anyone else in the enterprise who speaks the same language. Tools for entity-relationship modeling (e.g. ERWin) are buried in the DBA's toolbox, hidden by scripting and client tools (e.g. TOAD), like a specialized wrench reserved for out-of-the-ordinary tasks.

Contrast this with the state of the middle tier. There's a cacophony of voices whenever someone proposes an object design. Good UML tools, both commercial and free, help to make even the gnarliest whiteboard session intelligible and clean.

It has dawned on me that data are still the crown jewels of every business. Client and middle tier technologies continue to evolve rapidly, coming and going at a rapid pace. But the data lives on forever.

Data warehousing and OLAP were huge in the late 90s. Kimball and Inmon slugged it out for dominance. Articles and books were written; consultancies prospered; businesses gobbled up advice and software as quickly as their pocketbooks would allow.

Has the animal spirit gone out of this field? The furor has died down and moved elsewhere, but the need to manage information and deliver it in a timely way to customers is still with us. I don't know if the problem is "solved" so completely that it's become routine everywhere but at my current employer.

But from where I sit today, knowing relational databases and dimensional modeling very well looks like a good bet if you want to have a rare and valuable skill.