A New Old Idea

This blog entry is different from all the others I've posted to date. Whenever I've uploaded a photo, I've gotten it by searching Flickr.com Creative Commons for something that fit the theme of the post.

I took the picture that accompanies today's entry. I wanted to try something new.

Writer's block is a problem for me. I've had trouble for a while with finding my voice here. I love to write, but I've had a hard time making up my mind what I should focus on. A purely technical blog, like Jeff Atwood's Coding Horror, would be worthwhile, but I have a hard time filtering out more personal and non-technical thoughts. I feel a little exposed putting too much personal information out on the Internet. The frequency of posts shows my problem: when I have long gaps, I'm having trouble coming up with a topic.

The funny thing is that inspiration is all around us. I see all kinds of small details that are interesting to me. But they're often forgotten in the bustle of getting through busy weeks.

I used to keep an electronic journal. I have entries dating back to 1994 that comprise a special personal history. Sadly, it's fallen into decay. I don't have the same inspiration for it.

At one time I thought that learning to draw from Betty Edward's wonderful "Drawing On The Right Side Of The Brain" would be my inspiration. I did the exercises faithfully. I would draw random things while sitting in meetings, just to hone my skills, my eye, and the shift to right-brain mode. I loved it - until I hit the chapter on portrait drawing. My left brain was too critical. I couldn't find a way to quiet it. I would still love to find a way over the barrier, but to date I've been unsuccessful.

I have learned how to adopt ideas from people who are smarter than me. I have no problem emulating and following when I see someone doing something that I admire.

I went to Boston this past weekend to see the just-completed renovation job on my beloved second sister's house (it's spectacular). My beloved oldest sister was taking photos using a Fuji digital camera that had a big view finder, took great photos, and fit into a shirt pocket. I've never been a photographer. I've tried to concentrate on experiencing the moment rather than preserving it. But watching her snap away unobtrusively made me think "I could do that, too."

So I went out the other night and picked up a Fuji Finepix J38 - same model as hers; same color, black. Did I say "no problem emulating"? That meant "slavishly copying." I grabbed a case sturdy enough to protect the view finder and still slide into a pocket without looking too bulky. ("Is that a Fuju Finepix J38 in your pocket or are you just glad to see me?")

My idea is that I'll try to have the camera on hand as much as possible. If I see something interesting, I'll snap it, upload it later, and perhaps write about it here. I'm not going to worry so much about what comes out; I'm going to just keep practicing and writing.

I failed at drawing; now this jewel of technology will be my cache for ideas. It's got to piss off every truly skilled photographer who has spent a lifetime mastering film, technology, light, developing techniques, and the all-important artist's eye. Digital cameras have become so cheap and so good that any fool can trick themselves into thinking that they're Ansel Adams.

I tried it this morning when I went to work. The light and the view when I came out of the pool before work was enticing, so I unselfconsciously took out my camera, stood on the sidewalk, and snapped away. I took a few on the way out as well. I liked this picture the best of all.

I'm going to emulate my beloved, beautiful, brilliant eldest daughter, too. She's been writing a blog on street art for almost a year. The amazing thing about it, besides the content, is that she posts an entry every single weekday, without fail.

How does she do it? By treating it like a job. She lines up her sources, writes the pieces, and queues them up for daily release. Her dedication, discipline, and work ethic are as impressive as the material she elicits from all over the world.

I'd like to try that, too. I need to be more focused on what I'm pumping out there. I shouldn't have only one or two posts per month. I can do better.

What does all this mean? Probably nothing. I'm just another guy on the Internet, taking pictures and blabbing about himself, putting it out because Blogger makes it easy, thinking that it's terribly interesting and world-changing. How self-indulgent and boring! Right?

I'll concede that point.

I like the mental stimulation of trying to do something that I've never done. I want to keep thinking and challenging myself. I prefer this to coming home and settling in front of a television every night. I don't bloody well care if anybody notices. I'm doing this solely for myself, for its own sake.

Binary Tree Iterators

In my last post, I presented code (and unit tests) for a binary tree implementation in Java. I alluded to the fact that there was a depth-first iterator in one of the test classes. This time I'll discuss iterators and present more practice code.

Iterator is one of the behavioral patterns described by the Gang of Four in their classic "Design Patterns". It provides the means for walking through a data structure without having to expose the details of how it's done.

Timothy Budd provides an excellent explanation of binary trees and iterators in chapter 10 of his "Classic Data Structures In C++".

Binary trees are interesting data structures. If the n nodes in a binary tree are independent, then there are n! = n*(n-1)*...*2*1 different orderings by which one could visit every node.

But the fact that the tree is arranged so that each parent has, at most, a left and right child limits the number of choices for walking the tree to six:

1. Process value, then left child, then right child
2. Process left child, then value, then right child
3. Process left child, then right child, then value
4. Process value, then right child, then left child
5. Process right child, then value, then left child
6. Process right child, then left child, then value

Subtrees are usually traversed from left to right, so the first three possibilities are the most common. Each is given a name that may be more familiar and memorable. The first is called preorder or depth-first traversal; the second in-order or symmetric traversal; and the third post-order traversal. There is also level order or breadth-first traversal, where all the nodes at one level are visited before proceeding to the next.

Java has an Iterator interface in its java.util package that defines the methods that all classes that implement it must provide. The next() returns a generic value. For my binary tree iterator I knew there'd be times when I wanted to get the next Node instead of the value, so I extended the Iterator interface and added a next method that returned a Node<T>:

package tree;

import java.util.Iterator;

public interface BinaryTreeIterator<T extends Comparable<T>> extends Iterator<T>
{
Node<T> nextNode();
}

Then I created an abstract class that provided default behavior for all methods except the ones that provided the next Node value and whether or not the walk through the tree was complete:

package tree;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import java.util.Iterator;

public abstract class AbstractBinaryTreeIterator<T extends Comparable<T>> implements BinaryTreeIterator<T>
{
public static final Log LOGGER = LogFactory.getLog(AbstractBinaryTreeIterator.class);

public void remove()
{
throw new UnsupportedOperationException("cannot remove from a binary tree");
}

public T next()
{
Node<T> nextNode = nextNode();

return nextNode.getValue();
}
}

The post-order traversal examines the left child first, then the right child, then the value in a recursive manner. It uses a last in, first out (LIFO) stack to hold the nodes in the opposite order they are to be visited. The iteration starts by pushing the root node onto the stack and recursing down the tree:

package tree;

import java.util.NoSuchElementException;
import java.util.Stack;

public class PostOrderIterator<T extends Comparable<T>> extends AbstractBinaryTreeIterator<T>
{
protected Stack<Node<T>> stack = new Stack<Node<T>>();

public PostOrderIterator(BinaryTree<T> tree)
{
init(tree.getRoot());
}

public void init(Node<T> root)
{
if (root != null)
{
stack.clear();
stackChildren(root);
}
}

private void stackChildren(Node<T> node)
{
stack.push(node);
Node<T> next = node.getRight();
if (next != null)
{
stackChildren(next);
}
next = node.getLeft();
if (next != null)
{
stackChildren(next);
}
}

public Node<T> nextNode()
{
if (!hasNext())
{
throw new NoSuchElementException();
}

Node<T> x = null;

if (!stack.empty())
{
if (LOGGER.isDebugEnabled())
{
LOGGER.debug(stack);
}

x = stack.pop();
}

return x;
}

public boolean hasNext()
{
return !stack.isEmpty();
}
}

Of course there are unit tests:

package tree;

import org.springframework.test.context.ContextConfiguration;
import static org.testng.Assert.assertFalse;
import static org.testng.AssertJUnit.assertEquals;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.NoSuchElementException;

@Test
@ContextConfiguration(locations = "classpath:app-context.xml, classpath:app-context-test.xml")
public class BinaryTreeIteratorTest
{
public void testHasNextEmptyTree()
{
BinaryTree<String> empty = new BinaryTree<String>();
AbstractBinaryTreeIterator<String> iterator = new DepthFirstIterator<String>(empty);

assertFalse(iterator.hasNext());
}

@Test(expectedExceptions = UnsupportedOperationException.class)
public void testRemove()
{
BinaryTree<String> empty = new BinaryTree<String>();
AbstractBinaryTreeIterator<String> iterator = new DepthFirstIterator<String>(empty);
iterator.remove();
}

public void testNextDepthFirst()
{
Integer[] data = {5, 3, 9, 1, 4, 6,};
BinaryTree<Integer> tree = new BinaryTree<Integer>();
tree.insert(data);

String expected = "5,3,1,4,9,6,";
AbstractBinaryTreeIterator<Integer> iterator = new DepthFirstIterator<Integer>(tree);
String actual = createCommaSeparatedString(iterator);

assertEquals(actual.toString(), expected);
}

public void testNextPostOrder()
{
Integer[] data = {5, 3, 9, 1, 4, 6,};
BinaryTree<Integer> tree = new BinaryTree<Integer>();
tree.insert(data);

String expected = "1,4,3,6,9,5,";
AbstractBinaryTreeIterator<Integer> iterator = new PostOrderIterator<Integer>(tree);
String actual = createCommaSeparatedString(iterator);

assertEquals(actual.toString(), expected);
}

private static String createCommaSeparatedString(Iterator iterator)
{
StringBuffer actual = new StringBuffer(1024);
while (iterator.hasNext())
{
actual.append(iterator.next()).append(',');
}

return actual.toString();
}

@DataProvider(name = "emptyTreeIterators")
public Object[][] createEmptyTreeIterators()
{
BinaryTree<String> tree = new BinaryTree<String>();

return new Object[][]{
{new BreadthFirstIterator<String>(tree)},
{new DepthFirstIterator<String>(tree)},
{new InOrderIterator<String>(tree)},
{new PostOrderIterator<String>(tree)},
};
}


@Test(expectedExceptions = NoSuchElementException.class, dataProvider = "emptyTreeIterators")
public void testNextEmptyTree(AbstractBinaryTreeIterator<String> iterator)
{
iterator.next();
}

public void testNext()
{
String[] data = {"F", "B", "A", "D", "C", "E", "G", "I", "H",};
BinaryTree<String> tree = new BinaryTree<String>();
tree.insert(data);

Map<String, String> expected = new HashMap<String, String>();
expected.put("depth-first", "F,B,A,D,C,E,G,I,H,");
expected.put("in-order", "A,B,C,D,E,F,G,H,I,");
expected.put("post-order", "A,C,E,D,B,H,I,G,F,");
expected.put("breadth-first", "F,B,G,A,D,I,C,E,H,");

String name = "depth-first";
AbstractBinaryTreeIterator<String> iterator = new DepthFirstIterator<String>(tree);
AbstractBinaryTreeIterator.LOGGER.debug(name);
assertEquals(name, expected.get(name), createCommaSeparatedString(iterator));

name = "in-order";
iterator = new InOrderIterator<String>(tree);
AbstractBinaryTreeIterator.LOGGER.debug(name);
assertEquals(name, expected.get(name), createCommaSeparatedString(iterator));

name = "post-order";
iterator = new PostOrderIterator<String>(tree);
AbstractBinaryTreeIterator.LOGGER.debug(name);
assertEquals(name, expected.get(name), createCommaSeparatedString(iterator));

name = "breadth-first";
iterator = new BreadthFirstIterator<String>(tree);
AbstractBinaryTreeIterator.LOGGER.debug(name);
assertEquals(name, expected.get(name), createCommaSeparatedString(iterator));
}
}

Tests are great, but being a visual person I like to be able to see a picture. There's a terrific package called graphviz from AT&T that lets you generate graph plots in an elegant way. I wrote a quick program to walk the binary tree { F,B,A,D,C,E,G,I,H } using an iterator and spit out the dot representation, like so:

digraph simple_hierarchy {
F->B [label="L"]
F->G [label="R"]
B->A [label="L"]
B->D [label="R"]
D->C [label="L"]
D->E [label="R"]
G->I [label="R"]
I->H [label="L"]
}

I can use this as the input to dot.exe to render the tree as .png or .svg file:



I'll post the other iterators next time.

All this work just to answer a single interview question! This tells me that doing it justice would be hard in such a short period of time. I would flunk such an interview.

Thanks to my friend Steve Roach, who was the first person to bring graphviz (and so many other things) to my attention. He's one of the most talented developers I've had the pleasure of working with, but it's his teaching ability that's his greatest strength. He's one of those people who makes everyone around him better. Such intellectual generosity is rare indeed.

Practice, Practice, Practice

I spent several hours last week interviewing Java developers. We needed to bring in some contractors. It was decided that we'd ask candidates to write some Java code as part of the interview process. We had a list of twenty quiz questions that ranged in difficulty. Since our time was limited, we'd restrict ourselves to one or two quiz questions per candidate.

I had mixed feelings about the quiz questions. I'm familiar with Joel Spolsky's Guerrilla Guide to Interviewing. I didn't want to be a Quiz Show Interviewer.

But there was something else nagging me. How would I fare if presented with this quiz?

One of the questions was: "Traverse a binary tree in depth-first order." I'd bet that anybody fresh out of a good data structures class would be able to whip this out quickly enough to be able to fit into the scope of a 30 minute phone conversation.

But what about a guy like me?

The only data structure I was taught as a mechanical engineer was a FORTRAN array. I was keenly aware of my ignorance when I started down the software path, so I did what I always do: sign up for a degree and start taking courses. I started down the path of an Master of Science degree in computer science. It included basics like data structures, but they taught it using Eiffel. And that was ten years ago. I'd have to dredge up those memories and express them in Java.

Even worse, I've been working as an architect for years now. The big firms that employ me tend to take a dim view of architects that write code. Programming is considered a low level, commodity, low skill task that's best left to the least expensive off-shore individuals you can find.

It made me nervous to think about how foolish I'd look taking my own interview.

So I started working through the interview questions, one by one. I spent some time this weekend on that binary tree implementation. I got through it using all the best practices I knew: Java, test driven development using TestNG, IntelliJ and refactoring, etc. It took more than thirty minutes to finish, but I'm pleased with the result. I am not fast, but I think I'm conscientious and thorough.

Practice will help with my speed. Part of the value of this exercise is to practice, practice, practice. Individuals that I have a great deal of respect for advocate the concept of constant practice, calling it code kata.

I think it's especially important for someone with "architect" in their job title. How on earth can you represent for "best practices" when you're so woefully out of practice yourself?

The other benefit? It's fun and satisfying. There's still a great sense of satisfaction, of an aesthetic for mathematical beauty, whenever I manage to pull myself through whatever rabbit hole I've fallen into. There's frustration, too, when I struggle and fall and fail. But when it works, there's nothing like it.

It's no different from any other profession. We all have to keep learning, struggling, adding new skills, re-sharpening old ones.

If you're not a programmer, you can stop reading here. (Thank you for coming at all and getting this far.)

If you are a programmer, and you're still interested, here's my solution to the first part of the problem: a binary tree in Java. I started with a Node class that encapsulated a value plus left and right child references:

package tree;

public class Node<T extends Comparable<T>>
implements Comparable
{
private T value;
private Node<T> left;
private Node<T> right;

public Node(T value)
{
this(value, null, null);
}

public Node(T value, Node<T> left, 
Node<T> right)
{
this.setValue(value);
this.left = left;
this.right = right;
}

public T getValue()
{
return value;
}

public void setValue(T value)
{
if (value == null)
throw new IllegalArgumentException("node value 
cannot be null");

this.value = value;
}

public Node<T> getLeft()
{
return left;
}

public void setLeft(Node<T> left)
{
this.left = left;
}

public Node<T> getRight()
{
return right;
}

public void setRight(Node<T> right)
{
this.right = right;
}

public boolean isLeaf()
{
return ((this.left == null) && 
(this.right == null));
}

public int compareTo(Object o)
{
Node<T> other = (Node<T>) o;

return this.getValue().compareTo(other.getValue());
}

@Override
public boolean equals(Object o)
{
if (this == o)
{
return true;
}
if (o == null || getClass() != o.getClass())
{
return false;
}

Node node = (Node) o;

if (!value.equals(node.value))
{
return false;
}

return true;
}

@Override
public int hashCode()
{
return value.hashCode();
}

@Override
public String toString()
{
return value.toString();
}
}

Of course I wrote a TestNG class to unit test it:

package tree;

import static org.testng.Assert.*;
import org.testng.annotations.Test;

@Test
public class NodeTest
{
/**
* For any non-null reference value x, x.equals(null) 
* must return false.
*/
public void testNotNull()
{
Node<String> x 
= new Node<String>("test");

assertFalse(x.equals(null));
}

/**
* It is reflexive: For any reference value x, 
* x.equals(x) must return true.
*/
public void testReflexive()
{
Node<String> x 
= new Node<String>("test");

assertTrue(x.equals(x));
assertEquals(0, x.compareTo(x));
}

/**
* It is symmetric: For any reference values x and y, 
* x.equals(y) must return
* true if and only if y.equals(x) returns true.
*/
public void testSymmetric()
{
Node<String> x 
= new Node<String>("test");
Node<String> y 
= new Node<String>("test");
Node<String> z 
= new Node<String>("something else");

assertTrue(x.equals(y) && y.equals(x));
assertTrue((x.compareTo(y) == 0) 
&& (y.compareTo(x) == 0));
assertFalse(x.equals(z));
assertTrue(x.compareTo(z) > 0);
}

/**
* It is transitive: For any reference values x, y, 
* and z, if x.equals(y) returns
* true and y.equals(z) returns true, 
* then x.equals(z) must return true
*/
public void testTransitive()
{
Node<String> x 
= new Node<String>("test");
Node<String> y 
= new Node<String>("test");
Node<String> z 
= new Node<String>("test");

assertTrue(x.equals(y) && 
y.equals(z) && 
z.equals(x));
assertTrue((x.compareTo(y) == 0) && 
(y.compareTo(z) == 0) && 
(z.compareTo(x) == 0));
}

public void testHashCode()
{
Node<String> x 
= new Node<String>("test");
Node<String> y 
= new Node<String>("test");
Node<String> z 
= new Node<String>("something else");

assertTrue(x.hashCode() == y.hashCode());
assertFalse(x.hashCode() == z.hashCode());
}

public void testToString()
{
String expected = "expected";

Node<String> node 
= new Node<String>(expected);
assertEquals(node.toString(), expected);
}

@Test(expectedExceptions = NullPointerException.class)
public void testCompareToNull()
{
Node<String> x 
= new Node<String>("test");

x.compareTo(null);
}

public void testCompareTo()
{
Node<String> x 
= new Node<String>("x");
Node<String> y 
= new Node<String>("y");
Node<String> z 
= new Node<String>("z");

assertTrue((x.compareTo(x) == 0) && 
(x.compareTo(y) <  0) && 
(x.compareTo(z) <  0));
assertTrue((y.compareTo(x) >  0) && 
(y.compareTo(y) == 0) && 
(y.compareTo(z) <  0));
assertTrue((z.compareTo(x) >  0) && 
(z.compareTo(y) >  0) && 
(z.compareTo(z) == 0));
}

@Test(expectedExceptions = IllegalArgumentException.class)
public void testNullValue()
{
Node<String> x   
= new Node<String>(null);
}

public void testIsLeaf()
{
Node<String> x 
= new Node<String>("test");

assertTrue(x.isLeaf());

x.setLeft(new Node<String>("left"));
x.setRight(new Node<String>("right"));

assertFalse(x.isLeaf());

assertEquals("left", x.getLeft().getValue());
assertEquals("right", x.getRight().getValue());
}
}

Then I wrote a BinaryTree:

package tree;

import java.util.Arrays;
import java.util.List;

public class BinaryTree<T 
extends Comparable<T>>
{
private Node<T> root;

public BinaryTree()
{
this(null);
}

public BinaryTree(Node<T> root)
{
this.root = root;
}

public Node<T> getRoot()
{
return this.root;
}

public boolean contains(T value)
{
return contains(this.root, value);
}

private boolean contains(Node<T> node, T value)
{
// empty tree can't contain the value; value 
// cannot be null
if ((node == null) || (value == null))
{
return false;
}

if (value.equals(node.getValue()))
{
return true;
}
else if (value.compareTo(node.getValue()) < 0)
{
return contains(node.getLeft(), value);
}
else
{
return contains(node.getRight(), value);
}
}

public void insert(T value)
{
this.root = insert(this.root, value);
}

public void insert(List<T> values)
{
if ((values != null) && 
(values.size() > 0))
{
for (T value : values)
{
insert(value);     
}
}
}

public void insert(T [] values)
{
if ((values != null) && 
(values.length > 0))
{
insert(Arrays.asList(values));
}
}
private Node<T> insert(Node<T> node, 
T value)
{
if (node == null)
{
return new Node<T>(value);
}
else
{
if (value.compareTo(node.getValue()) < 0)
{
node.setLeft(insert(node.getLeft(), value));
}
else
{
node.setRight(insert(node.getRight(), value));           
}
}

return node;
}

public int size()
{
return size(root);
}

private int size(Node<T> node)
{
if (node == null)
{
return 0;
}
else
{
return (size(node.getLeft()) + 
1 + 
size(node.getRight()));
}
}

public int height()
{
return height(root);
}

private int height(Node<T> node)
{
if (node == null)
{
return 0;
}
else
{
int leftHeight = height(node.getLeft());
int rightHeight = height(node.getRight());

return (Math.max(leftHeight, rightHeight) + 1);
}
}

public boolean isEmpty()
{
return (root == null);
}
}

And a TestNG unit test:

package tree;

import static org.testng.Assert.assertFalse;
import static org.testng.Assert.assertTrue;
import static org.testng.AssertJUnit.assertEquals;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

import java.util.Arrays;
import java.util.List;

@Test
public class BinaryTreeTest
{
private BinaryTree<Integer> tree;
private Integer [] data = { 5, 3, 9, 1, 4, 6, };

@BeforeTest
public void setUp()
{
tree = new BinaryTree<Integer>();
tree.insert(data);
}

public void testIsEmpty()
{
assertFalse(tree.isEmpty());
assertTrue(new BinaryTree<Integer>().isEmpty());
}

public void testSize()
{
assertEquals(tree.size(), data.length);
}

public void testHeight()
{
int expected = 3;
assertEquals(tree.height(), expected);
}

public void testGetRoot()
{
Node<Integer> expected 
= new Node<Integer>(data[0]);
assertEquals(tree.getRoot(), expected);
}

public void testContains()
{
for (int value : data)
{
assertTrue(tree.contains(value));
assertFalse(tree.contains(value*1000));
}
}

public void testInsertList()
{
// When you insert a list, you get it back without 
// alteration using a pre-order, depth-first traversal.
List<String> data 
= Arrays.asList("F","B","A","D","C","E","G","I","H");
BinaryTree<String> tree 
= new BinaryTree<String>();
tree.insert(data);

// Check the size
assertEquals(tree.size(), data.size());

// Now check the values
BinaryTreeIterator<String> iterator 
= new DepthFirstIterator<String>(tree);
int i = 0;
while (iterator.hasNext())
{
assertEquals("i = " + i, iterator.next(), 
data.get(i++));
}
}

public void testInsertArray()
{
// When you insert a list, you get it back without 
// alteration using a pre-order, depth-first traversal.
String [] data = {"F","B","A","D","C","E","G","I","H",};
BinaryTree<String> tree 
= new BinaryTree<String>();
tree.insert(data);

assertEquals(tree.size(), data.length);     

BinaryTreeIterator<String> iterator 
= new DepthFirstIterator<String>(tree);
int i = 0;
while (iterator.hasNext())
{
assertEquals("i = " + i, iterator.next(), 
data[i++]);     
}
}

public void testInsertNullList()
{
List<String> data = null;
BinaryTree<String> tree 
= new BinaryTree<String>();
tree.insert(data);

assertEquals(tree.size(), 0);
}

public void testInsertNullArray()
{
String [] data = null;
BinaryTree<String> tree 
= new BinaryTree<String>();
tree.insert(data);

assertEquals(tree.size(), 0);
}
}

This was the ground work. The real solution meant writing iterators to traverse the BinaryTree. If you're reading closely, you'll see that I used a DepthFirstIterator in the unit test for BinaryTree.

I'll post those next time. In the meantime, I'll keep practicing.

Frightening Update

I recently wrote about how frightening the financial situation of the United States was. How could it be that an entity as complex as the United States economy could employ cash based accounting, the same method used by hot dog stands?

After posting that blog I had breakfast with one of my oldest and best friends. I met him early in my engineering career. We branched in graduate school after both of us completed Masters degrees. I went on with mechanical engineering, while he pursued an MBA. We branched again when I abandoned engineering and ran down the software track.

We've managed to stay in contact in spite of the fact that we no longer work together. We'd meet for lunch until the noon hour stopped being considered a sacred "meeting free" time. Then we switched to monthly breakfasts before work. These are coffee-fueled discussions about all our favorite topics - engineering, politics, religion, movies, sports. I can't think of many people in my life with whom I could spend an hour this way. I look forward to them like nothing else.

My friend is a dyed-in-the-wool leftie who's against all things Republican. I tend to be left of center as well, but it's easier for me to slip into "a pox on both your houses" mode when I think that Democrats aren't living up to their ideals. It's a position that's easy to assume these days. Except for the change in tone I don't see much difference between the current administration and the Republican regimes of the last eight years, with two wars in progress and being funded by "off budget" expenditures, the Treasury taken over by Goldman Sachs, the Patriot Act in place, Glass-Steagall rolled back, and Guantanamo in operation.

During our last breakfast I relayed the essence of my distress: Why doesn't the US government use GAAP? I thought myself very clever when I said that cash based accounting made us no better than a hot dog vendor.

My friend drew on the wellspring of knowledge he accumulated during his MBA and shot me down: "Cash-based accounting is the most honest form there is. You look into the till and report on how much cash you see!" He pointed out that GAAP is loaded with loopholes and tricks. Depreciation makes all kinds of shenanigans possible.

My argument was shot down. I was an easy target, because I've never studied accounting.

I'd appreciate it if someone could explain to me where I went wrong. I realize that it won't be possible to relate or absorb such a vast subject in the space of a web comment, but I could use a nudge in the right direction.

Perhaps the real answer is that no accounting system is a guarantee of honesty and transparency. Scandals too numerous to list, such as Baan, Enron, etc., have taught us that outright lying can be done using any accounting system. GAAP won't stop you from booking loans as income, or moving sales back from one quarter to another to make a bad quarter look better, or shipping product to a warehouse you own and treating them as sales, or ignoring an IOU to Social Security and Medicare because you spent the cash accumulated within back in the Vietnam days.

The real problem isn't the way the Congressional Budget Office is tallying the numbers. It's the way we're misled about what the numbers are in the first place.

One reason for the housing boom is people taking on debts beyond their means, assuming that the value of the house would always appreciate at a rate that would make it possible to flip the house at a profit before paying for it became a problem.

I think our current fiscal situation is based on an equally flawed assumption: that the United States will always be the world's pre-eminent economic power, that the rest of the world will never catch up, that our economy will always continue to grow fast enough to make it possible to pay off the obligations we're piling up.

I keep waiting for the Obama administration to start telling us the truth, but it's like waiting for Godot.

Why Is UML So Hard?

I changed careers back in 1995. I jumped from mechanical engineering to software development. I've worked hard to try and learn what object-oriented programming is all about, what advantages it brings to helping to solve problems in software design and implementation.

First I learned C++, the great new language that Bjarne Stroustrup gave us. I thought that figuring out pointers when I moved from FORTRAN to C was hard; wrapping my brain around objects was much more difficult.

Then Java came along. I took a one-week class, but I didn't really get it.

Then I moved along to a company that wrote client-server accounting software using Visual C++. One day the CTO asked if I was willing to tackle an assignment that required Java. "Oh sure, I know that language," I said. I really had no business taking on that problem, but I muddled my way through it well enough to deliver something.

That company was struggling with the transition from client-server to a distributed, three-tier architecture. They had a long history with the Microsoft platform, but they liked Java's "write once, run anywhere" promise. Their clients were banks and businesses, not all of which ran on Windows. They also wanted to get away from the tight coupling between their user interface and the database tier. They had all their business logic tied up in stored procedures. This meant that they had to support Oracle, DB2, Microsoft SQL Server, Informix, Sybase - any flavor of stored procedure language that a client wished to run. They had a "can do" cowboy attitude that said hacking stored procedure code on site for a new customer was just good business, even if it meant that every installation was a custom. Why let an out-of-synch source code repository stop you from saying "Yes, sir!" to the customer?

The CTO brought in a bunch of folks to try and help them move to a more object-oriented approach. He bought several licenses to the most well-known UML tool of the day. He hired a consulting firm from the Washington DC area to come up and give us a week's intensive training in the use of this UML tool. When the pressures of keeping the production version rolling out the door subsided, he took us all to a hotel conference room, away from the office, and had us spend two weeks locked away with our UML tool, flip charts, and markers. When we were done, we'd have an awe-inspiring object-oriented design for the 21st century accounting system.

As you can guess, the two weeks were a disaster. No object-oriented design came out of those sessions. The company didn't get their distributed accounting system.

What went wrong?

We lacked a strong leader with experience at object-oriented design. We were still learning the tools. Domain knowledge in accounting and experience with the product varied among the participants.

Each session would start with something like "Let's do one use case." We'd draw stuff on flip charts and quickly veer off the road. Every discussion would descend into a dizzying argument that was a roller coaster ride from the heights of abstraction to the stomach-churning drop into implementation details. I was trying to persuade them to list the steps for accounts payable when one old hand smirked and said "I can tell you what accounts payable is! Pay me!", holding out his hand with palm facing up.

The developers would scowl and listen quietly until one of them would stomp out of the room, tossing something like "If you don't make up your mind soon, I'm just going to start coding what I want" over their shoulder as they headed towards the soda machine.

We couldn't agree on what words meant. We'd have bike shed arguments for hours about what "customer" meant. We couldn't agree on how to drive from a meaningful description of the problem at hand to artifacts that a developer could use to produce working code. It's as if we'd get bored or frustrated doing that very hard work and give up before the payoff.

I left the company soon after those sessions ended. There was a layoff within six months. The CTO was forced out in a power struggle with the other two founding partners.

Fast forward eleven years. I'm working for another large company that is struggling with a transition from an older platform to a more modern one. UML has been championed as the cure for what ails us. Licenses to another UML tool have been procured. Training will commence. A large cross-disciplinary team has been convened to go through the UML design process. Consultants have been hired to shepherd us along the path of righteousness.

The funny thing is that it feels just like those sessions I sat through eleven years ago. Every discussion descends into a dizzying argument that's a roller coaster ride from the heights of abstraction to the stomach-churning drop into implementation details. We can't agree on what words mean. We have bike shed arguments for hours about design minutia. We can't agree on how to drive from a meaningful description of the problem at hand to artifacts that a developer can use to produce working code.

We'll see if we get bored or frustrated doing that very hard work and give up before any payoff comes through.

This might be the growing pains of a new team. But what if it's something wrong at the heart of UML? This object-oriented notation for rendering design decisions, codified and maintained by the Object Management Group, was born out of years of notation wars among the Three Amigos - Booch, Jacobsen, and Rumbaugh. They created a notation (UML), a company (Rational), a software tool (Rational Rose), and a software development methodology (Rational Unified Process) before selling out to IBM.

Agile approaches have largely discredited heavy approaches like a full-blown UML design.

Maybe somebody has found that this is a good way to develop large software systems in a team setting, but I haven't seen it yet. Things don't seem to have improved a bit in the past eleven years.

Frightening

Articles like "Risk Free Is Not Without Risk" at Rude Awakening scare the living hell out of me. I've included their image of the US government's outstanding liabilities, calculated using cash accounting and GAAP, to lead off this entry.

One paragraph from that column says it all:

The fiscal condition of the United Sates has deteriorated dramatically during the last several years. On the basis of current obligations, U.S. indebtedness totals “only” about $12 trillion. But when utilizing traditional GAAP accounting – the kind of accounting that every public company in the United States MUST use – U.S. indebtedness soars to $74 trillion. This astounding sum is more than six times U.S. GDP. (GAAP accounting includes things like the present value of the Social Security liability and the Medicare liability – i.e. real liabilities.)

The $12T figure is scary enough. It's the basis for all the rationalizations that our politicians are using for their monetary policies: "We can afford this deficit, because it's still a small percentage of our GDP. America is still the mightiest economy in the world." Our overall debt is roughly one year's GDP now.

But with a true debt that's six times our GDP it feels we're in an airplane that's nosediving and in gravity's grip. Pulling out will take all of our strength.

Does it bother anyone else that our government can hector industries about their poor practices - deservedly so - yet continue to use cash based accounting? Isn't that the style that one would use to run a small, cash-based business like a hot dog stand? I suppose that would be fine, as long as we were talking about the world's mightiest hot dog stand.

There's a fight going on between "fresh water" and "salt water" economists about the wisdom of continuing to run a large and growing deficit to stimulate the American economy. The Keynesian school says that massive deficits are the only way to restore our prosperity.

This argument is based on the kind of wishful thinking that brought about the recent collapse of the real estate market. It assumes that deficits are a temporary condition and that the economy will always grow. What politician has ever shut down a program once it was put in place? What if the economy collapses under the weight of all that debt?

The only time I can recall running a surplus was at the end of the Clinton years. Even that was suspect, because it was fueled by an unanticipated windfall from capital gains taxes that were based on "irrational exuberance". Remember Al Gore and his "Social Security lockbox"? How did that work out? How much of our debt was retired after that adventure in creative finance?

I fear that our situation has become so unmanageable that the economy will never be able to grow its way out of debt. Neither our people nor our government show any inclination to cut back on consumption and retire our debts. We continue to hear about new rights (e.g., good jobs, universal health care, retirement with dignity, etc.), but no one wants to pay for them or face up to the obligations we've already incurred.

The only alternative we have is inflation. If we continue to debase our currency, perhaps all those dollars that the Chinese and Japanese are sitting on will become worthless. That'll teach them a lesson! Too bad that the dollars in my 401(k) will be suffering the same fate. The joke will be on us.

I'm glad that the Republicans are out of office after an eight-year nightmare. I like Mr. Obama and wish him well.

But the Democrats have done nothing with their filibuster-proof majority to date. And neither political party is telling us the truth or doing anything significant about the real problems that we all face. Our news media would rather tell us about Jon and Kate and their unholy brood rather than let us know how dire our finances have become. We're encouraged to continue to spend, as if it was our patriotic duty.

I fret about how right-leaning religious nut cases who believe that our wealth and happiness are mandated by divine right will react if the tide turns.

We're all going to have to learn to live within our means; our means will be declining. If we can't do it ourselves, perhaps the Chinese and Japanese will force the lesson on us. If they decide to stop buying our bonds we're going to see interest rates rise whether we like it or not.

We won't be a great economy if we don't make things that the rest of the world wants to buy. Contrary to what a lot of people think, God didn't make the United States the greatest economic power of the 20th century.

I think it's misguided to think that a service-based economy can allow us to maintain our pre-eminent position in the world. If we continue to see industries collapse, without something new coming along to take their place, things could get very ugly here in the coming years.