Many times I find myself wondering about the future of the way we develop software currently. I guess if you ask different people their opinions you'll get just as many different views.

Back in the good old days we used to use punch cards to feed into the machine, from which the program is read and executed. Each time you want to execute a program, you have to load it using the punch cards again. Lateron came assembly language, where at least you could express yourself at the level of the CPU's opcodes, and save the programs for execution in an assembled, linked state called an executable application.

People used to build highly sophisticated systems using the archaic assembly language, under which operating systems, big parts of the original Windows NT kernel, etc. are examples.

One thing I am constantly fighting with while developing software systems, is the issue of when to do automated testing, and when not to. Some evangelists of automated testing procedures will try and convince you to always automate your testing.

I believe differently. Like everything in life, there should be a balance between using automated procedures and manual testing. Web based systems are growing at a very rapid pace, and this is a prime example of where a delicate balance is needed. Automating back end logic is much easier than automating front end logic (i.e. the GUI). This is simply because the user interface is not easily accessible using programmatic methods such as API's, services etc.

Most (clever) people adopted the coding style of separating business logic out of the UI, into a separate business layer. This eases automated testing dramatically, since now most of the functionality can be automated and only the thin UI layer needs to be tested manually.

Ever got frustrated by virusses (virii??) and Windows? Well, I am subscribed to Bruce Scheider's Cryptogram and found this link which I think is a great read.

I remember I used to work happily in Wordperfect 6.1 for Windows 3.1, back in the 90's. I had only 8MiB of RAM back then, so it took a few seconds to load. But once it was loaded, it was amazingly fast. Put it this way - I could never outtype the word processor.

Windows 3.1 booted in a few seconds on a high end PC. DOS booted even quicker - under a second. Yeah yeah I know - DOS and Windows 3.1 does not have true memory management, no pre-emptive multitasking, no networking subsystem etc. But then again those PC's were 486DX2-66 MHz machines with 2-8MiB of RAM...

Why am I ranting like this? Because I am frustrated seeing my G4 1GHz 17" PowerBook with 1GiB RAM run into 90% CPU utilization just because I have Microsoft Word 2004 on Mac OS X 10.3.4 open. In specific, I have one document open based on the new Notebook type document. For the rest - I am not touching the notebook. It is just sitting there chewing up 90% of my CPU - doing nothing. After some troubleshooting I found 70% of the problem. I have about 20 entries, each with one of those fancy aquafied icons (checkbox, exclamation mark and question mark) to indicate the kind of entry. That is why those icons are there for, isn't it???

I always knew DOM and XPath were not as fast as SAX, but I have never realized exactly how slow they are when applied to a huge XML document.

By huge I mean 5MB+. I had a 32MB XML document consisting of a root element, that contained about 148000 child elements each containing 8 attributes. I needed to iterate over all of these nodes to retrieve the values and write them away to a database. I know the right tool for the job is SAX, but I insisted on using DOM and XPath as I am well acquainted with them. Needless to say, the parsing (extrapolated) would have taken 46 days - for a task that needs to happen once a day that was obviously unacceptable. Here is an exert of that code:

int vCnt = Utils.getInt(pXML.valueOf("count(/" + XML_DOCUMENT_NAME + "/article)"));
for (int i = 1; i < = vCnt; i++) {
  String vArtNo = pXML.valueOf("/" + XML_DOCUMENT_NAME + "/article[" + i + "]/@article_no");
  String vArtUOM = pXML.valueOf("/" + XML_DOCUMENT_NAME + "/article[" + i + "]/@article_uom");
  String vDescription = pXML.valueOf("/" + XML_DOCUMENT_NAME + "/article[" + i + "]/@description");
  // ...
  // Do something with them
}