Battle of Midway

Sixty-four years ago today, the naval battle now known as “The Battle of Midway Island” commenced. It lasted for four long days:

From the U.S. Navy’s official Midway history site:

The Battle of Midway, fought over and near the tiny U.S. mid-Pacific base at Midway atoll, represents the strategic high water mark of Japan’s Pacific Ocean war. Prior to this action, Japan possessed general naval superiority over the United States and could usually choose where and when to attack. After Midway, the two opposing fleets were essentially equals, and the United States soon took the offensive.

Japanese Combined Fleet commander Admiral Isoroku Yamamoto moved on Midway in an effort to draw out and destroy the U.S. Pacific Fleet’s aircraft carrier striking forces, which had embarassed the Japanese Navy in the mid-April Doolittle Raid on Japan’s home islands and at the Battle of Coral Sea in early May. He planned to quickly knock down Midway’s defenses, follow up with an invasion of the atoll’s two small islands and establish a Japanese air base there. He expected the U.S. carriers to come out and fight, but to arrive too late to save Midway and in insufficient strength to avoid defeat by his own well-tested carrier air power.

Yamamoto’s intended surprise was thwarted by superior American communications intelligence, which deduced his scheme well before battle was joined. This allowed Admiral Chester W. Nimitz, the U.S. Pacific Fleet commander, to establish an ambush by having his carriers ready and waiting for the Japanese. On 4 June 1942, in the second of the Pacific War’s great carrier battles, the trap was sprung. The perserverance, sacrifice and skill of U.S. Navy aviators, plus a great deal of good luck on the American side, cost Japan four irreplaceable fleet carriers, while only one of the three U.S. carriers present was lost. The base at Midway, though damaged by Japanese air attack, remained operational and later became a vital component in the American trans-Pacific offensive.

The full story of Midway Island could be a Tom Clancy novel for all its drama, technology, larger-than-life characters, luck, turning point nature, and heroic actors. It’s very sad for this history buff to know how few young Americans know this story…

From an historical perspective, of course the most important thing about the Battle of Midway Island is that it is the beginning of the Allied victory in the Pacific war against the Japanese Empire. Since the preceding December 7, when the Japanese struck Pearl Harbor, the Americans and their allies had been subject to six months of almost unrelentingly bad news in the Pacific. There were many knowledgeable and informed people at the time who were uncertain the Allies would be victorious — the Japanese Army and Navy had a definite air of invincibility about them. Those were dark days indeed for Americans. About the only positive news before Midway was Jimmy Doolittle’s amazing raid on Tokyo, just two months earlier. That raid, while terrific for morale, had little military significance. Midway, on the other hand, was an unqualified and very significant military victory:

Battle of Midway: 4-7 June 1942: Combat Intelligence Released as of 14 July 1942

21. The following is a recapitulation of the damage inflicted upon the enemy during the battle of Midway:

(a) Four Japanese aircraft carriers, the Kaga, Akagi, Soryu, and Hiryu were sunk.

(b) Three battleships were damaged by bomb and torpedo hits, one severely.

(c) Two heavy cruisers, the Mogami and the Mikuma were sunk. Three others were damaged, one or two severely.

(d) One light cruiser was damaged.

(e) Three destroyers were sunk and several others were damaged by bombs,

(f) At least three transports or auxiliary ships were damaged, and one or more sunk.

(g) An estimated 275 Japanese aircraft were destroyed or lost at sea through a lack of flight decks on which to land.

(h) Approximately 4,800 Japanese were killed or drowned

22. Our total personnel losses were 92 officers and 215 enlisted men.

Of course I was not there at the time — I wasn’t born until 10 years after Midway — but I have read extensively about the period. One thing comes through loud and clear: both in the military sphere and in the civilian world, the victory at Midway had an electric effect. It was a proof point that America and her allies could prevail; that the Japanese were anything but invincible.

Midway was also a triumph for our intelligence services, in particular the cryptanalytic team who cracked the Japanese codes. For anyone with a technical bent, the story of how that team cracked the codes is fascinating (and entirely declassified — so the complete, authoritative history is available). Likewise, the story of how they deliberately used disinformation to figure out that a particular code phrase referred to Midway Island is just as thrilling.

But most of all, when I think of the Battle of Midway Island, I think of the “Greatest Generation” — the courageous generation of my parents — who lived through such dark days (Hitler, Stalin, Hirohito, and Mao all at the same time!), rallied and persevered through months and months of unmitigated bad news, until finally one day they were victorious. Can you even begin to imagine the joy and relief every American must have felt on that happy day?

The Battle of Midway was the first uplifting step on the path to that day of victory. And today is the sixty-fourth anniversary of the first step…

Able Ajax

Geek alert!

If you’re not a Javascript/Web developer, skip quickly to the next post — that will minimize the pain. If you are a Javascript/Web developer, you might be interested in this.

I’m working on an application now that has several different needs (and almost certainly more to come) that all fit the same pattern: quite large tables (100s of thousands of entries) that need to be viewed, scanned, filtered, and sorted. The entries in these tables represent real-world electronic trading events; it’s vital for the information to be up-to-date and accurate.

My first stab at a solution for this took advantage of the fact that modern web browsers are fully capable of displaying truly gigantic HTML tables in an IFRAME element, making a nicely scrollable (horizontally and vertically) UI element. However, there were quite a few shortcomings with this approach. First, the HTML was generated at the server and delivered to the client in a GET; this means that every change in the view required a complete download of that portion of the table that was being viewed — definitely not a zippy operation, and quite a server load as well. Second, a little thing but very annoying: the table headers scrolled right off the screen unless you were at the very top of the table. Third, and probably worst of all: when the table had more than about 8,000 rows, it was completely unreasonably slow — and the client’s CPU utilization went to 100% (always a bad thing!).

Some investigation on the third problem led me to an interesting discovery: the browser (Firefox 1.5 in my case) uses a fairly inefficient algorithm to insert a row into a table. It doesn’t matter whether the row is being parsed from an HTML document or is being inserted by client-side Javascript code operating directly on the DOM — it’s still slow. From the behavior, I think it’s an n^2 algorithm; I speculate that they are copying the entire table on every insert operation.

After a little thought I realized that the real root of my problem was that I was trying to render thousands of table rows just to see 30 or 35 of them. There had to be a better way!

The approach I chose relies heavily on Ajax techniques. First, I wrote some stuff to fetch the underlying data and store it in Javascript arrays. I used JSON as the encoding on the messaging layer, as this is very lightweight on the client side (as it’s natively supported by Javascript). This code queries the server on a configurable interval to get the data; the end result is that the array in client memory is up-to-date within that interval. This stuff was easy to write (prototype.js is my friend), it is very fast, and it imposes almost no measurable load (other than a few 10s of megabytes of RAM consumed).

Then came the really tricky Javascript part — tricky for me, anyway, as I am no Javascript guru. I created an object (I call it ViewManager, and it has some associated objects for filtering, sorting, and so on), whose purpose in life is to maintain a small (e.g., one screenful) table that is a view onto the much larger underlying array of data. Using HTML and CSS, I created a vertical scroll bar that is entirely operated by my Javascript, rather than by the browser. The HTML table seen by the browser has only 30 or 40 rows in it (the size can be changed by the user by dragging a corner) — this renders blindingly fast, with practically no CPU consumption. When the user scrolls around in the table, what’s happening under the covers is that the ViewManager object gets told to move around on the large array of data, and it simply re-renders the entire HTML table as needed to simulate this.

Because my code is managing the table’s presentation, it’s a snap for me to keep the headers on top — that works fabulously well. I handle filtering and sorting by keeping a secondary array whose elements represent the items actually being viewed, in the order that they should be viewed. Each element of this array contains an integer index to an element in the main data array. This arrangement allows me to sort and filter without actually modifying the data table — and this means I can very conveniently always return to a normal unsorted, unfiltered view. With a data array containing 500,000 items, a sort and/or filter operation takes well under one second — awesomely zippy compared to the initial method. And best of all, the server is completely uninvolved in this!

I was pleasantly surprised at how easy all this code was to implement. It’s very reminiscent of low-level GUI implementations, especially in the event-driven nature of it. The trickiest bits turned out to be accurately figuring out where an object happened to be rendered on the screen — in a browser like Firefox, the user can do things like change the font size, or resize the window, and that will re-render the screen, sometimes in unexpected ways. I’ve not found good documentation on stuff like this, so I ended up doing quite a bit of reverse-engineering and experimentation — but that’s ok, I learn a lot every time I do something like that…

The bottom line on this experience: a modern web browser, using Ajax techniques, can do just about anything that an old-fashioned WIN32 GUI can do, with just as much interactivity and zippiness. The only exception I am currently aware of is for graphical elements that you’d really like to create on the client — Javascript provides no tools for manipulating or creating images. But the vast majority of the business applications I’ve worked on had no such requirements. Without such a requirement, for me choosing a web client over a thick-client GUI application is a no brainer. With such a requirement, I’d be looking very hard at other approaches, either server-side or with client side Java applets. I’d also be looking for someone’s add-on to Javascript for image manipulation…