Storing and transmitting monetary values...

Storing and transmitting monetary values...  Most financial applications these days are not standalone applications, but rather components of much larger and more complicated systems.  In a past life I was the CTO of a firm that made stock and option trading software, and I once made a map of all the systems that our servers communicated with.  There were about a dozen separate machines in our own server room that talked to each other.  There were also over two hundred other applications (in over thirty other organizations) that our servers talked to.  Those communications paths used a dozen or so different protocols, not counting minor variations.  In addition to all these connections, our servers also stored financial information: in memory, on disk, and in logs.  That system was not an outlier – I've worked on dozens of other systems just as complex, or even more so.

This complexity has lots of implications when it comes to representing monetary values.  What jumps right out to anyone examining all these needs is that there is no “one way” to represent a monetary value that is correct, best, or optimal.

Often the representation has to match an existing standard.  For instance, perhaps the monetary value must be in the form of an ASCII string, like this: $4532.97.  The standard might specify that it be in a fixed-length field, with spaces padding to the left as required.  Or the standard might specify that the value be in XML, encoded in UTF-16, with a tag of “AMT” for the amount, and “CUR” for the three-character ISO currency code.  Some older protocols have specified binary formats, with a numeric code for the currency and packed BCD for the value.  I could go on and on with these, because many clever and/or crazy people have devoted a great amount of time to inventing all these schemes.

When storing or transmitting large numbers of values, often a compact (or compressed) encoding is essential.  I once worked on a server that did algorithmic trading (that is, the program decided what stocks to buy and sell).  This server needed to store (both on disk and in memory) huge tables of stock pricing data.  On busy days this could amount to several hundred million entries.  The original design of the server represented the monetary values in a fixed length binary structure that occupied 22 bytes, so on busy days we were storing a couple of gigabytes of data – and moving that data between disk, memory, and CPU cache.  We observed that this was a bottleneck in our code, so we came up with a variable-length encoding that averaged just 7.5 bytes per value.  We rolled our own on this one, as the numeric representation we were using didn't have a compact encoding.  The performance impact was stunning, much more than the 3x you might naively expect.  The biggest impact came from fewer CPU cache misses, the second biggest from faster encoding/decoding time.  The latter we didn't expect at all.  The real lesson for me: compact encodings are very valuable!

Databases present another challenge for representing monetary values.  The numeric column types available often can be specified with the precision required – but the API will require converting the value to or from a string or an unlimited precision value (such as BigDecimal).  Sometimes the values don't need to be queried, but rather simply stored, as key-accessed blobs.  In those cases, a compact encoding is valuable, especially for large tables of values accessed as a whole.

From all this we can derive another requirement for a monetary value representation: flexibility of encodings.  There should be a variety of supported, standard encodings including (especially) a compact, variable-length binary encoding, a fixed-length binary encoding.  There should be support for conversions to and from unlimited precision numbers as well as native types.  There should be support for conversions to and from strings.  Also very important, for flexibility: support for adding encodings – because there's no way to anticipate all the strange things the world will require of you...

Paradise ponders, thermostat monitor, borosilicate glass, and excellent beef soup edition...

Paradise ponders, thermostat monitor, borosilicate glass, and excellent beef soup edition...  After all the work we've had done to our downstairs furnace, it still has the original problem that started last December: it cycles on and off every 5 minutes or so.  The changes made have greatly increased the air flow, especially in our basement cattery, and that's a very good thing.  I'm happy to have made the changes even if they didn't fix the issue.  Nevertheless, we still need to find (and fix!) the cause of this rapid cycling.

One of the possible culprits is our Nest thermostat.  When the furnace is all buttoned up, there isn't any exterior indication of whether the thermostat is “calling” for heat.  So yesterday afternoon I built a thermostat monitor.  This is a ridiculously simple piece of electronics: just three pre-packaged 24V (AC or DC) LED monitors.  One of them monitors the 24V power to the thermostat, another monitors the “call” signal, and the last one monitors the fan signal.  The fan signal is used only when we want to run the fan without heat.  I packaged them in a Ziploc food container, and mounted it to the furnace with a magnet.  The really important one is the call signal.  Here's some photos of my monitor in place:

After I put that in place, I sat and watched two complete cycles of the heater.  The call signal never wavered – which tells us that the thermostat is doing its job perfectly, and the problem is somewhere in the heater.  In the first photo above, if you embiggen it and look just to the left of those labels, near the bottom of the labels, you'll see a dim neon bulb showing.  That is normally solidly on, indicating that all is well.  I noticed something last night that we've all managed to miss before.  When the heater has been on for a couple of minutes, the burner goes out (gas turned off) while the fan is still going.  For the next 30 seconds or so after that, the neon light is blinking a two digit code: 3 short blinks, then 1 long blink, so “31”.  Then after the 30 seconds is up, the fan stops and the neon light turns back to solidly on.

That 31 code means a problem in the low pressure sensor.  This is a safety device that ensures the exhaust gases are going out the vent and not into the mechanical room – a good thing!  My nose doesn't detect any exhaust gases in there, so I suspect the problem is not a lack of draft, but rather a malfunction in that switch.  In any case, this is a certifiable clue as to the source of our cycling problem.  That means we might actually be able to get it fixed!  Yay!

Yesterday was Tuesday, and that means beef soup at Los Primos.  Debbie didn't feel up to going out, so I ran up and got two takeout orders of soup.  The folks there were disappointed not to see Debbie, and asked after her health.  They told me to make sure she shows up next Tuesday. :)  The food there is really special, but the people at the family-owned restaurant are even more so...

A few months ago, we bought eight of these tumblers.  I thought I'd posted about them earlier, but I can't find anything on them.  Debbie and I both tend to drink big glasses of stuff: water, milk, iced tea, and the like.  Cold drinks, mostly, and often iced.  These tumblers caught my eye because they're insulated – there's two layers of glass with a vacuum between them, just like an old-fashioned vacuum thermos bottle.  Plus they're big: 20 ounces.  The glasses are made from borosilicate glass, a material I know well for its unusual combination of strength and low thermal expansion coefficient.  It's also very clear.  The fact that they have a low thermal expansion coefficient means they're safe for both hot and cold drinks, plus you can throw them in the dishwasher.  We've been using them for all our cold drinks, and we both love them.  I'm especially appreciative of the rounded rim (comfortable in my mouth) and the sturdiness – we still have all 8 of them, despite daily use for 3 or 4 months now.  Looking on the Amazon page I see that there are a few very negative (one star) reviews, mainly for three reasons: their weight, the thickness of the rim, and their alleged fragility.  We actually like the weight and the thick rim, and I'm guessing that most people do as the reviews are overwhelmingly positive.  As to the fragility, I can only say that these are lasting longer than the majority of glassware we own. :)  Anyway, these are resoundingly endorsed by the Dilatushes of Paradise!