Timestamps and Floating Point Numbers in Financial Systems
Yes. Press back on your browser on read something else. It will be more exciting I promise you.
Timestamps are always one of those really annoying aspects of computer science and system development that irritate everyone, but there is no gold standard way to handle them. Being FinTech focused I’ll focus on aspects I’ve come across in financial systems.
Within the code you write, you have complete control, you are the king, queen or ruler of the universe. However, as we are constantly reminded, there are many external dependencies! Let’s use a treasury bond trade as an example and let’s assume you are executing via an RFQ on an external platform. (An aside, slightly peeved Medium doesn’t have table support for this!).
In order to understand how your trade impacted the market, you need to know the time that you first decided to do the order, the time you submitted the order to the platform, the time the dealer responded to your request, the time you executed, and so on. These first 3 are the most important.
You want to know when you decided to do the order as your decision will have been made on market data at this point in time. For quant models this may have been the time that the model decided to trade, and for us lowly humans, this is the last point in time that someone had the the opportunity to jettison the trade idea from their mind.
The time you submitted the trade to the platform is two-fold, the time you sent the order, and the time that they acknowledged the order. It may have been lost in the post like a $20 bill in an envelope going through the mail. This is important for debugging technical issues, but it’s also important to verify if submitting the order had an impact on the market. For example, if you submit a large buy order to 5 dealers and see market prices immediately increase, you can surmise that there is information ‘leakage’ somewhere (i.e. they are trying to eat your lunch!). This timestamp will need to be correlated with timestamps from your market data provider which constitutes a mapping problem.
Once the trade is executed, there may be a price move as the market reacts to the trade. You’ll need this timestamp, again, to correlate with market data. With all this data, you can do analysis to see how your trading patterns impact the market. This is critical if you are a large asset manager. When you execute with a dealer they may position their book ahead of time in anticipation of your trade, meaning they may buy some treasuries to balance their risk before selling some to you. This is reasonable as they are charged capital for holding risk so they want to flatten their books as much as possible. However, if they buy ahead of you they potentially raise prices, selling to you at a higher price. You need these timestamps to understand if that is happening and if either you or the dealer are moving prices. It helps you understand if your large trade is best split up into smaller blocks and split over time (potentially getting hit with more trading and back office fees).
Internal vs External Timestamps
Always store external timestamps in their native format. Find some extra storage space and architect your platform to handle this without impacting performance. Pressure external providers to include timezone in their timestamps. ISO 8601 is generally what I recommend but even then there are nuances:
This is a timestamp for UTC. It is a string and store it as such. The Z represents ‘no offset’ i.e. UTC, but you could also replace that with +00:00, -00:00, -0, +0, +0000, etc. Thankfully most open source timestamp libraries work with this format so best to use it.
In low latency systems a string is the devil, as there is ‘substantial’ processing to do to check the timestamp. Think about the first four chars, you have to load the bytes, translate them to unicode characters and then cast them to an integer or whatever representation your application needs. This is wasteful where your system needs to respond in a couple milliseconds! Because of this, some systems might send you a number representing milliseconds from a fixed point in history (e.g. milliseconds since Jan 1st 1970). Again, store these in the native format and go on about your day.
This storage cost is more than offset when you get to problems like debugging a live production incident with a trade. Being able to quickly work with an external partner and communicate the timestamp with them is valuable. Imagine this conversation where you are not saving the external timestamp:
- You (in the US): We a message 9:45am, we didn’t get an acknowledgement, so we resent. But then we got two events later so looks like it was duplicated on your side.
- Platform support (in Europe): We don’t see anything at that time, sir/madam.
- You: Erm. Well this is the US, so maybe I’m looking at EST. Try 2:45pm.
- Platform support: Nope.
- You: Well, this server looks like it’s in us-west so maybe its the system timestamp. Try 5:45pm?
- Platform support: Nope.
This doesn’t look so bad perhaps, but now imagine you’re on a conference call with 10 people, 5 from each side. Some are technical, some are not. Everyone is flustered. These things waste valuable time. The pressures is compounded when traders aren’t sure of their current market risk as a result, and may have to unwind the technical mistake.
This goes much easier when you can talk to external people in their timestamps. It also potentially helps regulatory reporting. In cases you and someone else may be dual reporting, or may accidentally both report when only one is supposed to. If you’re using the same timestamp its easier for other third parties to dedupe.
Back to the comfort of our internal domain we still have a few things to think about. I’ve already stated I prefer the ISO 8601 format despite its latency and storage penalties. The next step is deciding on a timezone.
Do you standardize the whole company on a single timezone? Your HQ perhaps? This is generally done by most older companies, though for new FinTech companies if you store in ISO 8601 you can do a just-in-time translation when presenting the data. Again, there will be a slight performance hit, but worth it in my mind. Over time companies build up tribal knowledge in how things are done. Let’s say the company started in 1980 and runs its end of day processes at night in US timezone, then at some point it created a division in Asia. This is very common and you might be surprised how many offices there are in Asia where everything resets in the middle of the day because US is telling them its a new day. Even big brand vendor platforms have this issue.
By using ISO 8601 you’ve perhaps sidestepped this ‘my timezone rules’ approach. You will need to make sure developers use this approach. I’ve seen stuff like checkstyle used in Java development to force explicit setting of timezone. This can feel draconian as forcing them to get the system timezone in every log message is a lot of extra keys to type (arguably). You could wrap your favourite loggers with convenience methods to make this easier.
There are many articles that talk about issues representing floating point numbers within computer science. The cause of this issue is easily Googled and would recommend you to spend time understanding this if you plan having a long career. Within finance this can be exhibited by weird rounding issues like seeing “2.499999999” when you entered “2.5”. Being exact within financial systems is very important as these numbers are key in legal contracts executed electronically. So again, my recommendation is to store external numbers in the native format provided to you. To reiterate it helps debugging, communicating with other companies and reconciliations.
In some markets, for historical reasons, prices are quoted in fractions instead of decimals. Storing the native format helps with this, in that a fraction based price stored as a string can be easily displayed. Again, storing as a string means you’ll incur a performance penalty but it is too small unless you’re building a truly low latency system.
Storing the external format of a number doesn’t mean you have to code to it constantly. You can select a global standard for storing decimals which is a purely technical decision outside the scope that I’m covering here. But as the data comes in you can translate (if required) and store to your chosen format. So in the case of US treasuries that are stored as fractions, you may receive a price of 99 7/32, which you can store as 99.21875 (either string, float, double, etc — whatever you decide).
In financial systems things get a bit more complex where we start dealing with monies. You’ll have to decide if and how to denote a number is a monetary unit. Here are a few options:
- Store the currency symbol with the number, and store as a string. This is painful as you have to have a global convention and every program will have to (at least under the covers) translate this.
- Store the decimal (in float, double, decimal, etc) in one field, store the currency in another field.
The latter is probably going to look better to most readers, as it should. The subsequent question is how to store the currency. If you use a single field called currency with an ISO currency code, this will work for single currency products, but falls down with cross currency products like FX cash.
During the financial crisis and again in 2011/2012 the USD and CAD currencies hit parity. When this happens, you need to be really clear about how a floating number relates to those two currencies. So let’s say the rate is 1.05 Candian dollars for every US dollar. In the FX markets there are conventions that are used to denote this. You would see this example noted as 1.05 USDCAD. You take the first currency and take 1 unit, and this would convert into the second currency at the announced rate (1.05 in this case).
So in your systems you have a few ways to save this. We’ll store the 1.05 in a field (let’s just call it Value for simplicity). The currency field(s) could be:
- Field 1=Base Currency (USD). Field 2 = [Other|Target|Far|whatever you like] Currency (CAD).
- Singular field called currency USDCAD.
In the latter you are relying on developers understanding these conventions. From one aspect there is a risk that a developer might switch the currencies by accident (disaster if you’re making trading decisions off this data), but on the other there is also a simplicity in that hopefully destination systems will understand it.
From a technical perspective the singular field approach may incur some additional CPU cycles. First you have to check the length of the field (3 chars = single currency, 6 chars = cross currency), then parse each individual currency.
If you decide to use a binary storage/transmission format you can save time/space by having unique binary values for each field. If this is core to your architecture (e.g. low latency) you may want to opt for the first approach. Using a singular field with 6 chars opens up A LOT of potential permutations that the binary formatter will need to understand. The binary format would need to understand both USDCAD and CADUSD, or you have to do some translation before writing the binary object.
So you may opt for storing 2 separate fields. In this case your UI developers need to make sure they understand how traders and business users would like to see the date. They just want to see USDCAD because this is how they mentally understand the data. If you show them base currency and target currency, you will waste screen real estate and force them to engage their valuable brain cycles on something that isn’t business focused.
Let’s say you go for 2 separate fields, do be aware that you are likely going to have many other currency fields later:
- Settlement currency. For example you may have a USDCAD trade but it is cash settled in EUR.
- Margin/Collateral/etc currency. If you’re trading a margined product, there may be an additional currency. E.g. you may be a Japanese institution trading USDCAD swaps with the collateral posted in JPY (yen).
- Reporting currency. Perhaps your institution is based in Switzerland, and when aggregating numbers up across all products they want to see it as CHF.
There are more cases, just make sure you’re aware that you are going to end up having lots of different currency values and paired fields. You’ll need to make sure your naming conventions make it easy to self-interpret what goes where, e.g.:
- settlement_currency=EUR, settlement_value=123
- margin_currency=JPY, margin_value=123
- and so on
As always, feel free to comment below if there’s anything I missed or any questions!