David Doherty
4 min readAug 13, 2021

Key Value Stores and the API Versioning Quandary

If you’re not into APIs and versioning you should close your browser now!

— Winston Churchill

This is for people who have a key/value store as a database and don’t support partial updates. An example might be a file system, or someone using DynamoDB that actively decided not to use UpdateExpressions.

We’ll use a service that provides security reference data as a context to discuss. The security model as three fields: a, b & c. The client wants to make an update to field b, as below.

Security Reference Data Update

Your client has previously read from your API but threw away non-pertinent data, as is coding best practice. It has 2 of the 3 fields and decides it wants to update one field, in this example b.

They look at the API definition and see that field C is required. You are forcing them to do an extra round trip as well as put additional load on your system (the superfluous read). This extra work on their side gives you a short-cut — your API/data store code is very simple — so maybe you like that trade-off. Your philosophy is:

“Clients must always send us the full update!”.

2 weeks later your product manager asks for a new field. You didn’t get the PM unicorn who knew every single field required on day 1 of the buildout. Darn.

Now the development team adds the new field to their model, and create a V2 API with the new field and everything is good. (Stop scrolling if you don’t want a spoiler).

Your V2

Some bright spark says:

Hey, what about the clients who are still using V1 of the API?

The team gets together and decides to write a backfill for v1, like so:

The backfill

As v1 of the API service code doesn’t have the ‘d’ parameter, so you must go and load it from your data store and add it to the object you got from the client and then save down the data.

Why is this horrible?

David Doherty

I write about Fintech, it's past & future, leveraging 20+ years of experience in leadership roles at large Fintechs