Hi everyone 👋
I’ve had to a series of emergency dental ops this week which have really knocked me for six, so I’ve been majority out of action. The team are cracking on without me though.
It’s been pretty much all hands on the replacement of Consul with the completion of the first stage of removing Consul from the application and subscription configuration process.
Gateway is now loading core apps and subscriptions from Stargate using a network native solution (rather than Consul), and watching for changes / deletes. It isn’t yet creating the caches or queues, but it will be soon.
Also of note is what the work will mean for updates to the network in future. At the moment when we release there’s effectively a race to update between all devices in the network. This can create a scenario whereby services drop out in regions as all devices are updating at once. It also means that a failed update requires a full rollout again. With the move away from Consul we will be able to update selectively, configuring a region by region and device by device cascade rollout schedule. This is a far more robust solution for our use case.
To give you some more insight in to the changes we’re making as move on from Consul, we’ve always used the Consul KV directory watch events as triggers for change, with Stargate writing to Consul KV and Gateway reading from it, so the fact we’re removing those watches and replacing them with a flat data structure is a big deal.
An example in core apps is that in Consul we stored nested values:
cdn > config
cdn > build
cdn > config > subscription > config
Now we’re decoupling the two, which is far more efficient. Changing the CDN build data will no longer trigger a watch event for each subscription config. The real complexity here is having the two work independently.
The performance impact is huge. Gateway full configuration is now down from ~20 seconds through Consul to 2 seconds with the native solution. Massive performance gains are coming online across the board as we move through this work stream.
Next up in the replacement of Consul is Host. Host will sync applications and config from Gateway. The code from Gateway will be reused here, which will make it quicker to integrate than the Gateway portion of the work.
We’ve also been working on the build and deploy process to streamline the configuration, reduce the build speed and automate the final deployment stage. This is expected to be deployed early next week.
The eCom/subs platform discussed last week progressed well, with a full launch planned in little under a month.
Our sales strategy advanced, as did the development of supporting materials. We’re a few weeks out from putting this lot in to action.
Jason from TNC is going to be doing another AMA with us soon. Date TBC, but you can start getting your questions in to BJay and Rick now.
Enjoy your weekends. I’m off to lie down some more 🤒