Edge Storage Deep Dive

Edge caching

The first delivery for Edge Storage was the creation of a short term distributed cache to be leveraged by all applications deployed in the network. Each VPC queues and distributes cache results proportionally across every available Host, reducing the number of Hosts and compute units required for your application to scale.

Image A: Picture of 6 Hosts, with 3 hosting an application and all 6 storing cached results for the application

Caching at the Gateway

The logical place for a short term cache is arguable on Gateways (the termination point of requests within the network), but this just doesn't scale. Let's imagine you have a single Gateway with 250 Hosts, a reasonable number on the network. The Hosts are each responding to hundreds of requests per second and it's not abnormal, especially when dealing with rich media, for each payload to be ~100kb. That's a lot of results to store in a cache, in memory, on a single machine.

Entirely removing the cache from Gateway creates a potential vulnerability when a high volume of requests for a single resource causes an unnecessary flood in the Gateway queue. To protect against this, Gateway operates the lightning cache, a resource specific in-memory cache that adjusts TTL in real time based on the frequency of requests across a moving window, with consideration for available memory and queue size to further optimize the balance between delivery speed and load distribution.

Image B: Gateway cache showing different sized caches based on traffic

Object storage

By using the same distribution and replication protocol as the cache we were able to facilitate a persisted long term object store, with the added option to distribute and coordinate outside of a single VPC. Though this may appear as a natural progression, long term object storage comes with a fresh set of requirements, most notable the issue of resilience.

Whilst cache corruption or invalidation due to service disruption can cause a significant CPU spike, the data is by nature ephemeral so nothing is lost that can’t be rebuilt. The same is not true for longterm object storage which added a fresh set of considerations to our approach.