While you see the changes to the explorer, a lot of what goes on actually happens beneath the surface. Yesterday, you may have noticed that all Hosts appeared offline in the explorer as well as in the dashboard, and remained that way for a few hours. This was actually part of a major update to our synchronization layer—you’ll learn about that more soon—which meant we had to put the explorer into a maintenance mode for a few hours while we upgraded the sync services that manage the aggregation of data from the network which powers, among other things, the explorer.
But how do we gather this data, where does it come from, and how does it work?
Follow me as we venture on a curiosity voyage into the depths of the network.
🔗Devices & The Network
Our first stop on this curiosity voyage is the network itself. This is where the data begins to form. When a device connects to the network, a session is created within the Stargate it connects to, and this remains active until the device goes offline.
This session contains information such as which device it belongs to, which service it pertains to (all services, the Agent service as well as the Host service, have their own sessions) as well as some other useful metadata too.
🔗Synchronization layer
Our next stop on the curiosity voyage is the synchronization layer. In this shadowy place, thousands of little robots work tirelessly to… okay, they’re just software programs. But these synchronization services run day and night, watching each and every Stargate for the smallest change to their sessions.
When a change is detected, these synchronization services—or sync—work out what has happened. Has a device come online? Has a device gone offline? Has it done both?
Once the sync service has determined both what has happened, and which device it has happened to, it records this data in the contributor database. When a device comes online, a new session is created with information such as location, time, the Stargate the device has connected to, and some device specific information such as the CPU architecture. When a device goes offline, the sync service closes this session, recording the exact date and time that the device left the network. And when it comes back online again, the whole cycle begins again.
🔗Turning data into information
The last stop on our curiosity voyage may be familiar to you, it’s where we turn the data into information, linking up data about devices with data about their activity, to bring you the Edge Network Explorer.
Previously, the explorer would be querying the network directly, running through Stargates to get an idea of where devices were, what they were up to, and when they were last seen. This wasn’t very efficient so we could only update the data roughly every ~2.5 minutes.
Now that the data is managed by the sync service layer, the explorer is now able to query the data more often, meaning we can drastically increase the refresh rate. We’ll be trialing some different numbers in the coming weeks but we’re starting with a 60 second refresh rate. That’s 250% quicker than before, meaning the data you see in the explorer is closer to the heartbeat of the network than ever before. Kapow!
In addition to this, the data we’re seeing is more reliable and up to date than ever before, with over 130 online devices being seen yesterday, their every activity viewable in real time.
🔗More where that came from
If you think that’s great, you’re right. But there’s more where that came from. Earlier this month we made some major improvements to our earnings and payout layer, and coming soon you’ll start to see your earnings appear day by day, so stay tuned, as we’ll be writing more about that.
