Edge Latest articles from the team en 2019 Edge Network Technologies Ltd. All rights reserved. Sun, 15 Dec 2019 02:44:45 +0000 2019-12-15T02:44:45+00:00 en 2019 Edge Network Technologies Ltd. All rights reserved. Edge https://edge.network/assets/img/dadi-logo-colour.png https://edge.network Finding a Stargate https://edge.network/knowledge/network/finding-a-stargate First, some background information about DNS and BGP

We use BGP for DNS which means we can have a number of Stargates that live under a single subnet IP. When your browser requests a DNS record from the network, BGP directs it to the closest Stargate so that the response is returned as quickly as possible. BGP has been around for a long time, and is reliable enough to make this process fairly straight forward.

Where BGP doesn’t work well

Stargates are responsible for network coordination as well as DNS which means the same device that talks to Hosts and Gateways is also responding to DNS. Because of this, we initially decided to let Gateways and Hosts find their closest Stargate in the same way the DNS requests I mentioned find the closest Stargate - BGP.

BGP is reliable, but not entirely accurate. Sometimes a request made in Singapore will be resolved by the Stargate in Frankfurt. This is okay for DNS lookup, perhaps adding 10ms to the response time, which is less than the impact of moving from a wired connection to WiFi. However, when using this lookup method for Gateways and Hosts to find their closest Stargate, the issue is greatly exaggerated. If a Gateway in Singapore connects to a Stargate in Frankfurt, the Hosts that connect to it are likely to also be in Frankfurt, meaning all of those images for Content Distribution are travelling a great distance, and that’s something that can occasionally cause a timeout, or 504 Gateway Timeout error that some of you have seen.

Big corporations like Google have the same issue and the way they resolve it is by buying up a lot of the internets’ backbone at a huge cost, but for the rest of us this isn’t really a sound resolution. BGP just isn’t accurate enough to be relied upon for service discovery, so what we are deploying today is a more innovative fix for this.

When a Gateway looks for a Stargate it no longer uses BGP, but instead requests a single DNS record from the Stargate that BGP reports as the nearest, which contains a list of IP addresses for all other Stargates. It takes these IP addresses and runs some connectivity checks. These checks allow the Gateway to confirm which Stargate is the closest to it in terms of connectivity.. Once they completed, the Gateway connects directly in to the chosen Stargate.

For those who have asked “Why does X Gateway connect to Y Stargate when Z is closer?”, now you know.

]]>
Thu, 12 Dec 2019 14:30:00 +0000 5df1f273e5287b1e040ba6b3 Arthur Mingard 2019-12-12T14:30:00+00:00
AMA Recap: December 06th, 2019 https://edge.network/knowledge/network/ama-recap-december-06th-2019 Question 1

What’s the status of Gateway onboarding? I understand the first wave will be selected handovers of existing Gateways, yes? When will we be able to onboard gateways ourselves?

Arthur Mingard: There are a bunch of items that need to be lined up for self on-boarding of Gateways to be enabled, key of which is the replacement of Consul with a custom solution at this layer in the network. Timing for this is TBC, and will be outlined in the coming update to the roadmap (which has been slightly held off to allow for the sequencing of new requirements such as this). And yes, rentals will be open up in the first instance.

Question 2

How far away are we from fully automated self on boarding of hosts?

Adam K Dean: Expected Q1 2020.

Question 3

What’s the status with utilising host GPUs in CDN to speed up the image processing?

Arthur Mingard: The community behind vips have run extensive benchmarking comparisons between GPU/CPU image processing and have concluded that the performance benefits are negligible. We suspect that this will change over time, and are keeping a close eye on developments.

Question 4

For how long will the name dadi be used and why?

Joseph Denne: In Github for example, simply because we haven’t found the right moment to fully migrate legacy repositories relating to web services yet. And of course, it exists in the token contract which, isn’t something that we change. The only way to fully remove it from the token would be to issue a new contract, but this would require a token swap, which isn’t something that we think that we should be doing just for a rebrand. It will happen in time, but we’re not sure when yet.

Question 5

How is Tiipr App doing? Are they still in the race for making the world a better place with cashless tips?

Joseph Denne: Very well! They now have two apps (one for customers and one for service staff, powered by API and making use of CDN for component delivery), and a portal for businesses/venues in the works (built on Edit). You can expect to hear much more from them in 2020.

Question 6

I was wondering what behaviour I can expect when I exceed the limits of the Free tier. What status code can I expect and will this result in immediate termination of service or will assets continue to be served at a lower level of priority? How can I best monitor this?

Adam K Dean: We’re working on exposing telemetry for usage. There’s no hard cut off, rather a cool down period to allow for switch over. We expect to be exposing telemetry for use in Q1 next year.

Question 7

Is there any overall goal in terms of host capacity within self onboarding phase?

Joseph Denne: There’s no set goal for capacity overall as the requirements are a moving target depending on service utilisation. As this side grows we expect to get a feel for the balance of resource / capacity, which in turn should allow us to publish the requirements in the network, which could go as far as showing which areas are in need of capacity and even looking at methods to incentivise the back filling requirements quickly.

Question 8

What I’m curious about is the earning potential. $4.69 is what is reflected on the edge website. We earned like 15-25 for the last months till self onboarding started. But how does edge get to that $4.69 number?

Joseph Denne: The figures on site/in legacy docs are not updated in real time and do need a refresh. The average annualised ROI against the POS for Hosts in November was 40.73%, which is ~3.39% per month.

Question 9

Looking at the roadmap we have several network ready services such as API, Publish and Storage. When will we see those going live on the network? How about containers?

Arthur Mingard: API + Publish = Edit. Storage should be in Beta in Q1. Containers are in use now, but opening up the framework is complex and is not expected to be in testing until the back end of next year. The updates to the roadmap will provide more detail.

Question 10

A little bit longer than a month until the end of the new year. Is the network code about to be open sourced?

Arthur Mingard: Not in full as we’re not quite ready for that yet. But we are starting to open the code for a bunch of components and tools, and will be opening up key service dApps as well.

Question 11

The Q4 2017 DADI marketing strategy presentation; Brand Development was planned to be outsourced to creative agency. Was the current edge website/brand delivered by an agency? What agency? When agency?

Joseph Denne: We delivered this in house, including the re-brand to Edge. We have one of the best brand designers in the UK as our design director and he is continuing to move forward our visual style.

Question 12

How well does the network currently operate? When I extrapolate my earnings, I calculate a total per month to around 150k tokens which with today’s valuation is like 7.5k USD. This feels very low, like the team is holding back the process of shifting over traffic to the edge network. Reason? Just being careful or bugs? Is it possible to get some elaboration about this (sensitive) topic?

Joseph Denne: I can’t comment on specific figures or assumptions as this is commercially sensitive information. However, the platform is live and performing well. There are no known bugs in the current deployment of either DNS or Content Distribution. We have had some issues with Consul, which we’re working to resolve. But in the main our focus is on service enhancements (which is informed by a feedback loop of live data), and moving to a self-service footing.

Question 13

As I understand, with CDN the Gateways are the nodes in the network that have the contact with the clients. Will it be like that for all other services as well? Or will there be services where the hosts and clients speak directly?

Arthur Mingard: As a general rule everything passes through a Gateway. This isn’t always a direct hand off however. It could be the channelling of a data stream. Because of the realities of mixed home and business environments, and the security that exists around these networks, it is not possible to consistently connect an end user to a Host without both parties first having software installed. This is the key barrier that the architecture of the Edge network solves.

Question 14

How many Gateways will be available for rentals and will self-onboarding available for Gateways?

Joseph Denne: All Gateways will be made available. And yes, but the timing for self on-boarding is not yet set.

Question 15

Does CDN utilise the OpenGL ES API? If not, will it be considered since it could increase the performance of image processing by using GPUs on ARM devices?

Adam K Dean: CDN doesn’t use OpenGL ES API at the moment since libvips has no direct support for GPU processing, but we’ll consider using it when we add video streaming.

Question 16

Is Eduardo Bouças still with us? Just saw he contributed in Stackbit repo. Is this project affiliated with EDGE Network somehow?

Joseph Denne: Eduardo has stepped down from his role on Edit and has now left the company. His work has been picked up by Jean-Luc, who has been doing a brilliant job moving the platform forward. On which, I had planned to share the new interfaces with the community this week, but the issue we saw with Consul in the week had the effect of pushing that back. I’ll be sharing them next week instead.

Question 17

How are things going with edit dot com? Bringing in any traffic yet to the network?

Joseph Denne: It’s moving and starting to bear fruit. 2020 is shaping up well, but as ever there’s a lot to do! Everything that runs on Edit touches the network to a greater or lesser extent. There are components of functionality that are directly integrated. And we look to the future we expect the core data layers to move to the network as well.

Question 18

Are there plans for building a product for affiliate and/or drop shipping sites, similar to Shopify?

Arthur Mingard: Not at the moment, although you could our tools to create one if you wanted!

Question 19

How will Edge increase its reach in regions outside of central Europe and North America? Is there a long term strategy for getting more people to run hosts?

Arthur Mingard: This completely comes down to supply and demand. If there is end user demand in those regions, it will ultimately be profitable to run a node in them. We have loads of ideas for driving adoption where there is a shortfall but where there is also demand however.

Question 20

What is the benefit of having hosts in the current CDN system considering that - a) the average host upload speed is probably less than the average download from source speed, making cache misses very slow - b) On cache hits: having a 1TB SSD disk cache on the gateway is orders of magnitude faster than retrieving from 100 hosts over the wire - c) modern GPU software can decode, down sample, encode up to 100-1000 images per second, making a GPU-based transformation pipeline on the gateway much more efficient than relying on comparatively slow image transformations on the hosts.

Arthur Mingard: In our benchmarking, GPU based transformation is nowhere near as fast for image manipulation. The only way to build a scalable decentralised model is to distribute work, which is why Hosts resolve the requests and return them to Gateway, rather than a single Gateway doing all of the work, which is a huge bottleneck. On scale, a single Gateway with 250 hosts connected will manage far more requests than if the Gateway were to have to request all of the source content and process it.

Question 21

The current process of node update for Founding Nodes is extremely complicated, involving shipping of sd cards (which could be corrupted) or having ports open on routers and expecting a remote update. Is there a plan to release a self-diagnostic tool or a document for us to download and fix our Founding Nodes, if it’s not reported as online in the dashboard? Want to avoid losing on $EDGE over extended periods of time.

Joseph Denne: It’s not really that complicated, but I accept that’s it less than optimal. Unfortunately, it’s a facet of the Founding Node being the first in-home and in-business device on the network. The flip side of this is the gauntleted base payment for Founding Nodes. So yes. we’d like it to be simpler… it take time this side as well. But it’s a question of priorities and slotting it in to what is already a packed roadmap.

Question 22

When will I be able to host a website on edge? Is it going to be Serverless only or docker based containers?

Adam K Dean: You can cache an entire site for delivery under your own domain via Content Delivery in the network now. Running a site in full in the network will need to wait for the Edge Compute service.

Question 23

If I want to trial a site on edit.com, how do I go about this?

Adam K Dean: Drop an email to hello@edit.com and we’ll follow up with you directly.

Question 24:

Where is the promised roadmap update for the next 18 months?

Joseph Denne: The road map update is ready, and has just been held back to allow us to sequence in new requirements, such as the need to replace Consul. We expect to be releasing it ahead of Christmas. (As a side note: we don’t make promises. We give indications of expected timings for in plan deliveries. I hate the use of this word in crypto – it’s misleading; ignores the realities of product development and R&D; and contributes to the general background of FUD that is holding the whole sector back.)

Question 25

Who is VP Strategy & Business Development?

Joseph Denne: My co-founder, Chris Mair.

Question 26

I assume that there’s not enough gateway spots available to fill the list of people who want a Gateway. While the list may not be up to date, can we know till how long the 50,000 deal will be honoured, and if it will work based on lottery who gets a Gateway?

Joseph Denne: Timings are not set yet. We’ve still not decided how best to choose Gateway stakers equitably. There’s the option of the lottery which isn’t the fairest, there’s also FCFS which disadvantages fairly newer members of the community. We’re waiting until we achieve a certain level of utility that will allow them be profitable for the stakers. We have also been thinking about the potential for creating a community stargate that will those who don’t get into Gateways can stake with.

Question 27

Why doesn’t the dadi.tech website redirect to edge.network?

Joseph Denne: Good spot. And a quick fix. Resolved.

Question 28

What are your ambitions for edit.com? Do you see it moving to a more self-onboarding model like Prismic? Prismic is really lovely to use. Feels like edit.com could play in this space if you want / produce excellent documentation like Prismic.

Joseph Denne: Edit is currently targeted at SMEs and Enterprises, and provides an incredibly flexible and fast development and editorial experience. We’re also working on a self-serve proposition, but there are no timings available for this yet.

Question 29

Is it the gateway or host that downloads the original image to the CDN transformed from the source?

Arthur Mingard: Hosts handle this portion of CDN operation.

Question 30

Will Gateway’s be self onboarded or rental of existing ones?

Arthur Mingard: In the first instance, rentals, like Stargates. Why? Because we’re early in the lifecycle of network roll out and we need to ensure performance. As the network grows in scale and in usage, and the tools needed to fully monitor the performance of self on boarded Gateways become available, we will open this up.

Question 31

How is the new video going? (mentioned after launch new website)

Joseph Denne: Held back to allow us to focus on more pressing items such as the Edge Console. We’ll get to it, but it won’t be until we’re on a full self-service footing with Content Distribution and we can ensure that the messaging is in line with the service offer.

Question 32

Having a timing-allow-origin headers set in CDN response would be a nice-to-have. Any chance this could be enabled? (as an option switch in dash for example)

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Timing-Allow-Origin

Arthur Mingard: All source headers are passed through CDN so if you’d like to use this, just define it on the origin. It wouldn’t be sensible to CDN to assign this as it could conflict with headers from the origin.

]]>
Mon, 09 Dec 2019 17:00:00 +0000 5dee79633eaeb31e32123efb Bolaji Oyewole 2019-12-09T17:00:00+00:00
Weekly Update: W/C 25th November, 2019 https://edge.network/updates/announcements/weekly-update-wc-25th-november-2019 Hi everyone 👋

We passed the 400 node barrier 💥

Image

It’s awesome to see the growth of the network, and I’m loving some of the photos being shared by the community.

Equally the effort put in in the creation of bots to enhance the ecosystem has been fantastic to watch. We’re working to better support this sort of thing, through access to APIs and of course through the opening up of the code base, which remains high on our agenda.

The network team has been refactoring the DNS service on Stargate to use EDNS, a method which introduces location lookup to queries and improves accuracy.

If you like a technical read, this one’s for you: https://tools.ietf.org/html/rfc6891

This is an exciting development, which will help the network move beyond BGP, speeding up performance and building a far higher level of intelligence in to the core routing layer of the network, enabling real-time routing decisions at a top level based on network telemetry.

Work on a Disk Cache solution designed to significantly reduce the load on Gateways memory kicked off. This is vital as we continue to onboard devices and begin to scale up customer use of the Content Distribution service.

The team have also spent a block of time working on the migration requirements ahead of moving some of the current backbone devices. We’re reshaping the network slightly for better distribution of capacity.

And work has been ongoing on the Edge Console, which is progressing nicely.

A range of small bugs within the editorial interfaces have also been squashed, tightening up performance and the overall user experience.

We’ll be sharing the new editorial interfaces with you next week.

Youseff wrote about the conversion of the CDN service to Go.

The tech is an amazing piece of work. You can read about it in detail here: https://edge.network/en/updates/network/go-content-distribution/

Finally, we will be holding our next AMA on Friday the 06th. Arthur and Jean-Luc will be attendance, so get your questions in now.

And that’s it for this week.

Enjoy your weekends.

]]>
Mon, 02 Dec 2019 14:30:00 +0000 5de535ad36e2c61e60c3e778 Joseph Denne 2019-12-02T14:30:00+00:00
Go Content Distribution https://edge.network/updates/network/go-content-distribution As you may already know, the first app to be released to the Edge Network was CDN, a web service written in Node.js which predates the network in its current form, and that has been used in production for over five years by a wide range of content providers, from low traffic microsites to international publishers. It’s easy and quick to configure and comes packed a wide range of modifiers that provide comprehensive *just in time* asset manipulation, as well as CSS and JS compression. Its ability to rapidly crop, resize, compress, and reformat images and quickly serve cached resources made it ideal for an increasingly mobile consumer that expects high quality visuals with minimum delay.

Image

When we first rolled it out into the network there was very little to change. It already had multi domain support, meaning a single instance could exist on each Host but be configured to support each domain slightly differently, with different sources, caching, and header configurations. CDN originally required its config to be stored on disk, so we added an endpoint that allowed it to be configured without a restart. If you’ve been running a Host in the last year, you’ve been successfully running a very powerful app that we are hugely proud of.

Those of you that have been paying close attention to the impact on your system may have noticed one rather glaring problem with CDN: it’s huge. Well, that’s not strictly true. It’s ~500mb on disk, which isn’t aggressively large, and certainly doesn’t affect operations when running on a single machine, however in a network where a high level of distribution is required, and where available disk space per device is relatively low, it can become a limiting factor.

When we release a new build of CDN, 390+ devices globally all need to pull a 0.5GB docker image. That’s 190GB from the registry for every release, and that just doesn’t scale. And on a 16GB machine that’s ~3% of the entire volume just for a single application. Those who have followed the more technical posts from the team will know that the network applications are primarily written in Go, and those that have been following the project since the early days may remember that the original prototype for the network was actually written in Node.js. Switching to Go wasn’t a difficult choice. It’s stricter, faster and more efficient than Node, and the resulting binary is far smaller.

Much of the logic from the Node.js build of CDN was so clearly written that the run up to rebuilding it in Go was far less daunting than we originally expected.

Feature migration

The Node.js build of CDN was built for a very different production environment, so migrating to Go was the perfect juncture to take stock of which features to carry over and which to drop. Here’s a short breakdown.

Features to carry over

  • Multiple domain configuration
  • Remote file source for image
  • Remove file source for CSS and JS transformation, as well as all other filetypes

Features to drop

  • SSL support, which is now handled at Gateway level
  • S3 file source which will be replaced by Object Storage in the network
  • Local filesystem source
  • Redis cache which is replaced by Gateway-level caching

Keeping it all production ready

We’ve built Content Distribution using the same logic as the previous version without introducing any breaking changes. Whilst the majority of users to the network are using our technology for the first time, there are a number of existing clients that have opted to migrate from their Cloud hosted CDN to the network, so it was of vital importance that the move was pain-free and invisible to the end-user.

The benefits of writing in Go

A single compiled binary

As mentioned earlier using Go to rebuild Content Distribution wasn’t a difficult choice, especially when we knew how significant the reduction in binary size would be. The newly released docker image is currently ~140MB, a number we intend to half over the next few releases. To put that into perspective, the Node.js build was ~568MB. This reduction in largely down to Go being a compiled language. In addition to the reduction in filesize, we are also seeing a 30-50% reduction in response times. The significance of a 100% increase in performance is a great achievement, and one we’re really proud of.

Image

Reducing potential for errors with Go

Compared to JavaScript, Go is a typed and strict language. At compile-time, the Go compiler detects problems that JavaScript only sees during runtime. This also allows for the earlier detection of errors that in turn speeds up development and protects the deployment process.

Better resources utilization

Node.js uses a single CPU core so to take full advantage of a multi-core system, we rely on process multiplexers such as PM2. In Go the default application behaviour is to use all available CPU cores, and optional limits can be set with the environmental variable GOMAXPROCS .

Node.js has a performant event-loop which helps maximize CPU utilization, but that comes at a cost. Go on the other hand uses goroutines – simple, scaleable subroutines for concurrent and parallel workloads, without the high resource utilization.

Benchmarking in Numbers

Go

Thread Stats Avg Stdev Max +/- Stdev

Latency 564.91ms 406.14ms 1.93s 70.07%

Req/Sec 7.72 5.21 20.00 75.25%

Latency Distribution

50% 475.04ms

75% 781.49ms

90% 1.16s

99% 1.85s

1295 requests in 30.04s, 32.77MB read

Socket errors: connect 0, read 0, write 0, timeout 1

Requests/sec: 43.11

Transfer/sec: 1.09MB

Node.js

Thread Stats Avg Stdev Max +/- Stdev

Latency 636.30ms 415.39ms 1.99s 74.95%

Req/Sec 4.28 3.67 20.00 77.07%

Latency Distribution

50% 555.32ms

75% 735.90ms

90% 1.26s

99% 1.93s

519 requests in 30.04s, 55.79MB read

Socket errors: connect 0, read 0, write 0, timeout 64

Requests/sec: 17.27

Transfer/sec: 1.86MB

What’s next?

We will be moving to use gRPC for communication between the Content Distribution app and Gateway, which will allow us to considerably increase our network performance by reducing latency and concurrent http/2 connections. We’ll also be further optimizing modifier performance and introducing new modifiers for video, audio, and live streams.

]]>
Wed, 27 Nov 2019 14:30:00 +0000 5ddd9eb78429dd1e468543ab Youssef Bouhachem 2019-11-27T14:30:00+00:00
Weekly Update: W/C 18th November, 2019 https://edge.network/updates/announcements/weekly-update-wc-18th-november-2019 Hi everyone 👋

We’ve had good news on the new business front this week, with a new engagement confirmed today.

The network team have been working on a series of refactors for the queue process, adding some additional clarity and a rebalancing in the repetition of jobs, increasing overall performance.

Cache pre-warming methods have been added to Content Distribution. This is a system that Gateway uses to re-issue requests for items that are heavily accessed during their time in the cache, meaning that they are in effect persisted in the cache in a way that removes the first load delay for new objects entering in to the service layer.

This is seriously cool tech, and a really powerful and unique feature.

Arthur will be writing more about this in the coming week or so.

The Content Distribution service has been extended to include a header to identify whether a result was created by a user triggered action or by the pre-warming system.

The team also added functionality that enables the pre-warmers ability to persist an expired cached object until it has been successfully re-warmed, removing the potential for service outage and ensuring consistent object delivery.

A new command was introduced to edge-cli, check, which performs a series of connectivity and system checks to make sure the Device is fully capable of delivering as a Host.

You can read more about this here: https://edge.network/en/knowledge/network/edge-cli-connectivity-check/

And that’s it for now!

Enjoy your weekends.

]]>
Mon, 25 Nov 2019 14:30:00 +0000 5ddc05283eaeb31e32123ef2 Joseph Denne 2019-11-25T14:30:00+00:00
edge-cli: Connectivity check https://edge.network/knowledge/network/edge-cli-connectivity-check Whilst the benefits of an increase in geographic spread, machine classification and processing architecture are already being felt by the network, this hasn’t come without its fair share of issues. The devices themselves require a little preparation in the form of the installation of an OS, and of Docker.

Docker is widely supported, but making sure the machine has the correct version with the right permissions can take a little technical expertise. Firewall configurations can on occasion pose an issue, although this is often easier to diagnose.

Whilst we can’t provide support for third-party applications, to help our community members detect common operational issues, and to help us diagnose unhealthy nodes we’ve added a new check command to edge-cli which provides a clear list of requirements and performance tests to cover the majority of common snags, and we’ll be rolling out a lot more over the coming months.

Update your current version

sudo apt update
sudo apt install edge-cli

Running the check command

Image

sudo edge-cli check

Image

Docker setup check

A failed docker setup check is a sign of improper docker installation.

DNS resolution check using system default resolver

A failed DNS resolution check using the system default resolver is a sign you might want to change your system resolver. Perhaps setting it to 8.8.8.8, or 1.1.1.1

DNS resolution check using the alternative resolver (1.1.1.1)

If DNS resolution using the system default resolver fails and this passes, we suggest changing your system DNS resolver to 1.1.1.1

Reachability check

A failed reachability check is a sign there might be a firewall setting that is blocking access to the network.

Latency check

A failed latency check is a sign to check your network setup, perhaps changing from wireless to wired, or moving closer to the router. It’s also a sign that with the current network setup your device may be less competitive than others in term of request dequeuing.

-

See a check missing? We’d love to hear your experience with device onboarding and look forward to introducing more helpful diagnostics tools to keep your Host healthy.

]]>
Fri, 22 Nov 2019 22:00:00 +0000 5dd867474f60661e371d3e6c Meher Assel 2019-11-22T22:00:00+00:00
Weekly Update: W/C 04th November, 2019 https://edge.network/updates/announcements/weekly-update-wc-04th-november-2019 Hi all 👋

A short and punchy update this week.

I wrote an article on the future of IoT and Edge computing. You can read it here: https://www.gigabitmagazine.com/cloud-computing/iot-will-unlock-potential-cloud-edge-and-blockchain

The network team have been working on diagnostics and benchmarking commands for CLI.

They’ve also been working on the Go version of Content Distribution, which is currently in heavy testing.

On the back of testing we’ve been refactoring and improving Content Distribution, and have found and resolved a number of bugs, including some that relate to extension case, hot config reload and filetype detection.

Work has been ongoing with the new Edge Console which is due to launch early 2020.

And a number of updates to DNS on Stargate have been made.

We’ve also made a number of updates to our roadmaps for the next 12-18 months, which we’re close to finalising for release.

We’ve had s series of great customer discussions this week, including a number of new business opportunities.

And we’ve moved to production testing of content distribution with a key customer. I should be able to share more on this in the run up to Christmas.

And that’s it for this week!

Enjoy your weekends.

]]>
Mon, 18 Nov 2019 14:30:00 +0000 5dd4327d0aeb751dffc67ee3 Joseph Denne 2019-11-18T14:30:00+00:00
Weekly Update: W/C 04th November, 2019 https://edge.network/updates/announcements/weekly-update-wc-04th-november-2019 Hi everyone 👋

First up this week: we’ve released a Beta version of the self-serve interfaces for Content Distribution 💥

You can access this now under the interfaces on my.edge.network and take the technology for a spin.

This includes access to the full imaging pipeline in the platform. We’ll be adding documentation for this to the main site soon.

The Beta gives you access to spin up the service for a single domain, with an additional domain unlocked for every device that you’ve contributed to the network.

The team have also been working on connectivity tests within Edge Cli.

And they’ve put the finishing touches on to the new Go version of the Content Distribution app, which will be deployed to mainnet next week.

The required builds for this have been prepared. And the required methods in the package that we use for caching – big-cache – have been created. This makes it possible to introduce conditional cache pre-warming on Gateway based on the frequency that a cached resource is accessed.

Which is a fantastic bit of tech.

We’ve released a series of performance improvements for Gateway, which includes a bi-directional RPC stream. This means a reduction in required concurrent http/2 connections and a reduction in latency between Host and Gateway.

There’s an outreach article that covers this, for those who fancy reading the more technical details: https://edge.network/en/knowledge/network/gateway-bi-directional-streaming/

And that’s it for now!

Enjoy your weekends.

]]>
Mon, 11 Nov 2019 14:30:00 +0000 5dcadeb17dcfb81e19265992 Joseph Denne 2019-11-11T14:30:00+00:00
Content Distribution: self serve available now https://edge.network/updates/announcements/content-distribution-self-serve Registered users have been given access to the configuration page for Content Distribution, available in the menu on https://my.edge.network. The Beta Free tier starts with a single zone, plus one additional zone per contributed network node. You’ll get a healthy 2GB data transfer, along with 5m requests per month and it’s really easy to get started.

Image

You’ll need to tell us the domain (and subdomain if applicable) you’d like to use, the source url for your images and optionally that of your assets. We’ve set out some TTL options, so you can choose whether you want a longer cache for quicker responses, or a shorter one for content that’s likely to change frequently.

The Free Tier is running the latest version of Content Distribution, replicated globally, and will not receive direct support. As previously mentioned, this is a Beta, so uptime and availability are by no means guaranteed, although we’ll do our very best to reach out ahead of each major deployment. For those of you wishing to discuss the Enterprise option, we’ve also included a contact link.

Enjoy.

]]>
Fri, 08 Nov 2019 18:00:00 +0000 5dc5a25a36e2c61e60c3e75e Arthur Mingard 2019-11-08T18:00:00+00:00
Gateway Bi-Directional Streaming https://edge.network/knowledge/network/gateway-bi-directional-streaming This week marks the release of a significant refactor as part of a focussed round of efficiency improvements set out to target the low level communication interfaces used by Gateway and Host. This specific change comprises of a change to the RPC (Remote Procedure Call) protocol used to distribute requests and receive responses from Host. RPC uses the client-server model to define procedures which suspend the caller whilst executing on the receiver. It allows two machines to perform a single synchronous operation when connected over a network where latency is otherwise a crippling factor.

In the last round of improvements we significantly reduced the number of RPC connections between Gateway and Host by streaming requests to the Host rather than sending one-by-one. This reduced latency, but didn’t affect the way Host returned payload to Gateway, which still required a connection per response.

In this release we removed the response method and adjusted the protocol that streams requests to the Host to also accept a stream of responses back, reducing the entire process to a single stream. This is made possible through gRPCs support for bi-directional streaming which takes advantage of the multiplexing capabilities of the http/2 protocol whilst structuring payload with Protocol Buffers, a serialization method with its own markup interface definition language.

In slightly less technical terms, this allows Gateway to asynchronously send unfulfilled requests to each Host whilst simultaneously receiving responses, all over a single connection. This reduces the number of open connections to the Gateway, removes the latency caused by establishing a new http/2 connection and only requires headers to be sent once.

One obstacle when moving to a fully bi-directional stream was migrating the concurrent request limiter method from Host to Gateway. With a single stream it was possible for a Host to suspend the RPC method by limiting the number of asynchronous operations using Channel Buffering. When the stream is bi-directional, the logic which previously provided a strict coupling between the RPC method and Hosts local request buffer became too fragile, so it was moved to Gateway.

The new logic stores the number of requests that are reserved by the Host, placing a temporary pause on the RPC method until the Host has satisfied enough requests to fall below the threshold. As some of you may have noticed last week, there were some teething problems with the first release. Under certain conditions where latency from one Host causes the request to be reassigned to another, Gateway was not successfully deducting the count of requests from the first, eventually causing the RPC method to indefinitely lock. After a quick refactor I can now report that the process is working as expected and having a significant positive effect on latency and a reduction in Gateway CPU usage.

]]>
Fri, 08 Nov 2019 12:30:00 +0000 5dc55cf607b2421e1ea48798 Arthur Mingard 2019-11-08T12:30:00+00:00
Weekly Update: W/C 28th October, 2019 https://edge.network/updates/announcements/weekly-update-wc-28th-october-2019 Hi everyone! 👋

It’s been another jam packed week here 🏃‍♂️

Earnings for October were pushed to network dashboards, with Stargates returning an average annualized ROI of 18.48%, and Hosts returning an average annualized ROI of 33.83%.

Daily earnings have been enabled in the network, with earning data now pushed to network dashboards once every 24 hours.

You can read about this in more detail here: https://edge.network/en/updates/network/daily-earnings/

API version 6 was released: https://github.com/dadi/api/releases/tag/v6.0.0

There are a whole world of updates and enhancements in this release, and we’ll be writing about them next week.

Publish version 3 was released: https://github.com/dadi/publish/releases/tag/v3.0.0

Note that this release requires API version 6.

The web service team continued work on improvements to the rich editor in Edit. Among other updates they implemented nested lists, improved keyboard navigation and the behaviour of the link prompt.

They also spent time planning for the implementation of third party data sources within the dashboards of our editorial interfaces.

The team continued work on the Edge Console framework, including closing out account login, creating a datasource plugin similar to that used in Web, and kicking off work on the Edge Console API layer.

And they continued work on Edge CLI connectivity tests and the integration of benchmarking into Network CDN for Host.

The team also integrated Slack reporting into Earnings service for internal monitoring.

A series of earnings service improvements were made along with a few bug fixes.

We’ve seen good growth in Hosts since multi-device onboarding kicked off, with some new nodes in far-flung spaces!

We remain on track to have interfaces for content distribution in open Beta for the community around the second week of November (TBC). We’re currently integrating front end templates, which will be released under my.edge.network in the first instance.

Finally, I expect to be releasing an update to our roadmaps next week.

And that’s it!

Enjoy your weekends 😊

]]>
Mon, 04 Nov 2019 14:30:00 +0000 5dc51985d7531f1e14772493 Joseph Denne 2019-11-04T14:30:00+00:00
Daily Earnings https://edge.network/updates/network/daily-earnings This month we have been trialing daily earnings calculations. Every day, in the early hours of the morning, while we sleep soundly in our beds, our robotic underlings diligently calculate the earnings for the previous day for every staked Host, Gateway, and Stargate on the network. While trialing the daily earnings process, we have kept the data for October hidden from users, but it is now visible for all to see.

Image

Daily earnings will be visible as they are calculated from this point on. The process kicks off around 02:00 UTC every day, and your earnings will be visible anytime thereafter. Payouts will remain monthly for now.

If you haven’t set your wallet address in your account be sure to do so right away, by going to my.edge.network/account/wallet in order to avoid missing the payout window for this month.

Masternodes

We are happy to announce that we now have 6 Stargates staked, and they have been active for the entire month of October. We’d like to thank our community for their ongoing support. We are now in the process of getting the Gateways ready for community staking, and will have more information on that for you soon.

We are happy to report that Stargates are currently running at an annualized ROI of 18.48%.

Hosts

We now have an incredible 311 staked and operational Hosts in the network. Since we announced Multi-device Onboarding last week, our community been busy adding devices to the network from all around the globe. Just look at the increase in Hosts below:

Image

Hosts are currently running at an annualized ROI of 33.83%, earning an average of 166 $EDGE each this month.

The amount Hosts earn per day depends on a number of factors, including the number and value of jobs completed, which in turn is driven by demand in the network. Because of this your earning will differ from the average, as well as between nodes (if you are running more than one). We are continuing to build out telemetry services, integrating more metrics and extracting more information from the network data available. We will be exposing more of this data as we go, and expect to be able to show a count of jobs completed over time in the near future.

What to expect in November

You can expect to see your earnings reported every day in November. And payouts for October will be completed next week. Remember, if you haven’t set your wallet address in your account be sure to do so right away, by going to my.edge.network/account/wallet. If you do miss this months payout window, not to worry, you’ll simply receive the earnings next month.

]]>
Fri, 01 Nov 2019 16:00:00 +0000 5dbc5320c8ada71e0f37ebfa Adam K Dean 2019-11-01T16:00:00+00:00