Distributed Edge Computing: New Kid in Town
Medium-sized companies are being increasingly challenged to figure out how their product portfolios or operations can become smarter with the help of the IoT technology. A growing number of connected devices produce trillions of gigabytes of data per day (at the edge). They are facing bandwidth infrastructure limitations, growing costs for data transfer and cloud, but also increasing data privacy and security requirements.
We see a new type of software, which enables decentralized edge computing for IoT. Edge computing, data streaming, and the blockchain technology are the driving trends for decentralized edge computing. It seems to enable endless possibilities but also an immense complexity.
In this article, you can learn more about the benefits and challenges of decentralized edge computing and what technologies and companies we recommend you to watch.
Defining Distributed Edge Computing
Edge computing provides the functionality to locally run workloads on IP-enabled devices for processing, analyzing, and visualizing data at the edge. As opposed to a centrally located server, decentralized edge computing consistently delegates computational workloads to autonomous devices located at the data source. Control, access, data storage, or ownership are spread across a network of nodes that run decentralized applications. Clients can connect to any of these nodes.
In contrast to centrally located servers operated by a single company, decentralized systems consist of operators who hold a portion of the overall storage or computing, thus creating a resilient system. The goals of it are to be faster, safer, and more open. Such a system can be implemented in a blockchain-based application or a peer-to-peer-based network.
Before we discuss the advantages and disadvantages of distributed systems and technologies which enable it, let us picture a business-relevant use case.
In the manufacturing industry software systems are implemented to automate, track, monitor, and supervise manufacturing processes and materials on the production floor. These systems must understand complex conditions and make precise decisions to optimize and improve the production output. It is crucial that these systems are resilient. If the software is no longer working, then the production is standing still. This is the worst-case scenario for any manufacturer.
No matter how good a system is engineered, sooner or later, a defect of the software or hardware will eventually cause it to fail. This means that the system will not meet one or more quality requirements (e.g. capacity, performance, availability, interoperability, reliability, robustness, safety, security, and usability). This is where resilient software with no single point of failure creates a tremendous benefit.
Now, let us take a look at the benefits and challenges of distributed edge computing.
Benefit 1 | Higher performance: Device resources (e.g. storage or computation) can be utilized better through the automated distribution of workloads. This leads to a cost-effective investment in the on-premise landscape. Resource-limited devices can form a powerful network to perform computation-intensive tasks, there is no need to go back and forth with your central system.
Benefit 2 | Lower operational cost: The hidden costs of cloud resources are typically growing with more and more data being pushed into the cloud, even if it could have been stored locally. In a distributed network of devices where any device could fail without causing any disruption, operators can use low-cost devices without worrying about their long-term durability.
Benefit 3 | High flexibility: Distributed systems should be designed so that the coupling between modules is as loose as possible. This allows components to be replaced with alternative implementations which provide the same services. In times of constant evolution, it is beneficial when a system is less contained to a platform, language, operating system, or build environment.
Benefit 4 | Always-on system: In the event of a failure, there is no centralized system that can jeopardize the whole operation. Incoming data could be rerouted to a fully functional device. End users will be still able to access the applications, network issues will be less common.
Challenge 1 | Risk management: It is hard to mitigate risks in a decentralized system. In centralized systems, risks are easily understood, defined, and measured. From a security standpoint, data stored on multiple devices can be troublesome to secure. Therefore, a consistent risk management plan and well-organized central support are required.
Challenge 2 | Higher complexity: Decentralized systems are harder to set up and operate than centralized systems. It is not uncommon that in decentralized systems an increasing number of dependencies develop over time. The number of moving parts can become overwhelming and lead to higher complexity which requires more skilled personnel.
Challenge 3 | More personnel required: The risk management and higher complexity challenge require more personnel with good engineering and operation skills of distributed systems.
Challenge 4 | Data integrity: It is challenging to provide data integrity and to maintain consistency across the network of nodes. Partially completed transactions should not result in invalid data or inconsistencies within the data.
Technologies to Watch
In this section, we want to highlight interesting open-source technologies that are helpful in building a distributed edge:
libp2p | P2P Networking Stack: libp2p provides modules and libraries that enable the development of peer-to-peer network applications. Applications like IPFS are built upon libp2p.
Akka.NET | Distributed Actor Model: Akka.NET is a toolkit and runtime for building distributed event-driven applications with .NET & Mono. The original Akka framework is built in Java/Scala.
IPFS | P2P File Sharing: IPFS is a peer-to-peer network for storing and sharing data in a distributed file system.
OrbitDB | P2P Database: OrbitDB provides a decentralized database based on top of IPFS.
Ceramic | P2P Data Streaming: Ceramic provides a decentralized platform for data streaming.
Etherum-powered Dapps | P2P Applications: Dappps are applications that run on a decentralized peer-to-peer network.
Companies to Watch
In order to realize the benefits of distributed edge computing, companies often choose to develop their own systems from scratch. With no ready-to-use edge technologies to build upon, these systems are hard and expensive to build. However, start-ups are working on making this technology more accessible, with fewer implementation efforts and faster installation.
There are many companies which invest in technologies enabling distributed edge computing. We suggest watching the following three start-ups. We can also be sure that big players like Siemens are investing in the technology too.
Watch 3 | Crate.io, Inc.: Crate.io provides a distributed SQL database.
Watch 1 | Actyx AG: Actyx provides an platform to build and run distributed applications for the digital factory.
Watch 2 | ObjectBox Limited: ObjectBox provides tools for developers focusing on the Mobile and IoT industries with an edge database synchronization solution.
Distributed edge computing can definitely provide a viable extension to the currently dominant platform-as-a-service IoT model. The technology for building resilient message-driven applications is maturing and evolving. In the future, we will see more applications which require real-time data processing at the edge, and thus a growing number of easy-to-use tools and frameworks which will support developer efforts.