Relay Mining: Incentivizing Full Non-Validating Nodes Servicing All RPC Types

2305.10672

YC

41

Reddit

1

Published 4/30/2024 by Daniel Olshansky, Ramiro Rodr'iguez Colmeiro

🌐

Abstract

Relay Mining presents a scalable solution employing probabilistic mechanisms, crypto-economic incentives, and new cryptographic primitives to estimate and prove the volume of Remote Procedure Calls (RPCs) made from a client to a server. Distributed ledgers are designed to secure permissionless state transitions (writes), highlighting a gap for incentivizing full non-validating nodes to service non-transactional (read) RPCs. This leads applications to have a dependency on altruistic or centralized off-chain Node RPC Providers. We present a solution that enables multiple RPC providers to service requests from independent applications on a permissionless network. We leverage digital signatures, commit-and-reveal schemes, and Sparse Merkle Sum Tries (SMSTs) to prove the amount of work done. This is enabled through the introduction of a novel ClosestMerkleProof proof-of-inclusion scheme. A native cryptocurrency on a distributed ledger is used to rate limit applications and disincentivize over-usage. Building upon established research in token bucket algorithms and distributed rate-limiting penalty models, our approach harnesses a feedback loop control mechanism to adjust the difficulty of mining relay rewards, dynamically scaling with network usage growth. By leveraging crypto-economic incentives, we reduce coordination overhead costs and introduce a mechanism for providing RPC services that are both geopolitically and geographically distributed. We use common formulations from rate limiting research to demonstrate how this solution in the Web3 ecosystem translates to distributed verifiable multi-tenant rate limiting in Web2.

Get summaries of the top AI research delivered straight to your inbox:

Overview

  • Relay Mining presents a scalable solution to estimate and prove the volume of Remote Procedure Calls (RPCs) made from a client to a server.
  • This addresses a gap in distributed ledgers, which are designed to secure permissionless state transitions (writes) but lack incentives for servicing non-transactional (read) RPCs.
  • The solution leverages digital signatures, commit-and-reveal schemes, and Sparse Merkle Sum Tries (SMSTs) to prove the amount of work done, introducing a novel ClosestMerkleProof proof-of-inclusion scheme.
  • A native cryptocurrency and feedback loop control mechanism are used to rate limit applications and dynamically scale the difficulty of mining relay rewards.

Plain English Explanation

Relay Mining tackles the challenge of incentivizing distributed networks to handle non-transactional "read" requests, in addition to the more commonly addressed "write" transactions. This is important because many applications rely on these read requests, but the current distributed ledger systems don't provide good incentives for handling them.

Relay Mining's solution uses a combination of cryptographic techniques, including digital signatures, commitments, and a novel "closest merkle proof" scheme, to allow multiple independent service providers to prove the amount of work they've done in servicing these read requests. This is enabled through a native cryptocurrency that is used to rate limit applications and dynamically adjust the difficulty of earning rewards for servicing requests, based on network usage.

By leveraging these crypto-economic incentives, Relay Mining aims to reduce the coordination overhead costs and introduce a more geopolitically and geographically distributed mechanism for providing these essential RPC services, compared to relying on altruistic or centralized providers.

Technical Explanation

Relay Mining's solution employs probabilistic mechanisms, crypto-economic incentives, and new cryptographic primitives to address the challenge of incentivizing full non-validating nodes to service non-transactional (read) Remote Procedure Calls (RPCs) in a distributed ledger system.

The key technical components of the Relay Mining approach include:

  1. Digital Signatures and Commit-and-Reveal Schemes: These are used to prove the amount of work done by RPC providers in servicing client requests.
  2. Sparse Merkle Sum Tries (SMSTs): A novel data structure that enables efficient proofs of inclusion for the work performed.
  3. ClosestMerkleProof: A new proof-of-inclusion scheme introduced by the authors to support the SMST-based proofs.
  4. Native Cryptocurrency and Feedback Loop Control: A built-in cryptocurrency is used to rate limit applications and dynamically adjust the difficulty of mining relay rewards based on network usage, drawing from research on token bucket algorithms and distributed rate-limiting penalty models.

By leveraging these cryptographic primitives and crypto-economic incentives, Relay Mining aims to reduce the coordination overhead costs and introduce a more geopolitically and geographically distributed mechanism for providing essential RPC services, compared to relying on altruistic or centralized off-chain providers.

Critical Analysis

The Relay Mining paper presents a compelling solution to a significant challenge in the distributed ledger ecosystem. By addressing the incentive gap for servicing non-transactional read requests, the authors have the potential to improve the overall usability and robustness of Web3 applications.

However, the paper does not extensively discuss potential limitations or caveats of the proposed approach. For example, it would be valuable to understand the performance characteristics of the SMST data structure and the ClosestMerkleProof scheme, especially as the network scales and the number of RPC requests grows. Additionally, the authors could provide more insight into the potential security implications of the crypto-economic incentive model and how it might be vulnerable to manipulation or abuse.

Furthermore, the paper could benefit from a more in-depth comparison to related work, such as proof-of-useful-computation approaches and self-custody non-ledger-based solutions, to better contextualize the novelty and trade-offs of the Relay Mining approach.

Overall, the Relay Mining paper presents a promising solution to a significant challenge in the distributed ledger ecosystem. However, further research and analysis would be valuable to fully understand the practical implications and potential limitations of the proposed approach.

Conclusion

Relay Mining introduces a scalable solution to address the incentive gap for servicing non-transactional (read) Remote Procedure Calls (RPCs) in distributed ledger systems. By leveraging cryptographic techniques, crypto-economic incentives, and a novel ClosestMerkleProof scheme, the authors have developed a mechanism to enable multiple independent RPC providers to service requests from applications in a permissionless network.

The key innovation of Relay Mining is its ability to prove the amount of work done by RPC providers and dynamically adjust the difficulty of mining relay rewards based on network usage. This approach has the potential to reduce coordination overhead costs and introduce a more geopolitically and geographically distributed RPC service ecosystem, compared to relying on altruistic or centralized providers.

While the paper presents a compelling solution, further research is needed to fully understand the practical implications, performance characteristics, and potential security considerations of the Relay Mining approach. Nonetheless, this work represents an important step forward in addressing a significant challenge in the distributed ledger landscape and enabling more robust and scalable Web3 applications.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Related Papers

NotNets: Accelerating Microservices by Bypassing the Network

NotNets: Accelerating Microservices by Bypassing the Network

Peter Alvaro, Matthew Adiletta, Adrian Cockroft, Frank Hady, Ramesh Illikkal, Esteban Ramos, James Tsai, Robert Soul'e

YC

0

Reddit

0

Remote procedure calls are the workhorse of distributed systems. However, as software engineering trends, such as micro-services and serverless computing, push applications towards ever finer-grained decompositions, the overhead of RPC-based communication is becoming too great to bear. In this paper, we argue that point solutions that attempt to optimize one aspect of RPC logic are unlikely to mitigate these ballooning communication costs. Rather, we need a dramatic reappraisal of how we provide communication. Towards this end, we propose to emulate message-passing RPCs by sharing message payloads and metadata on CXL 3.0-backed far memory. We provide initial evidence of feasibility and analyze the expected benefits.

Read more

4/11/2024

Dynamically Sharded Ledgers on a Distributed Hash Table

New!Dynamically Sharded Ledgers on a Distributed Hash Table

Christoffer Fink, Olov Schel'en, Ulf Bodin

YC

0

Reddit

0

Distributed ledger technology such as blockchain is considered essential for supporting large numbers of micro-transactions in the Machine Economy, which is envisioned to involve billions of connected heterogeneous and decentralized cyber-physical systems. This stresses the need for performance and scalability of distributed ledger technologies. Sharding divides the blockchain network into multiple committees and is a common approach to improve scalability. However, with current sharding approaches, costly cross-shard verification is needed to prevent double-spending. This paper proposes a novel and more scalable distributed ledger method named ScaleGraph that implements dynamic sharding by using routing and logical proximity concepts from distributed hash tables. ScaleGraph addresses cyber security in terms of integrity, availability, and trust, to support frequent micro-transactions between autonomous devices. Benefits of ScaleGraph include a total storage space complexity of O(t), where t is the global number of transactions (assuming a constant replication degree). This space is sharded over n nodes so that each node needs O(t/n) storage, which provides a high level of concurrency and data localization as compared to other delegated consensus proposals. ScaleGraph allows for a dynamic grouping of validators which are selected based on a distance metric. We analyze the consensus requirements in such a dynamic setting and show that a synchronous consensus protocol allows shards to be smaller than an asynchronous one, and likely yields better performance. Moreover, we provide an experimental analysis of security aspects regarding the required size of the consensus groups with ScaleGraph. Our analysis shows that dynamic sharding based on proximity concepts brings attractive scalability properties in general, especially when the fraction of corrupt nodes is small.

Read more

5/27/2024

Proof-of-Learning with Incentive Security

Proof-of-Learning with Incentive Security

Zishuo Zhao, Zhixuan Fang, Xuechao Wang, Xi Chen, Yuan Zhou

YC

0

Reddit

0

Most concurrent blockchain systems rely heavily on the Proof-of-Work (PoW) or Proof-of-Stake (PoS) mechanisms for decentralized consensus and security assurance. However, the substantial energy expenditure stemming from computationally intensive yet meaningless tasks has raised considerable concerns surrounding traditional PoW approaches, The PoS mechanism, while free of energy consumption, is subject to security and economic issues. Addressing these issues, the paradigm of Proof-of-Useful-Work (PoUW) seeks to employ challenges of practical significance as PoW, thereby imbuing energy consumption with tangible value. While previous efforts in Proof of Learning (PoL) explored the utilization of deep learning model training SGD tasks as PoUW challenges, recent research has revealed its vulnerabilities to adversarial attacks and the theoretical hardness in crafting a byzantine-secure PoL mechanism. In this paper, we introduce the concept of incentive-security that incentivizes rational provers to behave honestly for their best interest, bypassing the existing hardness to design a PoL mechanism with computational efficiency, a provable incentive-security guarantee and controllable difficulty. Particularly, our work is secure against two attacks to the recent work of Jia et al. [2021], and also improves the computational overhead from $Theta(1)$ to $O(frac{log E}{E})$. Furthermore, while most recent research assumes trusted problem providers and verifiers, our design also guarantees frontend incentive-security even when problem providers are untrusted, and verifier incentive-security that bypasses the Verifier's Dilemma. By incorporating ML training into blockchain consensus mechanisms with provable guarantees, our research not only proposes an eco-friendly solution to blockchain systems, but also provides a proposal for a completely decentralized computing power market in the new AI age.

Read more

5/1/2024

📈

Replacing Cryptopuzzles with Useful Computation in Blockchain Proof-of-Work Protocols

Andrea Merlina, Thiago Garrett, Roman Vitenberg

YC

0

Reddit

0

Proof-of-Work (PoW) blockchains have emerged as a robust and effective consensus mechanism in open environments, leading to widespread deployment with numerous cryptocurrency platforms and substantial investments. However, the commonly deployed PoW implementations are all based on solving cryptographic puzzles. Researchers have been pursuing the compelling idea of replacing cryptopuzzles with useful computing tasks for over a decade, in face of the substantial computational capacity of blockchain networks and the global pursuit of a more sustainable IT infrastructure. In this study, we conduct a comprehensive analysis of the prerequisites for alternative classes of tasks. We provide insight into the effect of introducing usefulness and of transitioning to task classes other than cryptopuzzles. Having distilled the prerequisites, we use them to examine proposed designs from existing literature. Finally, we discuss pertinent techniques and present research gaps in the current state-of-the-art.

Read more

5/15/2024