Angler: Dark Pool Resource Allocation Deploy to multi-edge, multi-cloud, or multi-potato, securely.

github - paper - slides - contact

Dark pool resource allocation is a way to consume multi-provider infrastructure in a privacy preserving way, i.e. without broadcasting how many resources are requested and without providers needing to share how many resources they have. The goal is to support a “dark pool” of resources where no one knows how much each provider contributes to the pool and requests of the pool are fulfilled sharing the minimum amount of required information. The term dark pool comes from a dark market, or a market for which the order book is not visible to participants.

Angler is a system which supports resource allocation from dark pools. Angler manages the complexity of finding infrastructure providers nearby (in terms of network latency), and allocates Kubernetes namespaces on the providers platform based on the customer’s request.

How Does It Work?

Say you want to deploy an application on a managed Kubernetes service with 32 vCPUs and 32 GB of RAM.

1. Angler hashes the definition of the resource requested, i.e. a Kubernetes namespace with 1 vCPUs and 1 GB of RAM, 32 of which are required. This hash is looked up in the Angler DHT to find a list of providers nearby who offer Kubernetes namespaces at a granularity of 1 vCPU and 1 GB of RAM.
2. Angler runs a placement function under a cryptographic protocol called MPC. This hides the total size of the request (32 vCPUs and 32 GB of RAM) as well as the available capacity of the providers.
3. Finally, the requester and the matched provider(s) learn the output of the placement function e.g. allocate half on GCP at price X and the other half on AWS at price Y. The winning provider(s) provision the requested namespaces and the other providers learn nothing other than they were not selected to satisfy this request. The Angler Kubernetes Operator provisions these namespaces automatically and delivers credentials to the requester. At this point the customer is ready to deploy their application!

The steps above corespond to the numbers in this diagram.

Security Goals

Angler protects the privacy of infrastructure providers by hiding what they contribute to the global pool. At the same time, Angler protects privacy of requesters by only sharing request details, e.g. number of vCPUs, with provider(s) selected to satisfy the request. In order to keep these secrets, Angler leverages the WRK protocol from the AGMPC EMP library.

Kubernetes Integration

The “resources” allocated by Angler are in fact Kubernetes namespaces. Infrastructure providers each are expected to run a Kubernetes cluster and when a customer workload is assigned to them, the customer receives a namespace within their cluster. The namespace has an associated ResourceQuota to ensure the customer only uses what they requested and presumable paid for. Customers do not learn the total number of resources each provider contributes to the pool (how many vCPUs, GB of RAM, GPUs, etc). Only the matched provider learns the details of the request (price, how many vCPUs, RAM, etc). The other providers who were not matched learn nothing.

Disclaimers

Angler is a research prototype and is likely not suitable for production purposes. Payment between customer and infrastructure provider is out of scope of this work. Angler requires manually setting network coordinates of each node, which is required for discovery purposes.