Applications and Architecture for Blockchain-style Messaging

In my previous article, I described third generation blockchain systems that will connect with the real world to do trading and ledger transactions at high volume. The enabling technology is “blockchain-style messaging.”

We can use ideas from blockchain ledgers to make a great messaging system. A blockchain-style messaging system will collect messages about a transaction, share them, and save the whole stream in permanent blockchain-style storage.

With blockchain-style messaging:

• All bits are publicly shared
• It’s easy to join and participate
• You get security and privacy through encryption
• You have a trustworthy record
• It’s easy to look at, debug, and audit any transaction
• It has the force of law in setting up agreements and transactions

Applications

Legal documents
You can unilaterally throw in documents and recordings for future reference — liability disclaimers, NDA’s, wills, easements, bills of sale, etc. The message store becomes a flexible digital signature system.

Simple agreements
We can make agreements. If I say that I will do something for you, and you say you will do something for me in return, that is legally a type of contract. We don’t need a smart contract. We just need a record of this exchange.

OTC transactions
One type of agreement is an “over the counter” trade, where two parties agree on a security and a trade by phone or email or some other communication. These message types can be attached to a description of a trade in a blockchain-style messaging system.

Banks settle trades by agreeing on delivery for payment
The core of settling a trade is asking banks to move money, IF the other guy’s bank delivers the securities. The banks can use blockhain-style messaging to negotiate this handoff, and deliver securities and money immediately. For example:

• “I have Y dollars to trade for X stock in Megacorp.”
• “I have the stock. I will transfer it to your correspondent account if you confirm your delivery of the money.”
• “OK. I transferred.”
• “OK. I transferred. Transaction finished. Send to block builder.”

Straight through processing
The instructions for settlement can come from a process called “Straight through processing”, where a securities buyer or seller takes a trade from an exchange and attaches the settlement instructions — the accounts where money and securities will be transferred. Here is a picture that compares the old messy bilateral way of doing this, with the blockchain way.

Deliver assets to a blockchain ledger
The buyer and the seller might add settlement instructions in the form of signed transactions for a blockchain ledger. Then, the blockchain messaging system would deliver the transactions to the blockchain ledger.

Regulatory monitoring
Regulators often ask for records from banks, online trading, and online gaming. With blockchain-style messaging they have a simple way to get reliable records in real time.

Using the system

I promised that it would be easy to join and participate in this system. We can make it easy to participate in this system by providing some shared infrastructure. A typical user can connect to the system and do a few simple things:

• Join by sharing public keys and routing addresses
• Receive and send messages with associated transactions
• Use encrypted message channels
• Look up past transactions and get the related data

Underneath this simple API, a lot is happening.

Requirements

Big Volume
We see near term applications that require 1M messages per second, and the system should expand to accommodate any number of messages. We can look at the example of US equities trading to see a typical financial market requirement. There are about 25M US equity trades per day, or about 1000 per second. This stream normally bursts up to 20K per second at open and close, and it could be as high as 100K per second. If we need 10 messages to settle a trade, we want to support 1M messages per second. The total data stored in this blockchain would be about two megabytes per second or 50 gigabytes per day. This is comparable to the volume handled by a big cloud-based message system such as Twitter, where the “firehose” runs at 6000 tweets per second and 100 GB per day. It is much higher than the volume of existing blockchain systems.

Distribution
We need to provide a routing system that will send individual messages to recipient addresses. We will also satisfy our mission of providing shared data with a “firehose” of all transactions and messages and publication of the final blocks.

Delivery guarantees
We want to be sure that if we put a transaction in, it goes into our records because it is marked as finished, or because it times out.

Private encrypted data
Participants should be able to exchange private, encrypted messages.

Key management
We will need a way to keep track of the public keys for each participant, so that we can send them private messages and verify instructions. Each participant will need to save secure private keys for reading private messages, signing instructions, and reading historical private information.

Real Names
We want to know what a real person or a real company said. We will want to know the real name associated with each public key.

Immutability
When we retrieve a published block, we should be able to show that it has not been changed. We can do this by looking at block hashes and Merkle hash trees saved in an immutable storage such as the bitcoin blockchain.

Infrastructure

We’ll need some shared SaaS infrastructure to deliver all of this to a simple API. The shared infrastructure will include components like:

* An endpoint that accepts new transactions and messages

* A router that delivers messages to the next participant

* A firehose that distributes all of the data, for the few participants that want it and can handle it

* A block builder that assembles and signs storage blocks containing transactions with their messages. This centralized block builder can run at a much higher speed than consensus-based systems like bitcoin. If you aren’t sure whether you trust the block-building service, you can audit it by observing the firehose and finished blocks for known messages. We can guarantee that data will not be changed, with the same level of security of bitcoin, by taking hashes from our big block store and putting them into bitcoin transactions.

* A SaaS service for storing blocks and looking up past transactions

Architecture

I won’t share all of the details of the architecture and data structures that we are working on. But, at a high level, we know how to build this kind of system from modern cloud components.

Public key encryption
We will use elliptical public key encryption for authentication, signing, and for passing encrypted secret keys and addresses.

Open source code
As a trusted, encryption-based system, it will be built on open source code that the user community can examine and improve.

Cloud Infrastructure
A high-volume messaging system is a creature of the current era of big data, not the previous era of P2P networks. It will require cloud infrastructure, and most participants will access it through centralized services.

Scalable number of message pipelines
We will use the on-demand cloud architecture to expand the number of message pipelines as needed.

Microservices
The infrastructure will evolve through continuous improvement of microservices.

A continuous integration testbed
A continuous integration testbed will be available for contributors to test modifications and integrations. We will deliver working code, not standards. This will allow the system to evolve at high speed compared with an older standards process or governance process.

Following up

Please contact me with your related ideas and applications. If you want to learn how my company, Blockchain Clearing Corporation, is organizing real time (TØ) delivery of US Equities, please come and see us at the ISITC meeting in Baltimore.