This is a comprehensive list of all things scalability. Skim the summaries and focus on what interests you most. If unsure, start from the beginning.
Introduction to ScalabilityIntro lecture [slides]
Sharding FAQ – I recommend reading about the first half of this Ethereum wiki article for a comprehensive understanding of the “scalability” paradigm, and to help build a mental framework for analyzing solutions, such as sharding. Explains the “scalability trilemma”. Read full FAQ for information about sharding, its attacks, and Ethereum’s implementation. – 1 hour read
The DCS Triangle – 7/10/16 – Describes a different scalability trilemma dubbed the “Decentralized Consistent Scale Triangle” and provides background on looking at tradeoffs that technology stacks face. This post focuses on BigchainDB, but does provide a big-picture view of the scalability paradigm. – 14 minute read
Scalability FAQ – Bitcoin wiki article containing “[a]nswers to commonly-asked questions and concerns about scaling Bitcoin, including ‘level 1’ solutions such as increasing the block size and ‘level 2’ solutions such as the proposed Lightning network”; pretty comprehensive, especially on block size theory, but doesn’t contain information about SegWit.
Information Propagation in the Bitcoin Network – 2013 – Comprehensive analysis by Decker and Wattenhofer on propagation rates, block creation rates and consequent fork rates in the Bitcoin network. This paper is significant for understanding scaling via changing block time. – 10 pages
GHOSTSecure High-Rate Transaction Processing in Bitcoin – 2013 – Original Sompolinsky and Zohar paper introducing the GHOST protocol, a modification to the “longest chain” selection algorithm of conventional blockchains in order to maintain security at faster block times. Includes analysis of decreasing block times in Bitcoin, security proofs, and implementation details. An accessible and informative (not to mention groundbreaking) academic work. – 31 pages
Toward a 12-second Block Time – 7/11/14 – Technical description of Ethereum’s GHOST variant, which differs from the original proposal by including a block reward scheme for uncle blocks. A more concise description of the protocol can be found in the respective sections of Ethereum’s whitepaper and design rationale. – 21 minute read
Block Size DebateBlock size limit controversy – Useful Bitcoin wiki article with compilation of arguments against and for a block size increase in Bitcoin, including related Bitcoin Improvement Proposals. – 5 minute read
BitFury: Block Size Increase – 9/6/15 – Goes through arguments for and against block size increase far more in-depth, including all the proposals as of September 2015. Includes some mathematical modelling; results from this study used to argue against increasing block size past 4MB. – 18 pages
On Scaling Decentralized Blockchains – 2/26/16 – Cornell study analyzing scaling bottlenecks and current solution proposals. Famous result from this study is that scaling block size up to (and not exceeding) 4 MB can be adopted by 90% of Bitcoin nodes securely; some contradicting results from the BitFury study. – 20 pages
Roger Ver vs Johnny Dilley – 4/26/17 – Heated debate on Bitcoin Unlimited (increasing block size) vs Segregated Witness. Very good representation of both sides; discussion of censorship, ideology, what gives Bitcoin value, r/Bitcoin, appeals to authority, and various social and economic factors involved in the debate. Less focus on technical details, though by no means ignorant of those details (you should have this knowledge before watching, see SegWit section below). – 2 hour video
Bitcoin Forks Chronology – The Ultimate List of Forks – 10/30/17 – Chronology of Bitcoin forks and extensive description of each.
Bitcoin Nodes Summary – Number of nodes running each Bitcoin implementation.
See slides for list of BIPs and forks.
Segregated WitnessPieter Wuille: Segregated witness and its impact on scalability – 12/14/15 – Bitcoin Core developer Pieter Wuille introducing the Segregated Witness proposal and answering questions. The original slides can be found here. A great compilation of resources and transcript can be found here. – 1 hour video
BIP 141 – 12/21/15 – Segregated Witness specification.
Segregated Witness, Part 1: How a Clever Hack Could Significantly Increase Bitcoin’s Potential – 12/19/15 – First of three-part series; highly recommended introduction to SegWit. Gives a rough technical explanation of Bitcoin transaction mechanics, and goes on to introduce the segregated witness soft fork in the context of the prior explanation. – 10 minute read
Part 2: Why You Should Care About a Nitty-Gritty Technical Trick – 12/22/15 – While the previous part explained how SegWit works, this part explains what it does. Describes how SegWit improves transaction throughput, and more importantly, the other advantages of SegWit, i.e., fixing transaction malleability, enabling Schnorr signatures, and making fraud proofs for SPV nodes possible. Draws a lot from the original Wuille presentation. – 7 minute read
Part 3: How a Soft Fork Might Establish a Block-Size Truce (or Not) – 12/29/15 – Describes the blocksize debate scene at the time, specifically the concerns of the “block-size progressives” and the “decentralists”, and the extent to which SegWit satisfies their respective concerns. Goes through the pros and cons of implementing a block-size-related hard fork. – 11 minute read
What Are Segregated Witness Benefits – 1/21/16 – Uniquely comprehensive information on advantages of SegWit (malleability fix, linear scaling, etc) and “who benefits” from each feature; written by Trace Mayer. – 11 minute read
SegWit: How We Got Here and What Happens Next – 7/13/17 – B@B presentation slides by Casey Rodarmor on the various proposals, BIPs, technical details, significant events/dates, and political contentions involved in the SegWit timeline as of July 13. You can find the corresponding video recording here – ~2 hours
SegWit, BitcoinCash: Technical details explained – 8/22/17 – Brief description of the split in the Bitcoin community between Bitcoin Core and Bitcoin Cash. Includes information about the BCC hard fork, and a detailed technical explanation of Segregated Witness and transaction malleability. – 12 minute read
The Power of Schnorr: The Signature Algorithm to Increase Bitcoin’s Scale and Privacy – 4/16/16 – Introductory explanation of how SegWit enables Schnorr multisignatures, how they work, and how they affect the size of the blockchain. – 7 minute read
Bitcoin Core Technology Roadmap – Schnorr signatures and signature aggregation – 3/13/17 – A slightly more technical description of Schnorr signatures, concerns about its use, and a helpful compilation of further resources. – 6 minute read
Why Schnorr signatures will help solve 2 of Bitcoin’s biggest problems today – 7/4/17 – A short blog post on Schnorr in the context of the scalability paradigm. Goes on to describe how Schnorr also relieves privacy and spam issues. – 6 minute read
Lightning, Raiden, and Plasma
- Lightning Network Whitepaper – 1/14/16 – Technical draft of the Lightning Network. The authors Poon and Dryja claim that Bitcoin’s scalability bottleneck is recording micropayments on the blockchain, and they propose pairwise “contracts” that would enable micropayments to be made off-chain. In their solution only the net transaction would be recorded on the blockchain. Using a chain of pairwise contracts the authors claim that the Lightning Network can offer extremely fast and light transitive payments. – 59 pages
- Fyookball Lightning Network Criticism – 6/26/17 – Fyookball claims that the Lightning Network would require a high degree of centralization, requiring banking hubs and ultimately going against the spirit of decentralized currencies. He then presents a case for why transitive pairwise transactions in the Lightning Network cannot scale. – 15 minute read
- Murch Lightning Network Criticism – 6/27/17 – In response to Fyookball, Murch criticizes Fyookball’s assumptions. Together with Fyookball’s argument offers a more nuanced mental model for how Lightning Network would work. – 5 minute read
- Lightning Network Simulation – 7/3/17 – In response to Fyookball and Murch, Reynolds created a simulation program for Lightning Network, specifically targeting the pairwise transactions Fyookball focused on. The simulation ultimately demonstrates that the network would scale. – 7 minute read
- Rusty Russell’s LN Blog – 3/30/15 – Excellent 3-part series on the Lightning Network. Russell works full time on Lightning Network the series covers revocable transactions, hashed timelock contracts, and channeling contracts. The bottom-up structure of the series moves from the simple “building blocks” to how they will construct the network.
- Bitfury’s Lightning Network Routing Proposal – 7/6/16 – “This paper describes a hybrid routing algorithm Flare, which could be used for payment routing in Lightning Network. The design goal for the algorithm is to ensure that routes can be found as quickly as possible. This is accomplished at the cost of each node proactively gathering information about the Lightning Network topology…” – 40 pages
- Raiden Network Overview – Raiden website summarizing its objective and current implementation details. Raiden does not have a whitepaper and appears to be following the lead of the Bitcoin Lightning Network team. The blog includes up-to-date information on Raiden’s ICO and major vision updates. Raiden Network documentation here.
- Plasma: Scalable Autonomous Smart Contracts – 7/11/17 – “Plasma is a proposed framework for incentivized and enforced execution of smart contracts which is scalable to a significant amount of state updates per second … Similar to the Lightning Network, Plasma is a series of contracts which runs on top of an existing blockchain to ensure enforcement while ensuring that one is able to hold funds in a contract state with net settlement/withdrawal at a later date.” – 47 pages
- Polkadot Whitepaper – Ongoing – “Chain for chains” that allows blockchain platforms to interoperate by using a central relay chain. Proposes a new set of actors and incentives to secure the relay chain and enable current blockchain platforms to adopt Polkadot.
- Chain-Fibers – Spring 2014 – Original concept for Polkadot. Provides the starting point of the idea with its original intentions and actors.
- Introduction to Polkadot: what it is, what it ain’t – 7/9/17 – Provides a series of comparisons between Polkadot and existing systems. This is an excellent resource if learning through examples/reference points is best for you. Written by the founder of Slock.it and former CCO of Ethereum. – 11 minute read
- Polkadot Gavin Wood Talk – 7/10/17 – High-level walk-through of the Polkadot protocol. – ~1 hour video
- Cosmos Whitepaper – Sept. 2016 – “The Cosmos network consists of many independent, parallel blockchains, called zones, each powered by classical Byzantine fault-tolerant consensus protocols like Tendermint. Some zones act as hubs with respect to other zones, allowing many zones to interoperate through a shared hub. The architecture is a more general application of the Bitcoin sidechains concept, using classic BFT and Proof-of-Stake algorithms, instead of Proof-of-Work.”
- Enabling Blockchain Innovations with Pegged Sidechains – 10/22/14 – The “sidechains whitepaper” from Blockstream; “We propose a new technology, pegged sidechains, which enables bitcoins and other ledger assets to be transferred between multiple blockchains. This gives users access to new and innovative cryptocurrency systems using the assets they already own. By reusing Bitcoin’s currency, these systems can more easily interoperate with each other and with Bitcoin, avoiding the liquidity shortages and market fluctuations associated with new currencies…” – 25 pages
- Sidechains Explained – 10/10/16 – Introductory explanation (first two minutes) and forking and security issues behind sidechains (last two minutes). – 4 minute video
- Understanding Blockstream – 1/12/17 – Recommended summary if limited on time or need more information understanding Blockstream; includes interview with the creator of Blockstream. – 8 minute read
- Drivechain – The Simple Two Way Peg – 11/24/15 – “Agenda: (1) Explain the principles of the sidechain-concept, and what problem must be solved to make sidechains possible. (2) Explain how an SPV proof works. (3) Propose [original] SPV proof (Drivechain), and explain the relevant economic and technical tradeoffs. (4) FAQ at the end” – 45 minute read
- Explanation of what an OP_RETURN transaction looks like – 1/6/15 – A Stack Exchange thread on the history of the OP_RETURN transaction and its role in the SPV proofs of sidechains.
- Why Every Bitcoin User Should Understand “SPV Security” – 5/28/17 – A step by step introduction to explaining how sidechains and SPV can be used to solve the current issues regarding scalability. – 5 minute read
- Purse Proposal Touts Extension Blocks as Bitcoin Scaling Solution – 4/4/17 – Simple reading on the history and the future of sidechains and explains “extension blocks”, a combination of sidechains and Lightning. – 4 minute read
- How Two New Sidechains Proposals Could Change Bitcoin’s DNA – 11/4/16 – Explains the differences between drivechains, a new method of sidechains, and generic sidechains and the potential issues with either method. – 6 minute read
- Sidechains, Drivechains, and RSK 2-Way peg Design – 4/5/16 – Summary of two-way pegs and how they can be implemented; accessible read with useful diagrams. – 9 minute read
- See Sharding FAQ in introduction
- Programmer explains Ethereum Sharding – 5/10/17 – For sharding, begin watching at 7~ minutes as the beginning discusses other methods of scalability while the second half explains sharding in general. Highly recommended for those interested in how sharding can be used in Ethereum. – 15 minute video
- Blockchain Sharding, and correlations with Proof of Stake with Vlad Zamfir of Ethereum – 6/5/17 – A very specific and in-depth discussion about the relationship between sharding and proof of stake. Provides information on what may happen when Ethereum switches to proof of stake and how sharding could impact and assist in this change. – ~90 minutes of videos
- How Sharding-Based Blockchains Could Handle More Transactions Than Visa – 10/20/17 – Describes sharding in the context of scalability, less technical. – 3 minute read
- Why Scaling Bitcoin With Sharding Is Very Hard – 11/24/15 – Explains the implementations of sharding and compares it to other scaling solutions, like zk-SNARKS. Presents flaws of sharding, why it isn’t implemented in Bitcoin. – 13 minute read
- Service-Oriented Sharding for Blockchains – 11/21/16 – “This paper introduces Aspen, a sharded blockchain protocol designed to securely scale with increasing number of services. Aspen shares the same trust model as Bitcoin in a peer-to-peer network that is prone to extreme churn containing Byzantine participants. It enables introduction of new services without compromising the security, leveraging the trust assumptions, or flooding users with irrelevant messages.” – 8 pages
Directed Acyclic Graphs
- What is a DAG – DAGs are a type of data structure used in mathematics, statistics, and computer science. This is an introductory, but still quite technical, pdf on what a DAG is, what rules they obey, and their underlying structure. – 10 pages
- DAGs and Cryptocurrencies – 11/2/17 – An overview of how DAGs can be used as a replacement to the traditional blockchain structure, what advantages they bring, and a brief intro to two new DAG initiatives. – 4 minute read
- Blockchains to BlockDAGs – 7/2015 – The initial research that suggested the DAG as an alternative data structure to the traditional blockchain.
- CAP Twelve Years Later: How the “Rules” Have Changed – 5/30/12 – CAP Theorem is a long-standing concept that any networked data-sharing system can have, at most, two of three desirable properties: consistency, availability, and resistance to network partitions. BlockDAGs tend to have a high-susceptibility to partition attacks. – 28 minute read
- Intro to IOTA – A surface description of IOTA by the developing team that introduces the concept, underlying data structure, and key benefits. – 5 minute read
- IOTA White Paper – 10/01/17 – An updated version of IOTA’s whitepaper which describes a platform for machine-to-machine networks built on top of a customized emerging cryptocurrency structure: the directed acyclic graph (DAG). – 28 pages
- Cryptographic Vulnerabilities in IOTA – 9/07/17 – The report of an academic research group into flaws in the IOTA Tangle. The re\\\port specifically found crucial errors in the implementation and design of IOTA’s self-built hash function. (The full vulnerability report can be found here.) – 7 minute read
- Why IOTA is Deeply Alarming – 9/26/17 – A broad overview of issues with the IOTA project and team beyond the broken hash function. – 5 minute read
- Centralization Concerns and Response – 6/15/17 – IOTA uses a centralized coordinator in order to protect the network against 34% attacks. These two blog posts explore the danger that poses, the justification, and what IOTA claims as its roadmap for this coordinator. – 15 minutes altogether
- SPECTRE: A Fast and Scalable Distributed Ledger Protocol – 12/18/16 – This is a Medium article that breaks down the SPECTRE protocol, a DAG that achieves the same consensus as a blockchain with the same security standard, but with a significantly faster block generation rate. – 8 minute read
- SPECTRE: Serialization of Proof-of-work Events: Confirming Transactions via Recursive Elections – This is the full whitepaper for the SPECTRE protocol. Very technical. – 66 pages
- SPECTRE Discussion – This is a Reddit thread between active community members and Aviv Zohar, one of the authors of SPECTRE, that dives into the details of the protocol and discusses some possible attack vectors. – 7 minute read
- SPECTRE Creators Seek VC Backing for Blockchain-Free Cryptocurrency – 10/25/17 – An indication on the direction of the SPECTRE protocol. – 4 minute read
- Tree-chains preliminary summary – 3/25/14 – “Treechains would allow organizing the blockchain in a tree, not as a linear sequence. This would help increase transaction throughput that the network could handle without increasing the block size limit.” [source]
- A Fast and Scalable Payment Network with Bitcoin Duplex Micropayment Channels – 8/4/15 – Presents “a protocol for duplex micropayment channels, which guarantees end-to-end security and allow instant transfers, laying the foundation of a network of payment service providers.” – 16 pages
- Can Flexcaps Settle Bitcoin’s Block Size Dispute? – 11/5/15 – Flexcaps are “flexible caps” on the block size, determined by the miner who mines the block. This article presents the technology in the context of the block size debate. – 8 minute read
- Bitcoin-NG: A Scalable Blockchain Protocol – 11/11/15 – Presents Bitcoin-NG, “a new blockchain protocol designed to scale. Based on Bitcoin’s blockchain protocol, Bitcoin-NG is Byzantine fault tolerant, is robust to extreme churn, and shares the same trust model obviating qualitative changes to the ecosystem.” A more accessible overview can be found here. – 21 pages
- Soft-forking the block time to 2 min – 1/23/16 – Vitalik’s “primarily silly and academic (but seemingly effective) entry to the ‘increase the blockchain’s capacity in an arbitrarily roundabout way as long as it’s a softfork’ competition”; Reddit post. – 10 minute read
- Mimblewimble – 7/19/16 – Original pseudonymous Mimblewimble whitepaper; “Imagine that we can combine [transactions] across blocks so that when the outputs are created and destroyed, it is the same as if they never existed. Then, to validate the entire chain, users only need to know when money is entered into the system (new money in each block as in Bitcoin or Monero or peg-ins for sidechains) and final unspent outputs, the rest can be removed and forgotten. Then we can have Confidential Transactions to hide the amounts and [simplified] OWAS to blur the transaction graph, and use LESS space than Bitcoin to allow users to fully verify the blockchain.” – 11 minute read
- Mimblewimble 2.0 – 10/6/16 – Adds further scaling improvements to the original. “Mimblewimble shrinks the transaction history such that a chain with Bitcoin’s history would need 15Gb of data to record every transaction (not including the UTXO set, which including rangeproofs, would take over 100Gb). [The original author] left open a problem of how to reduce this; we solve this, and combine it with existing research for compressing proof-of-work blockchains, to reduce the 15Gb to less than a megabyte.” – 19 pages
- Scaling Tezos – 4/19/17 – “TL;DR: scaling isn’t really quadratic, a very high throughput can be sustained without significantly hurting decentralization, and a far more powerful technique is right around the corner anyway.” (recursive SNARKs); also goes through arguments for and against sharding. – 9 minute read
- TumbleBit: An Untrusted Bitcoin-Compatible Anonymous Payment Hub – 7/31/17 – “This paper presents TumbleBit, a new unidirectional unlinkable payment hub that is fully compatible with today’s Bitcoin protocol. TumbleBit allows parties to make fast, anonymous, off-blockchain payments through an untrusted intermediary called the Tumbler.” – 36 pages
- On-Chain Atomic Swaps – 9/20/17
- Bulletproofs: Efficient Range Proofs for Confidential Transactions – 11/4/17 – “[A] new non-interactive zero-knowledge proof protocol with very short proofs and without a trusted setup; the proof size is only logarithmic in the witness size.” Will reduce the size of transactions using confidential transactions. – 40 pages
- Demystifying Hashgraph: Benefits and Challenges – 11/8/17 – Quick explanation of how hashgraph consensus achieves scalability. – 6 minute read
Scaling Bitcoin presentations – Archive of all presentations from the annual Scaling Bitcoin conference.