Horizontal and Vertical sharding
Large datasets can be scaled using database splitting techniques like horizontal and vertical sharding. Whereas vertical sharding separates data across servers according to columns, horizontal sharding splits data across several servers according to rows.
Also Read About What Is ewasm Ethereum WebAssembly In Blockchain
Horizontal Sharding
What it is: Divides a table into smaller tables with fewer rows but the same columns and distributes them to other servers.
When to use: It works well when queries frequently yield a subset of jointly grouped rows, such as data based on date ranges.
Pros: Enhances scalability by making it simple to add new servers as data expands, and improves query performance by restricting the search to a smaller portion of data on a particular server.
Cons: May need more cross-shard communication for queries that span multiple shards, and it can be more difficult to implement and maintain.
Vertical sharding
What it is: Divides a table into multiple by moving columns to new tables. Thus, each table has fewer columns but equal rows.
When to use: It works well when a query regularly returns only a subset of columns, such as names and addresses.
Pros: Helps queries that employ a fraction of the columns by letting each shard focus on a subset.
Cons: Schema updates and data transfers are difficult with complex shard interactions.
Vertical vs horizontal sharding
| Feature | Horizontal Sharding | Vertical Sharding |
|---|---|---|
| Definition | Splits data by rows | Splits data by columns |
| Data Structure | Each shard has the same structure (schema) | Each shard has a different part of the schema |
| How It Works | Different rows go to different shards | Different columns go to different shards |
| Example | Users A–F in Shard 1, G–L in Shard 2 | Shard 1 = user info, Shard 2 = user transactions |
| Best For | Distributing users or records | Separating frequently accessed vs less used data |
| Scaling | Easy to scale out by adding more shards | Less scalable, can become complex |
| Query Speed | Fast if queries match the shard’s data | Slower if query needs multiple shards |
| Complexity | Simpler joins; complex routing | More complex joins between shards |
| Use Cases | Blockchain, distributed apps, user data systems | Large systems with different data types |
Also Read About Drizzle Blockchain: Simplify DApp Front-end Development
Sharding challenges
Despite its advantages, sharding raises a number of issues and complications.
Complexity
It is anticipated that it will “take time to create a workable scheme” to implement and manage a sharding blockchain.
Security Issues
Sharding might create new vulnerabilities even though its goal is to preserve security and decentralisation.
Decreased Security per Shard
When a network is sharded, its security is “sharded” or broken down into separate components, which means that each shard is protected by fewer validating copies.
Single-Shard Takeover Attacks
Because a single shard has fewer validators or less hash power than the entire network, there is a chance that hostile actors may take control of it. By carefully and randomly allocating or reassigning nodes to shards, this risk can be reduced.
Collusion
Members of the committee might band together to report nefarious transactions.
Cross-Shard Communication/Transactions
Because messages transferred across shards might not be synchronised, cross-shard transactions necessitate coordination and might add complexity. Effective data sharing requires secure communication techniques.
State Management
It can be difficult to manage the blockchain’s state (code, storage, and balances) over several partitions, particularly if users are regularly switched between shards.
Data Inconsistency
An inconsistent network state could result from one shard perhaps recording different data than another. To overcome this, mechanisms like as fraud proofs and transaction ordering are suggested.
Also Read About DBFT Delegated Byzantine Fault Tolerance Def And Advantages
Ethereum’s Approach to Sharding (Historical Context)
Originally slated as “phase three” of the Ethereum 2.0 (Serenity) roadmap, sharding was a crucial component of Ethereum’s scalability solution. Although a layer-2 centric strategy (danksharding) has mostly superseded direct sharding of the mainnet, the original plans effectively demonstrate the idea.
Beacon Chain
A key element is the Beacon Chain, which acts as the coordination layer or master chain. In addition to managing validators, coordinating the sharding process, and recording the status of shard chains, it generates the randomness necessary for a working sharding system.
Shard Chains
Initially 64, shard chains would have been deployed in Phase 1 of Ethereum’s sharding, serving as “data depots” to enhance data processing capabilities. They were supposed to develop into code execution layers in subsequent stages, each functioning as a mini-mainnet and having its own independent “state” (such as distinct smart contracts or account balances).
Committees and Validators
The Beacon Chain assigns validators, who deposit 32 ETH, to shards at random. A “collation” a collection of transactions, akin to a block is produced by a chosen “proposer” validator. Other “notaries” or “committees,” who serve as validators, vouch for the collation’s accuracy. It is sent to the Beacon Chain through a Validator Manager Contract (VMC) if two-thirds of notaries attest.
Cross-Links/Collation Headers
Only “collation headers” cryptographic proofs of validity that include metadata and Merkle roots of transactions are added to the Beacon Chain in place of the whole collation. These headers serve as “cross-links,” allowing communication across shards and providing a high-level picture of the network without being a part of each shard.
Additional Sharding Examples
Other initiatives that investigate or employ sharding techniques include:
- Utilising network-level dynamic sharding, Zilliqa modifies the number of shards in response to network traffic.
- NEAR: Enables low-end devices to function as nodes by concentrating on real-time cross-shard transactions without raising client load.
- Omniledger.
- The “Hydra” Layer 2 solution from Cardano.
- Polkadot Parachain: An alternative method of distributed database sharding that links to relay chain security.
- MultiversX (previously Elrond): Processes thousands of transactions per second in parallel using adaptive state sharding.
- Shardeum: Assigns dynamic account spaces to nodes across shards and strives for linear scalability with each additional node, implementing dynamic state sharding from the ground up.
Also Read About The History Of PAXOS Consensus Algorithm And Objectives
By dividing the burden among parallel operating segments, sharding is essentially a potent on-chain solution that seeks to greatly increase a blockchain’s capacity to manage a higher volume of transactions and users.