March 17, 2018 . 10 min read
The purpose of this test is twofold:
Testing to date of DyPoS has been focused on the consensus efficiency in 4-node and 8-node validator setups. Load Test #5 greatly extends the number of validators to 21, further increasing the decentralization of the system. In other blockchains observed, poor performance results in increased centralization as the number of validators increases. This load test seeks to confirm that this is not the case with DyPoS.
The 21 validator nodes are geographically distributed across 4 regions of the United States. Housed in Amazon Web Services (AWS) data centers, 6 validator nodes are hosted in Northern California and 5 validators each are in data centers located in Virginia, Ohio, and Oregon. While there may be several validator nodes sharing a region, they are deployed independently of each other. They do not share hardware or other resources.
Clients sending transactions share a node located in an AWS data center located in Canada. Figure 1 below shows the test setup.
TEST 1 - 2 CLIENTS, 50 BURSTS, 500 Txs/BURST, RANDOM TRANSACTION SIZES, 30 SECOND PAUSE BETWEEN BURSTS
In this test, 2 clients are used to generate and send transactions to each of the 21 validator nodes. Each client generates a burst of 500 transactions of random sizes. The transaction sizes are randomized as described in table 1 below.
Each client sends 50 bursts of 500 transactions to each of the 21 validator nodes with a pause of 30 seconds between each burst. A total of 21,000 transactions (2 clients x 500 txs/burst x 21 validators = 21,000) are sent per burst. A total of 1,050,000 transactions (21,000 transactions x 50 bursts) are sent to the 21 validators.
The consensus efficiency is measured for each burst. The results are captured as follows.
Please see https://drive.google.com/file/d/1iD9lVyJH9jtC4sUKHCjZjgdYY_C3ZXwU/view?usp=sharing for the results are for all of the bursts.
TEST 1 results for consensus efficiency or throughput ranges from a low of 4,750 to a high of 7,759 transactions.
After analyzing the results, there are two factors affecting consensus efficiency. They are:
TEST 2 - 2 CLIENTS, 50 BURSTS, 500 Txs/BURST, RANDOM TRANSACTION SIZES, 120 SECOND PAUSE BETWEEN BURSTS
This test is identical to Test 1 above with the exception that rather than a 30 second pause between bursts, there is a 120 second or 2 minute pause between bursts. The longer pause enables the validators to stabilize before taking on another round of transactions.
The purpose of this test is to verify the “transaction accumulation” claim made above.
The results are available at https://drive.google.com/file/d/15gKpc96hpDwnRoV3ubxmH0IAHiXWtYGG/view?usp=sharing
TEST 2 shows consensus efficiency, captured as “Txs/Sec,” range from a low 2,348 transactions per second to a high 2,654 transactions per second.
Test 1 and test 2 are virtually identical. The only difference between the two is that in test 2, there is a 120 second pause between bursts whereas there is a 30 second pause in test 1. Yet, there is a big difference in consensus efficiency between the two. Consensus efficiency is lower in Test 2 with the 120 second pause between bursts because the validator nodes are idling more than in the previous test. This observation verifies the “transaction accumulation” claim. If transaction accumulation keeps the validator nodes occupied, consensus efficiency will be higher. This leads us to conclude that the lower consensus efficiency in test 2 is a result of under-utilized validator nodes.
TEST 3 - 8 CLIENTS, 50 BURSTS, 50 Txs/BURST, RANDOM TRANSACTION SIZES, 30 SECOND PAUSE BETWEEN BURSTS
The difference between Test 1 and Test 3 is that rather than using 2 clients to send transactions, 8 clients are used in Test 3.
The purpose of this test is to see how DyPoS performance is affected by the use of more clients to send transactions to validator nodes. Rather than 21,000 transactions (2 clients x 500 transactions x 21 clients) or 500 transactions per client per burst, there are 50 transactions per client per burst. As a result, a total of 8,400 transactions (8 clients x 50 txs/burst x 21 validators = 8,400) are sent per burst. For the entire test, 420,000 transactions (8,400 transactions x 50 bursts) are sent over 50 bursts.
The test results are available at https://drive.google.com/file/d/1WljDwSGBPybZPAomn_6iaS3FHBZ9O9q5/view?usp=sharing
Test 3 shows consensus efficiency, captured as “Txs/Sec” in the results, range from a low of 5,592 transactions to a high 10,282 transactions per second.
In Test 3, a larger number of clients generated transactions in smaller bursts. The resulting consensus efficiency is better than the previous two tests. Smaller bursts allow for larger concurrency, resulting in better consensus efficiency. In this case, the validator nodes are kept busy but not sufficiently so as to overwhelm them with large burst sizes. This results in Test 3 having the best performance of all of the tests.
In this set of tests, we evaluated the Storecoin DyPoS consensus engine for greater decentralization by testing network performance using 2 clients sending transactions to 21 validator nodes.
In typical blockchain deployments, greater decentralization severely affects consensus efficiency, resulting in centralized deployments. The tests performed here verified that Storecoin DyPoS is not affected by centralization. While consensus efficiency did drop from previous load tests where transactions were sent in a single burst, the drop in consensus efficiency is respectable in multiple burst tests.
In the test with large bursts of transactions, consensus efficiency was observed ranging from a low of 4,750 transactions to a high of 7.759 transactions per second. In the test with smaller bursts of transactions but with greater concurrency, consensus efficiency was observed ranging from a low of 5,592 transactions to a high of 10,282 transactions.
Failure scenarios of DyPoS have not yet been tested in the latest round of load tests. However, such scenarios can be inferred from the previous tests. As an example of such failures, validators can be saturated by increasing the number of clients and the number of bursts or any combination thereof. Such actions increase transaction volumes, affecting consensus efficiency negatively.
With initial testing of DyPoS complete, the behavior of validator nodes is now understood under a variety of load conditions. This behavior can be summarized as follows:
Nothing herein is intended to be an offer to sell or solicitation of offer to buy, Storecoin tokens or rights to receive Storecoin tokens in the future. In the event that Storecoin conducts an offering of Storecoin tokens (or rights to receive Storecoin tokens in the future), Storecoin will do so in compliance with all applicable laws which may include the Securities Act of 1933 and the rules and regulations promulgated thereunder, as well as applicable state and foreign law. Any offering for sale to US Persons in a regulated transaction will be pursuant to a registration statement qualified by the Securities and Exchange Commission, or an applicable exemption from the registration requirements.