On Continuous Integration for DigiByte and the v8 release

After a HUGE effort with the Unit Tests, Yoshi has taken a lot of the prior work from Marpme_ (Marvin from Vergecurrency) that he did for us and completed the Continuous Integration setup for DigiByte. Thank you also to those who’ve taken time to individually review all of them, as part of our new Gitflow workflow process as well.

The conclusion of the Continuous Integration means every time a new pull request is made against DigiByte, not only will it do a test compile to make sure it builds, but it will also run the Unit Tests as well to ensure that the changes don’t negatively affect the fundamentals of what makes DigiByte “DigiByte”!

Testing the CI with a build “In progress - This check has started…”

This is a HUGE milestone for the integrity of DigiByte, and we are grateful for all those who’ve been involved with writing / correcting / reviewing the Unit Tests, as well as the Continuous Integration.

These are major steps forward for us as a project that I’m super excited for.

The question I’ve been asked lately is:

Aren’t we working on DigiByte version 8?

Yes. All of this has been done because the DigiByte v8 release is shaping up to be SO massive, that it made sense to ensure we didn’t miss anything inadvertently.

The earlier MultiAlgo / MultiShield releases of DigiByte may have been the biggest in terms of fundamentals, but v8 will without a doubt be the biggest in terms of lines of code changed, performance enhancements, bug fixes, feature additions, and (what is probably most dear to me) future-proofing of the project to ensure the longevity of the codebase / blockchain.

So with the Continuous Integration and the Unit Tests, we’re effectively setting a benchmark to compare against for in the future.

You see, if we inadvertently made a change to DigiByte that allowed for SHA256 blocks to be “mined” from Block 1, then we’re opening ourselves up to a world of pain and potential attack vectors. SHA256 should only valid from the MultiAlgo upgrade. The Unit Tests will ensure “Are we only permitting only Scrypt from Genesis to 145,000th block?”, along with other similar fundamental tests and checks.

Now that we have these Unit Tests, it means we can forge ahead with certainty.

So what’s the plan? When do we get DigiByte version 8?

First, we’re releasing 7.17.3!

This will be the same as 7.17.2, but, with unit tests and continuous integration sorted, and that’s basically it. There’s no further fixes / improvements, however this is going to be the foundation we use for version 8. As such, if you’ve got 7.17.2, then there’s no practical reason for upgrading to 7.17.3.

However, what you’re going to see now on GitHub is more work being done on merging the Bitcoin Core 0.21 code. This will bring in a bunch of stability improvements as well as feature enhancements (Things such as Partially Signed DigiByte Transactions). However there’s also a number of changes to the way that mempool / retransmissions behave, so we’re also re-integrating Dandelion.

You’ll also see pull requests for algo swaps coming in after this. Barrystyle has almost finished the RandomX + ProgPoW code, we’ve seen demo videos in the past of RandomX etc done against some earlier. However, I want to reiterate that this code will still go through the same rigorous review as everything else, there won’t be any skipping of the process. The code is looking pretty great, however this will provide opportunities for people to provide feedback on:

1. Do we implement ProgPoW or RandomX first?
2. Do we even implement both? Or do we implement one but not another?
3. Do we implement another algorithm altogether?
4. Which algo(s) should we replace? SHA256? Or Qubit? Or Skein?
5. Are we seeking hard-drive mining?
6. Does Odocrypt need to be tweaked / upgraded?
7. Or do we not change anything?

In addition to this we’re also going to see some pull requests for some performance enhancements and feature improvements, I understand we’ve got some of the following things being worked on:

• Caching of block headers / PoW hash for exponentially faster launching (Roughly 500X speed-up)
• Implementing btree to reduce RAM utilization
• Syncing prefers “faster” peers that can feed more blocks, concurrency improvements
• Schnorr Signature implementation + Taproot
• Emissions curve fixes
• Additional RPC calls related to MultiAlgo

And more. Again though, with each of these, they’ll be going through a peer-review process from other developers, regardless of who commits them, along with discussion around it.

One thing to be aware of is that just because a pull request is made against DigiByte does NOT mean that it’s specifically going to be implemented.

For example, the caching of block headers / PoW hash will require a little more drive-space usage as a result (Roughly 350MB from early tests, growing as block #s grow). While that’s not much, it’s worth being aware that as part of the review process it’s entirely plausible that other developers might say “Hey what if we made this optional, with a launch flag similar to -txindex rather than being always-on? What if it was enabled by default? What if we disable it by default?” and have these kinds of discussions.

Once that’s all done we’ll wrap it all up nicely, tie it with a bow, again do some additional tests over and above the Unit Tests (As well as possibly adding some more unit tests along the way) and then get the DigiByte version 8 release out.

The good news is that a lot of the code has already been done for a number of these items and it’s simply going to require smaller amounts of modification for each to work with this new base. Expect that a lot of these will come pretty quickly now we’ve got an improved foundation to go off.

So as always I encourage you to go to github.com/DigiByte-Core/digibyte and click the “Watch” up the top-right to stay aware of all these developments as they happen, and have your input in them all!