Let’s talk about DigiByte v8 + fundraiser
At the start of the year it became clear that DigiByte needed to elevate it’s game and handle development better than had been in the past, with more accountability and transparency. This isn’t to say we believe the previous way was causing any problems, and DigiByte has done well for itself over the previous 7 years, but more importantly this was an opportunity to further improve the decentralization and trustless nature of the blockchain, adopting some industry best-practices, and spearhead DigiByte into the future.
However, this came at the expense of delaying RandomX, ProgPoW and a number of other things that were already in the works. I want to talk a little about where things are at now that we’ve got this new process in-place and DigiByte v7.17.3 out the door.
So, let’s start with RandomX / ProgPoW
Late October 2020, I was contacted by Barrystyle via Reddit who noticed we’d been struggling with the RandomX + ProgPoW implementation, and that there was the possibility of a bounty for RandomX. Kristy-Leigh Minehan had graciously volunteered to do the ProgPoW implementation for us but had been having a tough time with finishing it off. Coupled with her catching COVID-19 twice throughout the year, and previous health complications, she advised that she was fine for Barry to do the implementation there for ProgPoW. I’m still personally grateful for the time she’d spent, but more importantly I’m glad that she’s been looking after her health.
So towards the end of 2020 Barry began implementing RandomX + ProgPoW, originally on the 0.19 codebase that Jared & Gary had been working on back in Jan 2020. Before long, it became clearer it would be better to do it from a fresh base of 0.20 which had come out in the middle of 2020 (June if memory serves me right), as the DigiByte 8.19 was in a very rough state, partly due to the myriad of upstream changes. It was not compiling / building, and hadn’t really been touched much for the better part of 9 months, save for some fixes Yoshi had been slogging away at. As such, Barry advised me he was happy to work on 0.20 for DigiByte as part of the bounties for RandomX & ProgPoW.
Work was progressing well towards the end of 2020, with the DigiByte Core compiling + running happily, alongside the incorporation of a number of other major improvements. For example, in Bitcoin 0.18, they removed OpenSSL from their codebase. That has no major impact on Bitcoin, however if you look at the likes of Litecoin and other Scrypt-based projects, the OpenSSL libraries are used as part of the Scrypt hashing algorithm, meaning it’s not as easy to just “remove it”. Barry had rewritten the Scrypt hashing algorithm fully using Bitcoins libsecp256k1, which in and of itself is a brilliant feat, and this was working in the 8.20 code back in November / December! In December 2020, I’d paid out the bounties for RandomX & ProgPoW to him, as the work on the algorithms was all but done, with additional minor fixes needed on a stratum server and the final commit alongside the 0.20 codebase once it was in a happier state.
The DigiByte v8 development
In January, the 0.21 release of Bitcoin Core came out, and Barry asked me if it was worth us releasing from 0.20 given it was no longer the most recent. A fair question! Couple with the improvements from Schnorr + Taproot that the DigiByte community had been talking about for quite some time now (A year prior I’d given a presentation on it, in Feb 2020) were being prepared with the 0.21 codebase in-mind. As such, implementing Schnorr + Taproot would be far easier based on 0.21 rather than 0.20. On the 15th of Jan, Barry began to work on 0.21, which included some additional “fun” given the way Bitcoin now rebroadcasts transactions conflicts with Dandelion. Despite this, after just a couple of days in between his other jobs, it was working. He and I had it running on the network, which was when I made several tweets around the time showing it syncing.
However the other developers rightly raised that we needed to have some better checks in-place to ensure that we wouldn’t inadvertently break DigiByte as we’d done in the past with the likes of 6.16.5 not syncing from scratch. It’s a fair point, DigiByte is a billion-dollar blockchain, so we need to be smart about bringing in such big changes, be it merging / rebasing from Bitcoin, algorithms etc.
We needed to ensure that DigiByte remained “DigiByte” and that this new client based on the new Bitcoin Core codebase still functioned perfectly in the interim, right up until our first fork-date for v8. It also provided opportunity for us to review the existing code, and address some of the things that have been discussed in Gitter, including fixing the supply-curve issue I raised last year (Big thanks to Matthew especially for his help with fixing the code there, and Yoshi too for his input) amongst other things.
This all highlighted we needed to have our Unit Tests fixed and operational, which had been neglected and were in definite need of some loving attention. These were something that Barry was going to be also dealing with for DigiByte. However, after he’d fixed the first few Unit Tests for DigiByte he was unable to wait for the feedback / acceptance of those and got busy with some other jobs of his own. As such, we’re very grateful that Yoshi stepped up and completed the remainder of the Unit Tests (I’d like to also raise from $DGB for Yoshi in the future as well). These Unit Tests now all test correctly, and make sure the code that makes DigiByte “DigiByte” is behaving as expected. Though, any other additional things (Say for example if we show a specific button in the UI) which don’t affect consensus or expected protocol behavior are not tested.
We then got the release of 7.17.3 completed and out the door (big thanks to Yoshi / Gary and all others involved in those few days of testing / building). This 7.17.3 release contained all of the functional Unit Tests, so we have a solid testing base to now match against with the DigiByte v8 process. This is important because we know that DigiByte is still “DigiByte” up until the fork-date, at which point all nodes will be reorienting anyway, and using the new definition of “DigiByte”.
Now, no matter what, DigiByte needs to do a hard-fork so that we can resolve the issue with the supply-curve as discussed under Issue #20, and again a big thanks to Matthew and Yoshi for helping there with getting to the bottom of this after I raised the problem in late 2019. It makes sense then, if we’re doing one hard-fork, to also have the others (Algorithm replacements) done under the same code-release. If we stagger the forks under the one “v8 release” it will serve two purposes:
- Provide sufficient time in between implementation of RandomX / ProgPoW for MultiShield difficulty to even out and minimize any risks of attack attempts
- Greatly minimize the effort from exchanges / wallets / service-providers who would have to upgrade every single time otherwise, and risk having some of them dropping support for DigiByte if we were to do too many “mandatory” upgrades of the Core wallet.
Otherwise if we were to try and do a single release each time for each upgrade, there’s the very real possibility of exchanges / services simply deciding they don’t want to accept / support DigiByte, as we have seen happen with a small number in the past.
This brings us to the important question:
Where-to from here?
Well, we need to finish off DigiByte v8 based on 0.21, and the sooner we can do this the better it is for addressing the supply-curve. This is because as of right now the block-reward subsidy will drop from 84 DigiByte to 1 DigiByte in 2035. That’s a pretty sharp drop (Worse than any halving event) and after that it will carry on with an infinite supply (also not good). This needs to be done ASAP because the longer we kick the can down the road, the harder the adjustments are going to have to be, and we can avoid that by being prudent with our time.
Now as mentioned above, Barry has already done a lot of the leg-work so far and it’s largely operational. We as the broader DigiByte community need to simply finalize a few things such as the block-height we want to activate the likes of the supply-curve changes at, ProgPoW, RandomX, Schnorr / Taproot (Schnorr etc can technically just be done at the same time as any one of the other changes). This is also why I’ve mentioned in almost every single video that people should be subscribed to GitHub so they can discuss this. There’s also the Gitter channels where it can be discussed also.
However, as we’ve basically gone from it starting with 0.19, then to 0.20, and now 0.21, I wanted to do a bit more of a fundraiser to send funds to Barry for the completion of this, and so that he can dedicate some time away from his other usual projects. I’ve grabbed his DigiByte address which he’s put on GitHub, and I’d like to see us donate some DigiByte over the next few weeks to him towards the following:
- Finishing 0.21, with the working unit tests, Dandelion changes implemented, and pull request to the DigiByte-Core/DigiByte repository made. This is what most of the donations are for / about.
- Ensure that RandomX is operational and the appropriate stratum modifications available, and documentation / notes around implementation.
- Ensure that ProgPoW is operational and the appropriate stratum modifications available, and documentation / notes around implementation.
- Implementation of the supply-curve fixes.
- Have a date set for Schnorr + Taproot set to the same as the supply-curve changes.
I also just want to highlight that there’s also a few other things that Barry has already helped with of late too:
- Implementing memory improvements from Yoshi
- Improved sync times (I’ve done a full sync in under half the normal time)
- Significantly improved launch speed (Which is great if you’re testing things regularly)
- The full implementation of Scrypt in libsecp256k1
- Improvements to MultiAlgo block generatetoaddress on testnet
- Added algorithm stats RPC call
- A large amount of the DigiByte-specific codebase has being refactored into a segregated area which will allow for much easier improvements / iterations upon in the future
In addition to this, Barry has said that he’d be willing to also do some GUI work for us and refine the themes that are currently there but were never fully completed (There’s a lot of contrast issues and places where the themes weren’t quite polished off), as well as fixing up a few of the other issues such as the inability to send from some of the earliest wallets created on DigiByte when the node is sync’d past a certain point.
So I’m aware that we’ve already sent DigiByte his way last year for the algorithm implementations etc, however, given the regularly moving target this has been through no cause of his own, I want us to arrange for a little more to help get this over the line and as a bit of a thanks to him. This is more specifically for the work around DigiByte v8, both 0.20 / 0.21 and the additional features too. If you’re behind this, I’d encourage you to donate directly to him too: DGBaZ7bGFn6uifeSYp4wiAY4H4aWRpNCoz
I’ll be starting off the donations too with ~USD$150 of DigiByte.
I hope you will be joining me and doing likewise, with any amount big or small. I’m eager to see this finished, get it into the GitHub and get v8 ready to roll! There’s so much packed into this already, I’m very excited about the future of DigiByte, and this is a perfect opportunity to have your say and be involved in the production of the 8th major version of DigiByte!
How will this be delivered / what will successful completion look like?
The primary part here is the completion of an 0.21 based DigiByte v8. This will be something you can compile and test alongside us by going to github.com/digibyte-project/digibyte, where the branches / improvements being worked on are visible.
Over the coming weeks as this is finished off, preparation will be made for a pull request against the existing DigiByte repos. This will include information around the way that ProgPoW & RandomX have been implemented (such as details around the RandomX seed etc, and why it was built the way it was), the moving of the DigiByte-specific parameters into src/digibyte/, some of the testing details, and references to where the upgrade block-heights can be adjusted. For the non-techies, this is basically the same as somebody offering up their code changes and saying “Here’s what I’ve been working on, hope you like it”. All going according to plan, this should be a viable release in and of itself in that state.
Following this Pull Request, the other DigiByte reviewers will then be able to not only provide feedback, amendments, and other suggestions to the code, but further tweaks / improvements to it as well should there be a desire from them / the community to incorporate or remove anything else from the release of DigiByte v8.
After the review process, we should be good to go with DigiByte v8! That will naturally mean the fork-heights are finalized and then the community (myself included) will need to reach out to exchanges / wallet providers etc to ensure they’re running this latest release in preparation for the upgrades. This is something I’ll talk more about closer to the time, as a group of us have done this several times over in the past contacting exchanges.
Should you have any questions about this process, please feel free to reach out to Barry or myself on Twitter, or maybe shoot him a thanks for all that he’s done thus far on the v8 release / algorithms.
Also, feel free to join in the Gitter channels to ask any questions about the protocol etc and liven the place up a little. Keep in mind that this is specifically just for development-related purposes, not for general support / questions or any form of speculation.
