In pursuit of GPU mining, a timeline pre Odocrypt -> DigiByte v8
I’ve been asked a lot lately about what’s been happening with a CPU / GPU mining algorithm that we’ve all been waiting for what seems like years to implement. This will cover some of the history there, as well as the development of DigiByte version 8. I’ll also include a spiffy little timeline at the end, so we can clarify a few things and provide people with both some insight into the history as well as a comprehensive perspective of events.
The lead up to, and release of Odocrypt
DigiByte had evolved to be at a point where it was faced with 5x ASIC-mineable algorithms since all the way back in 2017. Immediately after all of the fervor of DigiByte briefly reaching the Top 10 in CoinMarketCap around July of 2017 had begun to die down, rumors began to swirl of ASICs (or at least highly optimized FPGAs) on all the remaining algorithms.
Everyone knew that SHA256 and Scrypt were ASIC mineable, this was the case when MultiAlgo was originally implemented in September 2014. However of the 5x algorithms, one was originally CPU mineable (Qubit), while the remaining 2x were GPU mineable (Skein and Myriad-Groestl). That changed in 2015 with Qubit being GPU mineable as well, however towards the tail end of 2017 in September / October, the relative ‘difficulty’ of the Skein and Myr-Gr algorithms rocketed upwards.
Come November 2017, the Baikal X10 Giant was released, with a power to hashrate ratio that destroyed even the likes of the GTX1080Ti by an order of magnitude. I recall being away on my honeymoon at the time talking with Jared, GTO90, Ycagel, DigiContributor, DiscovernowDigiByte and a few others about how we needed to bring back GPU mining.
Even back then, despite linking Jared to the buy-page for the devices, I recall he wouldn’t believe they even existing, iwth support for Myr-Gr, Qubit and Skein, 3x of our 5x algorithms.
I’ve talked a lot about why GPU / CPU mining is beneficial but here’s a few quick recaps of why it’s important:
- It vastly improves the distribution of the mined asset, compared with ASICs
- It greatly distributes the hashrate globally because of better distribution channels for CPU / GPU vs ASIC / FPGA
- It encourages a decentralization of mining thanks to participation from hobbyists or people testing the waters of mining
- Users can participate in the network without needing a costly investment in single-purpose hardware
- Ease of onboarding to the network, without requiring KYC purchases from an exchange
…just to name a few.
At the time in November / December 2017, I looked at the likes of Ethash & Equihash, both of which had placeholders in the codebase already for. I also began looking into CryptoNote, Lyre2RE, BlueMidnightWish, and SkunkHash-Raptor. You’ll note that some of those clearly achieve the goal of being a CPU / GPU focused algorithm and stood the test of time better than others.
Unfortunately though, not a lot happened throughout the next 12-odd months, in 2018.
We had a few minor updates in the 6.x releases which were basically all small bugfix releases across the course of 2018. Then, on the 5th of November 2018, MentalCollatz (the brains behind MultiShield) submitted a pull request for Odocrypt.
Collatz was the driving force behind the Odocrypt upgrade for sure! Originally the theory was that it could potentially also be GPU mined, though FPGAs were definitely the key focus. I began to work with him after we caught up on Telegram about a week after his pull request was submitted, testing it, and testing it, and testing it some more. I ordered several different FPGAs based on the Cyclone-V architecture so that we could test competitive mining against each other. It took a while but we got a fully simulated testnet setup!
Collatz also identified a minor security issue, and so the ReserveBits soft-fork was created with the 6.17.2 release. This was probably the 4th time in the previous 12 months that myself and half a dozen others had contacted Exchanges / Pools etc to get them to upgrade. This upgrade would mean a tighter upgrade in the future as we replaced Odocrypt, and replaced subsequent algorithms.
The ReserveBits upgrade got locked in without too much issue, in preparation for the Odocrypt upgrade, however because we’d been doing a bunch of bugfixes with the algorithm weighting issue, then the bitcoin inflation bug, now this, there was “upgrade fatigue” setting in for sure.
Still, Collatz and I began with a testnet-upgrade replacing Myriad-Groestl with Odocrypt. The first time around the initial difficulty was set way too low, meaning a whole bunch of Odocrypt blocks were found all in a row, so we adjusted it again setting the difficulty requirements to be much higher. This testing Collatz and I did throughout March / April confirmed the upgrade a second time, working perfectly with the adjusted difficulty, so mainnet was set higher in order to accomodate the fact that significantly more people would be mining.
There was also significant testing going on with Dandelion, and I was spending most days working with Collatz and Yoshi testing it. It took a bit to get to the bottom of a few crashes there due to the reference-implementation being for an older Bitcoin codebase, but we got there in the end and it’s been solid ever since, also included in the Odocrypt upgrade.
With the Odocrypt signalling, we missed the originally expected upgrade time because that mining signalling hadn’t reached the 70% threshold. Despite myself and half a dozen others reaching out to all possible mining pools (And there was approx 80% “known” sources for mining hashrate) it was difficult to get people to enact this upgrade. I believe this was mostly due to the aforementioned “upgrade fatigue”, as we never had any written / verbal pushback etc.
However a couple of signalling epochs later, the mining signalling threshold was reached. We had the one-week lock-in period, and we scheduled a count-down livestream in July 2019.
I recall on that day helping Jared get his Cyclone V Intel Starter Kit operational immediately beforehand, because he’d never setup Odocrypt mining. I was assisting him with the process of getting it setup right up until the start of the livestream, as all the testing had been Collatz and I.
If memory serves me right, there were 3x Odocrypt blocks in a row at the cut-over point (far better than our first testnet migration), the livestream was a success, the changeover was a success, Odocrypt was operational, and we had completed the first algorithm replacement in almost 5 years since MultiAlgo was activated back in 2014! As of 22nd July 2019, Odocrypt was live.
What happened to GPU support?
Well we very quickly noted even pre-activation of Odocrypt that it would be heavily favoring FPGAs to the point where GPUs couldn’t keep up. Had the Cyclone V based hardware been where things remained, it might have been a different story, however we were seeing even the AtomMiner coming out and being over twice as efficient, then BlackMiner announcing support for Odocrypt and being an order of magnitude more efficient. I immediately began the hunt again, even before the activation of Odocrypt, for a GPU-focused algorithm.
Looking back in hindsight over the other algorithms being investigated, it had become clear that a lot of them were not fit for purpose: Lyra2RE became Lyra2REv2, Equihash had ASICs, BlueMidnightWish was being mined as part of X16R on FPGAs, and even Ethash had ASICs (although they displayed the smallest benefit compared with GPUs).
Despite these ASICs, placeholders for Ethash and Equihash remaining in the codebase, it was clear we’d need something fresh!
In June of 2019, the two main algorithms I was pushing for adoption after Odocrypt were SIGMA for CPU mining, and ProgPoW for GPU mining, though consideration was being given to Cuckaroo / Beam algorithms as well. This was a little over a month prior to the activation of Odocrypt, and it has remained a priority (in my mind at least) ever since.
Enter: ProgPoW
Following on from Odocrypt (once activated on mainnet), I began a bit of a deeper dive in to the algorithms. Over the coming months, I discovered this video of Kristy-Leigh Minehan at Ethereums Devcon4 which had me sold on the idea of ProgPoW as the next major upgrade for DigiByte:
If you haven’t seen the video, I would highly encourage you check it out!
I reached out to Kristy-Leigh back in August / September, we got talking, and in October of 2019 she came on to my podcast for DigiByte and Friends. It became abundently clear we needed to implement ProgPoW, and she offered to assist us with the implementation of it all the way back in October 2019.
We gratefully accepted her volunteering, and she began to work on that in her spare time. There was a sense of progress finally, heading in to 2020 that we might be able to get something out the door in the new year that supported GPU mining again finally, further improving our dwindling mining decentralization.
Over the Christmas period of 2019, was a good opportunity for me to review the Community Whitepaper (now renamed to the Infopaper) I had begun to write the year prior, and reflect upon the long-standing desire (and even need) for a GPU-based algorithm.
Although this isn’t directly related to ProgPoW, I had mentioned it in the document when I drafted this originally in 2018, coupled with the DigiFacts I was was making around the same time. In fact, one of the original DigiFacts I wrote incorrectly stated DigiByte was an “early adopter” of it, as the prevailing theory was it would have been introduced sufficiently within the following few months prior to that DigiFact being sent out by DigiByteCoin twitter. This naturally has since been replaced with a different fact.
When the Infopaper was being written at the time in 2018, Jared had asked I not do any further work on it, but rather asked that I give him 4 weeks to finish his book in January 2019. Reason being he wanted to use parts of his book, Blockchain 2035, to make such an Infopaper. Blockchain 2035 ended up being released in November 2019, and the Infopaper coming out in Q2 of 2020 without any of Jareds involvement. For those curious about why I’d mentioned the Infopaper back in 2018 but it took 18 months to eventuate, this was why.
Carrying on though to the start of 2020, on January 4th 2020 I was writing about the implementation of ProgPoW and how following that I strongly believed that RandomX would be the next after ProgPoW. This is because RandomX was to CPUs as ProgPoW is to GPUs. However, we weren’t blindly charging in to algorithm upgrades just for the sake of it, with me intentionally seeking out people who would try and convince us not to change, to highlight down-sides and negatives. Still, it remained a priority for the community, for myself, and it simply made good sense to pursue!
The start of DigiByte 8
Jared, ycagel, DigiContributor and myself all jumped on a video call, as DigiContributor worked with Jared to do a whole lot of initial work on DigiByte version 8 on the 31st of January. The video call went on for around 4 hours, which I recorded as Jared was wanting to document some of the ways that he and DigiContributor worked through merging the code. Sadly, because I was at work on the day and remotely accessing my PC, I didn’t notice that I wasn’t recording desktop sounds, and so the entire recording had no audio.
This work on January 31st was merging Bitcoin Core 0.19 at the time, to give us version 8.19. Unfortunately at the conclusion of January, the code was left still completely non-functional, with 8.19.0 not compiling in any way. This was also the last time we would see Jared contributing code throughout 2020.
I sought out further assistance, predominantly with the 8.19 codebase. The theory was that if Kristy-Leigh was to implement ProgPoW against 8.19, we needed it to actually work. In April, SmartArray began working on tidying up the many outsanding issues, then in May I found AkshayNexus who also began to assist. They re-merged OpenSSL which was a main showstopper, as this library had been removed by Bitcoin Core and yet we needed this as the basis for our Scrypt (basically, the key parts of the blockchain).
April rolled around and I wanted us to do a fundraiser to show some gratitude for her effort going into ProgPoW.
We managed to fundraise to reach the goal amount before too long, and I’d been keeping in regular touch with Kristy-Leigh about ProgPoW and the progress being made. However, without a functioning 8.19, it made progressing quite difficult for all parties.
I continued to write about ProgPoW and then subsequently RandomX in DigiByte at the end of April. Again this was a clarifying article for people who were asking when it could be expected, clearly there was a lot of enthusiasm, and people just wanted to know “When can I mine with my graphics card again?”.
The one part that I wasn’t completely transparent about was that although I mention the need for 70% mining lock-in, even as far back as April 2020 I was discussing with others how we should proceed given we knew of only ~50% of the mining hashrate source (at the time) since the Odocrypt upgrade. Because there is no way to reach out and find the remaining 50% unfortunately, or at least 20% to push us over the 70% threshold, it would make any signalled upgrade incredibly difficult to find mining consensus on.
Fun fact: Odocrypt was the only signalled hard-fork DigiByte has ever done, the remainder were all activated at a block-height.
I also reached out to a friend I had gotten to know who predominantly worked on Verge (XVG) to see if he’d be up for helping. He committed some Continuous Integration testing for us at the end of June. This was fantastic as it identified a lot of the previous Unit Test issues we had not addressed, while also providing something to enable more rapid testing in the future. Very grateful for Marpme_ setting us on the right path there.
When things had completely stalled
Then in May 2020 we saw Jared getting furious at Exchanges for not “giving back” to projects such as DigiByte and announcing he would take a sabbatical from DigiByte. He had, however, not contributed code since January, but nonetheless choosing to intentionally step away seemed positive at the time.
It was indeed a positive for the community because despite him returning to Twitter almost immediately talking about non-blockchain things, Binance listed DigiByte the following month in June now that he was publicly stepping away from the project (given his prior highly public distaste for them). DigiByte was the last project in the Top 50 of CoinMarketCap to be listed on Binance (holding a position of 44 at the time).
Since then we have seen DigiByte plummeting down the list, even briefly exiting from the Top 100 for half a week in July of 2021. Although the Market Cap has quadrupled in size since the June 2020 sabbatical, due to more recent stifling of progress in 2021, DigiByte has found itself unable to keep up with other projects increases.
Even though there was a wave of excitement from the Binance listing that continued throughout Q3 of 2020, there was absolutely no development work happening on the Core wallet at that time, nor mobile wallets, nor DigiAssets (Aside from mctrivia working on DigiAssetX). Kristy-Leigh Minehan had unfortunately been unable to carry on with the work, coupled with fact that the 8.19 branch had remained broken since January, meant we were stuck.
Despite my best efforts to find developers and people who would be willing to contribute or help with either DigiByte version 8 as well as ProgPoW / RandomX, I was coming up short. DigiByte was effectively “treading water” at a code-level, with more and more centralization of mining occurring every day.
Progress starts up again in Q4 2020
In October I was contacted by someone through Reddit. This person, barrystyle, had heard we were looking for someone to help with a RandomX implementation and had begun work on it. Barry caught me up to speed on where things were at with what he’d done already and what more needed to be done. In particular he was doing a bunch of extra work to ensure that it would work “out of the box” with existing XMRig and other Monero-expecting mining software implementations.
So we took up a small fundraiser to pay Barry for his time implementing RandomX which I believe was reached just two days later, a real record.
Barry got to work right away basing it off version 7.17.2, and at the first milestone just 4 days later I sent half the funds his way when blocks were being minted with RandomX on a private testnet.
Progress was truly being made, for the first time in 9 months!
However, Barry quickly identified that the existing 8.19 was very broken and offered to assist further there. Given Akshaynexus had been attempting to re-add OpenSSL, and that all development had completely stalled since May (6 months with no end in sight), in November Barry asked if I’d mind him rebasing from Bitcoin Core 0.20.
I told him that sounded fantastic for a few reasons:
- More up-to-date codebase to work from of Bitcoins, zero objections there
- Bitcoin was using that as the basis for their Schnorr signature work (which we were also interested in), and it would minimize effort there
- The codebase Jared left in January for 8.19 was in quite a state, so much so that SmartArray and Akshaynexus were at a loss with how to proceed
It’s worth noting that both the Master as well as the 7.17.2 branch were both still fully functional throughout that time. I had previously seen a lot of untrue accusations about this. It was only the 8.19.0 that Jared had started and abandoned that did not ever work. You can still test this to this very day, as the branches are available, with the old Master and the 8.19.0 both still visible.
Nonetheless, things were truly starting to ramp up, and genuinely new achievements + accomplishments being done, the likes of which DigiByte had not seen in the longest time.
Fun fact: Odocrypt was the first “new code” written for DigiByte (by MentalCollatz) since the DigiSpeed upgrade in 2015. All other upgrades were purely merging of pre-existing Bitcoin code. Dandelion, which was also a proposal for Bitcoin, was also implemented by SmartArray and MentalCollatz.
Aisde from that, everything since 2015 was taking Bitcoin code and “making it work” with DigiByte.
So, Barry began to work instead on the Bitcoin Core 0.20 code in October as a more “stable” base for RandomX to be built against, which would mean we could also implement Schnorr more easily too.
But even better, Barry had begun to work on a number of other improvements, things that were genuinely “firsts” and not just code pulled from Bitcoin, such as:
- Improvements to the sync of the blockchain from scratch so it occurred in half the time on a normal machine, and in a fraction of the time where you were re-syncing to test code (important for developers)
- RAPID start up of the blockchain, taking it from 12 minutes on some test machines to 60 seconds flat.
That alone, outside of Odocrypt, is more new / fresh / unique code that had gone in to DigiByte in the previous 5 years.
Undertaking ProgPoW and RandomX
Throughout October / November, Barry managed to get a really good 0.20 base working for DigiByte, which was then being used for RandomX development. SmartArray had also made a pull request with a replacement of hashmap with btree, as he’d been loosely working with Barry at that time in the same Telegram channel.
I’d also been talking with Kristy-Leigh about ProgPoW throughout October about the progress there. She actually suggested towards the end of the month that if Barry was doing well with RandomX that he might like to also undertake ProgPoW. Kristy-Leigh had some personal things taking priority, so I guess not having me asking every week how things were progressing would probably be one less stress (my bad). We’re still on great terms and keep in touch on the regular, and I’d encourage anybody to follow her on Twitter too.
I ran the suggestion past Barry, he agreed, and then in November he undertook ProgPoW as well.
A lot of work went in to the Stratum for RandomX, which was where a lot of the magic happens to allow RandomX mining with existing miners (such as XMRig) where usually they would require some modification from the normal Monero blockchain. This again is a very unique implementation, with the stratum all being written from scratch, and will be another unique part to DigiByte that has not simply been merged from Bitcoin Core code.
ProgPoW happened throughout November / December, and I paid Barry out all of the outstanding bounties at this point given we were pretty close to completion.
We’d been doing mining on testnets we’d setup, we’d setup mining pools and tested rolling the blockchain forward to implement these algos, we’d tested mining on XMRig, SRBMiner-Multi and NanoMiner. Issues had been identified and were being cleaned up, we’d probably done a few hundred compiles at this point.
Barry had also done some other major work with things like the removal of OpenSSL. Bitcoin Core had removed the OpenSSL library from 0.19, which the Scrypt algorithm uses. This is part of what had caused so many issues with 8.19 for Jared back in January of 2020, and Akshaynexus had attempted to readd as a “quick fix” to get things working again. Instead of taking the quick route, Barry implemented a single-file scrypt copy, which had an integrated sha256d function, which got us by, so that OpenSSL would no longer be required. This was no mean feat, and not the kind of thing some junior coder could accomplish (We saw relatively skilled people simply trying to add it back in), nor is it the sort of thing someone could ‘fluke’. This is the kind of thing that requires a LOT of skill and intricate knowledge of how all of the pieces tied together, and making the hashes match what they should be.
We had a fresh implementation, with new code, and outside of Odocrypt, DigiByte had not seen this kind of improvement in the last half a decade.
Things were looking really good, and progress was being made!
Odocrypt upgrades
In December of 2020, I got together with Matthew, SmartArray, and Barry, with the goal of getting to the bottom of the Odocrypt “Is there an ASIC?” question.
Despite the fact that a large mining pool had just started up a month prior, and we knew they were using BlackMiner F1+ and F1-Ultra’s FPGAs, and were getting around 20% of all Odocrypt blocks, I took this very seriously as did the other gentlemen.
The theory being: If there are ASICs, then changing Odocrypt will stop them from being able to mine.
If there are no ASICs for Odocrypt, then because it’s a FPGA-targetted algorithm that changes every 10 days as it is, there’s no problems with tweaking it slightly, due to the fact it changes every 10 days as-is.
Then on the 3rd of January we created a Telegram group and invited the BlackMiner team (who specialize in FPGAs) to join us.
We got to work, and on the 26th of January, an initial submission for an upgrade of Odocrypt was created. It still needed some more work, but thanks to Matthew and Barry who were there and immediately replied.
You’ll note that the “other” developers didn’t respond for 6 months after it had been submitted, and even then only because I had talked with Laura asking why they were refusing to even acknowledge it. We even had the 250th DigiByte update on June 19th with Jared Tate on it, talking with him about Odocrypt v1.1, but it took another 4 weeks before “the” devs acknowledged it?
We’d got a proposal for an Odocrypt upgrade, it still needed a bit of work, but yet again: Progress was being made!!
Planning the release of 0.21, Jareds return
On January 14th 2021, Barry messaged me to say that 0.21 had been released by the Bitcoin Core team. Given things were working so well with 0.20 at this point, we were both kind of loathe to do things based off 0.21, but at the same time if doing a larger release such as we were with DigiByte version 8, it kind of makes sense to be as “up to date” as possible.
Unfortunately, Bitcoin changed a lot about the way that transactions are handled, doing rebroadcasts and the likes, which directly impacts how Dandelion transactions work.
Still, Barry got to work on it and had things mostly operational.
In early January of 2021, Jared claimed he would setup a new social network based on Mastodon, but he hadn’t been able to get it to work. He was claiming that Amazon could not be trusted, Telegram could not be trusted, which is especially ironic given he has since setup a Telegram channel “for his followers, because he keeps getting censored”. Despite all of our differences, in the second week of January, I unblocked him from Signal and offered to help him with libre earth. He gave me server access, and I setup the libre earth server for him so it was ready to go. The relevance of this will be clear soon.
At this point Jared had still not been involved with DigiByte since he spent half a day on the video-call working on 0.19, some 12 months earlier.
Now there was a signal group-chat created with a bunch of people to discuss how DigiByte 8 would be supported with it’s release, the testing that needed to be done, how we could contact exchanges / pools etc. This groupchat included the likes of Rudy, Hans & Rutger from the DigiByte Foundation, ycagel & js555, Laura & Michelle from DGBAT, DigiContributor (the gentleman who’s arguably committed the most code to DigiByte over the last half a decade), gto90 who originally started the Android / iOS wallets, and also myself.
As Barry wasn’t on Signal, he wasn’t participating in this, but seeing as these were mostly non-coding related discussions, he hadn’t been too concerned. After a little while, seeing as Jared had emerged from his slumber like a Michael Bublé christmas carol, he was invited to the group too.
Barry had been working on the 8.20 code on the DigiByte-Core GitHub, and while still not a completely perfect “release-ready” codebase, it was highly functional with just a handful of minor bugfixes needed, which had already been explained to the team in that chatgroup, Jared included.
Jared, to his credit, initially sounded like he wanted to help Barry with the codework being done on 8.21. He asked on the 18th of January if Barry could commit the work he’d been doing rebasing off Bitcoin 0.21 to the DigiByte-Core repository, to which Barry agreed with the disclaimer that it was far from operational and that the most functioning tree was 8.20. This was because Bitcoin Core had only released it on the 14th of January, a few days prior.
So with that disclaimer, we advised Jared (and the others in the groupchat) what we knew needed fixing still from 0.20:
We had advised there were issues with MacOS building which were being worked through, we had already identified the changes needed in Dandelion, and a few other things. However for being a rebase on top of code that was released just a few days prior, it was still very much operational!
From the above messages of his, we can see Jared initially seemed like he would work with Barry and help with the 8.21 codebase. Instead things very quickly took a turn for the worse, we saw the very opposite, with allegations of “nothing work” and being broken in “thousands of places”:
Which was funny given I had it working, SmartArray had it working, Barry had it working, and I’d even posted screenshots on Twitter about it. We later learned that Jared had been talking with ycagel, and advised him that Jared had broken his build-environment on his Mac, and he apparently couldn’t even get DigiByte version 7 building (Which was why he never participated in 7.17.3, but more on that later).
Things VERY quickly became hostile, with Jared throwing out things such as:
However, we have concrete proof, because Jared has been working with BTC 0.22 codebase for two weeks now at the end of July ’21 (with SmartArray doing most of the heavy lifting) and it’s still not as operational as the work that Barry had done. Jared has since also stated this to both me and ycagel, in May 2021:
A far cry from the 3–5 hours.
The unfortunate thing is that there was nobody else technical enough to call out Jared in the 8.21 groupchat (as gto90 usually simply ignored it, and DigiContributor rarely checked in) when he would make ludicrous statements such as that.
In addition, I have to wonder, if it would only take 3–5 hours to get things working, why Jared left 8.19 in such a mess after the 4 hours of effort in January of 2020 the year prior?
I was on a video-call with Jared, ycagel, and DigiContributor as they did the work, in January 2020. There is nothing further from Jared, on his 8.19 branch:
It also raised the question of why Jared hadn’t done this if it was such a priority over the previous 12 months, despite allegedly working on DigiByte? To this day we still haven’t been able to find any commits from Jared after January of 2020 throughout that year.
Nonetheless, back to 2021 we can see that Barry committed code for 8.21 originally on the 19th of January, just 5 days after Bitcoin released it, and only the day after Barry had begun working on it:
Jared looked at it and decided to throw it all out, that he didn’t want to work with Barry, SmartArray or myself, and would start over from scratch himself:
Jared seems to imply that I “wouldn’t let him communicate” with Barry? It’s worth clarifying I had advised Jared several times that Barry was on Telegram, but not Signal:
Meanwhile, Jared was on Telegram, and trying to promote it on Twitter:
But oddly enough later on in July of this year, Jared tried to say that he doesn’t trust Telegram, and didn’t want to be on Telegram, despite never mentioning that at the time (as he was busy promoting it?):
The irony that he’s now started a Telegram Channel for one-way communications with people after claims of being censored, does not escape me.
So there was actually no attempt by Jared to reach out to Barry at all, but yet he says that I’m trying to stop Jared communicating with Barry? If you don’t understand how all these kinds of mental gymnastics takes place, then good, we’re both on the same page.
Back to January though, we had the division really starting to show once Jared had started to work on things without anyone else. It was this following comment that specifically had the others in the groupchat most worried, because Jared had clearly gone off, on his own at this stage:
This was the first time Jared drew a line in the sand, creating the division.
Jared decided to throw out all of the work that (predominantly) Barry had been working on, and on the 20th of January, Jared began to work on his *own* unique DigiByte update, without anyone else (Almost 12 months to the day after he had been working on 0.19). Again this flies in the face of his comments above saying that nobody wanted to work with Barry, quite the opposite when it was Jared who started working on his own.
This is very important because the narrative of “I’ll work with people, let’s all work together” at this point was well and truly shattered, and we could see it was just lip-service while Jareds actions showed a completely different story:
Jared did some very basic fixes over the next 3 days, but everything was still VERY broken and wouldn’t even compile. All this, despite claims from Jared that it would work after 3–5 hours. The final commit he did was on the 23rd of January:
Again, we were told he wanted to be “working together”, just not working with SmartArray, Barry or myself. Jared, instead, opted to work solo, of his own volition.
This has become a common theme, that things must be done “Jareds way” and under his name, or otherwise he throws a tantrum. This original code is still available online and still does not compile despite us being told it would take 3–5 hours (Or was it 2–3 months?).
Despite all of this, the version 8.20 that Barry had been working on was mostly done, with a few minor bugfixes still needing to be polished off. Again this code too is still available and can be compiled by anyone to test as we had been doing extensively.
More delays, and the new dev process
Following on from this drama from Jared, several members eventually expressed grave concern, predominantly from Jareds aforementioned comments about two competing codebases now that he was starting solo. This concern was mostly due to the fact that outside of myself and Jared (who were clearly at odds), gto90 wasn’t commenting on Jareds antics, and DigiContributor had been busy and not around the groupchat (as DigiContributor is usually uninterested in the ‘personality politics’ and just likes to write good code). There was nobody else who could really independently advise them on if Jared was telling the truth when he said things would take 3–5 hours or not. Similarly there was nobody who could advise if the 8.20 / 8.21 that Barry had been working on was “broken in thousands of places” vs the existing codebase (or that which Jared had started on himself).
Oddly though, despite the fact Jared claimed to want to work with others, no alarm bells went off for the others in the group after Jared threw out all the work of Barry, myself, and SmartArray, and started solo?
Despite sharing evidence of things such as this, with DigiByte 7.17.2 failing the tests in 14,000+ ways, nobody else in the groupchat would question Jared.
Instead, on the last week of January, the other non-technical people basically asked for a “time-out” on development. Basically, that no further development from anyone continue until they had a better understanding of things (makes perfect sense) and that Jared wouldn’t cause any further division.
As a move to have something more ‘independent’ that even non-technical people could understand, a suggestion was floated around the first week of February for a new development method where all code would be reviewed by independent sources from the people who wrote it. Not only would one person need to review it, but two would. GTO90 was spearheading this suggesting, and said that he would put together a presentation discussing why this was needed, and how it would all work.
In the middle of February, he put together a document, predominantly asking questions around “how do other projects do this?”.
Although there were no direct proposals in this document itself, the suggestion eventually became “we should basically adopt the same method as Bitcoin, and stick with the Atlassian GitFlow Workflow process”. This is something I’ve been a fan of, as having code audited and checked by others is always a good idea. Again, this just makes good sense.
However, getting everyone together to discuss this didn’t happen throughout February. March rolled around, and no further progress was being made around what process should be adopted, just more waiting.
April rolled around, and by then I was personally incredibly frustrated that nothing more had happened. I’d been advised the PDF would be updated with more details, a more in-depth description, but 7 weeks later there was nothing. Talking with Js555, and then GTO90, they confirmed the proposal was for the Atlassian GitFlow Workflow method, and I suggested to them that we needed to just go ahead with it and make it happen. There was no sense in waiting longer for a more in-depth document, nobody had any objections (we just hadn’t all been able to get on a video-call to discuss it at the same time) and frankly I was incredibly frustrated with so much waiting for 7 weeks with seemingly no advancements. So, it was a case of “Let’s just do it”, and we could tweak and adjust things if we needed to down the line.
This was what spurred the Medium article talking about a new development era for DigiByte at the start of April.
Fantastic, so this new document is out, nothing has happened for a few months, now what?
Fixing the Unit Tests
Well, in March, Barry had already begun to work on fixing some of the unit tests (despite the request of no further coding from the others in that groupchat, but still…) in a version 7.17.3, based on 7.17.2. Then, SmartArray continued this in April. The goal was to have 7.17.2 effectively re-released but with fixed and working unit tests.
These fixed unit tests would provide us with a good baseline to work from, as we would need to ensure DigiByte version 8 also adhered to those same tests, and wouldn’t break the consensus inadvertently before whatever upgrade-point. However, with going up a major version, and the plan to hard-fork and bring in a new algorithm means that the tests would have to be modified regardless.
Clarification: Hard forking basically means that previous clients won’t work after the fork-date, not specifically that the upgrade is contentious. For example, replacing SHA256 with ProgPoW, is a “hard fork” because 7.17.3 and 7.17.2 don’t understand the hashes generated by the ProgPoW algorithm.
However, basically all of those who were previously in the now disbanded DigiByte 8.21 channel were all in agreement that we needed baseline to work against, with the exception of Jared who has stated that he doesn’t feel 7.17.3 should ever have existed:
Yoshi volunteered to carry on with fixing up all of the other unit tests, and this was all done in April following the new development process, as Pull Requests with independent reviews before accepting the code.
During this time most of the unit tests were reviewed with an ACK from DigiContributor, and a utACK from GTO90. Although we should have an ACK from two people as part of the process, having one ACK and one utACK was enough in a pinch given we were unable to find anybody else technically competent to go through and review them all. This would have held us up, however, it was a concession even GTO90 was happy to make, adjusting our process a small amount so we could proceed.
Allow me to explain the meanings: ACK (short for “acknowledged + accepted”) means “I have looked over this all, I have reviewed the code, I have tested it thoroughly myself, and this is good to proceed”. On the other hand, utACK (short for “untested acknowledgement”) means “I have not tested this at all, but agree with the principals behind this that are being proposed”. This is roughly enough the industry standard for these meanings.
Why is this important?
The infinite supply issue
Jared had already disappeared again since the January events, a move not uncommon for him. However, he returned halfway through the unit tests being completed to ACK just two requests, one of which was the fabled infinite supply issue:
You see, it was very odd for Jared to be ACK’ing code (Meaning it had been tested), when he’d recently mentioned he had broken his development environment on his Mac. He also confirmed to ycagel when asked shortly thereafter that he never actually reviewed the code that SmartArray had contributed.
However this really came back to bite him when it turns out this was the code that he had “reviewed”, that confirmed there was actually an infinite supply for DigiByte (not capped at 21bn) and that the supply curve was actually VERY far off what it should have been (emitting 90DGB in 2035, not the near-zero expected).
Despite asking for where the 21bn supply-cap was in the code, Jared was unable to show this. Even though there were now copious amounts of people stating this was an issue, he blindly denied it, and nobody else who had looked into things could find the supply-cap. Jared attempted to state that there was no hard proof, though SmartArray had already contributed that:
This GitHub “issue” (which Jared was pointed to, countless times) had both the code where the issue was, charts showing the supply curve, detailed breakdown of the issues and more:
All of which Jared “allegedly” reviewed when he ACK’d Pull Request 33.
Clearly not.
You see, Jared had been in such a hurry at the time to be able to say he contributed to 7.17.3, that he hadn’t actually realized what he was doing, and the code that he was acknowledging and approving. He has later since stated “I was simply trying to show gratitude” amongst other changes in his story, none of which are at all related to the industry standard meaning for “ACK” when it comes to reviewing code.
Unfortunately, despite Yoshi, Barry and Matthew all acknowledging the infinite supply issue, not to mention other developers who had never been involved with DigiByte were able to see the issue very quickly, it was not enough for Jared and he consistently told his Twitter followers the contrary.
Jared outright denied this was an issue until at the end of July when he finally changed his tune, but instead, implied this was something miners can bypass with a single line change:
Which is crazy because the supply-cap + emissions curve IS a consensus level change. This was simply an attempt to discredit the fact that me, a non-developer, had managed to use basic arithmatic to re-add the maximum 12bn supply cap back in, now meaning we only needed to have the supply curve smoothed once more as Matthew had done the math for in January.
Jared unfortunately went through and removed all of his Tweets related to the matter, prior to disabling his own Twitter profile.
Still, despite all of this on the 13th of May, we managed to get DigiByte version 7.17.3 out the door, with help from js555 and ycagel, SmartArray and DigiContributor.
Where is DigiByte 8 then?
Following the release of DigiByte version 7, nothing at all happened for several weeks.
GTO90 had setup gitter channels back in the start of May, and Jared finally joined 6 weeks later in the middle of June. It was dead though, no discussion was happening, no progress was being made.
Jared was still missing from anything coding related, aside from causing drama over his inability to acknowledge the infinite supply issues. Nobody else was commiting any other code.
A simple one-line update / fix from Jolly-Pirate had gone unacknowledged for weeks, until I specifically mentioned to SmartArray I had checked it on the day it was submitted and it was fine. That was when he also checked it over:
Nothing was happening, so I figured in July if nobody was going to do anything, again it’d fall to me to push things forward and so I’d ask Barry again if he would finish off version 8.21.
Barry had already started it, but, had stopped all the way back in January when he was asked by all the non-technical people in the aforementioned groupchat.
Cue the fundraising for version 8 on July 7th!
This however caused further problems when ycagel decided to write a message from the DigiByteCoin twitter page specifically directing people away from the fundraiser, stating that the “official” fundraisers took place in Gitter, as did discussions around development.
Regardless of this intentional stiffling from DigiByteCoin, and from Jared (who was actively re-sharing the “don’t support Barry” message), funds were raised and a few thousand in DigiByte was reached after a week.
This really seemed to… well, shall we say “motivate” Jared? Given he immediately took the DigiByte code and began, again from scratch, on his own.
At no time, despite knowing fully-well where all the existing code was already (On the DigiByte-Core GitHub no less), despite knowing Barry was on Telegram, regardless of the fact many in the community had just contributed to pay Barry to get a Pull Request for version 8 ready, did Jared ever reach out to Barry about any of the code / work already done.
On the 21st of July, Jared announced he would be taking two weeks of full-time work to give “110%” effort towards DigiByte Core. Jared actually gave part of 4days, before he gave up (or at least, has stopped with any commits).
Barry has done most of 8.21 already, and the work is still further than the current state of Jareds (which is predominantly SmartArray) at this point. I’ve even posted side-by-side screenshot comparisons:
Again despite Barrys code being online at github.com/digibyte-project/ to use / work with / help with, Jared instead decided to work on his own.
So we now have Jared, doing the very same thing he got mad at Barry for doing: Forking the code, working on it, and trying to get a pull request ready.
We now have Jared, doing the very thing he got mad at me for doing: Giving people an update on where development was happening, before a pull request had been made back to DigiByte-Core organization.
We now have Jared doing the very thing he got mad at Barry for doing: Nobody else aside from SmartArray looking over the code prior to the pull request being made.
You know what would have been better a better solution from Jared / DigiByteCoin?
If they really needed that history in the same branch, given Barry had already done this twice with DigiByte in relation to BTC 0.20 and BTC 0.21, would have been for them to reach out to him (or me even) and say “Hey can we contribute to this fundraiser (for the first time) and ask Barry if he’d be willing to do it as a merge rather than a rebase?”.
No such luck though.
Where does this leave us?
Well that’s a really good question. As of now, Jared has created a second codebase that he’s been super lucky somebody else has managed to finish off . After the 26th of July, SmartArray has very clearly done most of the work (And we’re lucky he’s such a kick-ass developer).
We’d fundraised for Barry who’s put some more time into it and get things finished off with the 8.21 codebase he’d been working on, but it’s pretty clear at this point Jared / GTO aren’t interested in that and are instead hiding behind their new rules that have come in to play since after Barry did most all of the work. Despite this, the version of 8.21 that Barry has been working on has ProgPoW, RandomX, lowered memory requirements, startup optimizations, sync improvements and more, compared with what Jared has done.
A lot of this stems from the issue that Jared and GTO90 don’t want to change branches to look at commit history. You see we have some fundamental problems with DigiByte Core at the moment that we know of:
- Infinite supply curve (they’ve not merged in my fix yet)
- Supply emissions are skewed
- Users aren’t able to send from older wallets
- Fee guesstimates are not working
This stems from the fact that over the years we’ve merged and merged and merged and merged, without reviewing the code. Talking with both Barry, SmartArray, GTO90 and DigiContributor, they all agree that the current DigiByte code needs reviewing. We’ve found issues, there’s likely more we don’t yet know about.
So there’s two ways of going about these upgrades: Rebasing, or merging.
Here’s the trouble: We have limited developers, limited talent, and limited hours. We all agree on that. What we don’t agree on is what to do with the 7.17.3 codebase.
We have two competing theories:
- Take Bitcoin Core 0.21 (or v22 now) and rebase off it. This is where it’s the Bitcoin codebase is taken as-is and used as a sound starting point. Then, we bring in ONLY the neccessary changes to make it “DigiByte”, getting rid of a bunch of guff and outstanding issues in the process. This would require a review of JUST the changes, and the parts that make DigiByte “DigiByte”. Developer history would be in a separate branch and require a 5 second change to “look it up” (Even better, if you really need, keep a separate terminal window open with it and just alt-tab).
- Take DigiByte 7.17.3 and merge each patch that’s gone in to Bitcoin 0.18, 0.19, 0.20, 0.21 and then 22, leaving everything in DigiByte still in there, including the developer history (with the benefit of requiring no additional time when referencing). This would mean that all of the code, across the board needs reviewing, every single line in every single file, including all that’s being changed with merging-in Bitcoin to ensure that how it was done did not bring in any inadvertent issues. It would also require reviewing all of the DigiByte code too, both historical and current, which is where the majority of the time would be spent.
So option #2 will clearly take a LOT longer up-front, there’s literally millions and millions of lines to go through of both DigiByte and Bitcoin codebase, as opposed to only reviewing the “DigiByte” side of things.
For reasons I cannot grasp, this is the hill that GTO90 and Jared seem to want: to die on: Rather than changing git branches and checking the git history, they would rather spend far more hours doing a merge instead of a rebase, and then countless more reviewing every single line (IF it even happens) of code that goes in to DigiByte and that has historically ever been in it.
When we have other developers who’ve done the work and are fine with that commit-history being elsewhere, it’s clearly not a requirement for contributions from people.
In the Gitter discussions, DigiContributor seemed to agree with Barry and I that rebasing did make sense. SmartArray has also said that while a rebase isn’t without issues it’s still potentially a good idea.
So we’ll likely have a completed “version 8” which is still realistically version 7.22 at that point, given there’s no changes aside from bringing in Bitcoin Core updates. There’s none of the RandomX, none of the ProgPoW, none of the updated Odocrypt, none of the aforementioned performance enhancements. Those are still all absent from Jareds repository despite being mostly finished in Barrys.
But it’s clear at this point to me that it’s Jareds coin, and nobody is willing to speak up against him, which is why I’ve decided that I’m stepping away.
We tout decentralization of mining, despite it being more and more centralized every day.
We tout security, despite the fact that SHA256 is a present and immediate risk.
We tout being forward-thinking, but Jared roadblocks everything that he’s not done directly himself. We’ve had just two unique upgrades in the last almost 6 years now: Dandelion & Odocrypt. Everything else like Bech32, SegWit, they’re all merged from Bitcoins existing released codebase.
So until Jared actually steps away and allows the project to flourish, I’ve decided I’m no longer interested in humoring his facade.
I don’t know what will happen with the ProgPoW and RandomX work, I’m truly sorry. I don’t know what will come of the beautiful new branding / logo that spiral created for us that would truly help bring some refined branding and a glorious new styling to DigiByte.
Perhaps Jared can finally get down from his high-horse and talk about the code with Barry like he should have almost a year ago now. But I doubt it.
Timelines as promised
So here’s a longer timeline of all the work since the DigiSpeed upgrade which I mentioned previously, was the last time prior to Odocrypt we had “new” code in DigiByte:
And a more zoomed in one for the last few months, if anyones wondering why things have been taking so long (Hint: Every time Jared returns and throws out anything that was done without him):
Barry stopped working on 0.21 when he was asked by js555 in January, pending the development process decision. Even once that had been done in April, Barry started to help with Unit tests but had other day-job commitments that prevented his full-time attention.
Still, Barrys code remained online, and Jared could have helped with it at any time. He could have pulled it, found the code compiles, and then helped him with the last few fixes which we saw I’d told him needed addressing in the screenshots above. The whole rebase vs merging has only come about in the last couple of months, it was never mentioned back in January.
Instead, twice now, Jared has decided not to play with others, even though myself and SmartArray had already been working with Barry on things.
So it’s pretty clear this is “Jareds coin”, and until that changes, well I guess that I’m out.
Nothing has changed in the last 8-odd months, people still pander to Jared. Nobody will call him out. Nobody else questions him. When you do, at point blank, he’ll ignore you, make things up, or go on the attack (and ignore the issue):
Maybe one day that will change?
Maybe one day DigiByte will realize the aspirations of being truly forward-thinking?
Maybe one day DigiByte will actually sort the gaping security issues?
Maybe one day DigiByte will genuinely be welcoming of new people, and will actually collaborate with them?
Maybe one day DigiByte will fix the mining that grows more centralized with every day?
Maybe one day DigiByte will have more brand recognition, and won’t be confused all the time with Dash as it is currently (probably why DigiByte fell from the Top100 briefly in July ’21 while Dash has remained stable in / near the Top50)?
At least I can say: I tried. I tried really damn hard!
I’ve been here for the last two years trying to push things forward despite Jareds predominant absence. Jared hasn’t pushed anything forward, nor has he contributed anything aisde from 4-odd interviews in those same two years. He’s stiffled progress at every step that he can.
One day I hope DigiByte can proceed without him pulling the strings, calling the shots, roadblocking all progress. Until that day, I wish the DigiByte community the best, but I’m out.
And for the record: Jared was told about the vulnerability with SHA256 April 5th, 2018. The very vulnerability that permitted the DigiByte blockchain to be doublespent on numerous times in mid-late 2018.
I came to Jared on July 31st to advise him it looked like DigiByte had been double-spent on earlier in the month (which I noticed while I was reviewing the Orphans chart for a Medium article I was writing). Jared spent the next week trying to claim that it was a Sybil Attack, even sending me back to exchanges to tell them that, because he refused to acknowledge the issue (sound familiar in 2021?):
The reality is there was nothing else going on at that time, and that too was made up by Jared. You can go back and look over GitHub even.
For the record: Rebasing would likely have avoided that in the first place, where merging increases the chances of these kinds of issues.
This is also why ALL code, existing and new, needs reviewing.
You’ll note that despite these messages being in April 2018, work on 6.16.3 didn’t start to happen until August 6th, a whole week after I told Jared there were double-spend issues occurring on the blockchain: https://github.com/DigiByte-Core/digibyte/commits/archive/6.16.3
Now naturally when I told Jared that SHA256 was a major issue in December 2020, and I’d done the math to prove it, he ignored it and tried to say I was crazy. He still refuses to acknowledge it to this day, and we know how well that went last time, despite me bringing it up regularly:
Just remember, Jared doesn’t actually understand how MultiAlgo nor MultiShield work, we had Matthew correct him earlier this year (Matthew was right, in this video @ 2:14:07):
So good luck with dealing with Jared, because I no longer can.
As such, I don’t think I can be around while he’s participating and carrying on with all the lies and misdirection.
