What's Next for the L1: Part 1 Clean Code

This is the first in a three-part series that will go over the background and reasons for an upcoming set of improvements to our ecosystem. While I hope that the context for the changes will be helpful and interesting, they are not required to understand the final outcome. These posts will be published over the next couple of days and are summarized as follows:

Part 1: The Need for Clean Code

How and why so much work has been done that is not clearly visible to our community.

Part 2: Blossoms of Clean Code

What the future may hold for a code base that has had its code cleaned up.

Part 3: The Future of our L1 and How it Benefits You

What you can look forward to as a participant in the Divi ecosystem as these improvements take place.

So let’s get started with this series!

As we grow and adapt as a crypto project, we have to accommodate several forces simultaneously. While we could probably come up with a long list of these forces, it is pretty safe to say that there are three main ones: the state of the crypto market, the regulatory landscape, and our zeal for betterment, uniqueness, and utility. So, we will find from time to time that significant changes may happen in our ecosystem. We have had them in the past and will always have them periodically going forward for as long as this is a vibrant project.

To provide the most significant contribution to the community to date, we will take some necessary actions to improve the Layer 1 blockchain and lower infrastructure costs so that more grant funds can go towards new use cases and other groups and businesses.

Background

Let me take a moment to (re)introduce our lead blockchain engineer, @DeathByJanitorialCombat, as he wishes to be referred to in this public document. He joined Divi almost four years ago. He has a Master's degree in Mathematics with strong experience in algorithmic modeling, statistical modeling, and software development. He didn’t join with a crypto background, but he’s now one of the rare, unique minds shaping the future of blockchain. His relentless passion for learning and ingenious approach associated with a very high sense of ethics makes him one of, if not the biggest treasure of the Divi Project. This statement is my opinion but his temperament won’t allow him to agree.

@DeathByJanitorialCombat has worked diligently on our core for over three years. Forking another project's core means forking its code with all its features, poorly executed functions, and poorly written code, making it extremely difficult to upgrade, add features, and maintain. The foundation has done this diligently for years, but the last three years have seen so much code improvement that the original core is barely visible. Soon, we will deliver a post by @DeathByJanitorialCombat on why clean code is important and how it leads to significant benefits in the future, which we are already starting to see internally. But until then, let’s dive in just a little bit.

There are many resources on the internet available that you can go to regarding ‘Clean Code.’ There are entire textbooks written on it and multiple principles and protocols for achieving it. However, the compass he related to me is called SOLID, and if you wish to learn more about it, you can head over to that link or any of hundreds of relevant links about it. There is a holy mantra that all good programmers should adhere to: "The first step towards change requires that you make changing easy" (paraphrase from Kent Beck, software developer).

‍

Don’t worry if you don’t understand every aspect of this. It’s important to understand that starting from Bitcoin 0.1.0, the code everyone forks and iterates on has continued a slow march toward looking more and more like spaghetti rather than getting closer to a SOLID paradigm. Let me explain with a picture.

These pictures indicate the architecture of our core, before and after code cleanup. The image on the top is the ‘before’ for the Divi core. Now I get that this seems simpler than the ‘after’ image on the bottom, but it is not. Both have virtually the same functionality. But you can see that the functionality of the core has been broken down into sub-modules. In fact, the image on the right could even show more modularization within each of the smaller boxes. Why is this modularization good? Because if there is a problem with the mempool, our developers can go straight to the mempool code to debug it. Where do you go in the old model inside the Chain Extension Logic block? Those are not modules listed there. It is functionality. The offending code could be anywhere in there! Where do you start?

There are other benefits of cleaning up code (which is more than just about modularization). For example, by grouping functionalities properly, developers can ‘see’ better if the functionality is achieved efficiently and securely, far better than trying to trace code through a gauntlet of different functions grouped blindly together. Another important aspect of having it cleaned up is that it is easier to be creative about what the functionalities do and how to apply them. Let’s just say mobile masternodes would have been impossible to implement without the code around masternodes being cleaned up and modularized. They were linked to other aspects of the code that were irrelevant to masternodes.

@DeathByJanitorialCombat refers to aggregating poor code from version to version and fork to fork as “Technical Debt.” Let’s just say Satoshi, while brilliant, was a capable but not an expert coder and every project that forked his code inherited that technical debt and added more. To excel and prosper, we are working to end that cycle for our project. We are doing it with our internally built code for the back end and the wallets.

The core has seen over 4600 commits, and around 240,000 lines of code have been refactored. The similarity of our core code to its original fork is close to zero. It’s hard to see this change as a Divi user, it’s hard to imagine the work, and there is great difficulty delineating change and improvement while this effort goes on. Another reality we must embrace along this way is that the core developers are working with a codebase and need to hot-swap old with new while not losing a single fraction of a Divi for any user. It’s tricky and requires a tremendous sense of responsibility. Further, there is lore in the world of blockchain that it can solve every problem if you just put it on the blockchain. But almost all of these features have chain and immutability destroying side effects! So we have to be careful as to what the core actually does, and what needs to reference the core instead.

Our follow-up article to this will be on how this clean code can help Divi into the future in creating better and stronger architecture for our blockchain and be able to deliver better and more useful products for you… Stay tuned for the article landing tomorrow.

‍