Intro
Although I didn’t post anything lately, don’t worry – I’m still working on a proposal for dependency management. I arrived at the conclusion that not only dependency management needs to be taken into account and standardised. Also building software with the defined dependencies need to be taken into account when creating a standard for dependency management.
We all know that the C++ standardisation Committee goes to great lengths to provide standards that are backwards compatible. Dependency management is a different case because there is no standard yet. Implementations of tools for dependency/package management and building software do exist. But it will be undoable to create a standardisation proposal that will be backwards compatible with existing tooling and projects. They all have their own background and starting points, these tools don’t have a common ground of a previous standard. People have done a lot of intelligent and magic stuff to get dependencies fulfilled during a build (some people applied even more magic when they needed a cross platform build) and to get their toolchain build the software they wanted to have. I will not propose magic neither will I create a proposal that takes all the magic into account. A standardisation proposal for dependency and build management (hey, do you see that I’m using a broader scope now – I wil get back on that in the next paragraph) is unique in the sense that this will be the first standard for dependency and build management for the C++ ecosystem (yes, ecosystem not platform). Because it will be the first standard, backward compatibility doesn’t come into play.
Nevertheless, I will take a lot of existing good practices (and tools) into account. I take the lessons from all the things that the community has tried seriously and use the knowledge rooted in these lessons to create a standardisation proposal. As a community we know what works and what doesn’t work. But I definitely feel like I have a clean sheet of paper to start drawing.
The whole community is explicitly invited to provide me with ideas and feedback, so we can end up with a proposal that supports the majority of C++ software development.
Scope
The scope of my proposal for standardisation is declarative dependency and build management. This is a broader scope than envisioned in my first blog on this subject. The broader scope is needed, because dependency management has a strong coupling with build management. In the end dependency management is needed to create stable and reliable builds for software packages.
A package repository is not in scope, although I do understand that such functionality is needed for actual implementations. The same goes for build systems. I will not take build systems in scope. And yes I do understand that an implemented build system is needed to produce the artefacts.
The end result of the standardisation process will be a definition of ‘stuff’ to be used by tool builders and end-users (developers, etc) to create and understand tooling needed to create reliable and reproducible builds for software written in C++.
Declarative
I take the same stance as Mathieu Ropert in his article about ‘Simplifying build in c++ (part 3)’. The scripting route is a dead end from a standardisation perspective. Although scripting can be more flexible it is very difficult to specify such an approach. It is also more difficult for end users to maintain a build system based on scripting. In the previous mentioned article it is clearly stated (and I agree to what is written there in this regard) what the issue is with a scripting based build system.
The big difference between the two (ed; script vs declarative) is that the first one is much more complex. Since it’s a program by itself, it’s very hard to analyse for correctness and can’t easily be modified by a machine to handle something the author didn’t expect. The more logic and branches it contains, the higher the chance that it will fail on a particular setup or environment.
And of course I understand that underneath a declarative build system there is a program or script that uses the declaration to create the artefacts, but from the user’s perspective the declarative approach is much easier to use and understand.
First iteration
Alas, I don’t believe in first time right! The approach for creating a standardisation proposal is an iterative process. I like to start with the first iteration today and we (yes, you the reader and I) will have several iterations in the upcoming months to create a proposal for SG15.
Where to start?
A lot of things need to be addressed. It is tempting to start with discussing a file format and just start listing all items that need a setting with regard to dependencies and building. I resist that temptation an start by defining categories. These categories contain items that need to be addressed. The categories will be designed in a follow up post, for now I like to get to a list that is fairly complete.
- Package declaration
- Source rules
- Default behaviour
Package declaration
When building software a lot of settings/configuration is needed to be able to create executables or binaries. All these settings are part of the package declaration. Package declarations can exist in several places. From the perspective of a dev that builds an application that has some dependencies, the first package declaration will the one that he or she creates for the application. This package declaration will contain dependencies. These dependencies itself will also have a package declaration (which can have dependencies, etc). Items in this category are about source and include organisation, C++ standard (versions supported), dependencies and dependency versions, etc.
Source rules
The rules that need to be obeyed to make sure that the sources can be used by tooling that uses the dependency and build standard. It will contain rules that will prohibit issues with name clashes when an executable or library is built with sources from several authors.
Default behaviour
This is a textual description of the standardised (expected) behaviour when declarations for items are not provided, can’t be found on the platform or when declarations of items are not compatible with declarations in dependencies that need to be fulfilled.
Next step
First of all I like to know whether these categories make sense to you as a (library) developer and tool developer?
Secondly, what do you think is missing in this list?