This looks very similar to how I test my configuration management tool Holo: I just run the compiled binary in a chroot and compare its output and the resulting filesystem tree to the recorded expectation. [1]
So no unit tests at all. I think that unit tests work very well for pure functions, but their utility decreases the more side effects your code has. Also, unit tests usually just add confusion when the unit tested is really an internal implementation detail, rather than refering to some object in the application's or library's external domain model.
I personally see that as a unit test; Plenty of unit tests use real files on the filesystem in their mocks, you're hermetically sealed from the rest of the system, how is it not a unit test?
I'm just arguing vocabulary, though. There seems to be a spectrum, pure unit tests on one side, then integration tests, with Canaries way off to the other side.
> I assume that the reason why most of these failing tests are failing is not a regression in core functionality of Make but rather because of new features were added to GNU Make since 2006.
There were a number of bugs in the 3.81 release that I assume they now have tests for, since 4.x has been rock solid. (Off the top of my head, if you made a certain shape in the dependency graph with .PHONY targets, then .DELETE_ON_ERROR stopped working)
The author already changed the link (as mentioned below). On that note that---with some GNU projects use GitHub as a mirror---GitHub fails GNU's ethical repository criteria for hosts:
Savannah is home to the core development team's repository. The author mentions this in the article. However, the source code is available in more than one place. With GNU projects, it is not uncommon.
You got to be kidding. Please read VERY CAREFULLY what is Git [1] and what is GitHub [2]. The fact that GitHub only hosts Git repositories is only a matter of choice, Git repositories can be hosted in any server without any (big) restriction. Please restrain yourself from making useless comments if you don't have enough information about it.
I feel badly for the person who didn't realize you were sarcastic, passionately defended the idea that git != github, and got down-voted as their congratulations.
So no unit tests at all. I think that unit tests work very well for pure functions, but their utility decreases the more side effects your code has. Also, unit tests usually just add confusion when the unit tested is really an internal implementation detail, rather than refering to some object in the application's or library's external domain model.
[1] https://github.com/holocm/holo/tree/master/test for the core parts