Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Adding Modules to C in 10 Lines of Code [pdf] (nwcpp.org)
138 points by pmarin on June 27, 2022 | hide | past | favorite | 88 comments


ImportC is shaping up to be one of the greatest feature of D

Being able to just import your C code, and call it from C, or when you need to go back to C and resume your work by just importing your D modules

I tried it, while it still need some finish, it worked very well! (you had to put your files through a preprocessor though, hopefully this step won't be needed in the future)

Every modern language should include a C compiler, it is stupid to give up the entire C ecosystem!

No wonder Walter's name is "Bright"!

To the people who still doubt about D "because of the GC", it doesn't exist for me, i am making an online 3D game targeting WASM without it, with all the language features!

For cases when you need one "if you use D as a scripting language", it will be very helpful to have!

https://www.kdom.xyz/

If you love C, but need something modern with more safeties, give D a try!

D is a pragmatic language, very powerful, covers everyone's needs!

If you tried it before, now is the time to give it another try!


D is by far my favorite language. I put in a lot of effort to learn the language and thoroughly enjoyed discovering D's elegance. It has super clean solutions to all sorts of language issues (e.g. obj.foo() is just syntactic sugar for foo(obj), which gets you both type extensions and OO-looking methods on structs, which I miss in C).

That said I really tried to use D for my projects, but I had to give up for a rather surprising reason: the C interop is so good, that 1) most libraries provide a 1-1 translation of their C APIs, which ends up being ugly, non-idiomatic D that forces me to think in both C and D when coding; and 2) debuggers are not aware of D types and idioms, so when debugging, I have to again think in both C and D. Both of those add up to about 90% of the coding time, which is to say that, 90% of the time, when using D, I felt I had to code in 2 languages at the same time.

I'll skip some of the other issues I ran into, because I think a lot of the problems with D would go away if it had a large active community that would put the work in to maintain the D ecosystem, but that's a bit of a chicken and egg problem.

In the end, I decided that for me the reduced language overhead, solid ecosystem and modern conveniences of gnu17 C were more valuable in practice than the sweet features that D had to offer, and that made me a little sad, but I'm hopeful that one day D will make a strong comeback.

I know Walter gets notifications when D is mentioned on HN, and I imagine that if he read through this he'd shake his fist at me for saying interop-so-good-its-bad, but, if I could make a parallel with Java, I'd say that in code that uses many 3rd party libraries, D feels a bit like coding with JNI all the time (sorry). Ironically, in my opinion, D would benefit from having a community that rewrote popular libraries, instead of primarily relying on C interop.


The neato thing about obj.foo() goes even further. Ever seen code like:

    a(b(c(d(e),3))
? Not very readable. UFCS (Universal Function Call Syntax) enables it to be written as:

    e.d.c(3).b.a();
Reading it flows naturally left-to-right.


Thanks Walter, that's a great example and I like that, because parens are optional, it could also be written as

    e.d.c(3).b.a
which is even cleaner.

For those who are thinking UFCS is a trivial detail, consider that the shell and some other languages have pipe operators (|>) to make the code flow intuitively the same way as the data.

In my opinion, in a C-like language, managing to squeeze so much functionality out of the '.' operator without any downsides is the mark of a well thought out, elegant language.

Thank you for creating D.


It is cleaner, but unfortunately it is not explicit. It is a function or a variable? I used to love those things until I noticed its defects.

For example, in C++, a = b can invoke anything. Not sure it is a good idea (except for generic code, there it is useful).

Zig has a philosophy of nothing hidden that I think it is mostly good.

That said, I find D a very nice language, the only problems are:

1. small ecosystem 2. last time I tried, packaging of download and use was... improvable.


Isn't it amazing how the . operator can cleanly replace :: and -> too?


Absolutely! Speaking of operators, out of curiosity, what's the reason for using '!' with templates?

Naively, I would think that making template instantiation look the same as a function call would be a desirable feature, with ambiguous calls needing to be resolved by the user.


Not many characters left in ASCII. Reuse of an existing operator almost required. Binary operators cannot be use as it would be grammatical ambiguous and would need resolution at the semantic pass which is a big no-no (that's why C++ is so slow at compiling, it cannot be parsed without semantic analysis). This left only the two exlusively unary operators !, ~. As ~ was repurposed for string concatenation, only ! remained.

    templ!thing(a,b)   
I would have thought that templ(thing)(a,b) would have been a good solution, as it is what is used in the declaration/definition side of templates, but this would have made removing redundant () not possible in UFCS expressions.


i would have preferred:

    templ<thing>(a, b)

because with that:

    templ!thing(a, b)
is it a function?, or are you calling thing's function?

you can do:

    templ!(thing)(a, b)
but did you mean?:

    templ!(thing(a, b))()

i personally always use !(), no matter what, and it's annoying to type, i don't want to waste time constantly trying to figure out what is what, it's mentally draining


debugging has significantly improved, it works great for D types https://github.com/Pure-D/dlang-debug

kotlin became very useful for focusing on being able to consume Java code

it allowed them to have a huge presence on android, that's enabler

it profits Zig as well

not everything needs to be ranked #1 in TIOBE index

there is value in being the way it is, it's organic, and no companies get to control its faith

> the C interop is so good, that 1) most libraries provide a 1-1 translation of their C APIs, which ends up being ugly, non-idiomatic D that forces me to think in both C and D when coding;

what do you mean? it's the same, function and data

    struct Data {}

    do_this(&myData);

this is valid D, it's also valid C

the problem i think you have is you are abusing OOP and think it's the only way of doing things, which is wrong, and this explain the sad state of software nowadays ;)

but it's weird when you then say you decided to stick with C, you contradict with yourself


> ... you are abusing OOP and think it's the only way of doing things

> ... when you then say you decided to stick with C, you contradict with yourself

Figments of your imagination.


Thanks for your comment. Please rest assured I am not abusing OOP, and am absolutely not contradicting myself by choosing C.


those pretty-printers are useful, thanks!


The latest ImportC incarnation will run the preprocessor automagically for you. It will also snarf up all the macro definitions, and turn the ones it can into manifest constant declarations.


I've been learning Zig and it seems that Zig and D have a lot of things in common. With Zig one can readily import C code as well, and with `c-call-convention` any Zig function can be made callable from C.

However, I've run into lots of bugs in Zig so I got a little disillusioned... how do you think D compares with Zig? Is it able to produce as efficient and small binaries, cross-compile to most platforms, metaprogramming etc?


Zig is still new, so it has room to improve

While i like Zig, it has many ergonomics issues that i just can't deal with.. and it has the tendency to make your code unnecessary verbose

- no operator overloading for your math type, you end up chaining methods..

- no function overloading, you have to do things like 'add(comptime T: type, x: antype, y: anytype)`

then: add(f32, myX, myY);

then be prepared with a shit ton of @floatCast(), it can't guess what type you use so you forced to be doubly explicit..

you end up with code that is barely readable

- unused as error, this one hurts a lot.. you can say goodbye to fast iteration time, if you make a game it'll be painful

Zig is still great, i plan to use it for some project, knowing how to use more language helps train your skills and understand pros/cons of features better

You can crosscompile with D too

    ldc2 -mtriple=x86_64-windows-msvc -c foo.d
Metaprogramming is great, you get more power with D, mixin, proper templates, type introspection

You can link without the runtime if size is a concern, i haven't had any issues with it


While D is great, it also has some issues

- float default is NaN, why... just WHY!!!!!

- char default is 0xFF....

when you expect things to be just 0, it is a pain to deal with and to remember...

Also the standard library, while useful, i wish it would make use of allocators

They have ``std.experimental.allocator`` packages, for some reason it still is experimental..

There is no tagged union.. you have to import a package for it ``std.sumtype``.. wich is bad when all other languages have built in support for that

Walter if you read this, please! union are useful, but lack of tagged union mean potential bugs in your union when you pick the wrong value...

Proposal for you:

  enum MyTag { A, B, C}

  union MyTaggedUnion: MyTag
  {
      DataA A,
      DataB B,
      DataC C
  }

  struct DataA{}
  struct DataB{}
  struct DataC{}

  auto tgu = MyTaggedUnion.B;
  switch (tgu)
  {
   case A:
   tgu. /* implicit DataA */
   break;
   // need to implement every tags, or error
  }


> float default is NaN, why... just WHY!!!!!

> char default is 0xFF....

Invalid values, perhaps? Other than making initialization mandatory, this seems like a reasonable way of catching them. For integer types it's not really possible because all their values are valid.


I grew to agree, but that doesn't mean that's not annoying.. i'm nitpicking


Why is your proposal better than this:

  import std.sumtype;
  
  void main(){
   struct A { int a; }
   struct B { string b; }
   struct C { bool c; }
   
   alias TaggedUnion = SumType!(A, B, C);
   
   auto tgu = TaggedUnion(B("hi"));
   int i = tgu.match!(
    (A a) => a.a,
    (B b) => b.b.length,
    (C c) => c.c * 5
   );
   assert(i == 2);
  }
If you miss out a `match` handler for any of A, B, C you get a compile-time error. Or you can use a generic handler which will be instantiated for any type not explicitly handled. https://dlang.org/phobos/std_sumtype.html


I'd take a language that have built in support for tagged union over having to import a module and rely on templates

I know of sumtype, and i think it is a mistake for D to rely on this for such important language feature

I'm not a language dev, so i can't do much to help, using switch/union/enum is more natural, is cleaner, and is easier to add support for IDEs, everyone on the same page

That's in areas like this where the language falls short, and i can see people instead choosing alternatives

https://github.com/dlang/phobos/blob/master/std/sumtype.d

a mess of templates and imports to std.traits

using this will make your compile speed tank.. another reason to avoid

it's very sad that people recommend this instead of asking for built-in version, makes me sometimes reconsider my choice to use the language

if they expect language improvements to be templates, what atrocity will be added next?

this is on the same level as std::bool from C++


https://forum.dlang.org/post/[email protected]...

> The good news is, I have not put a lot of effort so far into micro-optimizing the compile-time performance of match, so there is almost certainly room for improvement.

it is shame that it ended up in the standard library

once Walter will be gone, i will have no faith in the language anymore


bool is a built-in type in C++...


> float default is NaN, why... just WHY!!!!!

All operations that are fed NaN as an operand produce a NaN result. This makes it very obvious when the initialization of a float has been neglected. Defaulting to 0.0 means uninitialization bugs are nearly impossible to detect.

> char default is 0xFF....

Same thing. It's intended to flush out uninitialization bugs.


> how do you think D compares with Zig?

Disclaimer, I've never used Zig, so the following are not comparative.

> Is it able to produce as efficient and small binaries,

There are compiler flags to disable linking to the D standard library and D runtime if you need small binaries (you can still use templates that end up in you binary) though most of the time that is more of a loss than gain (in terms of productivity), its not bloated unless you abuse templates a lot as would happen in C++. As for speed of execution, don't use DMD if you care about the it. LDC (the LLVM D Compiler) and GDC (which is part of the GCC) are both strong optimisers.

cross-compile to most platforms,

Yes, use LDC. GDC, like GCC, is not a cross compiler by default though you can build it to be, DMD is X86(64) only.

> metaprogramming etc?

Oh, yes. Compile time function evaluation (referred to usually as CTFE) in combination with mixins (interpret string as code), `static foreach`, `static if` and (much safer than C++) templates makes for a very potent package.


If I want to use D as a scripting language, I would like to have access to at least the core parts of the standard library (strings, lists, hashmaps, etc.), which I can’t without GC. I really don’t have time and energy to implement my own data structures when trying to prototyping gameplay code.


> If I want to use D as a scripting language, I would like to have access to at least the core parts of the standard library (strings, lists, hashmaps, etc.), which I can’t without GC.

How is it a "scripting language" if you don't want a GC? I use D for the vast majority of my scripting these days, but it would be odd for me to avoid the GC. I would not even think about using it if I had to deal with that.


I wasn’t talking about GC in general, but the GC in Dlang. With the GC turned off in D you can’t use most of the standard library.

You can certainly use the std without relying on GC in languages like Rust, Swift, etc, but the important thing is that you can’t in D. I’m sure it can be technically done, but nobody has actually put the effort to do it.


The plan is to finish https://dlang.org/phobos/std_experimental_allocator.html

Once finished, everything in the std will be able to make use of it

If you can't wait, you can use this package already with the allocators: https://github.com/dlang-community/containers


Ok, I’ll try it out after the feature stabilizes.


How much ram does your script use?

Note that many game engines do use garbage collection for this purpose


All the scripting languages I know of rely totally on the GC.


D's standard lib is also a lot more mature than Zig's, obviously because of its age. I'd recommend D above Zig unless you want to contribute bug fixes to Zig.


But isn’t most of the D standard library unusable with BetterC mode (without GC)?


BetterC is not no GC mode, BetterC was made to make sharing D code easier with C

GC only exist if you call ``new MyThing``

Just use your allocators with malloc/free, that is what i do (you do the same in zig, allocators)

if you want to enforce no GC use in your APIs, there is the @nogc attribute you can use


> Just use your allocators with malloc/free, that is what i do (you do the same in zig, allocators)

If you do that, is it RAII like C++/Rust? Objects get destructed at end of scope, and members get destructed recursively? Or do you have to manually call `free()` on everything? (which, granted, is significantly easier in languages with `defer` mechanisms)


D has defer

    scope(exit):
    scope(failure):
    scope(success):
https://tour.dlang.org/tour/en/gems/scope-guards

You can do RAII with D too, it's your choice, based on your use case


Yes, it is RAII and RAII is fully implemented in D.


But only for structs.

We really need to get actual reference counting into the language to cover classes properly as well.

It has been on my todo list to write a DIP for this for a while now. ~rikki.


Class instances can go on the stack too using `scope`:

  scope obj = new Object;
  ...
  // destroyed at end of scope


RAII, though there is also `scope(exit)`


BetterC is a hack designed to make it easier to migrate C code to D. It turns off everything that requires a runtime not just the GC


Another C alternative that people might want to check out or keep an eye on is Vlang (https://vlang.io/, https://github.com/vlang/v/blob/master/doc/docs.md).


I just completed changing the D druntime library C code to be compiled with ImportC. Dogfooding forever!


> No wonder Walter's name is "Bright"!

And he made a video game called Empire. I wonder if his character name is Heisenberg....


This is a neat little prototype, thanks for sharing. And the Naruto running makes it an instant winner in my book :)


Game doesn't seem to respond to touch input on my phone, Chrome Android.


Input system is still being worked on, i've been focusing on mouse/keyboard right now, i should treat touch the same way as mouse input, it basically act the same anyways


Cool! On keyboard, I can't seem to move north or west.


Interesting.. what is the layout of your keyboard?


I have a US keyboard that's part of a recentish MacBook Pro. I'm playing from Japan but my request headers should say I'm en-us.


I've done something different before:

    // main.c
    #include <module.c>
    int main() {
        moduleFunction();
    }
    
    // module.c
    #include <stdio.h>
    void moduleFunction() {
        printf("calling modules");
    }
This has its issues (you can't do parallel compilation anymore; you don't have unexported functions, etc), but it works fine for small things. Super helpful when I want to make a small thing and don't want to bother with headers.

You just include all the C files in one file and be done with it.


It also automatically enables "global optimisation" in most cases.

This style is called "amalgamation" and SQLite is distributed (but not developed) in this way.


Does this impose limitations on the code? I think all functions, even private, should be "namespaced" (i.e. in C - prefixed) then? Kind of awkward, but otherwise amalgamation is a great idea.


C doesn't have "private" functions in that way, what it has is "static" functions, and this is indeed the biggest limitation of doing this kind of amalgamated build.

If you mark a function as static, it means that has internal linkage and is only visible inside the translation unit was defined and no symbol is exported for linking. This is not the default: if you just define a function with some name, that symbol is in general visible to all other translation units that gets linked together by the linker (though with no type information attached, just the name of the symbol. Making sure the type information is correct is the purpose of header files).

That means you can have any number of functions marked static in different translation units with the same name, and it's fine, they don't see each other. If you're doing an amalgamated build like this, that no longer works: all symbols have to have globally unique names.

In addition, of course, you don't get the "separation" that "static" provides. You can't call a static function across a translation unit boundary, but since this just one big translation unit, anything goes: all functions can call all other functions, even if they shouldn't.


> Does this impose limitations on the code?

It does. Any technique that relies on features specific to translation units, such as taking advantage of internal linkage to reuse symbols to hold data specific to translation units, cannot be used anymore.

Also, even though global builds are faster, you forego the ability to do incremental builds which makes working over a bug a more time-consuming task.


TIL it has a name. Doing what sqlite does gives you the best of both traditional and single header projects I imagine.


> This style is called "amalgamation" and SQLite is distributed (but not developed) in this way.

These are actually called "unity builds" or "jumbo builds", and some build systems such as CMake offer support for this.

https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.htm...


Afaik this is exactly what Unity builds are. https://en.wikipedia.org/wiki/Unity_build


Yeah. Though most projects still use header files with unity builds, so I decided not to call them that. It wasn't the concept of a unity build that I was emphasising, it was the idea of not forward-declaring everything.


> Yeah. Though most projects still use header files with unity builds, so I decided not to call them that.

There is nothing inherently special in header files. They are just a convention where you use a specific extension in source files you mostly use to pass declarations. Nothing stops you from using the .c extension on all files, and there is nothing special in passing declarations in one place and definitions in another place.


Of course! But, saying 'unity build' puts emphasis on the fact that you compile all the C files 'in unity', in a single CU. What _I_ wanted to put emphasis on is that you can skip the header files, and that coincidentally means you also have to unity build.

The fact that most unity built projects still use headers made me hesitant to use the term, as that would put emphasis on building a single CU, as opposed to not forward declaring in headers.


Build systems like Meson can even enable/disable this with an option ootb.


For smaller C++ projects this tends to be way faster than the conventional wisdom of having many CUs, mostly because all the library bloat will only be parsed and instantiated once.


Also reduces link time.

I find that projects up to around 50K lines compiles instantly as a single file/"unity build" and is my preferred way for most of my projects.


> Also reduces link time.

It's important to stress that it does more than reduce link time: unity builds completely eliminate linking when plugging together submodules, and the final project linking simply has far fewer object files to link.


This really should be solved at the compiler level, rather than forcing people to restructure their projects.


I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language.


It's including another C file, normally you'd just include a "header file" in C and compile each C file separately.


For those who don't use C: header files are more or less an interface description and (usually) don't include any code. The code is added from external objects during the second "linking" step.


Oh, I see: including a header file injects forward declarations into the calling file. Each c file gets compiled separately into object code unless you directly import them like the code above is doing. In that case it essentially dumps the full text of the file into the calling file, which causes serial compilation of a single object file.

Unless I'm mistaken that clears some stuff up for me. Thank you!


That's my understanding as well, so I think you are correct but I am no C wizard myself :)


> I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language.

Unity builds are a very old and widely established concept. I dare to speculate they are older than the regular header and source file split, as C required a full blown preprocessor to split code in separate files and also someone had to come up with the include guard trick to get around the limitations of including the same definition multiple times.

They aren't employed that often because there are some tradeoffs and in large projects it's benefitial to split them in smaller translation units to limit the scope of incremental builds.


On the topic of the presentation, I'd be happy to have a way to kill the preprocessor. Not that I expect that to happen. Reading C code is like reading literature that only makes sense if you know eight languages, four of which are known to fewer than 100 people.


I've often wondered why C++ hasn't deprecated the preprocessor yet. D has hygienic replacements for all the things the C preprocessor does (except the nutburger stuff), and we're quite happy with that.


My guess is that they won't because it would dramatically decrease the subset of code that is both valid C and valid C++ in many environments. As you are no doubt aware, most C -- not all, and not with perfectly preserved semantics -- is valid C++ in practice. I think the C committee would have to do something first. But I'm only an outside observer.


C has already added things not in C++, like _Generic.


C would be a lot harder to write without the preprocessor. I have only really had one truly awful experience with the preprocessor in the last 15 years and it was when reading OpenSSL's code. Even then, I didn't have too much trouble to grok it with just Vim.


I discovered my C code was a lot better when I backed out all the clever metaprogramming preprocesser usage. C could be easily extended with:

1. modules

2. manifest constants

3. conditional compilation

and the preprocessor can be deprecated.


For me, it is hard to imagine to program in C without preprocessor metaprogramming. Obviously, it would be better if generic types and metaprogramming were available directly in the language. Alternative is either copying code and adapting it manually, or depending on generic code with many void * arguments.


If one really needs metaprogramming, it's time to move to a more powerful language. There are plenty of them.


> If one really needs metaprogramming, it's time to move to a more powerful language.

It's wholly wrong to try to frame this as "more powerful", specially considering the fact that C already supports generic selection.

https://en.cppreference.com/w/c/language/generic

> There are plenty of them.

There really isn't, at least in the application domains where C dominates, such as embedder programming.

You'd be hard pressed to find any producrion-level tool chain for embedded development which supports anything other than C or even C++, and any decision to pick up even C++ over C for embedded development requires a lot of soul searching.


This kind of insight was a driving force in the design of https://odin-lang.org/


Nah, the C preprocessor is a disgusting hack. I really don’t understand why they didn’t opt for a more logical, AST-based macro system when the base language has such bad expressivity in itself.


This differs from web applications how? (not that you said it did, but the implication seems to be that C is an exception in software development in this regard)


I'm not a webapp dev, but C/C++ always felt unique via the preprocessor functionality that essentially does text substitutions on source code before handing off to the compiler.

Actual flags are useful, e.g. -codegenfoo -linkflagbar, but I hate digging through a maze of -DFLIPOPTION1 and -DUSERSUPPLIEDTEXT="hahaha" to unscramble what code the compiler actually sees, and what went wrong because of the various user supplied strings (i.e. #define overwrites).

If other languages also have this, well ugh sorry. I have grown to despise it.


Why eight?


The title is a bit misleading. This is about extending C grammar used in D.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: