Of course you will be able to give a brittle, half-assed work around for any issue that someone throws at you. But you know, not all people are obsessed with LISP, not all people are ok with half-assed workarounds.
I think you are being very dishonest to yourself if you think that LISP is universally the best just because you can theoretically turn it into anything. You can't, really. Good tooling matters. Error messages matter. Standardization matters. Syntax matters (to some extent). Mindshare matters (a lot). There's a lot of subtle qualities that you can't just fake.
When I write performance sensitive code I'll personally just pick C. I'm sure there's a Common Lisp solution available to specify machine types and compile efficient code. (Probably there's many, which is another problem). But what's the point in putting lipstick on a pig when I can just use the real deal that works perfectly for what I'm doing?
"Half-assed" is in the eye of the beholder. If you appoint yourself as the ultimate arbiter of half-assedness then your position becomes unfalsifiable.
But I think you might be surprised at how un-half-assed these things can be.
If you think we should not "fall prey to the sunk cost fallacy" and write future software in Common LISP, then I recommend you to start working on improving the Common LISP systems-level coding situation. Because I'm not willing to replace this
unsigned long update_crc(unsigned long crc, unsigned char *buf,
int len)
{
unsigned long c = crc;
for (int n = 0; n < len; n++)
c = crc_table[(c ^ buf[n]) & 0xff] ^ (c >> 8);
return c;
}
That is not exactly an apples-to-apples comparison. You've taken a very simple C function and compared it to Lisp code written in two different ways to take advantage of features available only in specific compilers. Equivalent C code would be chock-full of #ifdefs and look just as ugly. I could produce an equally contrived example where to do the same thing as two lines of Lisp would require pages and pages of C.
Second, these code snippets are not comparable in their functionality. The Lisp code generates the crc table, the C code assumes it has already been done. If you wanted to add bounds checking to the C code you would have to make major changes. If you wanted to add bounds checking to the Lisp code, all you would have to do is change the optimization settings. (This is one of the reasons that today's computing world is a swiss-cheese of security holes.)
Third, there are any number of ways to make the Lisp code look significantly prettier. Comparable code would look something like:
if you wanted do. You could also embed all this in a little DSL so that the resulting semantics and even the generated code would be exactly the same as the C code.
Fourth, most Lisp systems have a foreign function interface, so if you want to you can call C functions directly and so get all the benefits of C (such as they are) from within Lisp.
I don't think it's generating a table, just calling generate-crc32-table. Let's remove that line and the difference is still ridiculous (even if you look at only one of the two implementations).
You really don't see how ridiculous that is? You are chasing exactly the target that is unholy in your eyes. Except, you will never reach it (the lisp version is really ugly, probably has really bad tooling, poor error messages, poor standardization and mindshare, and so on)...). While you could just use the right tool for the job.
> and compared it to Lisp code written in two different ways
question is, WHY was the Lisp code written in two different ways, but not the C code? Go figure.
> If you wanted to add bounds checking to the C code you would have to make major changes.
If you want automated and perfect bounds checking, you need a few more flavours of GC/OOP (because you need some automated notion where the "length" field is, and that length field needs to be authoritative), and you will end up with less modular, less portable code (because you can't just pass sub-arrays / smaller lengths, but always need to pass object handles. Can't support sub-allocators, etc). It's a tradeoff. If you go for automated bounds checking, that's a total different game. Personally I don't like to do GC, and I like to manually insert explicit "bound" checks at strategic locations. But YMMV.
You can easily find C code in the wild written in two (or more ) ways. Search almost any project for #if and #ifdef.
That CRC code is lucky that it's doing right shifts. The unsigned long type has an implementation-defined width, so if any bits were spilling leftward, the result would have to be clamped to 32 bits. If that were neglected on on a platform where unsigned long is exactly 32 bits, it would be a portability bug. In modern C, there is a uint32_t type in <stdint.h>. Oops, wait, no there isn't; it's only required to be there if the platform has a 32 bit type. To test for it, you can see whether the PRIu32 macro is defined (the conversion specifier for printing it with printf).
> question is, WHY was the Lisp code written in two different ways, but not the C code? Go figure.
It seems that someone saw, "hey LispWorks has various extensions that could be used" and rolled a LispWorks specific version.
I don't see the downside. Lisp is a very high level language in which you can easily write highly portable code. And, unlike in most languages in that class, you can also get fast code without leaving the language, at the cost of adding some verbiage.
Honestly, if I had to write a large Lisp application which consisted of almost nothing but the kind of code shown in that CRC function, I would reconsider using Lisp. But that's a strawman situation. Pretty much no application is like that. At best some specialized middleware.
> The unsigned long type has an implementation-defined width...
That's true, but well, it's not doing a left shift but a right shift. For left shifts, you'd usually write something like (c << 8) & 0xFFFFFFFF, or just use a fixed-width type (as you say).
> Oops, wait, no there isn't; it's only required to be there if the platform has a 32 bit type.
Most languages just have the standard integer size be 32 bits, and pretty much all machines/compilers today have uint32_t. If you just make uint32_t a requirement you get basically what you get in these other languages. Alternatively stick with (unsigned) int/long/long long, possibly wrapped in a typedef. I don't see what's the problem here. Plenty of options, pick your portability / explicitness tradeoff.
> You can easily find C code in the wild written in two (or more ) ways. Search almost any project for #if and #ifdef.
Sure, but not for this regular kind of bit-shoveling code. Mostly for platform integration. And yes, a little bit of setup code to support different compilers.
That's 14 LOC, and 8 of those are declarations. For an example that you cherry-picked to be a perfect match for the kind of task that C was designed to do.
> You really don't see how ridiculous that is? You are chasing exactly the target that is unholy in your eyes.
You are completely missing the point, which was:
> It's not that they are without flaws, it's that it is a lot easier to fix/ignore/work-around the flaws than in other languages, where the flaws are unchangeable (except by the standards committee) and constantly in your face.
And specifically, you were responding to this:
> I'll bet that for any flaw you name, I'll be able to show you an easy way to work around it.
You haven't actually done that, you've just exhibited some nice clean C code and some ugly CL code. I was just showing you some of the possible ways that the Lisp code could be improved. I threw the infix in there not because I was advocating it, just to show you that it was possible.
So what exactly is it about that code that makes it "ridiculous" in your eyes? Whatever it is, I'll bet I can fix it with very little effort.
For example:
> Can't see any machine types here.
That's right, I left them out (because you can do that in CL if you want). Is that what you want to see, a cleaner syntax for type declarations? Because that's trivial, a few DEFTYPEs and a two-line macro.
> WHY was the Lisp code written in two different ways, but not the C code?
Because the Lisp code was trying to leverage system-specific optimizations while remaining portable. Do you seriously doubt that I could find an example of ugly #idef-laden C code that does the same?
> If you want automated and perfect bounds checking, you need [a bunch of stuff]
That's right. And why is it that you don't see that in C? Because adding that to C is hard. Really really hard. So hard that in 47 years it still hasn't been done in any standardized way. Yes, it's true that CL is not a particularly good systems programming language out of the box. But turning Lisp into a good systems programming language (if that's what you want) is relatively easy, almost an elementary exercise. And it can be done at the user level. No need to wait for a language design committee.
You want to know what is ridiculous? A language that after 47 years still doesn't have a linked list as part of its standard library because the language design makes it impossible.
C doesn't have linked lists in its standard library probably because there is a myriad of ways of doing linked lists with different performance, stylistic and other trade-offs.
Should it be macro-based, like that <sys/queue.h> thing from BSD? Or just pure declarations?
Many C programs that maintain lists favor the "intrusive container" approach: mixing in the link node into their own payload data structure and then using that structure itself as a list node.
C++ certainly provides lists. C++ is opinionated: they pick a representation (or at least API) and stick it in.
> C doesn't have linked lists in its standard library probably because there is a myriad of ways of doing linked lists with different performance, stylistic and other trade-offs.
No, that's not why. The fact that there are myriad ways to implement linked lists is a fact independent of any programming language. And yet Lisp somehow manages to offer linked lists as a native part of the language.
The reason C cannot be extended in this way is that its memory model precludes it. You can't add GC to C because of the way that pointers work.
> C++ certainly provides lists.
Yes. Many languages do. Nonetheless, my original claim, that it is easier to work around Lisp's limitations than those of other languages, stands.
C's memory model also precludes decent string handling also, yet it has a string library. But that's it; the only other data structuring provided is the qsort function for sorting arrays and bsearch for searching them. I think it's deliberate. It looks like ISO C tends to avoid specifying new library features that can (and likely would be) be written entirely in C.
Making the point. C is not a language for convenient canned solutions. But I'm not too sad about that. There are a myriad ways of doing strings, and most high-level programming languages are still stuck in the 90s with UCS-2 encodings or such, and offering only solutions that don't scale well for, say, dozens of megabytes of strings. C isn't stuck, at the cost of convenience.
>> I don't think it's generating a table, just calling generate-crc32-table.
> Well, yeah. What do you suppose a function called generate-crc32-table could plausibly do?
...
> That's 14 LOC, and 8 of those are declarations.
It's considerable boilerplate, so much that I think you must really be in love with LISP to be able to deny that it is forbiddingly more work.
It's 202 characters in C vs 573 characters in LISP if we remove the initial whitespace from your LISP version. I'm not saying we should do APL, just that C is efficient at what it does best, and that's a good thing.
> For an example that you cherry-picked to be a perfect match for the kind of task that C was designed to do.
I didn't cherry pick at all, just got the first good looking code from github that implemented deflate, as in my comment before ("compression or imaging...").
All I'm aiming to do is to disprove the statement that LISP is universally the best programming language for anything and everything. I'm not the one being bold.
> Is that what you want to see, a cleaner syntax for type declarations?
Of course, that's what I wanna see. Haven't seen it yet.
> Because that's trivial, a few DEFTYPEs and a two-line macro.
Show me and tell me all the reasons why that's not worse than just writing it in C.
> That's right. And why is it that you don't see that in C? Because adding that to C is hard. Really really hard.
Absolutely. It's just not possible. You make tradeoffs when you write C code just as in any other language. And personally I like making the tradeoffs that come with C. That allows me to write self-contained, modular code, and to avoid a lot of boilerplate and duplication.
> You want to know what is ridiculous? A language that after 47 years still doesn't have a linked list as part of its standard library because the language design makes it impossible.
It's not that it's impossible to add one to the standard library. But there is enough crap already in the standard library. There are various ways to do such a thing as a linked list, each with different tradeoffs. And using off-the-shelf data structures just doesn't align very well with a typical C programmer's mindset. Often it's a good idea to just quickly code up your own so that you can integrate it better with the rest of your project. But have a look at Linux'es intrinsically linked list.h (or rbtree.h) for example, or at glib's extrinsically linked list if you want to depend on that and don't care about performance.
If you don't like that, you might just want to achieve different things than me. That's fine with me. Pick your own poison.
No, it's not boilerplate, it's actually functional. Type declarations in Lisp are optional (because they are there only to improve performance, not to change the semantics of the code).
> > Is that what you want to see, a cleaner syntax for type declarations?
> Of course, that's what I wanna see. Haven't seen it yet.
OK, here is one possibility:
(deftype u8 () '(unsigned-byte 8))
(deftype u32 () '(unsigned-byte 32))
(deftype array-of (type &optional size) `(simple-array ,type ,(or size '(*))))
(defmacro c-defun (name args &body body)
`(defun ,name ,(loop for (var) on args by #'cddr collect var)
(declare ,@(loop for (var type) on args by #'cddr collect `(type ,type ,var))
(optimize (speed 3) (debug 0) (space 0) (safety 0)))
,@body))
(defmacro c-let (vars &body body)
`(let ,(loop for (var nil val) in vars collect (list var val))
(declare ,@(loop for (var type) in vars collect `(type ,type ,var)))
,@body))
Now you can write:
(c-defun update-crc32-checksum (crc u32 buffer (array-of u8) end fixnum)
(c-let ((cur u32 (logxor crc #xffffffff))
(table (array-of u32 256) (get-crc-table)))
(dotimes (i end)
(c-let ((index u8 (logand #xff (logxor cur (aref buffer i)))))
(setq cur (logxor (aref table index) (ash cur -8)))))
(logxor cur #xffffffff)))
Better?
> Show me and tell me all the reasons why that's not worse than just writing it in C.
I didn't say it was better to write this code in Lisp. I said it was easier to make this kind of change in Lisp if you want to.
I can empathize with your enthusiasm that you can improve the situation a little on the surface, which feels like an achievement, but frankly, I think the code still sucks a lot... There's no way I could sustain writing the kind of code I like to write in this fashion.
From an engineering standpoint, using this was probably a poor choice (again, think error messages, tooling, mindshare...)
The metaprogramming code looks ugly and brittle to me, although I'm not in a position to judge.
I've already written so much and don't want to repeat myself anymore. Certainly the use of a macro that transforms basically all of the code must be one of the biggest problems, because it makes communication between compiler and programmer extremely hard (tooling, error messages, ...). Besides that, there is still considerable overhead compared to the C version (which was written once here, never changed, and is still effortlessly the clearest version), and I miss some punctuation (not just parens) to be able to make sure everything is closed correctly, indented correctly, etc. Having to manually space out variables and types as pairs is also bad ergonomics. And I miss some operators (assignment, subscripting, distinct function call syntax) that give visual cues that allow the skimmer to quickly find what happens in a function.
Fun story: Larry Wall once said, Lisp looks like oatmeal with finger clippings mixed in. I can't disagree, and it's a serious problem for ergonomics.
The wide-spread production practice of calling C libraries from dynamic (and other) languages is pretty much eating the world.
The naysayers are being tripped over and trampled by throngs of doers.
There are excellent reasons not to write the whole darn program in C, just because it happens to need a SHA256, CRC, access to graphics, or some number crunching or whatever.
I think you are being very dishonest to yourself if you think that LISP is universally the best just because you can theoretically turn it into anything. You can't, really. Good tooling matters. Error messages matter. Standardization matters. Syntax matters (to some extent). Mindshare matters (a lot). There's a lot of subtle qualities that you can't just fake.
When I write performance sensitive code I'll personally just pick C. I'm sure there's a Common Lisp solution available to specify machine types and compile efficient code. (Probably there's many, which is another problem). But what's the point in putting lipstick on a pig when I can just use the real deal that works perfectly for what I'm doing?