Seriously though, what other languages other than Lisp (all the mainstream ones at least) give you the freedom to change the language and/or create DSL's with the same ease?
And you can still do your 'bare metal' in C if you really really need to and bring it in.
I actually think the OP perfectly explains the core problem with Lisp: Part of the value of shepherding is getting everyone on the same page. When everyone's their own shepherd, nobody is on the same page.
Javascript is this on steroids: It wants to be inspired by Lisp, by Self, by Java, and the resulting melange smells like every paradigm while fitting none of them. Trying to write applications in Javascript is trying to corral the minds of hundreds of people who each had a different idea of what Javascript is and what it does and what it wants you to do. The resulting system is doomed to total incoherence.
JavaScript started out as a sort of "Java-flavored Scheme", actually
However, I'd say that modern JavaScript - the language - is much more like C++ than Lisp. It isn't a void of shepherding, so much as shepherding-by-committee.
However, due to how easy its ecosystem has made package management (in contrast with C++), much of that quagmire has been papered over with much more strongly-opinionated (shepherding) frameworks and dialects. This hasn't completely solved the "C++ problem", but it's gone a long way towards mitigating it. Working in modern JavaScript may be wildly different between frameworks, but it's reasonably consistent between projects that use the same frameworks.
In other programming languages there is a consistency between frameworks and a default way to do things: there's no shitload of frameworks each reinventing the same wheel.
Some things are easy doable without frameworks.
In Javascript land you don't learn the language, you learn frameworks. And, some "JS frameworks" like Angular do not even promote JS, due to how terrible the language is.
If your current framework goes out of fashion in favor of the next shiny thing, you are out of luck.
I am unfortunate enough to having had to learn one of JS frameworks because our web apps are APIs in the backend and I have to do the frontend too and people who started the projects were fans of a particular JS framework.
For upcoming projects I'll use Blazor after it becomes production ready, no more JS frameworks for me.
I don't dislike JS, I quite enjoyed old-school ES6 + jquery. I even like Vue because is very customizable. I have a strong dislike for big opinionated frameworks like Angular.
What's mostly resulted in Javascript land is people using opinionated (and often severely constrained) boilerplate generators like create-react-app, with opinionated linters like Prettier (alongside transpiling Javascript from other languages like TypeScript), that force a coding still despite the myriad ways to achieve things - I've actually found that to mitigate a lot of these issues well, but it can't deal with the boatload of "bad" advice online to wade through.
Even with opinionated generators, what happens when you pull in a library that comes from a very functional mindset, a second that treats JS like Smalltalk, and a third that was thrown together by a novice and has become the de facto standard for its purpose?
The issue doesn't just lie within the baseline of the code you write, but in how many disjoint dialects that code must interact with and partially conform or contort to in order to engage with.
If you have a word to say and if you have to use a framework, take a look at Vue. It's not opinionated, it doesn't get in the way, it doesn't come with the kitchen sink.
It's been said, "Lisp takes things that are technical problems in other languages, and turns them into social problems." Don't underestimate the human factor of team-based development.
>> I think the advantage to Lisp is that the programmer can generate and evaluate arbitrary expression trees at run time.
That's the one! Code is data - data is code.
While this can be done in other languages, it isn't done without considerable effort or going 'off road' so to speak, macros are Lisp.
cheers.
Isn't eval part of the cause of many security holes in other languages, e.g. Javascript? That is, not eval alone but combined with an unintended path from unvalidated user input to the snippet that gets eval'ed.
Is there anything special Lisp does to prevent user input from getting into eval'ed data-code? Or any sandboxing provided by Lisp's eval?
I thought this was super common knowledge, yet I reviewed a co-workers pull request the other day and found files filled with eval (easily 30 usages) as well as global state (treated and mutated as if it was thread local).
The most annoying part was that what they wanted to do didn’t even require eval and they refused to fix it even after I’d found a safe, non-eval way to do it.
This rule is true for Javascript and PHP too, yet it happened all the time. If Lisp relies heavily on eval, what specific protection measures does it employ?
Mainstream Lisp dialects do not "rely heavily" on eval. Its use is broadly discouraged in favor of other mechanisms like apply or macros.
Newbies sometimes learn about backquote before learning about apply, and when they need too pass a dynamic list of arguments to a function, they end up writing (eval `(fun 1 2 ,@args)) instead of (apply fun 1 2 args). Or doing some metaprogramming using (eval `(defun ...)) in the middle of a function, instead of making a function-defining macro.
In JavaScript and PHP, not to mention numerous other languages, eval is the only meta-programming you have. If you need to generate code, you end up using eval.
Moreover, eval is textual. Textual code requires very careful escaping to avoid injection problems. Lisp's eval is AST-based, so it doesn't suffer from that.
If I have a user-data variable that holds untrusted user data and put it into an evaluated code template, as in (eval `(list ... (some-api ',user-data) ...), I can completely trust that user-data will not "go live". It's inserted under a quote, and that's that. There is no way that content can bypass the quote, no matter what object it is.
Lisp has macros, read-macros, and eval. These things enable DSLs. You're saying strongly-typed functional languages have features of equivalent power for this purpose?
Yes, for example F# has quotations, type expressions and type providers.
Also .NET has attributes, compiler plugins and expression trees, which allow to do some kind of similar stuff even for C#, although not as straightforward as in Lisp.
On the Java side, you also get the attributes (aka annotations), compiler plugins, AOP (yay CLOS interceptors).
Haskell has Template Haskell and OCaml has ppx extensions.
It's all about ergonomics though. In Lisp, it's super easy, and even fun with macros. They can probably do roughly equivalent things in the other languages, but it's so fucking painful it's relegated to rare use and for relatively short code sections.
For Lisp users, it's like holding the knife by the handle. For them it's like holding the knife by the blade.
Not exactly equivalent, but monad-based DSLs have almost the same power as Lisp macros.
You can't escape the syntax restrictions, the same way you can't escape Lisp's syntax restrictions. And it's a bonus for Lisp (in power) that its syntax is much more flexible. But in semantics they are equivalent.
> what other languages [...] give you the freedom to change the language and/or create DSL's [...]
FORTH. It's a very similar language from that respect. One of the first things you learn how to do as a FORTH developer is to rewrite the interpreter/compiler words.
I really like Forth and if I ever get into IoT that's what I would use over C any day. Forth works best from a clean slate though I think (from my admittedly limited experience), working in with a host OS takes a bit of fun away I found.
almost all languages can pull in c, so that doesn’t differentiate.
being able to dsl, you have to ask how often is that useful? what happens when 10 dsls are built into a code base and you hire a new person? how hard is it to make sense of everything?
The comment about C was more to do with perceived speed issues with Lisp but for most situations, Lisps that compile to native code are more than fast enough for general application programming.
The point of the article rings true, languages like C/C++ and those based on them do shepherd you into how they work and if you're doing low level programming for drivers etc then they work well, that's their domain (and you wouldn't need half of C++ if it stayed in that arena!). When trying to 'express' or abstract a problem then you have to shoehorn your thoughts into the language.
>> being able to dsl, you have to ask how often is that useful? what happens when 10 dsls are built into a code base and you hire a new person? how hard is it to make sense of everything?
Probably not as hard as trying to decipher swaths of source code in a language that doesn't make it easy to create the DSL's, instead you end up with API's and code that tries to hide the ugliness of being pushed around by the language.
With Lisp you are not so much creating API's or DSL's, you're extending the language to suit the problem, not waiting for the language to catch up. This is freedom!
This is freedom and it does enable creativity and can increase productivity indeed. But I've also seen it hurt maintainability as well: the original programmer moved on and now nobody can decipher their genius DSL.
That's a shame, you hear this a lot about lisp code and I think the problem is that the people who really grok lisp and write 'genius' code can be a bit blase about documenting their code.
Simple documentation for standard function definitions is fine but macros definitely need special attention. Some say macros are over-used but I think it's more of a case they're under-documented.
Even if the name of the macro gives you a fair idea, documenting how it works and what is generates with examples goes a long way to deciphering them for maintenance.
cheers.
Absolutely. In fact I am writing a library in Elixir currently (for internal company usage) that mandates generating boilerplate to help a user project. It's a really fine balance to (a) not being too clever by making the macro code still readable and (b) documenting the intent of the macro, its input and output, and why it is actually useful.
Many people are like "I'll document it later" and it just almost never happens. Which is, as you said, a shame.
Seriously though, what other languages other than Lisp (all the mainstream ones at least) give you the freedom to change the language and/or create DSL's with the same ease? And you can still do your 'bare metal' in C if you really really need to and bring it in.