I really think Lisp's strength is also its biggest weakness. It's so flexible that it causes massive fragmentation. Everyone has their own toy interpreter. There's a massive schism between Lisp and Scheme. Nobody really seems to have rallied around Common Lisp, and thus it's not actually very Common in any sense of the word. On top of that there is no canonical implementation of "Lisp".
Lispers love to say "Lisp is a family of related languages" which is great but daunting to a beginner, and I think also deters people from contributing to the ecosystem, which is where I feel Lisp really falls down. There's just no ecosystem of reliable, standard-ish packages.
There's Ironclad for crypto, but in the readme it explicitly says not to consider it secure because there are trivial side-channel attacks against arithmetic depending on your implementation (again, fractured ecosystem)
There's lquery, CLSS, and plump for web scraping, but you can't scrape anything modern with them and the author seems to have abandoned the libraries
There's a GPIO library that simply doesn't work
Et cetera, et cetera, et cetera.
Compare that to something like Go or Python, where there are rich, polished, full-featured libraries for nearly every application you can think of.
I thought about writing a bot to scrape finance data in Common Lisp, but I didn't want to have to write the scraping and DOM traversal logic from scratch. I thought about using Lisp to have my raspberry pi monitor my hobby garden, but the GPIO library doesn't work at all and I didn't want to write my own.
I picked different languages and was immediately productive thanks to wonderful community libraries.
This is a weakness that is talked about more than it actually matters, to my mind.
In diving into libraries for Common Lisp, I'm finding more in the realm of "basically works as I would expect" than I would have expected based on how much this weakness comes up.
What is still jarring for me, is that image based development is just not taught or cultured. The iterative based approach that can be used in such an environment is just... so foreign to everything I have ever seen/done before.
Oddly, jupyter notebook style build ups are getting there. With the scary realization that a linear realization of the cells in your notebook is something that simply takes effort. Not something that is required for results.
I'm not sure what you mean by "no canonical implementation" -- there's the ANSI Common Lisp standard, of which there are multiple implementations. Is this somehow a bad thing to you?
> Lispers love to say "Lisp is a family of related languages"
Not really. I think it's slightly more common for Scheme or Clojure programmers to claim Lisp relations, but it's just advertising. If having s-expression-ish syntax (with nothing being compatible to CL like a real Lisp dialect would be, and was given how much code in pre-CL Lisps could be run in CL unmodified) makes it part of the "Lisp family", then why not speak of the ALGOL family? It's tiresome semantics and from what I've read Common Lispers at least don't tend to do it.
> There's just no ecosystem of reliable, standard-ish packages.
Yes, it's not as impressive as Maven. Yes, not everything is as well-documented as, say, PyGame (http://www.pygame.org/docs/ -- though not every Python library you'll find with pip is, either) or even the Common Lisp Hyperspec (http://www.lispworks.com/documentation/HyperSpec/Front/Conte...). Yes, some things are broken, or even abandoned, though in the latter case it may just be that they're "done" pending anyone coming up with a new issue or feature request. I've happily used web scraping libs to save my HN comments with their context, there's a good Twitter API lib I moved to when something broke the Python one I was previously using, I've orchestrated Selenium, I generated some FFI bindings to FMOD and had it working within a few minutes... My point here is just that there is an ecosystem for CL, and many things are reliable, when something works it's rare to stop working. Sorry about your GPIO experience, it happens, I had a bad experience trying to quickload a library that depended on iolib which opaquely depends on a custom C library the author also made and is not in my distro's package manager.
I'm not sure what you mean by "standard-ish" -- is Python's urllib3 "standard-ish"? What about urllib and urllib2? Do they qualify merely by being part of the standard library, or is it something else? My other guess is you mean something like "X is a common area of development in the current year, are there any packages dedicated to that area, what are they, and are they any good?" But this is a shaky categorization and in other languages changes all the time, not worthy of the attribute "standard" or even "standard-ish".
You're right though that what does exist is small and limited, and that's a natural limiting factor to growth, far more I think that what editors are available or easy to use. Quicklisp came about way too late. Yet the ecosystem continues to grow, so maybe you'll be willing to give it another chance in another 10 years.
I don't see how you can say with a straight face that no one rallied around Common Lisp given knowledge of Lisp history.
What do you think went wrong, then? (Assuming of course that you do think something went wrong!)
One observation I’d make is that both Scheme and Clojure seem to be much more widely used than Common Lisp. My perception is that Scheme is still somewhat popular as a teaching language, and Clojure is somewhat popular as a real programming language for large-scale projects, but I barely hear about anyone using Common Lisp these days.
If I’m right, I think that adds to the feeling of fragmentation, as the official standard isn’t the most widely used variant.
I think ML is in a similar situation. I hardly hear of anyone using Standard ML, but Ocaml is alive and kicking.
Maybe my perception is wrong, though? Are there industries I’m overlooking where CL is bigger than Clojure?
A lot has gone wrong in terms of achieving high adoption, but specifically about something going wrong with rallying around CL, I don't think anything went wrong. No more are Maclisp, Interlisp, Lisp Machine Lisp, Zetalisp, Franz Lisp, Portable Standard Lisp, Spice Lisp... They were all slain or subsumed by Common Lisp. (You might have seen Franz elsewhere in this thread; rather than continuing it Franz, Inc. just developed Allegro Common Lisp separately. Spice Lisp meanwhile changed to become a CL implementation, became CMUCL, which was later forked into the now most popular implementation SBCL.)
Le Lisp/ISLisp are interesting European competition that didn't fall in line, but I don't ever hear about them, I only know they exist/existed, they might nowadays be effectively dead for all I know. Emacs Lisp is probably the biggest success in not caving to CL. Not big enough to constitute anything "going wrong" though.
I think your perception is wrong in two ways. First is the idea that Scheme and Clojure are somehow "variants" or "dialects" of Lisp. Scheme was never a Lisp dialect, it was instead described as "a LISP-like language, SCHEME, based on the lambda calculus, but extended for side effects, multiprocessing, and process synchronization." Also notice neither Scheme nor Clojure even have "Lisp" in their name, unlike all those other languages that got eaten by CL. "Lisp" meant something, and "Common Lisp" unified that meaning and I think deserves to be synonymous with "Lisp"; many writers have treated it that way. But Common Lispers are giving up that fight, because it's tiresome but also an understandable confusion not helped by Scheme or Clojure's attempts at capitalizing on some primordial idea about the good name of Lisp or whatever drives them to associate with the term. (#lisp in Freenode used to be only for Common Lisp, now in Libera #lisp is for all Lisp-likes and CL has its own channel.) Anyway, Scheme and Clojure have happily had their own evolution and separate largely incompatible s-expressions. I don't think their continued existence is a flaw against CL any more than another random programming language would be. One aspect of Clojure that might sting a little is that its entire reason for existing was because the author couldn't win political fights about having CL in production instead of the JVM.
The second way I disagree with your perspective is on prevalence. Scheme has had some success in teaching (mostly thanks to SICP) though Common Lisp was/is also used similarly at various places, however I think that's hurt [Common] Lisp more than anything. (Basically CL gets taught like Scheme, and so whether CL or Scheme is used students come away thinking they "know Lisp" without ever really having seen its OOP power, its handling of types, its condition system, its easy-to-define macros, let alone the trivial things like LOOP or SETF that make imperative programming possible and easy. It's like C++ classes that teach it as C-with-classes, but worse.) Scheme has also had success as GNU's official extension language (with Guile, which goes a ways beyond standard Scheme to be useful) and you see Scheme pop up in places like GIMP plugins. Racket seems like the most popular modern Scheme, aimed at being suitable for real work, but it has gone waaaay beyond standard Scheme and slowly seems to be becoming as large as CL. Real stuff is presumably made with it, not just education stuff or scripts or extensions, but I'm less familiar with what's going on. It may yet eat CL's lunch.
Clojure of course has been a rising star and has enjoyed a lot of success in real stuff. It's popular, it's fashionable, and in terms of projects-per-second your perception is probably right that it's more widely used than CL right now. Where I would draw disagreement is in total pound-for-pound code that's Out There. CL has the benefit of decades of existence, so for example https://www.ptc.com/en/products/creo/elements-direct has been developed for a long time and is made of several million lines of CL code, and that's just one project. If you only used "active" (i.e. someone executed it over last month) code perhaps there's enough Clojure out there now to be an interesting race though there's no way to really tell; if you allow for all the CL that has been written and is no longer run, I don't think there's any contest, CL has such a rich history. (A random application being Mirai https://en.wikipedia.org/wiki/Mirai_(software) with demo https://www.youtube.com/watch?v=-IRsYGfr4jo -- has there ever been a 3D modeling program written in Clojure? Will there ever be?)
https://github.com/azzamsa/awesome-lisp-companies is an ongoing collection of companies known to be using CL. In terms of "industries", right now quantum computing companies seem particularly drawn to CL. Symbolic math historically also, with Maxima and Axiom being still-working/developed code bases. (The latter is a million lines of literate CL.)
But drawing on legacy again rather than last-few-years stuff, an old quote by Kent Pitman seems relevant: "Please don't assume Lisp is only useful for Animation and Graphics, AI, Bioinformatics, B2B and E-Commerce, Data Mining, EDA/Semiconductor applications, Expert Systems, Finance, Intelligent Agents, Knowledge Management, Mechanical CAD, Modeling and Simulation, Natural Language, Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web Authoring just because these are the only things they happened to list." We can of course add more things to the list if that would help, like Mars Rovers or video games, https://news.ycombinator.com/item?id=9510945 has a few more, in recent news I learned about https://www.reddit.com/r/Common_Lisp/comments/osnsgz/intels_... and https://github.com/shop-planner/shop3 was open-sourced in 2019. Is it dominant in any area (besides maybe quantum)? I don't think so. Against Clojure though I'm sure it dominates some areas while not at all winning at others, it'd be a case-by-case thing. Common Lisp is in a lot of places all over the world (it's had quite a legacy in Japan even), but not the most fashionable stuff, so it's also understandable that many people haven't heard about it, realized they're using it, or heard people talking about it. (Edit: one final example that's not too esoteric, if people have ever used pgloader for Postgres, it may surprise them to learn it's written in CL, and even more than that, used to be written in TCL and later Python before moving to CL...)
I think that name fight about the word Lisp is costing the CL community mindshare. Encountered as one of the first things about CL, it's not a good filter if you want to welcome people coming to the language curiously and pragmatically. Unless you especially want to select for curmudgeonly people.
> My other guess is you mean something like "X is a common area of development in the current year, are there any packages dedicated to that area, what are they, and are they any good?"
I think perhaps what Lisp lacks (or appears to lack) is tentpole developers such as - from the Python world - Mike Bayer (SQLAlchemy) or David Lord (Flask/Pallets), or the Django community, or Kenneth Reitz (Requests) who keep high quality 3rd party libraries going for a long period of time.
Eh, it's debatable. CL has got (github usernames) Shinmera, Fukamachi, ruricolist, edicl, among many others (I have to stop my list somewhere). There are some collectives that work on a handful of projects together too. What would help is more word about contributions that already exist -- blog posts (or if possible additions to the Lisp Cookbook (https://lispcookbook.github.io/cl-cookbook/)) and StackOverflow and so on. sabracrolleton has some amazing work for instance cataloging and comparing e.g. the various logging frameworks, test frameworks, json parsers, and more. (Example: https://sabracrolleton.github.io/logging-comparison.html) More of that I think will be more helpful than more tentpole devs, and more ways for Common Lispers to communicate what's out there even to themselves. A lot of unmentioned people are doing heroic work on those fronts already. Another thing that would help is just a critical mass of devs working in a similar area at the same time, but getting such critical masses is its own problem.
Shinmera is one of the big problems in that their web scraping library chokes on CSS selectors with parentheses and their gpio lib simply doesn't work.
Yeah, that's why it's debatable.. Some maintainers do a better job than others, or only with a subset of the libraries they've made. I found your bug (and the go-file-it-in-the-other-project non-response); in Shinmera's defense I'd want to be charitable and attribute most shortcomings to him devoting all his time to his video game.
One interesting advantage of Lisp (and dynamic languages in general, though Lisp's nicer) in cases like these is that if you can fix it yourself, it's easy to load the fix in. By that I mean, for example, suppose (I have no idea how this lib works) the lookup function has a pre-check on the string for invalid search characters before searching, but after inspecting the code you see no reason why it shouldn't work anyway. You can just jump to the code and edit the pattern or remove the error entirely, send the function to the REPL, and you're good to go. Save a copy in your own code if you need to persist your change. You could even do this while the debugger is active and not have to re-request the web page or lose whatever other state you'd built up until that point...
That's extremely cool, but if Lisp is more like Putty than Lego, and you sort of have to mould all features for yourself rather than plugging in some blocks, then it will struggle in this area, I think.
This is where Lisp should be able to kick ss. You can do literally anything you want semantically. I mean looking at the Stallman circuit Lisp [1] from 1975 should show the power language construction inside of Lisp. Most of the APIs I use don't have that expressive power. Another one that blows people's mind is the amb operator [2], and that you can construct new novel control flow directly in the language itself, including in libraries.
I frankly don't understand why the programmers of the world aren't demanding Lisp.
Lispers love to say "Lisp is a family of related languages" which is great but daunting to a beginner, and I think also deters people from contributing to the ecosystem, which is where I feel Lisp really falls down. There's just no ecosystem of reliable, standard-ish packages.
There's Ironclad for crypto, but in the readme it explicitly says not to consider it secure because there are trivial side-channel attacks against arithmetic depending on your implementation (again, fractured ecosystem)
There's lquery, CLSS, and plump for web scraping, but you can't scrape anything modern with them and the author seems to have abandoned the libraries
There's a GPIO library that simply doesn't work
Et cetera, et cetera, et cetera.
Compare that to something like Go or Python, where there are rich, polished, full-featured libraries for nearly every application you can think of.
I thought about writing a bot to scrape finance data in Common Lisp, but I didn't want to have to write the scraping and DOM traversal logic from scratch. I thought about using Lisp to have my raspberry pi monitor my hobby garden, but the GPIO library doesn't work at all and I didn't want to write my own.
I picked different languages and was immediately productive thanks to wonderful community libraries.