What does HackerNews think of clasp?

clasp Common Lisp environment

Language: Common Lisp

Putting on my architect's hat--what is it the customer really wanting to do? So..

"Why do you want to make Lisp more popular? If you were sucessful, what would be different in the world, and why is that desirable to you?"

Normally at this point I'd listen to the response, and ask more questions based on that. That would wind up with a very, very deep thread, so I'll break a cardinal rule and pre-guess at some answers.

This kind of question comes up pretty frequently. In many cases, I suspect the motivation behind the question is "Wow! Here's this cool tool I've discovered. I want to make something really useful with it. I want to do it as part of a community effort; share my excitement with others, share in their excitement, and know that what I'm making is useful because others find it desirable and are excited by it." The field could be cooking, sports, old machine tools, tiny homes, or demo scene. Its the fundemental driver for most content on HN, YouTube, Instructables, and such. It is a Good Thing.

If that is your motivator, then my suggestion is to find something that bugs you and fix it. You've already decided you're only interested in code, not other aspects. You said you preferred vim, but the emacs ecosystem has a very rich set of sharp edges that need filing off, and a rich set of tools with which to attack them.

One example: even after 50 years there's no open IDE which allows you to easily globally rename a Lisp identifier. I don't know about LispWorks or other proprietary environments, but you can't in emacs or vim do a right-click on "foo" in "(defun foo ()...)" and select a command which automatically renames it in all invocations. [Queue lots of "but you can..." replies here.] I don't think vim is up to the task of doing this internally. It would be possible in emacs; but would require a huge effort with lots of help from other people. If you emerged alive from that rabbit warren you'd join the company of Certified "How Hard Could it Be?" Mad Scientists such as Dr. "I just want to draw molecules" Meister [1] and "Wouldn't an OS in Lisp be Cool" Froggey [2].

[1] https://github.com/clasp-developers/clasp

[2] Mezzano https://github.com/froggey/Mezzano

Interestingly enough, a professor of organic chemistry[1] created a new Common Lisp implementation[2] just so that, among other things, he can use a very high level functional(ish) language in his research. His CL implementation is specifically designed to allow more seamless interoperability with libraries written in C and C++.

[1] https://drmeister.wordpress.com/about/

[2] https://github.com/clasp-developers/clasp

CLASP? CL on LLVM.

https://github.com/clasp-developers/clasp/

> Clasp is a new Common Lisp implementation that seamlessly interoperates with C++ libraries and programs using LLVM for compilation to native code. This allows Clasp to take advantage of a vast array of preexisting libraries and programs, such as out of the scientific computing ecosystem. Embedding them in a Common Lisp environment allows you to make use of rapid prototyping, incremental development, and other capabilities that make it a powerful language.

I wonder why not much is done to bring high performance scientific computing to common lisp. There are some interesting projects I was able to find like https://github.com/clasp-developers/clasp and https://github.com/marcoheisig/Petalisp and https://github.com/takagi/avm. But I guess it would be good to have a coordinated effort in this area.
The GDB JIT interface implementation is seriously flawed. I love GDB - but this causes me a LOT of grief when debugging clasp (Common Lisp implemented using llvm as the backend https://github.com/clasp-developers/clasp.git).

Every time a JITted object file is added using the API, the entire symbol table is sorted. If you have 10,000+ JITted object files as we do - it takes hours to days to weeks to register them all.

We use the Undo time traveling debugger that builds on top of GDB. It's awesome but we are crippled because of the JIT API implementation.

I'd love to see this get fixed - if anyone knows who to talk with about it - drop me a line.

Thanks for that patch - I'll watch this with great interest.

Is there a problem if dynamic libraries invoke dlopen/dlclose they also need to call the sync function - correct?

I'm asking because we've developed a Common Lisp implementation that interoperates with C++ and it uses exception handling to unwind the stack (https://github.com/clasp-developers/clasp.git). We hit this global lock in unwinding problem a lot - it causes us a lot of grief.

I like the term "Eternal Language" - programming languages where if you write code now - you will be able to compile and use that code ten years from now (a software lifecycle eternity). Common Lisp, C, C++, Fortran, (edit: Java) are close to eternal languages. After losing a huge amount of Python 2 code (I wasn't going to rewrite it) I implemented Clasp, a Common Lisp implementation that interoperates with C++ (https://github.com/clasp-developers/clasp.git) so that I can write code that will hopefully live longer.

I am one of the few software developers with code that I wrote 27 years ago that is still in active use by thousands of computational chemistry researchers (Leap, a frontend for the computational chemistry molecular dynamics package AMBER, implemented in C).

as i said i think that Julia is an exciting project in the scientific computing space. my statements were not directed against julia or any other language. i just happen to think that common lisp also has great potential for scientific computing, one that can appeal to scientists who want to be software engineers also. for those that just want to crunch numbers right now python is a first reference then julia, imho. however, i sincerely hope that julia overtakes python

having the ability to be self hosted is definitely not ideological. [* here is a hypothetical question that can be asked: would a julia programmer be more powerful if llvm was written in julia? i think the answer is clear that they would be *]

common lisp is a language specification. it does not have an official implementation, although by far the most popular one is sbcl. in fact there is an implementation CLASP[0] that is hosted on llvm

[0] https://github.com/clasp-developers/clasp

[*...*] enclosed part added to the response subsequently to initial posting

my background is also in physics and mathematics. in fact i think common lisp is well suited to a physicist-hacker type mindset. i could probably write a lengthy blog post about why i think this could be, but some of the key appeal factors are:

* unmatched interactive development which could be used for exploration of problems such as dynamical and complex systems

* although it is a very high level language you can perform low level optimizations in common lisp itself

* extremely powerful macro system that allows you to develop in a declarative style to reflect your problem domain. for example you can program in a syntax that is natural to general or special relativity, quantum mechanics, etc.

* fast and free implementations such as SBCL

* the language is standardized so your code will not need to be unnecessarily maintained (this is good for university professors that have repeating long-running courses)

i am not sure what type of references you are asking for but i will list few open source projects that might be interesting to you:

* CLASP - common lisp implementation build on top of LLVM. this project is headed by a computational chemist (prof. schafmeister) [0]

* PetaLisp - for HPC [1]

* MAGICL - Matrix Algebra proGrams In Common Lisp by Rigetti Computing [2]

* Quil - quantum instruction language developed on top of common lisp [3]

* CL-CUDA - use NVIDIA CUDA in Common Lisp programs [4]

* MGL - Common Lisp machine learning library [5]

* Maxima - a computer algebra system developed in common lisp [6]

i think these are plenty to show that there exists serious interest in using common lisp for scientific computing. it is possible that some of the people behind these projects are on HN and maybe they can further expand on this

[0] https://github.com/clasp-developers/clasp

[1] https://github.com/marcoheisig/Petalisp

[2] https://github.com/quil-lang/magicl

[3] https://github.com/quil-lang/quilc

[4] https://github.com/takagi/cl-cuda

[5] https://github.com/melisgl/mgl also https://github.com/melisgl/higgsml

[6] https://maxima.sourceforge.io/

There is also interesting CL implementation that is based on the LLVM framework - Clasp[1]. It can be natively compiled and designed for performance.

[1] https://github.com/clasp-developers/clasp

This reminded me of the chemical PhD who created Clasp, a full common lisp implementation using LLVM as a JIT compiler, able to call even template C++ code from Common Lisp (a feat which no other language I know of even tries, except perhaps .NET with managed C++).

[0] https://github.com/clasp-developers/clasp

I had a very different experience. I implemented Common Lisp using LLVM-IR as the backend (https://github.com/clasp-developers/clasp.git).

1. I started with a primitive lisp interpreter written in C++ and worked hard on exposing C++ functions/classes to my lisp using C++ template programming. LLVM is a C++ library, the C bindings are always behind the C++ API. So exposing the C++ API directly gave me access to the latest, and greatest API. That means you need to keep up with LLVM - but clang helps a lot because API changes appear as clang C++ compile time errors. I've been "chasing the LLVM dragon" (cough - keeping up with the LLVM API) from version 3.something to the upcoming 13.

2. I wrote a Common Lisp compiler in my primitive lisp that converted Common Lisp straight into LLVM-IR. I didn't want to develop my own language - who's got time for that? So I just picked a powerful one (Common Lisp) with macros, classes, generic functions, existing libraries, a community etc.

3. I used alloca/stack allocated variables everywhere and let mem2reg optimize what it could to registers. I exposed and used the llvm::IRBuilder class that makes generating IR a lot easier.

4. Then I picked an experimental, developing compiler "Cleavir" written by Robert Strandh and bootstrap that with my Common Lisp compiler. It's like that movie "Inception" - but it makes sense :-).

Now we have a Common Lisp programming environment that interoperates with C++ at a very deep level. Common Lisp stack frames intermingle perfectly with C++ stack frames and we can use all the C/C development, debugging and profiling tools.

This Common Lisp programming environment supports "Cando" a computational chemistry programming environment for developing advanced therapeutics and diagnostic molecules.

We are looking for people who want to work with us - if interested and you have a somewhat suitable background - drop me a message at [email protected]

Consider perhaps https://github.com/clasp-developers/clasp which is a common-lisp specifically designed to interop with C++.
A couple of others that may be of interest:

- https://ferret-lang.org/ "Ferret is a free software lisp implementation designed to be used in real time embedded control systems. Ferret lisp compiles down to self contained C++11" "Easy FFI (Inline C,C++. See Accessing C,C++ Libraries)" "Easily Embeddable (i.e Ferret fns are just C++ functors.)"

- https://github.com/clasp-developers/clasp "Clasp is a new Common Lisp implementation that seamlessly interoperates with C++ libraries and programs using LLVM for compilation to native code."

Some pointers: Scorecloud (https://scorecloud.com/), a music composition software (you sing, it writes music) comes from research, and is built in Common Lisp.

pgloader was rewritten from Python to Lisp: https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas... because of Python's poor performances and threading capacities.

A theorem prover used in the industry: https://github.com/acl2/acl2

A Lisp vendor selling semantic graph solutions: https://franz.com/ (see also its success stories for industry examples)

Here's a sadder story: a team switched from CL to Go for production, but still use CL for prototyping: https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-comm...

Some use CL in bioinformatics (and claim it's the best tool for the job). For example, they create a new Lisp implementation: https://github.com/clasp-developers/clasp

More success stories: https://lisp-lang.org/success/

Lisp (Common Lisp) beats all the other dynamic languages by a considerable margin. This is why I am developing Clasp - a Common Lisp implementation based on LLVM that interoperates with C++/C (https://github.com/clasp-developers/clasp.git) for scientific programming.

With Clasp, we get the best of multiple worlds. We get a dynamic language (Common Lisp) with automatic memory management and enormous expressive power that can directly use powerful C and C++ libraries. All three of these languages are "long-lived" languages in that code that was written 10 and 20 years ago still works.

Performance is really important to me and I have written a lot of code over the past four decades. I won't develop meaningful code in any language that falls below Racket in table 4 because these language implementations are too inefficient. I furthermore want to keep using my code over the years and decades and so I won't develop meaningful code in any language where someone else can break my code by changing the standard. My program "leap" was written 27 years ago in C and it is still being used daily by thousands of computational chemists. But it's really hard to improve leap because the code is brittle largely because of malloc/free-style memory management (brrr). For a compiled, high-performance, standard language with proven lasting power and performance - Common Lisp is the best choice.

I can testify to a lot of the difficulties Robert Strandh talks about here. We implemented a Common Lisp in C++ that interoperates with C++ (https://github.com/clasp-developers/clasp.git). I ended up implementing a lot more C++ code than I wanted to and then replacing much of it with Common Lisp versions (during startup) because it's easier to maintain and we can use type inference to improve the performance of Common Lisp code while C++ code is 'stuck' with the types you compile it with. Bootstrapping is hard.
I would rather vote for CLASP for which there are many applications in chemistry and molecular biology, see https://github.com/clasp-developers/clasp
B & C are good points, but...

A) Writing a new Common Lisp implementation is a major undertaking, but a new implementation was creates specifically for C++ interop[1], and that implementation is certainly post javascript being big.

D) Anything targeting JS can run the output through closure; I've done so with Parenscript, for example.

1: https://github.com/clasp-developers/clasp

While it's nowhere near as mature as Racket, those interested in this use case might also be interested in Clasp, a Common Lisp implementation built on LLVM specifically to enable the combination of Lisp-style dynamic development approaches with C++ libraries.

https://github.com/clasp-developers/clasp

https://www.youtube.com/watch?v=mbdXeRBbgDM

> Common Lisp still has some activity

CLASP is a relatively new Common Lisp implementation with deep C++/LLVM integration.

https://github.com/clasp-developers/clasp

Quicklisp adds new libraries every month.

http://blog.quicklisp.org

LispWorks and Allegro CL are commercially available Lisps since 30+ years, running on various platforms and still supported. The LispWorks 7.1 update from 2017 brought native ARM64 support (Linux, iOS), remote debugging of Android and iOS LispWorks, ...

Implementing a "simple virtual machine" for a particular task is greenspunning[1]. Introducing a JIT library is work, complexity, debug and portability issues. LLVM compiles much slower to native code as the Clasp[2] guys notice comparing to SBCL. And in the end, any of these would be at most "competitive" to the simple Common Lisp implementation in speed (which is even portable across the different implementations).

[1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

[2] https://github.com/clasp-developers/clasp

We are traveling down this road. We've implemented Clasp (https://github.com/clasp-developers/clasp.git) - an implementation of Common Lisp that interoperates with C++ and uses llvm as the backend. On top of this, we built Cando (https://github.com/drmeister/cando.git) - a computational chemistry programming environment for designing large molecules. The end of our story hasn't been written yet.

Clasp could be used for a lot of other things.

> whose C++ implementation from scratch...

You're aware of clasp?

https://github.com/clasp-developers/clasp

Which builds (in part) on embeddable common lisp:

https://gitlab.com/embeddable-common-lisp/ecl

See around 20-30 minutes: https://youtu.be/8X69_42Mj-g

A growing group of us are implementing Clasp (https://github.com/clasp-developers/clasp) - a new Common Lisp that uses Robert Strandh's Cleavir compiler, interoperates with C++ and uses LLVM as the backend. Clasp is the foundation of Cando - a programming environment for the development of molecular devices (https://www.youtube.com/watch?v=mbdXeRBbgDM&feature=youtu.be). Robert Strandh is an exceptionally clear thinker and a fantastic programmer. I can't count the times that Robert has implemented changes to the Cleavir compiler source code and without him being able to test it (long story) passed it to me and it ran flawlessly in Clasp. I implemented his fast generic function solution within Clasp based on another of his papers (http://metamodular.com/generic-dispatch.pdf). Edit: Basically - if Robert writes it up in a paper - I want to have the implementation.

Once we develop molecular robots - they are going to implement LispOS. :-)

Nice - I think we will make the switch to this. We wrote a Common Lisp kernel for Clasp (Common Lisp system that interoperates with C++ and uses LLVM as the backend -https://github.com/clasp-developers/clasp) We implemented ipykernel and ipywidgets in Common Lisp by translating the Python code. I think we could switch to Xeus and lower the maintenance burden and contribute to Xeus.
Wow!!! This is a wonderful talk.

The title is so old school though that need changing.

- How I use lisp to fight human aging or - why I choose lisp over python to combat protein generation - my protein generation using llvm was done all by lisp for. Reason

The talk is amazing. This one chemistry prof. who seems to be an IT expert but really just start lisp programming by building a lisp interpreter.

Wow!!!

https://github.com/clasp-developers/clasp And The cando one.

Counterpoint - I implemented a Common Lisp compiler that uses llvm as the back end (https://github.com/clasp-developers/clasp). I've kept up with the versions from 3.6 to 6.0 and I'm about to upgrade to 7.0.

I'm not sure why other projects find this difficult but I have noticed that if I get too far behind it does get a bit more difficult.

I exposed the C++ API to Common Lisp and write all of the compiler in Common Lisp. The Clang C++ compiler tells me what is broken when I upgrade to a new version of llvm and then I fix it. I talk about it here https://www.youtube.com/watch?reload=9&v=mbdXeRBbgDM&feature... at about 13:30

This is really nice! I wonder how well the ulisp compiler does with optimization - especially for code size. My group is developing Clasp Common Lisp and we use the Cleavir compiler (https://github.com/clasp-developers/clasp and https://github.com/robert-strandh/SICL). Cleavir is a new project to develop a powerful, portable Common Lisp compiler and it could be used to compile lisp dialects.
I'm a great admirer of Julia, but I doubt it would appeal to those seeking a "true" lisp/scheme. Even if it's has a sane macro system, and much of the same power. You can indeed see s-expressions (search for "s-expression") :

https://docs.julialang.org/en/v1/manual/metaprogramming/inde...

This nice talk mentions a bit on how Julia is (not) a lisp (if I recall correctly):

"Julia: to Lisp or not to Lisp?": https://m.youtube.com/watch?v=dK3zRXhrFZY

This might also be of interest: https://tpapp.github.io/post/common-lisp-to-julia/

As well as (maybe) clasp: https://github.com/clasp-developers/clasp

Just to counterbalance and give some hope: projects like CLASP (CL on LLVM) got university funds to development it (and the ecosystem on the way: LSP library,…).

https://github.com/clasp-developers/clasp

CL is still adopted by universities (for bioinformatics it seems), or innovative firms (Emotiq (blockchain), Rigetti quantum computing,…), and is still used by big ones (https://github.com/azzamsa/awesome-lisp-companies), so why not see fundings in the future…

This is a very nice web site describing a programming language with unparalleled expressiveness, power and permanence.

I am heavily invested in Common Lisp. We are developing a programming environment for designing new materials and molecules called Cando (https://github.com/drmeister/cando) using Common Lisp as a scripting language. Cando is running on Clasp (https://github.com/clasp-developers/clasp), a new Common Lisp implementation that interoperates with C++ and is based on LLVM.

What attracted me to the language, after 35 years programming in almost everything else, was how organically it lets one write software and how I don't have to worry about it fading like the next programming fad.