What I find promising about LISP is the ability to do term rewriting and macros.
But people write lisps in imperative style rather than definitions of desired behaviour declaratively. I don't think we've sufficiently solved how to define desired behaviour to a computer.
Term rewriting behaviours. What are your thoughts?
I started trying to implement term rewriting into my LISP parser, which is the idea that we can match on trees and transform them, subsume branches or move branches around arbitrarily. You kind of want the matching function and transformation function to be imperative or sometimes like a query.
So use ASTs for behaviour and relationships and imperative LISP macros for transformations and rewriting.
The reason I say this is because I dream of a compositional language where I can create a cell in a spreadsheet and say that it should have these behaviours
Lisp is not the language for that unfortunately, it is very much an imperative language with better syntax and _some_ macros.
Scheme is close but quotation isn't thought about nearly enough. It is generally a CS problem as logic systems with quotation are very much an open problem. I think that types have gotten too much attention and quotation way too little.
Macros are basically a way to deal with the fact that neither lisp nor scheme have first class quotation which you can evaluate at leisure. I understand why they've done it for efficiency reason, but I think we should have at least one language that gets quotation right.
I think a lot of confusion come from the fact that a lot of people (including, presumably, the parent) use "Lisp" to mean Common Lisp, whereas many others take it to mean what Common Lisp people might call "lisp" or lisp-family. You can easily have a substanceless argument this way, and many often do!
There is no member of the lisp family that treats quotation as a first class citizen of the language. Granted, lisp is one of the few language families where it's even a part of the language but it's at best an afterthought, which is why you need macros to manipulate unquoted expressions.
What I was thinking about in your post was the the statement "[Lisp] is very much an imperative language with better syntax and _some_ macros". Of course some lisp-family languages are very much not imperative and some get by without macros (e.g. Clojure).
Can you imagine a language with such a first-class quoting system?
What is the requirement defining "first class quotation"?
In support of ways of implementing Scheme hygienic macros, there exists an invention known as syntactic closures. In what ways does a syntactic closure fall short of being a "first class quotation"?
Sure, whatever; what I intended to bring up was the specific way of implementing hygienic macros using syntactic closures. Syntactic closures are a way of quoting code, with context. Hygienic macros don't have to be implemented with syntactic closures.
That author you cited seems to be working on something very similar to syntactic closure. The paper has a lot of references, but very few of them are anything Lisp or Scheme related. Looks like the author is working entirely on his own.
In a work like this, I'd expect syntactic closures to be acknowledged, with a discussion of how the work being presented is different.
Syntactic closures are a way to hide the fact that lambdas are the only abstraction in lambda calculus.
The work that I cited has types _and_ quotation as first class abstractions.
It's a bit like asking to explain proof theory in terms of Godel numbers. Sure you can technically do it. You're completely missing the point if you do however.
I think syntactic closures don't have to be concerned with type, because their job is just to ensure that the symbols included in a quote refer to what they are supposed to refer to, regardless to where that quote is moved. When the closures are evoked, reasoning about types can take place then.
Lots of term rewriting found in compilers. Usually based on pattern matching on trees followed by some guard function to deal with DAGs / graphs. Lisp/scheme don't have pattern matching in the core - possibly because it's much cleaner with a static type system doing some of the dispatch - but you can implement one or use one of the libraries.
Declarative programming is roughly that control flow is handled by the language runtime instead of the programmer. That's either some dedicated language (makefile, yacc) or a DSL in some non-declarative language. It's not lisp, but the tablegen programs used by llvm's backend are an interesting example of a concise declarative syntax turning into a load of C++.
I'd say both are examples of things lisp doesn't really do for you. They're more reasonable to implement in a lisp than in most other languages.
You think lisp is "weird and sinister or ghostly" and that 'we' are still wrestling with something 66 years later?
This sounds more like someone getting caught up in the pageantry of a niche that pragmatic people have left behind a long time ago. Lisp was very influential, but those advancement have made their way into practical languages and lisp has been impractical for many decades at this point.
Scheme has pioneered a lot of stuff over the last 30-40 years that is still fresh for most of the PL world, hygienic macros (like Rust is trying to implement), delimited continuations (which underlies Java's new virtual threads, see https://www.youtube.com/watch?v=9vupFNsND6o), efficient closure representations (used all over the place since everybody got lambda fever). Into formal verification? Scheme was there in 1995 https://www.semanticscholar.org/paper/VLISP%3A-A-verified-im...
And there is at least half a dozen things in Racket that I wish mainstream languages could get their ass in gear and copy, but no such luck.
Insert meme of Nolan Grayson, representing Lisp/Scheme/Racket programmers, pointing at fighter jets labeled "transpilers" and saying: "Look at what they need to mimic just a fraction of our power!"
I think the roles would be reversed when trying to make fast, small interactive software that people want to use. "Powerful" is interesting but clear, straightforward and fast is better. No web browser, database, video codec, or high end video game is written in scheme. At best it's inefficient ancillary software that someone wrote in scheme because they wanted to, not because that's what a user wanted.
A language that contains all Lisp features becomes identifiable as a member of the Lisp family, and is then removed from the discussion of languages that don't have all Lisp features.
Ah yes, all of the modern practical languages allow you to connect to a running system, redefine a class, and automatically update every existing instance of that class.
It's easy to just assign different functions to a metatable in lua.
That isn't the point though. You were saying lisp is some mystical thing and it is 66 years old. Its influence happened decades ago. It isn't about every language having every feature. Pretty much all software is made without lots of the features in lisp because not every tradeoff is worth it. Lisp itself is barely used because it isn't about a check list of features, but pragmatism of an ecosystem, syntax, actual compilation etc are all crucial. Lisp is not a modern tool, it is an influential invention from over half a century ago.
When people are stuck on an airplane, they don't try to watch citizen kane, they want to watch literally anything else. Influential isn't the same as being good by modern standards. People don't want to write lisp and people don't want to use software written in lisp. Sorry for the harsh reality check.
But people write lisps in imperative style rather than definitions of desired behaviour declaratively. I don't think we've sufficiently solved how to define desired behaviour to a computer.
Term rewriting behaviours. What are your thoughts?
I started trying to implement term rewriting into my LISP parser, which is the idea that we can match on trees and transform them, subsume branches or move branches around arbitrarily. You kind of want the matching function and transformation function to be imperative or sometimes like a query.
So use ASTs for behaviour and relationships and imperative LISP macros for transformations and rewriting.
The reason I say this is because I dream of a compositional language where I can create a cell in a spreadsheet and say that it should have these behaviours
Truly futuristic programming where we we program behaviours.I am asked to define the terms that these behaviours require.