If possible, do things without macros. Problem is - it's pretty much always possible to do things without macros.
No. If possible, do things with macros. And it is always possible to do things with macros.
Why? Because macros are the best way to implement simple, nice, clean, maintainable eDSLs. And any problem is best solved with its most natural language, where the very problem description is already a working solution.
Unfortunately, most people do not understand what macros are, and how they should be used to build multi-stage, simple DSL compilers. They're using macros to implement obscure syntax instead, all that awful LOOP macros, anaphoric ifs, etc., which was exactly what resulted in the bad reputation of the compile-time metaprogramming.
I have only ever heard the opposing argument, but I am interested in understanding to its fullest extent how to implement good DSLs with macros. I understand CL and defmacro; I am just looking for good patterns of use. Can you point me to any resources, documentation, or examples of this?
Firstly, macros must be simple and must essentially correspond to compiler passes.
E.g., if you're implementing an embedded DSL for regular expressions, one macro expansion pass should lower your DSL down to a syntax-expanded regular expression, the second macro would lower it into an NFA, another one would construct an optimised DFA, and then the next one (or more) would generate the actual automaton code in Lisp out of the DFA.
The best possible case is when your macro pass can be expressed as a set of simple term rewriting rules. And most of the transforms you'd find in DSL compilers can actually be represented as such.
Of course, there is also an alternative style, which may be preferable if you have debugging tools for designing your compiler passes at least equivalent to the Lisp macro expansion debugging. You can simply write your entire DSL compiler as a single function, and then wrap it into a macro, as `(defmacro mydsl (&rest args) (mydsl-compiler args))`.
This way, the same compiler infrastructure can be reused for an interpreted or partially interpreted version of your DSL, if you need it. Still, all the same rules apply to how the passes are implemented in that compiler function.
Another very useful trick is to make your DSLs composable, which is easy if you split your compilation pipeline into separate macro expansion passes. Multiple DSLs may easily share features of their internal IRs or even front-end languages, so any new DSL would simply cherry-pick features from a number of existing DSLs and wrap them into a simple front-end. This degree of composability is simply impossible with the function-based interpreted eDSLs.
A can recommend taking a look at the Nanopass [1] framework, which is built around this very ideology, or at my own DSL construction framework [2].
[1] http://andykeep.com/pubs/np-preprint.pdf
And some examples:
https://github.com/cisco/ChezScheme