Well written article, but this is HN so naturally I have a, "Why don't you just..." You know your situation better than anyone else, so take it with a grain of salt.

But, why don't you just not use a general-purpose language? No really. Hear me out.

It seems the main problem is the proverbial impedance mismatch between SQL and say, Perl or Python or whatever. Ok. One way to remove that mismatch is to remove the boundary altogether. Remove the general-purpose language and really embrace the "just use SQL" path. That may have drawbacks and we can talk about those or we could not, but one thing it certainly would do is remove the problem of the impedance mismatch.

As in, write your entire application in SQL? That sounds like a nightmare of epic proportions. Every time I've tried to use SQL for anything more general-purpose (like string manipulation, say) I've wanted to gouge my eyes out by the end.

Yes. Write your entire application in SQL, IF it's the kind of application for which this can be done. Is the application a flight simulator or a game engine or resource manager or a distributed task scheduler? No, that's probably not a good case for SQL. Neither would be a machine learning application that relies heavily on Python packages, and it's easy to think of other examples as well. But, is the application responding to user input to query and display data, perform transformations over those data, impose constraints and invariants over those data, and validate user input against those data? Those are tasks for which relational databases with SQL (and other kinds of databases as well, but here we're talking about SQL) are tailor-made. What then does, say, Python bring to the party?

The obvious thought that occurs to me is UI - how are users going to access this hypothetical application? If it's via the web, then you'll need a whole setup for generating HTML, and doing that in SQL sounds extremely painful. If it's a native app, then you've got to interact with the OS somehow to do things like draw to the screen, and I've never even heard of trying to do that from SQL.

More broadly, this just seems like it would sharply limit how you could extend the application in the future. What if at some point you want to query a web API for some additional data with which to enrich what you're returning from your database? (Not an unusual situation, in my experience.) You'd be stuck trying to make web requests from SQL, which again seems needlessly painful.

Note that I'm not against the basic idea of "do as much data-munging in SQL as possible" - in my experience that's a great way to ensure that your application stays fast and efficient. It' just all the ancillary things surrounding the data-munging for which I don't think SQL is the best fit.

- PostgREST (https://postgrest.org/): REST API for PostgreSQL

- PostGraphile (https://www.graphile.org/) GraphQL API for PostgreSQL

- pg_graphql (https://github.com/supabase/pg_graphql) GraphQL API for PostgreSQL

- Hasura (https://hasura.io/) GraphQL API for various databases

If you want to blend data from a web API and you're content with GraphQL, for some use-cases (not all, but some), there are options:

- Apollo Federation (https://www.apollographql.com/apollo-federation/)

- GraphQL Mesh (https://the-guild.dev/graphql/mesh)

- Hasura Remote Schema (https://hasura.io/blog/tagged/remote-schemas/)

If you want more control over the web API and you were going to fetch the data within your Python back-end and process it there, for some use-cases (not all, but some), there are options:

- pg_http (https://github.com/pramsey/pgsql-http)

Life is about trade-offs. Doing the work in SQL is not without its drawbacks, but it's also not without its benefits, and that's true for doing the work in a general-purpose language as well. Whatever the drawbacks of doing it in SQL, one of the benefits has got to be eliminating the impedance mismatch (for people who regard that mismatch as a problem, and the OP seems to be one such person). What I claim is that doing the work directly in the database shouldn't be ruled out in general (the specifics of a given use-case may rule it out in particular) any more than the other common patterns (API hand-written in Python, for instance) shouldn't be ruled out in general.