I agree, but only after you have a product that works. Else, you will see yourself drawn into maintaining the admin features, and not the real product. There can be a middle ground: design your API with ACL from the start, and then the CRUD admin features would just work out of the box. Or there's a cool shortcut: Hasura and Appsmith. Hasura connects to my PostgreSQL data and provides authorization, while Appsmith just consumes the REST endpoints created with Hasura.
There’s also the react-admin hasura connector if you need more control: