Can someone chime in why this hasn't worked in the past?

It seems a personal/portable CRM could be highly useful. The second aspect is the vertical aspect (professional or "personal")

One thing I keep hearing from people that have sold companies, exited etc. is that they are having a hard time operating within their networks without the CRM.

No experience, but there is https://github.com/monicahq/monica