A perfect solution for me would be something like ssh-add -c (which asks for confirmation on any agent access) but which could distinguish between local and remote agent access.

I want to add every key to ssh-agent on my local machine, be able to login into remote hosts with them but have a pop-up confirmation if there's an agent access from remote host where agent is forwarded.

The problem basically is the current agent forwarding protocol doesn't have a way to reliably identify the source and remote host that can't be spoofed. guardian-agent tries to do that using some extra software, this linked SSH proposal is to add that to SSH but it will require software upgrades even to the sshd of the intermediate and remote hosts - it's not ideal that it can't just work out of the box - but I welcome this we just need to get it done now for later.

I frequently finding myself thinking about adding useful things to software I want to use now and go well it will be years before its on every host I use and can be used reliably. I have had this thought on and off for more than a decade. Ship some new stuff, it'l be great later :D