I'm doing something very similar with my service: https://aytwit.com/thoughter
To the author, there is a simple trick you can pull in order to make the confirmation instantaneous and avoid caching. Have you figured it out? Let me know!
I also use a residential proxy service for all my profile requests, regardless of the identity provider. For some sites like Twitter, Facebook, etc. this is required, and for something like Hacker News it's simply future-proofing in case they decide to block scrapers at some point in the future.
Good work!
>for something like Hacker News it's simply future-proofing in case they decide to block scrapers at some point in the future
Why are you scraping? We have an API, it's linked at the bottom of every page[0].