How does the data get to BigQuery? Anything special/fun or just repeatedly polling the API endpoint?

I'm not sure if we are ready to document the process, but I can tell you that having an API based on Firebase helps a lot:

- https://github.com/HackerNews/API