What does HackerNews think of awesome-tunneling?
List of ngrok alternatives and other ngrok-like tunneling software and services. Focus on self-hosting.
I dont think (as the author suggests) that IPv6 everywhere is happening anytime soon.
Personally, I think the future of self hosting is going to happen through IPv4 tunnels[0] with SNI routing. You also get the added benefit of not exposing your actual IP address, and dealing with things like DDoS become the tunnel provider's concern.
That is what differentiates it from all the other options. There is no company behind it trying to make money by exploiting internet subscribers trying to connect with each other (not the so-called "tech" company).
Tor can have uses other than the ones normally discussed such as anonymity and evading censorship. Tor can provide reachability without use of commercial eavesdropping third party intermediaries.
For example, one can use Onion Services for advertising open IP:port information that is needed for peer-to-peer connections over other, faster peer-to-peer overlay networks, not the Tor network. The Onion Service can function as the "rendezvous" server for making peer-to-peer connection outside of Tor. Tor's Onion Services can be used to exchange IP:port information for making direct connections over the internet without using Tor. No need to use commercial third parties. Ngrok, Tailscale, etc. all require use of servers run by a commercial third party. Tor does not. There is ample free software that can establish peer-to-peer connections over the internet but in every case it requires some reachable server running this software on the internet, and for most users that means they have to run a server and pay a commercial third party for hosting. Tor has no such requirement.
Imagine being able to share content with family, friends, colleagues without the need for so-called "tech" companies^1 acting as intermediaries ("middlemen"). With a reachable IPv4 address this becomes possible. It would be nice if every home internet access subscriber received a reachable IPv4 address from their ISP. No doubt, some do. But on today's internet most do not. The so-called "tech" companies all have reachable IPv4 addresses. Hence they assume the roles of middlemen and use this position to exploit internet subscribers for profit.
Something like Tor provides a solution. Again, it is not always necessary to route all traffic over Tor. Tor can have other uses. When the goal is simply peer-to-peer connections, Onion Services can be used to bootstrap peer-to-peer overlay connections using the user's choice of software by providing a secure, reliable way to exchange IP:port information. Goal here when using Tor is not anonymity nor censorship evasion, it's reachability. Similarly, goal of peer-to-peer is not necessarily anonymity nor evading censorship either, it's bypassing commercially-motivated, eavesdropping middlemen known as "tech" companies, and avoiding the annoyances of advertising. A possible additional benefot of using Tor in this way is elevated privacy. Google, for example, cannot easily discover Onion Services. No one can discover Onion Services using ICANN DNS.
1. The term "tech" as in "tech company" means a company, usually a website, that collects data from and about people to support the sale of advertising services because advertising services are the only services the company can sell on a scale large enough to sustain a profitable business.
More reading/viewing:
https://github.com/anderspitman/awesome-tunneling
Tor Hidden Services (now called "Onion Services")
https://jamielittle.org/2016/08/28/hidden.html
As one author wrote on Github:
"onion-expose is a utility that allows one to easily create and control temporary Tor onion services.
onion-expose can be used for any sort of TCP traffic, from simple HTTP to Internet radio to Minecraft to SSH servers. It can also be used to expose individual files and allow you to request them from another computer.
Why not just use ngrok?
ngrok is nice. But it requires everything to go through a central authority (a potential security issue), and imposes artificial restrictions, such as a limit of one TCP tunnel per user. It also doesn't allow you to expose files easily (you have to set it up yourself)."
https://github.com/ethan2-0/onion-expose
As another Github contributor put it:
"With onionpipe, that service doesn't need a public IPv4 or IPv6 ingress. You can publish services with a globally-unique persistent onion address, and share access securely and privately to your own allowlist of authorized keys.
You don't need to rely on, and share your personal data with for-profit services (like Tailscale, ZeroTier, etc.) to get to it."
https://github.com/cmars/onionpipe
https://news.ycombinator.com/item?id=36734956
https://news.ycombinator.com/item?id=30445421
https://news.ycombinator.com/item?id=29929399
"Finally, onion services are private by default, meaning that users must discover these sites organically, rather than with a search engine." [Small websites with small audiences get buried by advertising-supported search engines anyway.]
https://nymity.ch/onion-services/pdf/sec18-onion-services.pd...
https://media.ccc.de/v/31c3_-_6112_-_en_-_saal_2_-_201412301...
https://wiki.termux.com/wiki/Bypassing_NAT (Termux recommends Tor over Ngrok)
It's exciting and important to have an open source alternative in this space.
To me, it seems that Cloudflare is just another additional abstraction layer / proxy -- between web sites / web apps / SAAS providers -- and the Internet.
There might be benefits from such an arrangement -- such as Cloudflare's ability to block DDoS attacks...
But there also might be drawbacks from such an arrangement -- such as what do if Cloudflare for whatever reason -- blocks/bans/or otherwise limits you?
Opinion: A good website / web app / SAAS -- would be as distributed as possible -- that is, it would use the Internet natively AND it would Cloudflare AND it would use N Cloudflare competitors/proxy services -- ideally all of them -- at the same time!
In other words -- let the user choose their own route to a provider!
Do you want to use Cloudflare?
Great, we have that!
Do you want to use the Internet natively?
Great, we have that!
Do you want to use a competitor to Cloudflare?
Great -- we have that too!
If it is technically impossible to do that with one domain -- then mirror your site/service/SAAS -- to multiple domains.
Let the user decide what they want...
Incidentally, towards that end, I found the following excellent list on GitHub:
https://github.com/anderspitman/awesome-tunneling
(https://news.ycombinator.com/item?id=30443747 -- for related discussion)
It's non-canonical -- but it's probably a step in the right direction...
Ngrok?
They enable you to move your selfhosted services from expensive, slow VPSes you don't control to fast devices in your own home[4]. IMO this is strictly better than a VPS in terms of privacy and data control. It's a step in the right direction, back towards the initial intent of the internet, but also forward with the lessons we've learned in the real world.
The reality today is that selfhosting is way too hard[1]. It shouldn't be any more complicated or less secure than running an app on your phone.
I think services like Tailscale are going to enable the first generation of selfhosting that approaches that level of simplicity. Once the market is proven, the second generation is going be designed for selfhosters and have features like end-to-end encryption, domain name integration, and simple GUI interfaces.
The other key pieces are strong sandboxing, which is now possible on all major desktop OSes through virtualization (mobile is coming[2]), and dead-simple cloud backups.
The technology for all these things exists, it just hasn't been integrated yet.
[0]: https://github.com/anderspitman/awesome-tunneling
[1]: https://moxie.org/2022/01/07/web3-first-impressions.html
[2]: https://twitter.com/kdrag0n/status/1584017653269958656?lang=...
[4]: I concede that the network upload connection is likely much slower, but expect that to improve over time.
The big question is going to be pricing. The current top player in this space is Cloudflare Tunnel, which is a loss-leader product that technically forbids selfhosting anything other than HTML sites.
Selfhosting media can use tons of bandwidth. Any service that doesn't charge per GB is incentivized to limit your speeds.
The technology exists to do all of these things, but no one has taken the time to glue it all together in a truly good UX (I'm working on it). Pretty much every solution in this space is targeted at the developer market, not self-hosters.
So for now I'd recommend using a VPS. Your main challenge is going to be learning a lot about security. There's currently no way around that. A VPS limits the scope of damage that can be done if you get hacked. Once you've learned enough you can move to your own hardware. At that point I'd recommend setting up tunneling[0] and using either Docker or QEMU/KVM.
EDIT: I see you're already using Tailscale. That can operate as a tunnel. Basically you'd want to run a reverse proxy like Caddy (recommended) or nginx on the VPS, and point it at services running on your other devices using the IP addresses from your Tailscale network.
Maybe headscale could run at home, served over a tunnel[1] to a VPS. But honestly, if I ever lost confidence in the trustworthiness of Tailscale the company, I would just connect my devices with some other overlay network like Yggdrasil[2] or Tor.
[1]: https://github.com/anderspitman/awesome-tunneling [2]: https://yggdrasil-network.github.io/
My favorite two projects currently in the tunneling space are sish [1] and bore [2]. sish is more full featured and utilizes ssh for handling proxies (disclaimer: I wrote sish) while bore maintains a client/server but all sources are available for both. They're written in Go and Rust respectively. There's even a publicly accessible instance of bore running. You should submit a PR to include CRProxy in the awesome-tunneling list, I know Anders might do that as well as he's normally around these parts :)
[0] https://github.com/anderspitman/awesome-tunneling
https://github.com/anderspitman/awesome-tunneling
Great to see more WireGuard implementations popping up. Most of the "no install" options are based on SSH tunnels, which are great, but WireGuard should have much better performance under certain conditions.
This can be tricky if you're stuck behind a CGNAT, which is becoming more common. I maintain a list of solutions to this problem here:
https://github.com/anderspitman/awesome-tunneling
It's interesting to me how many times this particular wheel has been reinvented. That said, it's a great exercise and one I highly recommend if you want to learn a bit more about networking.
I like the minimalism of this implementation.
I've run a split horizon DNS configuration for years and I've got to say that it caused more problems for me than it solves.
Luckily, you can just turn off DoH if it causes problems for you.
In my experience, the Chromecast and Google Assistant functionality isn't related to your DNS setup. Chromecast should work through broadcast or through an active request from HASS itself, and the Google integration has always gone through the cloud as far as I know.
Perhaps your setup is different than mine, but I don't think these issues are necessarily DNS related, unless Google's servers are getting your local IP when they query for the HASS domain.
With IPv4 you can do it with a port forward, but the IP listed isn't the same as the IP you'll actually receive the connection on. You can get your external IP relatively easily with `curl ifconfig.co`.
If firewalls or port forwards aren't available to you, pick any of the popular port forwarding methods (here's a list: https://github.com/anderspitman/awesome-tunneling). I'm a big fan of just SSH-ing into an internet connected machine myself (`ssh -R 0.0.0.0:3000:127.0.0.1:3000 server.on.the.inter.net` will expose localhost:3000 on server.on.the.inter.net:3000 and encrypt the traffic between you and the server as an added bonus!) but there are plenty of paid and sketchy alternatives if you don't have a server to SSH into.
You should be able to install a Nextcloud app on an old unused Android phone, go through a quick OAuth flow to connect it to a subdomain, plug the phone into power in a corner and be done with it.
Cloudflare is currently the closest to this ideal with their domains + Cloudflare Tunnel, but cloudflared is still CLI-only and all their products are built for developers. Plus they'll never offer end-to-end encryption.
I also maintain a list of more DIY tunneling services[0], but again CLI knowledge is currently a requirement. My main side project at the moment is seeing how far I can lower the barrier of entry.
And don't even get me started on web3.
I went ahead and beefed up the ZT entry in the list[0] a bit.
I'd have to dig into the details more, but something like this might allow you to implement a simple tunneling system based on WireGuard that runs in the kernel if you have the privileges, otherwise falls back to usermode and is no worse than QUIC in terms of performance. That would be awesome.
Main downsides to Cloudflare Tunnel are no e2ee (Cloudflare decrypts all your traffic) and technically anything other than basic HTML websites (ie media streaming) is against their free ToS, though I haven't heard of that being enforced in practice.
If you're the only one ever using your services then I'd recommend Tailscale instead, which sets up a VPN using WireGuard along with slick auto p2p setup (NAT traversal, relays, etc).
1. The Tor method mentioned in this article - Will probably be the worst performance but it's free and offers some privacy features.
2. VPN - WireGuard (Tailscale is popular), OpenVPN, IPsec etc. Best performance, but requires some low-level configuration on both the client and server. You'll probably need admin privileges on the client to configure network interfaces. This is usually automated in practice. If you're on network that blocks UDP, that can also be an issue. You also need a server with a public IP, which generally you'll have to pay for.
3. User-space tunnels - Solutions like SSH remote port forwarding, ngrok, etc. Convenient because on the client you don't need elevated privileges. Performance will likely be somewhere between Tor and VPN. Although note that unless you have very fast internet upload speeds, tunnel performance isn't likely to be your bottleneck. Many solutions multiplex over a single TCP connection, which can cause issues in lossy network conditions due to head-of-line blocking. You need a server with a public IP just like with a VPN.
I maintain a list of proxy tunneling solutions here:
We're going to have to prove that self-hosting is useful before people start demanding ipv6, public static IPs, and decent upload speeds from their ISPs.
https://github.com/anderspitman/awesome-tunneling
If you wanted to self-host a website from your home computer today I would recommend buying a domain from Cloudflare, and using Cloudflare Tunnel.
6 months from now I hope to be suggesting some variation of my open source alternative, https://boringproxy.io. It's not quite ready yet.
My only concern is if it gets popular enough that a lot of self-hosters start using it for things like Plex and Jellyfin, which I believe is against Cloudflare's TOS, they might crack down on that.
If you prefer a self-hosted tunneling system, I maintain a list here:
In this case, the PiBox is being offered by KubeSail[1], a company which specializes in services like tunneling (they're already on the list mentioned above). It will be very interesting to see if they can bundle it in such a way that people don't have to worry about how the tunneling works. Extra points if they make it easy for users to buy/use their own domains.
I'm hopeful we'll also see some robust QUIC-based tunneling tools over the next couple years.
I maintain a list of software to help simplify the networking bits:
What works now:
* Tunnel proxies[0] like ngrok. You can get an HTML page on the internet in 2 minutes with ngrok.
* Software like Caddy which simplifies certificate management.
What's needed:
* IPv6 and ISPs to drop NAT so we don't need proxies. Good luck.
* More upload throughput. Good luck.
* Services that make it much easier to register a domain name and manage DNS. Should be easier than buying and setting up a phone. This is ripe for the picking.
* Software designed with federated self hosting in mind. Like Inrupt Solid and similar projects.
OT: Text-to-speech has gotten really impressive these days. Took me a solid minute to figure it out.
https://github.com/anderspitman/awesome-tunneling
Always on the lookup for tools I may have missed.
Happy to add hoppy if you think you're a good fit and submit a PR.
A couple questions:
* Do you provide any sort of reverse proxy service, ie HTTPS termination with Let's Encrypt support, similar to OP?
* Do you provide any DNS integration or automation, or purely IPs?
EDIT: Also how would you compare hoppy to Tailscale?
https://github.com/anderspitman/awesome-tunneling.
There are a surprising number of tools that all do essentially the same thing, and it's often hard to tell how they're different from the other 30 options.
Making a really good solution to this problem has been the focus of my free time for the past month since I wrote this comment[0].
The project I'm working on:
* 100% open source and MIT licensed
* Strive for simplicity throughout. Very few dependencies. No config files, and only a single CLI parameter required to run the server.
* Clients can be remote controlled by the server, and each client can open an arbitrary number of tunnels. ie you don't have to start a separate ngrok process for each tunnel. Just set up the client on each machine you want to tunnel out of and forget about it. Everything is controlled through a simple web UI (or REST api).
* Automate as much as possible. Tunnel creation and Let's Encrypt are already working. Eventually you'll be able to add an API token from your DNS provider and that'll be automated as well.
It's not quite ready (launch planned for 31 Oct 2020), but you can check it out here: https://boringproxy.io/
I think technologies like WireGuard are going to play an important role in reducing that complexity (once we get some higher abstractions). I started a list of useful software I've found in this space here[1].