Because the code will literally not compile if it isn't correct. As opposed to weaker languages where bugs (run-time problems) can exist.

I'm still something of a layman on this, and I know less about formal specification languages like TLA+, but languages like Coq work by using things like dependent types.

Here's an example: if you have a method that takes (through specifying type parameters) an Int and returns an Array of Integers, and it compiles, then you have effectively proven that it is possible to take an Int and return an Array of Integers.

This is not particularly useful to the method author that is only using that method signature to write some more specific business logic. Dependent types allow you to spruce up that method signature until it actually reflects your business logic. So then you know that if it compiles, the actual intent of your method/function is bug free.

Yes, it is true that there is a "turtles all the way down" element to this kind of thing - "how do they prove that this real source code is correct?" The answer is that languages like Coq rely on a small kernel of code that cannot be proven. It's like their axiom from which everything flows. But the kernel is small enough that people verify it with eyeballs. Bugs can still happen there, and I think they even found one a few years ago, but it didn't have any impact on all the mathematical proofs that have been written using Coq.

I'm sure some experts can jump all over what I just wrote but that's the gist as I understand it so far.