In the case of Signal, I imagine people assume all of the following:
1. the protocol between client and server is setup in such a way, even if Signal wanted to store interesting information, they could not access anything interesting even if they wanted to (for example, messages), thus they don't store anything since it's useless
2. the app implements the protocol faithfully and this has been checked by people perusing the source code
3. the binary downloaded from the app/play store phone is compiled from the sources listed on github
This might seem like a naive question, but how is it possible to verify that 3 is true?
I get how it might be done in theory but real life is complicated. Has anyone attempted to do this?
I imagine it's non-trivial. I think it would involve (in case of iOS):
1. downloading the binary
2. jailbreaking the phone to extract the binary (pretty sure this is necessary on iOS)
3. check the version of the binary, then compile the original sources of the version
4. ??? compare the two binaries, this is likely the most difficult part, they won't be identical because of things like codesigning (and build flags, timestamps, ...)
I know noone that does this.
You don't compare builds because you probably don't actually have sources. What you do is use a special iPhone (a Security Research Device) that Apple grants some researchers or you use an emulator like the one from Corellium (to whom Apple recently lost a lawsuit over this emulator) to probe and step through the code. Find the key sections that do the real crypto work and make sure that they do what they are supposed to do and that they are getting the correct inputs.
There is a large group of people who do this sort of research, and some fraction of them do this research and actually talk about it or publish papers. If you could find a deliberate weakness in the security of an app like what we are talking about (or WhatsApp or iMessages) then you have just printed your own golden ticket to whatever mobile cybersecurity job you want for the next decade or two, so there is a bit of an incentive to publish if something like this was discovered...