Here's my problem:
Let's suppose that I, or anyone else that reads Hacker News (like many of the YCombinator companies) creates an app (as many of them have), and this app becomes very popular in foreign countries, and goes on to get tons of users in those foreign countries...
OK, now let's suppose that a foreign newspaper, like the New York times, but a foreign version of that -- all of a sudden prints an article like this one, alleging, but not showing a comprehensive proof (like the ones in mathematics or logic) that there are all kinds of things unspeakably wrong with the app...
Well... maybe all of those things are true, but maybe they aren't...
How would I, as the end-user of such an app -- know for sure?
If the article isn't true... think of the gigantic swath of economic damage that would be caused by an article like this...
And if the article is true... well, it could be argued that you've done a great thing, and saved countless users' privacy rights...
That's quite the dichotomy of potential outcomes, based on what the truth of the matter actually is...
So here's my question to the NYT: In this era of Fake News (which you are no doubt aware of!), how do I prove to myself (as someone who is 50/50, that is, a skeptic who is willing to believe, if the proper set of facts is presented to them) that what you're saying is true?
?
Here are two things for the NYT to think about:
"Semper necessitas probandi incumbit ei qui agit"
The burden of proof lays on he who makes the claim
and
"Extraordinary claims require extraordinary evidence" -Carl Sagan
https://en.wikipedia.org/wiki/Sagan_standard
Saying that this stuff is true because the Intelligence Community says that it's true -- is not unlike making an "Argument to/from Authority":
https://en.wikipedia.org/wiki/Argument_from_authority
I don't know who the Intelligence Community is (much like I don't know who other abstract entities are like "China" or even the "U.S." are) I don't know who that is!
I do know who a guy named "John Smith" is, and I do know that if John Smith raises his right hand in a court of law and under oath promises to "tell the truth, the whole truth, and nothing but the truth, under penalty of perjury", then John Smith's claim, when he makes it under those circumstances, carries a whole lot more weight (to me) than the claims of any nameless, faceless, unaccountable organization -- by whatever name they go by...
Due Process (enshrined in our Constitution, which the media does not seem to give these days to all parties, despite the Constitution giving them their right of Free Speech) presumes innocence until proven guilty, in a court of law!
Now, all of that being said... I am willing to believe this article -- but I require a higher standard of proof...
Security is about verifiability. If you can't verify it's secure, it's not trustworthy. I've been looking into secure communication stuff for almost ten years now, and there's a low bar that makes it easy to inspect whether the app developers are even trying.
You want security for two things: Metadata and content.
Content privacy is done with end-to-end encryption. You ask yourself
1) Can I find public key fingerprints from the app? If this is not available, the app is not end-to-end encrypted properly, and you should immediately abandon it.
2) If the app is end-to-end encrypted, can I inspect the source code to verify end-to-end encryption is implemented properly? If you can't do that, i.e. if the source code is not available, there's no way to trust the app. If you lack skills needed to do that, you can pitch into an audit by third party.
3) If the source is open, can I verify I'm using binary compiled from said source? I.e., does the client have reproducible builds.
If all these three are true, then you can be sure it offers quite strong protection for your content.
This is true for e.g. Signal.
Regarding your Sagan Standard, the extraordinary claim is not
"It has a backdoor because we can't check."
The extraordinary claim is
"It doesn't have a backdoor, it's secure, but no, you can't check that for yourself."
At that point you need to have extraordinary proof that can verify the security. There's no pro-security argument in keeping an app proprietary, only monetary.
---
Next, metadata:
You ask yourself
1) Can I register the service anonymously, i.e. without giving any of my details
2) Can I connect (both register and use the app) via Tor? If not, your IP reveals your identity.
3) Who's hosting the service? Does a third party learn how often UID #4346455432 sends messages to other parties. I.e. is the app peer-to-peer. If the app is not peer-to-peer, your peers' bad OPSEC might deanonymize you if server-side data is cross-compared with an other social graph.
4) Can I misconfigure the system and lose anonymity (i.e. is the system Tor-routed by design or can you accidentally connect without Tor), and can you accidentally turn off end-to-end encryption and say something that might deanonymize you to the server?
ToTok is not open source, it doesn't have public key fingerprints, and judging by the completely novice, non-technical name-dropping of a few primitives in their PR statement (https://totok.ai/news):
"Furthermore, we equipped ToTok with such high-security standards as AES256, TLS/SSL, RSA and SHA256, to diligently protect the user data. We also implemented a privacy framework that complies with the local and international legal requirements to safeguard our users at all times."
It's trivial to implement backdoored system with said primitives: TLS is encryption that uses AES256, RSA, and SHA256, and it's only encryption between user and server, so server has access to plaintext data which can be given to the secret police etc. Even if they say they don't comply they might a) lie or b) be hacked.
The only way to protect from that is to always use E2EE, something ToTok does not do. So no, there's no reason to trust them. They haven't even attempted to deploy E2EE or anonymity for their system. Even if they're not intentionally being evil, they're being ignorant about security and thus, ignorant about the human lives affected.
First off a few things: I appreciate your nuanced and well-written, well-thought out comments. You are a very smart person, and you are an excellent writer. I agree with what you said about 98%. We could stop here, and we could shake hands, or I could go on to describe the 2% of what you've written that I don't exactly agree with. With your permission, I'm going to discuss that 2%. Please understand that I mean you no personal offense in the following discussion.
First understand that I in no way support ToTok, I don't use it, I have never used it, and don't really care to potentially use it. I have no financial or interest in the company. I didn't even know it existed until I read the NYT article.
I do, on the other hand, have a very deep interest in Online Communities, Apps, Users, Users' rights, including but not limited to privacy, and Company's rights, including, but not limited to not being destroyed (financially and/or reputationally) by media smear campaigns, intentional or unintentional, explicit or implied.
Now you say: "Regarding your Sagan Standard, the extraordinary claim is not "It has a backdoor because we can't check." The extraordinary claim is "It doesn't have a backdoor, it's secure, but no, you can't check that for yourself."
No, the extraordinary claim is the title of the NYT article: "It Seemed Like a Popular Chat App. It’s Secretly a Spy Tool."
Now... here's the thing (and this is where things get really subtle, so follow along closely!).
That title, and all that the article alleges, might be 100% true!
So what's the problem then?
The potential problem looks something like this:
Let's suppose that an app developer in the U.S. does all of the things you say to to, for the sake of security/privacy.
OK, and now let's suppose that a Nation State Attacker -- our guys, their guys, whoever -- somehow subverts the security of the app (Browser/OS/Hardware/Network/? hacks) and gathers the apps' data.
Now... let's suppose that some foreign newspaper, the foreign equivalent of the NYT -- prints a story that our friend, the U.S. app developer -- who did nothing wrong, and tried to do everything right -- is actually gathering data for the Nation State Attacker (foreign or domestic), and thus is a "Spy" tool.
Well guess what! Under those conditions, the story is absolutely, absolutely CORRECT!
Except that there's a teensy little problem...
Call it a lie; call it an error of ommission...
The problem is as follows:
The problem is that this newspaper article's author/authors, being ignorant of the fact that there was NO INTENT of the app developer to have their app used as a "Spy Tool" -- yet that was the end effect, caused by circumstances and actors outside of the control of the app company!
That's scenario #1.
Scenario #2: The law/legal system of the country in question is used to force the intentional engineering of hidden backdoors for the Nation State Attacker, under penalty of a very long prison sentence...
Either way, the paper's story is "correct" -- but it causes a crapload of harm to the application developer, who only wanted to make a dollar by providing an equal-and-opposite value, and never intended for these things to happen!
Of course, if the ToTok developers intentionally engineered security holes while not being coerced to do so (and the NYT could present proof of this), THEN this article is clean, ethical journalism... otherwise the NYT should not have presented it until it had, and was willing to show to the world this proof...
Would you want a story like this to break in some other part of the world, some part where you do business, but some part where you might lack the language/cultural/legal skills to defend you and your company, when you and your company are acting with the best intentions in mind?
I'm not a Christian, but... remember that "Do Unto Others" quote...
That's why we have have Due Process, The Right Of An Accuser to Face The Accused (Cross Examination), and the Presumption of Innocence -- in our U.S. system of Law...
That's why in this day and age, I never assume that 100% of what the media says is true, 100% of the time.
I am sorry, but even if ToTok has betrayed its user trust (and I'm not saying they did or didn't) -- then certainly the U.S. media has, more often than not, betrayed mine...
I reiterate that the burden of proof -- is on the NYT.
"Either way, the paper's story is "correct" -- but it causes a crapload of harm to the application developer, who only wanted to make a dollar by providing an equal-and-opposite value, and never intended for these things to happen!"
They themselves took the risk when creating an insecure product. There are two ways companies can botch up security.
1) Introduce a bug that can be exploited by intelligence agencies. These happen sometimes and they're just a fact of life.
2) Design a crappy protocol. If you want to lock yourself out from your users contact's you must by definition implement strong, authenticated end-to-end encryption, and open source client with reproducible builds. That's the only way.
Once you've done that, any vulnerability will be a bug, and those are okay, they can be fixed when found.
The problem here is ToTok developers didn't do that. They didn't even try.
Now, as for metadata, VoIP capability makes Tor-routing by default impossible, thus we have need for two secure messaging apps on networked TCBs: Signal for E2EE VoIP, Briar for anonymous messaging.
Networked TCBs can not be made unhackable, thus you can't expect privacy from remote endpoint compromise, but that's again, OK, because it scales less, and is a fundamental limitation.
So no I do not agree with you because they could've done a LOT more but chose not to.
So here's what we know:
ToTok is not E2EE, thus we know for a fact their server has access to user data. The only way for NYT to independently verify the company is handing out data from the server, is to either obtain documents (which might never happen) or hack ToTok server to see such processes taking place. Which is illegal.
So I agree I'd rather see some whistleblower reveal documents, but we can't hope for that to happen if Signal etc. are blocked in UAE. If this leads to unblocking of more secure messaging apps, perhaps we'll see the evidence you need one day.
Your claim that "They themselves took the risk when creating an insecure product" is based on the idea that you know how to create a secure product. Your idea that you know how to create a secure product is based on the use of E2EE being secure. E2EE's security is based on the use encryption and secret keys, which must remain secret.
Well... if you read enough articles on HN, you'll see that there are a variety of ways that secret keys could potentially be compromised, for example, malware, spyware, what-have-you.
Most of these attacks are out-of-band. That is, there is no way that an application developer could possibly be responsible for them, should they occur to their app...
"They themselves took the risk when creating an insecure product" "So no I do not agree with you because they could've done a LOT more but chose not to."
Yes, but the jury is still out on those...
I have no arguments with you personally.
My skepticism is directed solely towards the New York Times.
All I know is that if I ever created an app and ran a company and acted with the best intentions for my users, I'd never want an article like this to appear about my app or my company in some foreign press...
That is why I give ToTok the benefit of the doubt until stronger more compelling evidence (which includes intentionality) is revealed.
"E2EE's security is based on the use encryption and secret keys, which must remain secret -- there are a variety of ways that secret keys could potentially be compromised,"
The argument can not be "anything can be compromised with sufficient resources, _therefore it does not matter if you don't follow best practices within limitations of the architecture_".
"Most of these attacks are out-of-band."
Agreed, but you also can't argue "the rest of the entire software stack isn't perfect, therefore I don't have to follow best practices". If E2EE fails because of vulnerability in OS, you aren't responsible. If it fails because of vulnerability in your app, then you are responsible. The same way, messaging app vendor is responsible for not using E2EE protocol which eliminates a huge gaping hole: e.g. the entire crypto community has criticized Telegram cloud chats that leave plaintext copies of messages on the server. Not knowing this is the same as not doing your job.
It's also the case you can design your software around more secure architecture to protect against remote key exfiltration, see my work on TFC for example: https://github.com/maqp/tfc
"All I know is that if I ever created an app and ran a company and acted with the best intentions for my users"
The question is, with E2EE becoming almost ubiquitous, would you really think not implementing modern encryption protocol is the same as acting with best intentions.
"That is why I give ToTok the benefit of the doubt"
I can see where you're coming from but having worked with secure messaging so long, to me it's the same as surgeon not verifying they were using industry standard materials for medical screws they use. Were they really acting in best interest of client? You can't excuse not verifying (industry best practice) just because you're "trying your best to minimize other complications".
You're working with sensitive data. You have industry checklists available. If you choose the business, there will be obligations. You don't get benefit of the doubt as a participation award when it's effin obvious you did not use the checklist.