Zoom CEO - I messed up

Hi
It seems there is a wsj interview where zoom Ceo mentioned yesterday that they will add end to end encryption soon. And full encryption in near future.
https://seekingalpha.com/news/3558448-i-really-messed-up-on-…

I use Cisco webex for my business.
My kids, wife, temple use zoom for meetings.
I plan to switch to zoom as paying customer after this security fix.

Newbie

6 Likes

Newbie,

There seems to be some confusion still about Zoom and E2E.

Zoom is and always has been true E2E by default for 99.9% of use cases.

I’ll admit that Zoom has been responding to everything by owning an issue and saying we are going to or have fixed it.

This blog describes it. Their infrastructure is set up to have E2E. I’ll bold it as it’s billed in their blog.

To be clear, in a meeting where all of the participants are using Zoom clients, and the meeting is not being recorded, we encrypt all video, audio, screen sharing, and chat content at the sending client, and do not decrypt it at any point before it reaches the receiving clients.

To have more context read this.

Zoom clients include:
-A laptop or computer running the Zoom app
-A smartphone using our Zoom app
-A Zoom Room

In this scenario, where all participants are using the Zoom app, no user content is available to Zoom’s servers or employees at any point during the transmission process.

Does everyone get this? If you are having a Zoom happy hour with 10 of your friends and everyone is on their phone, iPad, or laptop every stream is E2E by default. Same for meeting in Zoom room to Zoom room or zoom room to 100 remote workers. Or for your teacher to your kids.

By comparison here is WebEx by default.

Media streams flowing from a client to Cisco Webex servers are decrypted after they cross the Cisco Webex firewall. Cisco can then provide network-based recordings that include all media streams for future reference. Cisco Webex then re-encrypts the media stream before sending it to other clients. However, for businesses requiring a higher level of security, Cisco Webex also provides End-to-End encryption. With this option, the Cisco Webex cloud does not decrypt the media streams, as it does for normal communications.

Notice the difference. Normally Zoom does NOT decrypt as streams come into their servers, whereas that IS normal for Cisco.

Back to Zoom. There are 2 circumstances where Zoom decrypts and then re-crypts for not having full E2E. At one end of the call a client is using some sort of legacy system not capable of running the Zoom normal encryption. Like having an old Skype for Business conference room that Zoom can still stream to or a landline legacy phone system. The other is if you CHOOSE to integrate with certain third party systems. This occurs in Zoom’s data center. Your stream is decrypted sent to your third party software and back(often undergoing encryption going both ways there too) then re-crypted and sent on its way. If you want your Zoom meeting livecast for instance, this is just part of the process.

But these are very specific use cases and not typical.

So when you read articles like the one from The Intercept they are taking things out of context and using misquotes or straight up incorrect interpretations.

More about Zoom encryption.

By default, Zoom encrypts in-meeting and in-webinar presentation content at the application layer using TLS 1.2 with Advanced Encryption Standard (AES) 256-bit algorithm for the Desktop Client.

For dial-in participants joining by phone, the audio is encrypted until it leaves Zoom’s data centers and is transferred to the participant’s phone network.

Zoom’s encryption is entirely normal and appropriate. But they are going above and beyond to respond to concerns and Yuan is pulling out all the stops to make those that are concerned comfortable. Notice how he uses “I” and “me” and not trying to push it off on subordinates.

The irony is that Zoom probably winds up being a way more secure platform than other services whose similar or worse practices are largely being ignored by the media. Teams works just like WebEx above does if anyone was curious.

Darth

58 Likes

To be clear, in a meeting where all of the participants are using Zoom clients, and the meeting is not being recorded, we encrypt all video, audio, screen sharing, and chat content at the sending client, and do not decrypt it at any point before it reaches the receiving clients.

That is correct. But it appears even in that situation zoom holds all the encryption keys and can decrypt if they want to. I am not sure if this is common industry practice. See my post in an an earlier thread which I pasted below, specifically #1 and #4.

By default, Zoom encrypts in-meeting and in-webinar presentation content at the application layer using TLS 1.2 with Advanced Encryption Standard (AES) 256-bit algorithm for the Desktop Client.

This is what the Intercept article says “Zoom’s keys conform to the widely used Advanced Encryption Standard, or AES. A security white paper from the company claims that Zoom meetings are protected using 256-bit AES keys, but the Citizen Lab researchers confirmed the keys in use are actually only 128-bit. Such keys are still considered secure today, but over the last decade many companies have been moving to 256-bit keys instead.” Also see #3 in my post below. So it is a not an Intercept magazine just writing this up. They are quoting Citizen lab which is part of the University of Toronto.


  1. Encryption keys are developed by a type of zoom server called “key management system”. 5 out of 73 of these servers are in China. US traffic is routed through those servers in a small no. of cases. Chinese entities can get access to these keys and be able to listen in. UK cabinet meetings, US covid response, Navy. OPM, State dept. are some of the agencies using zoom. Zoom has said govt. customers were not routed through China. Perhaps this is not as much of an issue?
  2. Encryption keys are 128 bit not 256 bit which is what many companies now use.
  3. Encryption keys uses “an algorithm called Electronic Codebook, or ECB, mode, “which is well-understood to be a bad idea, because this mode of encryption preserves patterns in the input.” The article goes on to show a very good example.
  4. Zoom has access to all encryption keys and could decrypt any meeting. The company says they have internal controls against that and have not developed a process to do that. Is this a normal industry practice for a company to hold encryption keys?

Finally, the original article concludes:

As a result of these troubling security issues, we discourage the use of Zoom at this time for use cases that require strong privacy and confidentiality, including:
Governments worried about espionage
Businesses concerned about cybercrime and industrial espionage
Healthcare providers handling sensitive patient information
Activists, lawyers, and journalists working on sensitive topics

Issue #1 maybe a perception issue at this point. Issues #3, and #4 concern me most. Any thoughts why that should not be a concern?

https://theintercept.com/2020/04/03/zooms-encryption-is-not-…

https://citizenlab.ca/2020/04/move-fast-roll-your-own-crypto…

9 Likes

The attached article goes more into depth.

The noteworthy aspect of the debate is that WhatsApp is end to end encrypted, while GMail is just encrypted. What this entails: When we send an email or a chat message over Gmail, it is encrypted in transit over public internet. If a malicious hacker sniffs this message in transit they would see random text (which is how it should be). But when message reaches Google’s servers, it has the keys to decrypt the contents of the email. That’s how it can detect spam and show ads in Gmail. Basically the encryption is handled server side.
On the other hand, with end to end encrypted apps like WhatsApp (among many) the difference is in the ownership of encryption keys. WhatsApp does not own keys to decrypt messages on its servers, they are generated on-device for each message and not sent over network. Thus the only data sent over network and through WhatsApp’s servers is encrypted gibberish. The decryption happens on the end user device where the message is targeted.

https://sidstechcafe.com/gmail-vs-whatsapp-end-to-end-encryp…

5 Likes

With Zoom for Government (we are authorized Zoom in the Air Force) the Zoom servers are all contained in the AWS Government Cloud. That service is certified walled off for only the government. Being okay for government use is different for okay for classified use.

For encryption and Gmail vs WhatsApp. If the worst critics in their best argument claim Zoom is only about as good as Gmail, I think Zoom wins that round with most people/organizations.

Beyond that I’m probably in over my head. You’d have to make Zoom out to be the bad actors or to be compromised in some way. I trust lots of companies with all kinds of data. It’s just a fact of life.

Most users want Zoom to have that capability so that you can use services like recording, transcribing, or webcasting, etc. if you don’t select those services it stays encrypted the whole time. Gmail the same. They want Gmail to sort your spam or search for threats. Of course Gmail also wants to target you with ads. The difference is that with Gmail it’s all decrypted in Google’s cloud, whereas with Zoom you have to elect to receive a service whereby Zoom goes through the decrypt process.

Darth

5 Likes

But it appears even in that situation zoom holds all the encryption keys and can decrypt if they want to. I am not sure if this is common industry practice.

Isn’t it the case that, in order to offer recording of the session, the provider necessarily has to be able to decrypt? Do all such services provide recording? If they don’t, isn’t that a major weakness?

In any case, it seems to me that all of these criticisms are, at best, things easily fixed and not fundamental to the service being provided. I.e., at best a small rework, not a major revision.

1 Like

That is correct. But it appears even in that situation zoom holds all the encryption keys and can decrypt if they want to. I am not sure if this is common industry practice. See my post in an an earlier thread which I pasted below, specifically #1 and #4.

They would need two different encryption methods to be able to read one but not the other. I doubt that is the case now but it might happen with a different rate for each.

Denny Schlesinger

1 Like

Disclaimer: I’m not a security expert, but I have been involved with security aspects of software development and deployment as part of my job.

  1. Zoom does not support end to end encryption. Cramer said in the interview with Zoom’s CEO that Webex doesn’t offer end to end encryption either. That is incorrect. Here is WebEx’s page on that: https://help.webex.com/en-us/WBX44739/What-Does-End-to-End-E…

Media streams flowing from a client to Cisco Webex servers are decrypted after they cross the Cisco Webex firewall. Cisco can then provide network-based recordings that include all media streams for future reference. Cisco Webex then re-encrypts the media stream before sending it to other clients.

This is similar to what Zoom does. What’s different is that WebEx already offers an end to end encryption setting (https://help.webex.com/en-us/n4f016ab/Use-End-to-End-Encrypt… ). As WebEx states (first link):
However, for businesses requiring a higher level of security, Cisco Webex also provides End-to-End encryption. With this option, the Cisco Webex cloud does not decrypt the media streams, as it does for normal communications. Instead it establishes a Transport Layer Security (TLS) channel for client-server communication. Additionally, all Cisco Webex clients generate key pairs and send the public key to the host’s client.

The host generates a symmetric key using a Cryptographically Secure Pseudo-Random Number Generator (CSPRNG), encrypts it using the public key that the client sends, and sends the encrypted symmetric key back to the client. The traffic generated by clients is encrypted using the symmetric key. In this model, traffic cannot be decoded by the Cisco Webex server.

I can translate that.
If E2E Encryption is enabled in WebEx, then:

  1. TLS (what underlies the “https” we all know and love) is used to protect data flowing between meeting clients and WebEx. This stops internet eavesdropping.
  2. The host client (not the WebEx server) generates a Meeting Encryption Key using a good method (CSPRNG).
  3. Each client has its own Private Key that no-one else knows, not even WebEx.
  4. Each meeting client sends their corresponding Public Key to the host of the meeting.
  5. The host client encrypts the meeting key with each client’s Public Key and sends that to each client.
    Note: the way Private/Public keys work is that one key is used to encrypt, the other to decrypt AND that the key used to encrypt will not successfully decrypt, so it’s OK to share the Public key with everyone, as implied by its name.
  6. Now each meeting client has the Meeting Encryption Key, so they can all encrypt their feeds and decrypt everyone else’s.
  7. WebEx itself does not have the Meeting Encryption Key, so it cannot decrypt the meetings.

The whole point of E2E encryption is that no-one except the ends can decrypt. Zoom doesn’t have this capability today.

Some commentary on this:
A) It is a lot more than a bug fix to implement a scheme like this.
B) Some of the cryptographic capabilities needed in the clients will make them illegal to be exported to some countries. (US Gov regulations).
C) As a corporate customer, I would consider Zoom’s incorrect characterization of their encryption as “end to end encryption” a disqualifying flaw. Assuming it wasn’t intentional, even assuming the engineer’s designing end implementing the system knew better but product management decided it wasn’t a necessary feature, not having the internal mechanisms to prevent Marketing from incorrectly classifying the security of the product is itself indicative of a lack of necessary oversight.

In that last regard, when Cramer brought up Zoom hiring an outside security company (he mentioned Crowdstrike) to review Zoom’s security practices, Yuan didn’t say “We already do/did that.” He didn’t even say “We’re going to do that.” Using a third-party security company is not just de rigueur, it’s standard operating procedure. If Zoom hasn’t done that, it’s yet another corporate adoption disqualifying attribute. Another is that Zoom doesn’t produce regular transparency reports, another standard operating procedure.

  1. End to end encryption isn’t the only way to accomplish a high degree of privacy. Zoom has promised (https://blog.zoom.us/wordpress/2020/04/01/facts-around-zoom-… ) to provide features in the future to enable: certain versions of our connectors within their own data centers if they would like to manage the decryption and translation process themselves. It’s more trouble for customers this way, but having the meeting streams hosted entirely within a corporate’s network would provide enough security for many corporate customers. Note, however, this is a future feature promise, not a bug fix nor a current feature.

  2. The China server thing:
    Since Zoom holds the Meeting Encryption Key on its servers (remember, no E2E encryption option), they apparently have yet another bug where they don’t control which of their servers get which keys. If any of those servers are located in China, then by Chinese law, China already has access to the data on those servers, and hence has the keys. Period.

It is possible if the Chinese government wanted, to look in real-time at the Zoom Meeting Encryption Keys and find meetings to decrypt as they’re happening. That sounds impractical, but it’s indicative of bad security practices.

What we don’t yet know is whether meeting recordings also are being stored on servers in China. If so, the Chinese government by law already has access to those recordings. Even if they’re encrypted on the server, China may also have the encryption key since Yuan admitted those sometimes get to Chinese servers.

I have been involved with providing my company’s products/services to customers in China. Here’s a supporting data point: On a 60 minutes episode (https://www.cbsnews.com/news/electric-cars-chinas-drive-to-d… )in Feb 2019 on EV company Nio, they visited the Shanghai Electric Vehicle Data Center, a government agency which collects millions of bits of information every day on nearly 200,000 electric cars on this city’s streets. They’re looking a screen showing a portion of a map of Shanghai, with dots (some moving) representing cars they’re tracking.
Inside every electric vehicle in the city is a black box, automatically transmitting data to the center every 30 seconds.
Ding Xiaohua: For example, the speeds, the mileage, the battery temperature.
Holly Williams: And does that help the government plan for the future?
Ding Xiaohua: Yes, public charging points, how many public charging points? And where it is best place for the public charger.

Turns out all the cars were Teslas, and that they were looking at data from Tesla’s own servers transmitted every 30 seconds to the government. Tesla was required to do that. (I don’t now or ever have worked for Tesla). Drive a Tesla in China and the government knows where you are within 30 seconds or less. No subpoena needed.

  1. Just 5 days ago, Zoom issued a blog post (https://blog.zoom.us/wordpress/2020/04/01/facts-around-zoom-… ) that has been discussed much here. In part it says: To ensure this entire process meets the needs of our customers around the clock and around the world, Zoom currently maintains the key management system for these systems in the cloud. Importantly, Zoom has implemented robust and validated internal controls to prevent unauthorized access to any content that users share during meetings, including – but not limited to – the video, audio, and chat content of those meetings.

Well, we now see that these “robust and validated internal controls” are not so robust, and this makes the “validated” aspect suspect (see above for lack of third party scrutiny).

  1. Zoom has had a history of not considering security in their product. Back in June 2019, Zoom was found to have intentionally bypassed Apple security to enable users to join a call without clicking a Safari box confirmation to run the Zoom application. Here’s Zoom blog post from the time: https://blog.zoom.us/wordpress/2019/07/08/response-to-video-…

At first, Zoom denied this was a security issue and refused to patch. But, after much press, including explanations about how malicious attackers would leverage this by sending a tiny URL via email or text to someone, who would then click it, and it would start the meeting unbeknownst to the user, and thus his webcam would be on for the attacker to view and hear. Zoom initially tried saying that they would let people secure themselves by providing an option to not turn the camera on by default, but later relented and issued a patch to not provide this hackable local web server.

Unfortunately, Zoom’s engineering wasn’t on the ball. As Zoom admitted: We do not currently have an easy way to help a user delete both the Zoom client and also the Zoom local web server app on Mac that launches our client. The user needs to manually locate and delete those two apps for now. This was an honest oversight.

This episode cements for me that the company is focused on ease of use above all else. That is a good thing, if their security was good enough and not misrepresented.

  1. Yes, some of the criticisms have been overblown and misrepresented. For instance, some have shown a frame from a Zoom meeting with the UK Prime Minister as if that was hacked, when in fact it was a frame tweeted by Boris Johnson himself: https://twitter.com/BorisJohnson/status/1244985949534199808 . So there is some fake news on Zoom, but that doesn’t wash away the real security issues Zoom has.

In my view, what has happened is that Zoom is being forced to grow up more rapidly than they expected. Their focus on ease of use was great. However, their free limited account tactic has backfired on them in this travel lock-down era as they were clearly unprepared to support the security needs of many corporate and government clients. They compounded the issue by incorrectly advertising not just their security implementations, but also apparently their security practices, which by all apparent measures are insufficient. This has resulted in bad press even for people like my family, who are now questioning they should still hold things like Book Club on Zoom for instance.

I still have a moderate position in ZM, which is still up fairly nicely. But, I have lost all faith in management’s ability to address their security issues, and think that their blog posts and CEO visits on Cramer and the like will eventually not be good enough. At this point, they’re surviving by their ease of use advantage and freemium model. This leaves big holes for the competition to step into, but I don’t know if any will.

59 Likes

Isn’t it the case that, in order to offer recording of the session, the provider necessarily has to be able to decrypt?

No, it’s not the case.

If the meeting were end to end encrypted, then Zoom could simply record the encrypted meeting and also store the host’s public-pey-encrypted Meeting Decryption Key. Since only the host client has the private key necessary to decrypt the Meeting Decryption Key, Zoom itself would not be able to get that key and decrypt the recording. Even under government order.

If Zoom wanted more people than just the host to be able to access the recording, it could store the encrypted description key for each original participant. Additionally, for any other people wanting access to the recording Zoom could provide a means for them to contact the host to get the decryption key (which would be provided encrypted by the requester’s public key).

4 Likes

Smorgasbord1,

The link you posted also states that when end to end encryption is enabled Cisco Webex Meetings has some limitations. https://help.webex.com/en-us/WBX44739/What-Does-End-to-End-E…

"This End-to-End encryption option is available for Cisco Webex Meetings and Webex Support.

Limitations:
when end-to-end encryption is enabled, the following features are not supported:

Join Before Host
Telepresence Video End Points (formerly known as Collaboration Meeting Rooms Cloud)
Cisco Webex Meetings Web App
Linux clients
Network-Based Recording (NBR)
Saving session data, Transcripts, Meeting Notes, and etc…
Remote Computer sharing
Uploading shared files to the meeting space at the end of the Cisco Webex Meetings
Personal Meeting Rooms"

Just curious if you the missed the limitations of Cisco Webex Meetings in your post?

Cheers!
ron

long <OKTA, AYX, DDOG, NET, ZM, CRWD>

1 Like

Ron and Smorg,

We’ve already requested that the in-depth technical discussions on ZM’s security come to an end. Can you take them off-board, please?

Thanks!

Bear
Asst Board Mgr

1 Like

In that last regard, when Cramer brought up Zoom hiring an outside security company (he mentioned Crowdstrike) to review Zoom’s security practices, Yuan didn’t say “We already do/did that.” He didn’t even say “We’re going to do that.” Using a third-party security company is not just de rigueur, it’s standard operating procedure. If Zoom hasn’t done that, it’s yet another corporate adoption disqualifying attribute.

Hi Smorg
Interesting that you said that, since Crowdstrike’s CEO talked extensively in his Conference Call about his company’s use of Zoom, about how 70% of Crowdstrike’s employees work from home using Zoom, and about how he personally, and his chief of sales plan to call on 100 customers and prospective customers in 100 days by Zoom. Apparently security experts like Crowdstrike don’t see these “corporate adoption disqualifying attributes” as being as worrisome as you see them.
Saul

32 Likes

Damn! You got me breaking my own rule and talking about security again. I won’t again! NO MORE!
Saul

7 Likes

But, I suspect the primary reason for recording most sessions is to make them available to a larger group and to do so as simple video content, not a part of a Zoom session.

1 Like

Not only does WebEx have some limitations … including the recording issue … but I am quite sure that in one or more of the previous Zoom security threads (now hiding under a CEO subject line) that it was said that the default for Zoom was end to end encryption, assuming that all participants are on Zoom devices and the server-side encryption was used only when needed.

https://www.mlive.com/news/2020/04/michigan-supreme-court-wi…

https://zoom.us/meeting/attendee/uZAvdOugrT4u1tE-YTz7hpOt2Ow…

Zoom book club tomorrow with two emeritus UMich engineering department heads and 85 year old HOA board members using Zoom for board meetings.

Seems like valuable exponential growth “hidden” in plain sight. I’m adding to my position again.

Hi Saul,
Since the stopping of the discussion had the caveat on “no new information,” I took the Yuan’s subsequent appearance on Cramer as new information, which is this thread. However, if you still want, I’ll drop the discussion after this post.

Apparently security experts like Crowdstrike don’t see these “corporate adoption disqualifying attributes” as being as worrisome as you see them.

What I heard in the Crowdstrike conference call was that Crowdstrike recognizes Zoom’s security issues. Here’s Crowdstrike’s Kurtz in the conference call:

And if you’re working from home on Zoom you still need to be protected, right. So I think what they figured out very quickly is kind of pushing update, signature files through VPNs or overloading things, it’s just – the whole management doesn’t work.

But, Kurtz continues that Crowdstrike can be used to make Zoom more secure:
And I think by leveraging something like CrowdStrike they see how easy it is, it’s seamless, it doesn’t even have to go through their own network in terms of what we do and how we communicate with those endpoints…and they realize that just trying to jam everything through VPN back to the mother ship is not going to work. You’ll see more and more of Zero Trust which we’re a perfect fit in that overall architecture.
(https://www.fool.com/earnings/call-transcripts/2020/03/19/cr… )

So I would conclude that Crowdstrike is using Crowdstrike Falcon to make Zoom secure enough for their purposes. Whether other companies are willing to take those extra steps is something to discuss. I would guess people already using Falcon or similar would be OK going to Zoom and connecting them up, but I have a harder time seeing companies wanting to use Zoom to also purchase Crowdstrike just to make Zoom more secure. There is, of course, a middle ground where companies go to Crowdstrike for a variety of reasons, including making Zoom more secure.

Another thing is that I don’t know is whether the sales calls with potential customers are similarly protected, since presumably those people are not part of the Crowdstrike eco-system. I don’t know the mechanisms by which Crowdstrike can be used to make Zoom more secure.

As for why Crowdstrike uses Zoom in the first place, I would hazard an educated guess is that they don’t want to use WebEx, as Crowdstrike competes with Cisco on many fronts, such as Cisco’s AnyConnect (VPN), Cisco Stealthwatch, Cisco Threat Response, and Cisco AMP for Endpoints.

Remember, I don’t view Crowdstrike as being intentionally malicious. I view them as being laser-focused on providing a great user experience and are willing to do what it takes to reduce adoption/use friction. Unfortunately, the way they went about that has had the result of compromising the security of the product.

Of course, my view of Crowdstrike’s security posture is not everyone’s view. For another perspective on this being overblown, this is a reasonable Twitter thread: https://mobile.twitter.com/HackingDave/status/12455360008199…

And note that I haven’t said anything about some complaints, like security being off by default complaints. Like we saw with MongoDB a while back, this isn’t something about which to worry. I personally think it’s fine for Zoom have security off by default as long as it’s clear in the UI how to engage it. I don’t expect my house to lock itself automatically when I leave it.

20 Likes