That's only if a peer-to-peer connection can be made in the first place, which most of the time it can't because of NAT and other things. The Element client even has a checkbox to prevent you from making peer-to-peer connections forcing you to go through your homeserver's TURN server or Matrix's fallback TURN server.

Edit: To clarify the warning under the "Allow fallback call assist server" saying your IP address will be shared, it means it will be shared with matrix.org, not the parties you are calling.

Show