There are various public servers, which distribute the public key. It is quite a task to distribute the public keys between the receiver and the sender. If by chance a message-specific key is known to someone else, your message can be harmed. Let's dig into the question of how this is done.
