This is something I have learned is very confusing to people, as it was the most often asked question in my Security Basics for WSE talk. I have learned now to explain it as part of the talk, but thought to put it here as I had the same question asked in an email this morning.

You can encrypt data with either the public or private key.

It depends on your goal.

If your goal is confidentiality (i.e. only the recipient should be able to read/comprehend the message) then you want to encrypt with their public key so taht only their private key is able to decrypt the message.

If your goal is authentication (for example, when encryption is used as the process of digital signatures, the source of the message needs to be authenticated) then you need to encrypt with your private key. The recipient then uses your public key to decrypt and that will ONLY work with a message that was encrypted with the correct private key.

If you encrypt with a private key, you can only decrypt that message with the public key from the pair.

If you encrypt with the public key, you can only decrypt the message with the private key from the pair.

It is definitely confusing when you see some cases (eg: confidentiality) where the public key is used to encrypt and others (eg: digital signatures) where the private key is used to encrypt.