This article explains that while it is not possible to add or remove friends from a conversation that has already begun, you can unsubscribe yourself.

Once a conversation has begun, you are unable to add or remove friends from the conversation. If you have added a friend who does not want to receive emails or notifications about the conversation, they will need to unsubscribe.

To unsubscribe from a conversation:

1. Move your cursor over the Conversations tab.
2. Click the Options button at the top of the tab.
3. Click Unsubscribe from this conversation.

Once unsubscribed, they will no longer receive emails or notifications about any activity on this conversation.