The most important and the key difference between Buddhism and non-Buddhism lies in taking refuge in the Three Jewels. Accordingly, taking refuge is deemed a prerequisite for anyone who wants to learn Buddhism. However, it this has never been forced upon anyone. Only those who want to learn the Buddha’s teachings or take up Buddhist practice must complyadhere to this. Not taking refuge is toone remains an outsider (with regard to the Buddha’s teachings), is off the path to liberation, and cannot be deemed a Buddhist.

~ Luminous Wisdom 1 : The Three Differences