If you turn off peaceful, attack someone, and turn peaceful back on, you still hit them. (This is a holdover bug back from .89)