He uses it practically whenever he wants us to be afraid. It was always a sign that malicious ghosts were about in The Sixth Sense, and I think associated with their horrific, bloody wounds. He used the color to make you feel agitated, not warm and fuzzy.
In Unbreakable, both yellow and red are associated with unscrupulous people David contacts. Then the two colors come together when he finds the serial killer, as orange.
In The Village, although the creatures are a false threat, they still represent evil. And I don't think it's a coincidence that Noah takes on the red mantle of evil symbolically when he stalks Ivy in the woods. At that point he is consumed by hatred, not love. He is a monster. See what I mean? At the end of the movie, he lays at the bottom of the pit, defeated, but love triumphs. So whatever color love is, I doubt it is red.
And I don't think Mr. Walker hated the outside world, he was quite simply afraid of it. He allows Ivy to leave, because he loved her dearly. Yes, he made bad decisions, but it wasn't out of hatred. Just fear, I think. Remember everything he did was to preserve innocence.
But I can see where you are coming from. I do see a couple instances where red might be interpreted as love (like the red berries and the flower), but overall, red seems to be just the opposite. Bloody hands, a bloody knife, people's fears manifested in red-wearing monsters, and a young sadist (Noah) who gets a kick out of using the 'bad color' as a joke, because that is precisely what he thinks evil is: a joke.
- Sometimes we don't do things we want to do so that others won't know we want to do them. -