E-Brake Issue
 

When I release the e-brake on my car the passenger side occasionally will not release. I can walk around the car and reach under and push the bracket at the end of the cable and it will release though.
I put it up in the air today and used some PB Blaster on it since there was some rust discoloration around it, and it seemed to help, but then it did it again.
So does this require replacing the caliper, or is there something else I can do to try to make it work properly? The spring is in place and is appears to be putting plenty of tension on it, but it takes some force by hand to "pop" it loose.