I find this tactic to be more effective if you get a better rate quote from another service then report it to your current service, saying, "I'm going to switch unless you can do better on your pricing -- provider X is giving me price Y. I like your service, but not enough to pay an extra $Z for it."
If you just threaten to cancel, well, they hear that every day.
Another risk is they will actually cancel you when you say the words "I want to cancel". Especially if you show up on their screen as a "problem customer". It happens.