Draggin the clip for RETIME: sometimes it works, sometimes not. Why?

Hello there,

I love to retime clips and I don't like to type it down percentage, like 1% or 1000%, whatever. 
I like to drag it. It's so much more organic and of course, you get to see/visualize beforehand how 'physically' the clip is related to other elements.

Anyway, here's my question: sometimes the draggin simply doesn't work as it's supposed to be. Sometimes I can drag it from 100% to lets's say 2000% with a simply drag, and sometimes I keep trying to drag it, but it gets kind of stuck and it goes only about 10% each time I drag it.

Is there something I'm missing here? In the way I'm doing it? 

Thanks in advance!