It's pretty normal for a belt tensioner to move a bit, but you could have a pulley out of alignment somewhere.