You can remove them from your followers list as well, without blocking. Just click next to their name.When you block someone it automatically unfollows. This is what I do to get rid of people who o know that I don't want to follow me, I block them then unblock so it unfollows but they are not blocked