I would break it down like this:
Do I believe I am being manipulated? (Which you considered)
If no, then you just decide whether to participate or not.
If yes, I think you then have to ask why does the person feel he needs to manipulate me?
Then you have a decision on whether to call him (her) on the manipulation. KNowing someone is trying to manipulate you is very poewerful information. Particularly as it relates to the negative connotation--if a person has to manipulate you, it means they feel you are in too strong a position to just go after---