People interact differently with humans than they do with computers, but there is minimal research on what brings about these differences. Using agents labeled as either “another participant” or a “computer program”, we investigated the differences in people’s behavior and brain activity during the course of a negotiation paradigm. Our results indicate that people perceive human-labeled agents more human-like than computer-labeled agents, and the level of concession in the negotiations is dependent on agent type. We have also found that these differences can be captured in brain activation by showing that different brain patterns are observed in parts of the Theory of Mind neural correlates. We further demonstrate that brain activity can predict whether the negotiation agent was introduced as a competing human player or a computer program. Overall, our study suggests that different labeling of an interaction partner leads to significant impacts on one’s decision making.