Evolutionary biologist here.

This is actually a tricky one. Lying (and I’m going to fold the projection of false confidence in with that one because I’m talking about deception, intentional or otherwise, not a moral concept) is only effective if others believe you.

Humans, as the most highly social of the primates and ranking among the most highly social animals on earth, have adapted to believe each other, because this helps with trust, coordination, shared identity, learning, and so on. However, it also creates a vulnerability to manipulation by dishonest actors. Again, I’m not talking about a moral dimension here. There are species in which mating is initiated with the gift of a nuptial present (eg a dead bug) from the male to the female. Sometimes the male will give a fake present (already desiccated insect, eg) to trick the female, and sometimes it works. Deception and detection are an arms race, and it’s believed by many to be one of the drivers that lead to the development of human intelligence, where our information processing capacity developed alongside our increasing social complexity.

The problem is that when lying becomes the default, then the beneficial effects of communication cease. It’s like when you stop playing games with a kid that just cheats every time, or stop buying from a store that just rips people off. It’s a strategy that only works if few enough people play it. There’s tons of caveats and additional variables, but that’s the baseline. So why do we still see so much of it?

The first component of course is confirmation bias. If 90% of our interactions are trustworthy, the ones that stick out will be the deceptions, and the biggest deceptions will get the most notice. The second is that the deceptions as a whole have not been impactful enough, over time, to overcome the advantages of trust, either in biological time or in social evolutionary time. You will notice that more trust is given to in-group rather than out-group members, and a number of researchers think that has to do with larger social adaptations, such as collective punishment of deceivers - sending someone to jail for writing bad checks, say, is easier if they’re part of your community as opposed to a tourist from another country. We can also see cultural differences in levels of trust accorded in-group and out-group persons, but that’s getting into a lot of detail.

The third major operator is the concept of the self. This is a subject where we are just being able to start making scientific headway - understanding where the concept of a self comes from in terms of neurobiology and evolutionary dynamics - but this is still very much a new science layered on top of ancient philosophy. In the concept of the self there is a component of what I’m going to calll the physical integrity of the structure. People find being wrong painful - there are social situations that activate the same parts of the physical brain as physical pain and distress do. This is especially true of those ideas are seen as being held by other group members, because you now have the group structural integrity on top of the one in idea-space. That’s where you get people willing to literally die on the hill of Trump winning in 2020. For the evolutionary construction and nature of the self I’d recommend Sapolsky and Metzinger - it’s too new and too complex to get into here. If you want to just summarize it in your mind, call this component ego defense.

I think that’s most of what’s going on, at least as we understand it so far.