Aren't AI existential risk concerns just an example of Pascal's mugging?

Pascal’s mugging gets its name from Pascal's wager, a classic philosophical argument that one should believe in God regardless of how likely it is that God exists, because the rewards for believing, if God does turn out to exist, are infinite. The mugging can be seen as a finite version of the wager and aims to show that an unbounded expected utility (EU) maximizer is vulnerable to manipulation by a “Pascal’s mugger” who promises enormous rewards in exchange for e.g. a wallet. It may be very unlikely that the mugger will deliver, but our formalisms for calculating probabilities say the utility of the rewards can go up faster than their probability goes down—promising 10^100 times as much utility makes the offer less credible, but not 10^100 times less credible. That means the mugger’s offer seems worthwhile to the EU maximizer, despite being silly from a common sense perspective.

Some claim that taking actions to reduce AI existential risk is like being mugged in this way, because the risks are unproven and somewhat speculative, but potentially enormous in scope, threatening not only the death of eight billion people in the present, but also "astronomical waste", i.e. destroying all the value that could eventually be created using galaxies worth of resources.

However, calling the arguments for reducing AI existential risk a Pascal’s mugging mischaracterizes them. They generally don’t say that we should ignore the improbability of disaster because the cost of disaster would be so high. Rather, they say that disaster is actually quite plausible, and maybe even probable.^[1] Humanity being wiped out by a new technology that is smarter than us and that we’re not sure how to control is not the kind of event we should be extremely surprised by (in the way that we'd be extremely surprised if a Pascal's mugger's claims turned out to be true).