I claim that resolving N/A is bad, that it's usually worth the effort to untangle ambiguities and hash out what's fairest. That can be exhausting, invidious work and it's understandable if you're not up for it, but at least consider it a last resort to resolve N/A. And if traders are lobbying for an N/A resolution, that might be all the more reason not to do it!
But this is easier said than done. Sometimes you pose an innocent-seeming question and traders make different assumptions and bet accordingly. Later, the unanticipated ambiguities come to light and all possible resolutions feel grievously unfair to one set of traders or another. And it's possible that N/A is indeed what's least unfair. But here's a collection of reasons to resist it:
1. It's anticlimactic and throws away people's work
This one seems obvious and relatively minor. Resolving N/A unwinds everyone's trades as if they never happened. Less obviously, that includes clawing back profits of those who correctly predicted a price movement and then cashed out. This invariably surprises people.
2. It spoils incentives
This is kind of a corollary to reason #1 but is also a counterargument to a pro-N/A argument. Namely, that the real value of a prediction market is to see the market probability evolve. The final resolution, or lack thereof, doesn't undo that value. That's true but resolving N/A sets a bad precedent, making people less motivated to do the work to make market probabilities accurate in the first place if they think N/A is at all likely.
3. It can be a way to cheat
Here's an extreme example to illustrate how bad this can be. Imagine creating a "Will aliens arrive by Christmas?" market. You dump all your money on YES. If aliens show up, resolve to YES, win big. If aliens don't show up, you start stirring up fairness concerns. It wasn't specified whether you meant extraterrestrial aliens. Etc. Get the debate going and then suggest that maybe the fairest thing is to resolve N/A. Everyone gets their money back, no harm no foul. But it was totally rigged! You made a big bet on YES and then found a way to weasel out of paying up when you were proven wrong.
It's not normally so blatant but it's worth having that in mind. "Just give everyone their money back, no harm no foul" can be extremely incorrect. Totally harm, totally a foul. Be very skeptical of anyone suggesting that resolving N/A is what's fairest.
Rule of thumb: if the ambiguities that arose were no more likely to arise for any of the ways the underlying question might've played out, then N/A may be what's least unfair. It can be a judgment call.
4. It can distort market probabilities in decision markets
Say you want to know whether, if you buy Apple's VR headset, you'll actually use it enough that you'll be happy you bought it. So you make a market asking exactly that. You might naturally commit to resolving N/A if the market convinces you you'd regret the purchase. No purchase, no way to resolve the market. The problem is that this can spoil the incentives for traders and leave the market quite biased towards YES.
Imagine I'm pretty sure you'd regret getting the VR headset. If I drive the price to my true estimate of, say, 10% that you'll like it, my upside is cut off. If I'm wrong and the price goes back up and you buy the thing and have no regrets, I lose money. If I'm right then you defer to the wisdom of the market, don't buy the thing, and resolve N/A. I'd be paying up if I'm wrong and getting no payout if I'm right. Thus I won't drive the price down enough to convince you not to do the thing. I might not bet NO at all, in case others drive the price down that far.
What should you do instead? Maybe resolve to the market price in the case that you don't buy the headset. That does risk market manipulation. Driving the price to 1% is guaranteed profit if I think I have enough money to keep the price there. At that price I know you won't buy the headset and thus you'll resolve to the market's estimate of a 1% chance of headset happiness. Ka-ching. But my own meta-prediction is that that's too risky to try to exploit in practice. Especially if you don't commit to a particular threshold for doing vs not doing the thing. Any would-be manipulator has to worry that you might buy the headset despite the market seeming 99% sure you'll regret it. If you do, and don't in fact regret it, the manipulators lose their shirts.
5. Ambiguous outcomes are often better resolved to a probability
Did the thing being predicted kinda/partially/ambiguously happen? You don't have to endlessly debate YES vs NO and give up and resolve N/A when neither side can claim a decisive victory. Instead the creator can make a judgment call like "this counts as 75% YES" and just resolve to that. Resolve-to-PROB, it's called. It may seem weird but is pretty exquisitely fair if that 75% is the best reflection of the weird reality that ended up happening.
Exceptions
A. Questions that are fundamentally nonpredictive, like "Is Trump good?".
B. When the resolution criteria explicitly stipulate an N/A resolution if such-and-such happens. Like "Will Biden beat Trump? Resolves N/A if either of them dies before the election."
C. When an error was made in specifying the resolution criteria -- such as changing the criteria after some people already bet -- such that any possible resolution would be truly unfair to some subset of traders.
(I originally said some of this in a "will DALL-E be able to draw blue grass and a green sky?" market and a "will I regret bleaching my hair?" market. It was in the latter that user @Conflux pointed out the potential for market distortion in a "will I regret XYZ?" market. For more on trickinesses with decision markets, see Prediction Market Does Not Imply Causation.)
1,000
3.00reposting for reach: NA poll with a thoughtful writeup from @dreev on reasons it's best to resist the option
There's another behaviour that the NA option can facilitate, which is a variation of #3
Let's say that someone makes a market of medium complexity, and doesn't define the resolution conditions particularly well. But they certainly know what they intend to do for the majority foreseeable cases.
People bet and the market swings in a certain direction. Then a new bettor comes along, and realises there is an ambiguity. Instead of asking a clarifying question, they just bet the higher reward position, having already decided that if the creator 'rules' against them they will demand an N/A.
I think there is a general problem on Manifold where if a question seems poorly defined, and your good faith interpretation of what is asking suggests it is very incorrectly priced, the sensible thing to do is to ask the creator what the intention was, because it looks like you might be wrong.
However, if you ask, and the creator says "No no, your interpretation is correct", the chance that you will be the first one to see that information is probably not very high, and then someone else benefits from your insight and takes all your EV.
@JoshuaWilkes my solution is to make an initial bet on both the question and the probability that my interpretation is correct, and just take the hit like a manifolder when I'm wrong
Beautifully said, and all the more reason that if a trader is "demanding" N/A, that's a strong signal that it shouldn't resolve N/A. And I strongly agree that the Manifold norms are caveat emptor. Ask clarifying questions before trading!
Arguing ex post facto that the resolution criteria were ambiguous is no good.
/agentydragon/will-google-search-include-a-chatbo
I don't recall the exact details, there was a wording ambiguity involving by/at. Either YES or NO might have been correct. The question was still live and relevant. In situations like this, I generally expect people to ask for N/A but not get it.
/HelenD/will-miami-marlins-beat-new-york-me
The game in question was postponed due to weather and played the following day. Neither YES or NO seem correct. Circumstances had made the question irrelevant and moot. I generally expect N/A in these cases.
I think of mootness as a key factor in whether N/A will be used, but every situation is different.
@travis I would expect the 1st example to resolve N/A (ok, not expect, let's say I'd strongly prefer N/A). But I definitely wouldn't expect or demand N/A in the 2nd example. Except in very particular circumstances, like if those teams played for the "master cup" on the 26th and those same teams played for the "super cup" on the 27th and due to bad weather they switched the dates.
@Primer In the 2nd example, I would say leave it open if it were a football game, but for baseball a 1 day delay might cause the starting pitchers to change, so N/A seems better. I think the sports questions are easier to think about as part of a long term pattern of markets. Like people don't want to think about the baseball markets as being part weather forecast.
Here was one I thought should have been N/A: /DaltonImhoffBrey/will-forte-win-the-2023-kentucky-de
The horse in question was scratched the morning of the race. I think most people would be interpreting that question as "Will Forte win the race, conditional on running in it?" So the N/A resolution can be used to align the market to the expected interpretation.
@travis I disagree in the horse example, that should be a clear No.
Even if the Derby was canceled in 2023, I'd expect some traders go "Did Forte win? No. So this resolves No." and it would be interesting how @dreev would recommend to resolve those hypotheticals? Let's call this one #1.
Another interesting case to consider (#2):
Will Forte win his next race, the 2023 Kentucky Derby?
If Forte doesn't start in the Kentucky Derby, but wins the next race he starts in the next day? Suppose there are no additional criteria.
What about a slight variation (#3):
Will Forte win his next race (Kentucky Derby)?
Again, suppose there are no additional criteria.
What about (#4):
Will Forte win his next race?
and the description says
The next race is the 2023 Kentucky Derby.
Again, suppose Forte doesn't start in Kentucky, but wins his next race.
What if the description instead said (#5)
The next scheduled race is the 2023 Kentucky Derby.
?
@Primer I don't want to answer the hypos because I think it depends on a broader context. In horserace betting, N/A (voiding) is the common practice. On Manifold, the creator can make their own rules but that's part of the context. If the market is created far in advance, then it is clear that the market is about whether the horse will start and win, so N/A shouldn't be used. "Will this horse run and win?" and "Will this horse win conditional on running?" are both valid questions.
Market creators have a wide variety of goals when they create a market. It seems to me there is a technical way that N/A should be used that will align the market with the goals of the creator. Using N/A when it shouldn't be used or not using it when it should be can cause a distortion in the market. If the creator can communicate their goals, then the traders can anticipate how N/A should be used.
Here's another funny example from the other day:
/ReeMARKable/nba-basketball-games-live-20242025
The sub-market was "Thursday 1/9, 7:00 PM: (WSH) Wizards WIN at (DET) Pistons"
The problem is, it was the Warriors not Wizards that played at Detroit (and won) that day. The typo was obvious enough from the context that the traders bid the probability up and it resolved YES without dispute.
In horserace betting, N/A (voiding) is the common practice
Now I can understand why you'd expect another resolution. 90% of my horse betting knowledge comes from having watched Lucky Number Slevin. Thanks for the education!
@Primer Yeah, and you could ask why is it common practice? Horse betting exists for the entertainment of the bettors and bettors tend to be interested in question like "Which of these horses is the fastest?" and less interested in whether a horse might be removed from competition for some arbitrary reason. So the horse betting folks are using N/A (void) to align the market mechanism with its purpose.
Here's an example that reversed the usual order of these disputes: /getby/2024-us-presidential-election-winne
The original title was just "Who will win the election" or something and the creator was inactive. After Biden dropped out, people realized there was a significant chance it might resolve N/A because it was dependent multichoice so 1 choice had to be selected and Kamala couldn't be added.
Normally in an N/A dispute people try to figure out if the title and description imply that N/A should be used, but here N/A was potentially unavoidable and so the mods were trying to figure out how to change the title. Well, how to change the title depends on how you plan to use N/A. They eventually decided to N/A only if Kamala won and changed the title to reflect that. The other suggestion was to N/A no matter what because the market had confused hundreds of traders. That would imply that the proper title of the market all along might have been something like "Which of these people will win the election, conditional on both major party nominees being on the list?"
PS: I rearranged everything in the market description to be just a list of reasons not to resolve N/A, with the arguments in favor discussed along the way.
I'm still thinking about how to articulate a generalization of reasons 3 and 4. Something about needing the probability of N/A to be the same in all possible worlds, lest it be unfair and/or distortionary.
Say you want to know whether you'll regret it if you buy Apple's VR headset. So you make a market asking exactly that. You might naturally commit to resolving N/A if the market convinces you you'd regret the purchase. No purchase, no way to resolve the market.
In this case you might make a pair of markets, "Will I regret purchasing Apple's AR* headset" and "Will I regret not purchasing Apple's AR headset". You make a decision based on the spread between them and N/A the counterfactual after you choose. Either way, I think N/A is essential for conditional markets, and that's the only way I intentionally use it.
@Sailfish Hmm, but is it possible to regret not buying it? As soon as you started to have such regret, you'd go ahead and buy it. I'm not sure the additional market is accomplishing anything in this case.
Related to decision markets: https://manifold.markets/dreev/how-many-instances-of-prognootling
(Btw, I flipped the example in the description to be predicting happiness rather than regret. I kept confusing myself with the double negatives there.)
but is it possible to regret not buying it? As soon as you started to have such regret, you'd go ahead and buy it.
Then the regret market would resolve "Yes" and the condition for the "Regret purchasing" market would be triggered. I assume there's a resolution time. For instance, I say I want to know if I should purchase the headset in seven days. I make the pair of markets, and they both resolve in say two months. "Regret buying" is at 70%, "Regret not buying" is at 30%, and I decide not to buy it. Two weeks pass, and I feel like I do actually want to buy it, maybe I try it out somewhere and like it a lot or something. If I do end up buying it, say, a month later, I would have liked to have this information earlier and so I "Regret not buying" it. I agree the wording is kind of weird, at least, it's not a common way to use "Regret" but I think it's correct.
Maybe it's more intuitive to think that never buying anything isn't some secret strategy to optimize happiness. There are some cases where you'll think that you actually did want something, and would prefer to be in the counterfactual world where you did in fact buy it at some previous time.
I guess the key in my VR headset example is that we're more likely to get an N/A resolution in the universe where the VR headset sucks. The headset sucks => the market knows that => you listen to the market and don't buy it => you resolve N/A.
Similarly for the "will aliens land by Christmas?" scam. The market is more likely to resolve N/A in the universe with no aliens.
So the general principle may be that if you resolve N/A you should be certain that your probability of doing so is consistent across all universes that the market is predicting among. Something like that?
Easier said than done though. Counterfactual reasoning can be confusing.
If I have to vote, I will vote for "preaching to the choir", but, in defense of N/A:
On some markets I create, I know there is a low-odds thing that could reasonably happen (more often than 1 in 10,000 but less than 1 in 20), but I do not expect it to happen. Whether or not the low-odds thing happens is not valuable to me and generally not related to the important part of the question I am asking. Due to the structure of the market, it is difficult to price in the low-odds thing, so I will sometimes explicitly declare what the market will do if the low-odds thing happens, and it is often N/A.
Especially in cases where the odds of the low-odds thing are hard to determine, I'm usually much, much more interested in A vs B, than A vs B vs Low-odds-thing. The multiple choice question format sometimes allows explicitly factoring this in, but in my experience, users have a very difficult time pricing low-odds-thing because the site does not seem to want us to be able to price low odds events.
---
For cases where unforeseen events create ambiguities that could result in N/A, I think it is important to make sure traders understand they are trading in a situation of extreme ambiguity. If I know I'm trading on thin ice, I will have a lot less reason to feel betrayed if the market does not turn out the way I think it should. I think it would be good for the platform to support users by allowing creators, moderators, or users to post a warning flag so users can review the situation carefully before proceeding to bet on markets that have reached this state.
@Eliza Ah, this sounds like an important caveat that may be worth incorporating into my little essay here. If you pre-commit to a specific condition under which you'll resolve N/A and that won't bias the market probability (see the description here for an example of when it does! I have to think about how to characterize in fully generality when it does or doesn't) then resolving N/A is fine.
@TheoSpears Interesting! I haven't thought this through yet; are you thinking that that mitigates some of the CONs here? It at least offers traders a way to hedge. Maybe you didn't mean this as a serious solution, and my guess is it doesn't help too much. But it can't hurt!
@dreev It seems like a possible strategy some traders could use to mitigate the impact of an N/A resolution if it seems likely.