Notes on Decision Theory

Decision Theory

https://plato.stanford.edu/entries/decision-theory/

Context for what questions we might have

Blaise Pascal and Pierre de Fermat had a correspondance that began in 1654 related to formulating a mathematical conception of the expectation (likelihood) of getting a pair of 6’s if a pair of dice is thrown 24.

In 1662, a book called the Port-Royal logic was published that put a narrower measure on what one should and should not do. That you should not only choose the ‘more good’ option but that probability of these options happening should factor into your decision. This can be formalised as expected value.

This idea puts stress on our intuitions about ideas we feel we would make. It doesn’t seem that such a simple heuristic might govern our decision making. For instance, this formalism would say that in the next moment, we should be indifferent between a gamble to win or lose a million dollars with the flip of a coin vs doing nothing.

Daniel Bernoulli introduces this notion of moral value, that is value is based on the agent making the decisions, this is in 1738, so about 60 years later. This puts a value on outcomes. So I would value winning 1 million dollars but much less value losing 1 million dollars, so much so that it tips the balance in favour of maintaining the status quo.

The “Axiomatic Period” has two foundational texts aiming to formalise a general decision making process that we as rational agent use:

1. “Truth and Probability” (Ramsey)
2. “Theory of Games and Behaviour” (John Von Neumann)

In the following rationality is instrumental. An agent has some aim (outcome). A decision is rational if it has the most reason to choose a particular outcome for the agents aim.

we say that a decision is right if and only if its actual outcome is at least as good as that of every other possible outcome.

Decision Space

In order to set up a decision (what to decide about) we need a set that defines the acts, states and outcomes.

Details three levels of abstraction for a decision problem:

1. The decision problem itself.
2. The formalisation of that decision problem.
3. The visualisation of that formalisation.

Formalisations are malleable, in that representation of a decision problem can differ as long as the same information is being captured. Theres an interesting notion of rival formalisms.

• Outcomes: the primary focus of a rational agent is a state of the world where their aim is acheived.
• States: the relevant ‘future worlds’ where some outcome of interest lies. These states should not be causally dependent on your choices.
• Acts: A mapping from state space to outcome space.

Preferences and prospects (outcomes, options). Start from a notion that a preference indicates ‘choice worthiness’ between two options. Preference is always comparative.

We’re concerned with ordering the set which seems to be formalised by a binary relation. Taking in two elements and returning some mathematical object that represents ordering.

preference concerns the comparison of options; it is a relation between options. For a domain of options we speak of an agent’s preference ordering, this being the ordering of options that is generated by the agent’s preference between any two options in that domain.

Ordering operators must defined under:

• Completeness: That for each outcome, there is some relationship (in terms of order) between it all outcomes in the set. $\forall A,B\in S$
• Transitivity The argument for transitivity axiom is the money pump example.

Preference and preference ordering being two distinct things philosophically speaking. Defining a preference relation may not directly equate to some inner process of choice you possess.

Ordinal Scale

Cardinal Scale

• Interesting way of doing this. If you’ve an ordering of options $A\text{l}B\text{l}C$, create a new option L, that is a lottery between to options. If we want to determine how desired B is relative to C. We say that L is a chance to get either A or C and we change the chance of winning C until we determine an equality of preference with B. If there is a low chance to C but you still deem it just as preferable to B then you much prefer C as an option.

• For instance, if you are indifferent between Bangkok and a lottery that provides a very low chance of winning a trip to Cardiff, then you evidently do not regard Bangkok to be much better than Amsterdam, vis-à-vis Cardiff; for you, even a small improvement on Amsterdam, i.e., a lottery with a small chance of Cardiff rather than Amsterdam, is enough to match Bangkok

The orthodox normative decision theory, expected utility (EU) theory, essentially says that, in situations of uncertainty, one should prefer the option with greatest expected desirability or value.

Numerical representations of prospect ordering, utility functions. The idea of lotteries is key to quantifying and comparing preferences.

Decision Making

Formal representation is essential in decision theory. Decision rules only ‘work’ relative to a formalisation.

Ordinal vs Cardinal Scale

In an ordinal scale we just have a set of preference that have no measure of preference, just an ordering. Ordinal scales are mathematically equivalent (in that no set of numbers gives a change in information than the other) if a function transforms the scale but preserves ordering.

In a cardinal scale, I would assume that ordinal transformations would not be cardinal transformations as differences between numbers quantifies preference.

For instance, the scale 4,3,2,1 is that same as 100, 20,10,9 in an ordinal scale but such a transformation converys completely different information in a cardinal scale.

Cardinal scales reflect the differences between measured outcomes.

Preference relation

Lotteries

Lotteries have outcomes as ‘prizes’ that are obtained with a given probability.

What’s the point in preference ordering of acts? Is this to determine value in the decision tree?

It’s a preference ordering of outcomes, not actions.

Savages’ Theory

Savage presents a set of axioms constraining preferences over a set of options that guarantee the existence of a pair of probability and utility functions relative to which the preferences can be represented as maximising expected utility.

The primitives are states and outcomes. States are uncertain, the way the world is or will be. Outcomes are good or bad state of affairs that matter to an agent.

Actions are functions from the state space to the outcome space.

acts are functions from the state space to the outcome space, and the agent’s preference ordering is taken to be defined over all such possible functions.

The act $f(s_i)$ denotes the outcome for a state of the world $s_i$ that is actual. The expected utility of act $f$ is $U(f)={\sum }_{i}u(f(s_i))P(s_i)$