Cha p t e r 3

D e c i si on M a ki ng under U ncert a i n t y

In th e p revio u s l ectu re, w e c o n sid e red d ecisio n p r ob lem s i n w h ic h t h e d ecisio n m a k e r do e s no t k no w t he c o ns e q ue nc e s of hi s c ho i c e s but h e i s g i v e n t h e p ro ba bi l i t y o f e a c h co nse q ue nc e u n d e r e a c h c h o ic e. In m o st ec on om ic a p plica t ion s , s uc h a p r o b a b ilit y i s n o t g i v en . F or exa m p l e, in a g iv en g a m e , a p l a y er cares n o t on ly ab o u t w ha t h e p la ys b u t a lso a b o u t w h a t o t her p la y e r s p l a y . H en ce, th e d escrip tion o f con sequ e n ces incl u d e th e s tra t eg y p ro fi les. In tha t case, i n o rd er to fi t i n t h a t f r a m e w o r k , w e w o u l d n e e d t o giv e oth e r p l a y e rs’ m ixed strategy p r o fi les i n t he description o f t he gam e , m aking G am e T h eo retica l a na lysis m o o t. L i k e w i se in a m ark e t, th e p ri ce is form ed accord ing t o t h e c o l l ec t i v e a c t i o n s o f a l l mark e t pa rt i c i p a n t s , a nd he nc e t he pri c e d i s t r i b ut i o n i s n o t gi v e n.

I n al l t he s e probl e m s , t he de ci s i on m a k e rs ho l d s u b j e c t i v e b e l i ef s a b o ut t h e unkno wn asp ects o f t h e p r ob lem a n d u s e t h ese b e liefs in m a ki n g th eir d ecisio n s. F o r e xa m p le, a p l a y er c h o o ses his strategy a ccor din g to his b eliefs a b ou t w ha t o th er pla y ers m a y pla y , a n d h e m a y r e a c h t h e s e b e l i e f s t h r o u g h a c o m b i n a t i o n o f r e a s o n i n g a n d t h e k n o w l e d g e of pa st b e ha v i o r . T h i s i s c alled d e c ision m a k ing u n d e r unc e rtai nt y .

A s esta b l ish e d b y S a v ag e a n d th e o thers, u n d e r s o m e r easo na ble a ssum p tion s, su c h su b j ecti v e b e li efs c a n b e rep r esen ted b y a pro b a b ilit y d istrib ut io n , in th e s e n se tha t th e d ecision m a k e r fi nd s a n e v e n t m o re lik e ly t h a n an ot he r i f a n d o n ly if th e p rob a b ilit y di st ri but i o n as s i g n s h i g he r p ro ba bi l i t y t o t h e f ormer e v e n t t h a n l a tt e r . I n t ha t c as e , u s ing t he p r ob a b ilit y d is tribu t ion , on e c a n co n v e r t a de cision pro b le m u n d e r u nce r ta in t y to a d e c ision p rob l em un d e r r isk , a n d a pp ly th e a n a ly sis o f t he pr ev iou s lec t ur e. In th is lect ure , I w ill de scrib e t his p rog r am in d e ta il. In p a rticu l ar , I w ill d esc rib e

21

22 CHAP TE R 3 . DECI S I O N M AKI N G U NDE R UNCE R T AI NTY

• th e c on ditio n s s u c h c on sisten t b eliefs im p o se on the p r e feren ces,

• the e l i ci tati on of the b el i e fs f rom the p ref e rences, and

• th e r e p r esen ta tion of th e b eliefs b y a p ro ba bilit y d istrib utio n.

3. 1 A cts, S t ates, C on sequ e n c es, an d E xp ected U ti l - it y R e p r e se n t a t io n

Con s i d e r a fi nite set C o f co ns e q ue nc es . L e t S b e th e set o f all s tates o f t he w o r l d . T a k e a set F of act s f : S → C as th e s et of altern ativ es (i .e. , set X = F ). E a c h sta te s ∈ S d escrib es a l l t h e relev a n t asp ects o f t h e w o rld, hen c e t h e states a r e m u t u a lly exclu s i v e. M o reo v er, t he consequence f ( s ) of act f de p ends o n t he true s t a te o f t he w o rl d. H e nc e , th e d ecisi o n m a k er m a y b e u n c ertain a b ou t t he co n sequ e n ces of his acts. R eca l l tha t th e d ecisio n m a k e r c a r es o n ly a b ou t t h e con sequ e n ces, b u t h e n eeds to c h o o se a n a ct.

E x am pl e 1 (G am e as a D e ci si on P r ob l e m ) C o n s id e r a c o m p l e te i n f o r m a tio n g a m e w ith s e t N = { 1 ,..., n } o f pl a yers in w hich e a ch p l ayer i ∈ N has a s t r a t e g y sp ac e S i . T h e d e c isio n p r o ble m o f a p la ye r i c a n b e d escrib e d as f o l l ow s. Sinc e h e c ar es ab ou t t he str a te g y pr o fi l e s, t h e s e t o f c o ns e q ue nc es i s C = S 1 ×· · · × S n . S i n c e h e d o e s n o t k n o w w hat the o ther p l ayers p l a y, the s et of sta t es is S = S − i ≡ j / = i S j . S i n c e h e c h o o s e s am on g h i s st r a t e gi es, t he set o f a ct s i s F = S i , w h e re ea c h s t ra t e g y s i is r e p r e s en te d a s a fu n c tio n s − i 7 → ( s i , s − i ) . ( H e r e , ( s i , s − i ) is th e s tr a t e g y p r o fi le in w h ich i pl ays s i an d t he others pl ay s − i .) T r a d itio n a l l y, a c o m p l ete - in fo r m a t io n g a m e i s d e fi n e d b y a ls o i n c lu d i n g th e V N M u tility f u n ctio n u i : S 1 ×· · · × S n → R for e ach p l a yer. F i x i n g such a u til i ty f u ncti on i s e q ui val e nt t o fi xi n g t he p r e fer e n c es on al l l ot t e ries on S 1 ×· · · × S n .

W e w o uld lik e t o r epresen t t he decision m a k e r’s p r e feren c e r elatio n º on F by s o m e

U : F → R suc h tha t

U ( f ) ≡ E [ u ◦ f ]

(in t h e sen se o f (O R )) w h e r e u : C → R is a “ ut ilit y f u n c tion ” o n C and E is a n

exp ecta t ion o p e ra tor o n S . T h a t i s , w e w a n t

f º g ⇐⇒ U ( f ) ≡ E [ u ◦ f ] ≥ E [ u ◦ g ] ≡ U ( g ) . (E U R )

3 . 2 . ANS C OM BE - A UMANN M O DE L 23

In th e f or m u lati on of V o n N eu m a n n an d M orgenstern , t h e p r ob ab ili t y distrib u ti on (and he nc e t he exp e ct a t i o n o p e rat o r E ) i s o b j ectiv e ly giv e n. In fa ct, a cts a re for m u l ated a s lott eries, i.e . , p rob a b ilit y d istribu t io ns o n C . I n s uc h a w o rl d, a s w e ha v e s e e n i n t h e last lectu r e, º i s repres e n t a bl e i n t he s e ns e o f ( E U R ) i f a n d o nl y i f i t i s a co n t i n uo us

p r eferen ce r e latio n a n d s a t is fi es t h e I nde p e nde nc e A xi o m .

F o r t he ca ses of ou r c on cern in this lectu re, t here i s n o o b jectiv ely g iv en p r ob ab ili t y di st ri but i o n on S . W e t herefo re need to determ ine t h e d ecisio n m a k e r’s ( sub j ectiv e ) p r ob ab ili t y assessm en t o n S . T hi s i s d one i n t w o i m p o rt an t f orm u l a t i ons . Fi rs t , Sa v a ge ca refully elicits t he b e liefs an d r ep resen t s the m b y a p rob a b ilit y d istrib utio n i n a w o r l d w i th n o o b jec t iv e p ro ba bilit y i s g iv en. S ec on d, A n sco m b e a nd A u m a n n sim p ly u ses ind i ff er en ce b e t w een s o m e l otteries an d a cts t o e licit pr eferen ces. I w i l l fi rst d escri b e A n scom b e a n d A u m a n n ’ s t ra cta b l e m o del , an d t h e n p resen t S a v age’s d eep er a n d m o r e u sef u l an alysis.

3. 2 A ns c o m b e - Aum a nn M o del

A n scom b e a n d A u m a n n c on sider a tr actab l e m o d el in w h ic h t he d ecision m a k e r’s s ub - jectiv e p rob a b ili t y a ssessm en ts a r e d eterm i n e d u sin g h i s a ttitud e s t o w a r ds the l otteries (w ith o b j ectiv e ly giv e n p ro ba b i lities) as w e ll as to w a rd s t h e a c ts w i th u n certa i n c on se - qu en ces. T o d o this, t h e y c on sider t h e d ecision m a k e r’s p r e feren ces o n the set P S of all “a cts” w h ose o u tco m e s a re l o tter i e s o n C , w h e r e P is the set o f a l l l o tter i es (p rob a b ili t y di st ri but i o n s o n C ) . In the l anguage d e fi n e d a b o v e , t h e y a ssum e th at th e c o n sequences an d t h e decision m a k e r’s p ereferences o n t h e set of con sequ e n ces ha v e the s p e cial stru c - t u re of V o n- N e ua m a nn a n d M o r ge ns t e rn mo de l .

In th is set up , i t i s s traig h tfo r w a rd to determ i n e t h e decision m a k e r’s p rob a b ili t y assessm en ts. C o nsid er a s u b set A of S an d a n y t w o c on sequ e n ces x, y ∈ C wi t h x Â

y . C o n s i d e r t h e a c t f A tha t yield s th e s u r e l o ttery o f x on A , 1 an d t he sur e lottery

of y on S \ A . ( Se e F i g ure 3 . 1 . ) U nde r s o m e c o n t i n u i t y a s s um pt i o ns ( w hi c h a r e a l s o n ecessary for r epr esen ta b i lit y), t here exists som e π A ∈ [0 , 1] suc h tha t th e d ecision mak e r i s i ndi ff eren t b et w een f A an d t h e act g A th at alw a ys yield t h e l o tter y p A tha t giv e s x wi t h probabi l i t y π A and y w i th p r ob a b ilit y 1 − π A . T he n, π A is the ( sub j ectiv e )

1 That is, f A ( s )= x whenev er s ∈ A where t he l o ttery x assi gns p robabilit y 1 to the c onsequence x .

f A

x d 1 - x

x

y d 1 - y

y

A S \ A

g A

π A © * © x

© © ©

P A d H ©

H H H 1 − π A

H H j y P A

F i gu re 3.1: F i gu re f or A n scom b e a nd A u m a n n

p r ob ab ili t y the d ecisio n m a k e r a ssi g n s t o t h e ev en t A – u n d er th e a ssu m p tion tha t π A d o es no t d ep end o n w h i c h altern ativ es x and y a r e u s e d . I n t h i s w a y , o n e o b t a i n s a p ro ba bilit y d istrib utio n o n S . U s i ng t h e t he o r y o f V on N e umann a nd M o rge n s t e r n, on e t h e n o b t a i ns a r ep resen tatio n th eo rem i n t h i s e xten d e d s p a ce w h ere w e h a v e b oth su b j ecti v e u n certa in t y a n d o b j ecti v ely g iv en risk.

W h ile t h i s i s a tra c ta ble m o d e l , i t h as t w o m a j or lim ita t ion . F i r s t, th e a n a ly sis generates l i t tl e i nsigh t s i n t o h o w one s houl d t hin k ab o u t t he sub j e c tiv e b e liefs a n d t h e ir rep r esen ta tion thro u g h a pro b ab il it y d istrib uti o n . S eco n d , i n m a n y d ecision p ro blem s th ere m a y no t b e r elev an t i n t ri n s ic ev en ts tha t ha v e o b jecti v ely g iv en pro b a b i lities a nd ric h en o u g h to determ i n e t h e b e li efs o n t h e ev en ts th e d ecision m a k e r i s u n certa in a b ou t.

3. 3 S a v age M o d el

Sa v a g e de v e l o ps a t he o r y w i t h p urel y s ub j e c t i v e u nc e r t a i n t y . W i t hout us i n g a n y o b j e c - tiv e ly giv e n p ro b a b i lities, u n d e r c erta in a ssu m p tion s o f “ tigh tn ess” , h e d eriv es a u niqu e p r ob ab ili t y d i stribu tion on S th at rep r esen t t h e decision m a k e r’s b el iefs em b e dd ed i n hi s p ref e rence s , and t he n u s i ng t h e t he ory o f V o n N e umann a nd M o rge n s t e r n h e o bt ai n a r e p re sen ta tion t h e o rem – in w h ic h b o t h u tilit y fu nc tion a n d t h e b e liefs a r e d e r iv ed fr o m t h e p r e fer e n c es.

Ta k e a s e t S of st ates s of th e w orld, a fi nite set C of c o nse q uenc es ( x, y , z ), a n d t ak e th e set F = C S of act s f : S → C as th e s et of altern ati v es. F i x a relation º on F . W e w o u l d lik e t o fi n d necessa ry a n d s u ffi c i e n t c o n d i t i o n s o n º so th at º ca n b e r ep resen t ed by s o m e U in the sen se of (E U R ); i.e., U ( f ) = E [ u ◦ f ] . I n t h i s r epr esen ta tion , b o t h t h e

ut i l i t y f unc t i o n u : C → R an d t h e p r ob a b ilit y d istrib ut io n p on S ( w hi c h de t e rmi n e s

E ) a re deriv e d f rom º . T h e or em s 2 an d 3 giv e u s th e fi r s t n ecessa ry co n d ition :

P 1 º is a p r e fe r e n c e r ela tio n .

T h e s econd c ond i tion is the cen tral p i ece o f S a v age’s t heory:

3. 3 . 1 T he Sure- t hi ng P r i n c i pl e

T h e S u r e - th in g P r i n c ip le If a d e c ision m a ker p r efers s om e a ct f to so m e a c t g

w h en he kn ow s t hat s om e e vent A ⊂ S o c curs, a n d if he p r efers f to g w h en he kn ow s

th a t A do es not o c c ur, t hen h e m ust p r e fer f to g w hen he d o es no t k now w hether A

occ u r s o r n o t .

T h is is the i nform a l sta tem e n t o f the s u r e-th ing p rinci p le. O n c e w e d eterm i ne th e d ecision m a k er’s p ro ba b i lit y a ssessm en ts, th e s u r e-thin g p rin ciple w ill giv e u s the I n d e - p e nd en ce A x iom , A x iom 4, of V o n N e um an n a nd M o rgenstern . T h e f o l lo w i n g form u l a - t i on of Sa v a ge , P 2, not o nl y i m p l i e s t hi s i nf orm a l s t a t e m e n t , b ut al s o al l o ws us t o s t at e i t fo rm a ll y , b y a llo w in g u s t o d e fi n e con d i ti o n a l pr eferen ces. (T h e co n d ition a l p references a r e a l s o u s e d t o d e fi ne t h e b eliefs.)

P 2 Le t f, f 0 , g , g 0 ∈ F an d B ⊂ S be s u c h t h a t

f ( s ) = f 0 ( s ) and g ( s ) = g 0 ( s ) at e a c h s ∈ B

an d

f ( s ) = g ( s ) an d f 0 ( s ) = g 0 ( s ) at e a c h s ∈ / B.

If f ≥ g , t h e n f 0 ≥ g 0 .

3. 3. 2 C ondi ti onal pref erences

Us i n g P 2 , w e c a n d e fi ne t h e c ondi t i o n a l pref e r ence s a s f o l l o ws . G i v e n an y f, g , h ∈ F

an d B ⊂ S , d e fi ne acts f h an d g h by

an d

h ( s ) = f ( s ) if s ∈ B

| B h ( s ) ot herwi s e

h ( s ) = g ( s ) if s ∈ B .

| B h ( s ) oth e rw i s e

Tha t i s , f h

an d g h

agree w i t h f an d g , r e s p e c t i v e l y , o n B , but whe n B do es n o t o ccu r,

th ey yield t h e sam e d e fa u l t a ct h .

De fi ni t i on 6 (C ondi ti onal P r ef er enc e s ) f ≥ g gi v e n B i ff f h ≥ g h .

P 2 guaran t e e s t h at f ≥ g giv en B is w e ll-d e fi ne d, i . e . , i t do e s no t d e p end o n t he de f a ul t a ct h . T o see th is, t ak e a n y h 0 ∈ F , a n d d e fi ne f h 0 and g h 0 a cco rdin gl y . C h ec k

th at

| B | B

f h ( s ) ≡ f ( s ) ≡ f h 0 ( s ) an d g h ( s ) ≡ g ( s ) ≡ g h 0 ( s ) at e a c h s ∈ B

| B | B | B | B

an d

0 0

f h ( s ) ≡ h ( s ) ≡ g h ( s ) an d f h ( s ) ≡ h 0 ( s ) ≡ g h ( s ) at eac h s ∈ / B.

| B | B | B | B

T h erefor e, b y P 2 , f h ≥ g h i ff f h 0 ≥ g h 0 .

| B | B | B | B

N o te tha t P 2 p r ecisely sta tes t ha t f ≥ g gi v e n B is w e ll-d e fi ned. T o see this, t ak e f

an d g 0 arbi t r ari l y . S e t h = f an d h 0 = g 0 . C lea r l y , f = f h an d g 0 = g h 0 . M o r e o v e r , t h e

co nd ition s in P 2 de fi ne f 0 an d g as f 0 = f h 0 an d g = g h . T h u s , t h e c o n c l u s i o n o f P 2 ,

| B | B

“if f ≥ g , t h e n f 0 ≥ g 0 ” , i s t h e s a m e a s “ i f f h ≥ g h , t h e n f h 0 ≥ g h 0 .

| B | B | B | B

Exe r c i s e 2 S h o w th a t th e i n f o r m a l s ta te m e n t o f th e s u r e- th in g p r i n c ip le is fo r m a l ly tr u e : g ive n a n y f 1 , f 2 ∈ F , a n d a n y B ⊆ S ,

[( f 1 ≥ f 2 given B ) an d ( f 1 ≥ f 2 given S \ B )] ⇒ [ f 1 ≥ f 2 ] .

[H in t: d e fi ne f := f 1 = f f 1

f 1

1 | S \ B

, g 0 := f 2 = f f 2

f 2

2 | S \ B

, f 0 := f f 2

f 1

2 | S \ B

, a n d

g := f f 1

f 2

1 | S \ B

. N oti c e t hat y ou do not n e e d t o i n voke P 2 (expl i ci tl y).]

Nu l l E v e n t s I m a g i n e t ha t t he de c i s i o n mak e r r emai ns i n di ff erence to w a rds a n y c h anges made t o an ac t i on wi t h i n an ev en t B . N am el y , fo r a n y acts f an d g , t h e d ecisio n m a k e r rem a ins i ndi ff er en t b et w een f an d g , s o l o n g a s f and g are i d e n t ical on S \ B , n o m a t t e r ho w w i d e l y d i ff er o n B . I n t h a t c a se, it is p l au sible t o d edu c e t ha t t he decision m a k e r do e s no t t hi nk t h at e v e n t B ob ta i n s. S u c h e v e n t s a re ca lled nul l .

De fi ni t i on 7 A n even t B is sa id to b e nu l l if a n d o n l y i f f ∼ g given B fo r a l l f, g ∈ F .

R eca ll th at ou r a im is to d e v e lop a theo ry tha t relates t h e pr eferen ces on the a cts

w i th u n certa i n c o n sequences t o t he p r eferen ces on the c o n sequ e n ces. (T h e p r eference relation ≥ on F i s extended to C by e m b e d d i n g C in to F a s con s ta n t acts. T h a t i s, we s a y x ≥ x 0 i ff f ≥ f 0 wh er e f an d f 0 a r e c o n stan t a cts th at tak e v a lu es x an d x 0 , resp ecti v ely . ) T he next p o stula t e d o e s t h i s f o r con d i ti o n al p r eferen ces: