内容

目次

Part 1 Main ideas: decision processes; bandit processes and simplefamilies of alternative bandit processes; a first index theorem; jobs; theindex theorem for jobs with no pre-emption; knapsacks; different discountfunctions; stochastic discounting; ongoing bandit processes; multipleprocesses. Part 2 Central theory: a necessary condition for an index;splicing bandit process portions; equivalent constant reward rates andforwards induction for arbitrary decision processes; more splicing and proofof the index theorem for a SFABP; near optimality of nearly index policies,and the gamma - O limit; bandit superprocesses and simple families ofalternative superprocesses; the index theorem for superprocesses; stoppablebandit processes; the index theorem for a FABP with precedence constraints;precedence constraints forming an out-tree; FABPs with arrivals; minimum EWFTfor the M/G/1 queue. Part 3 General properties of the indices: dependence ondiscount parameter; monotone indices; monotone jobs. Part 4 Jobs withcontinuously-varying effort allocations: competing research projects;continuous-time jobs; optimal policies for queues of jobs. Part 5Multi-population random sampling (theory): jobs and targets; use ofmonotonicity properties; general methods of calculation - use of invarianceproperties; random sampling times; Brownian reward process; asymptoticallynormal reward processes. Part 6 Multi-population random sampling(calculations): normal reward process (known variance); normal reward process(mean and variance both unknown); Bernoulli reward process; exponentialreward process; exponential target process; Bernoulli/exponential targetprocess. Part 7 Search theory: a discrete search problem; two-person zero-sumgames; a game of hide and seek; hide and seek in continuous time. Part 8 Inconclusion: the Whittle index theorem; permutation schedules andsub-optimality; more about the Brownian reward process; more proofs,generalizations and extensions.