Stochastic games with short-stage duration

Abraham Neyman

We introduce asymptotic analysis of stochastic games with short-stage duration. The play of stage $k$, $k\geq 0$, of a
stochastic game $\Gamma_\delta$ with stage duration $\delta$ is interpreted as the play in time $k\delta\leq
t<(k+1)\delta$, and therefore the average payoff of the $n$-stage play per unit of time is the sum of the payoffs in
the first $n$ stages divided by $n\delta$, and the $\lambda$-discounted present value of a payoff $g$ in stage $k$ is
$\lambda^{k\delta} g$.
We define convergence, strong convergence, and exact convergence of the data of a family $(\Gamma_\delta)_{\delta>0}$
as the stage duration $\delta$ goes to $0$, and study the asymptotic behavior of the value, optimal strategies, and
equilibrium. The asymptotic analogs of the discounted, limiting-average, and uniform equilibrium payoffs are defined.
Convergence implies the existence of an asymptotic discounted equilibrium payoff, strong convergence implies the
existence of an asymptotic limiting-average equilibrium payoff, and exact convergence implies the existence of an
asymptotic uniform equilibrium payoff.

April, 2013