Exercise 5 3.c

Re: Exercise 5 3.c

by Ariane Delrocq -
Number of replies: 0
The elimination of gamma is independent of a=a'.
$\gamma$ only appears in delta_t as a factor of Q(s', a'). When using the semi gradient, Q(s', a') is considered 

fixed and independent of the weights. Therefore when deriving delta_t, \gamma multiplies the derivative of  Q(s', a') which is 0 (shown in the first equation of the answer).  Therefore this term disappear and only the derivative of  Q(s, a) remains.

Note that gamma still implicitly appears in delta_t.