I am working in a reinforcement learning program and I am using this article as the reference. I am using python with keras(theano) for creating neural network and the pseudo code I am using for this program is
Do a feedforward pass for the current state s to get predicted Q-values for all actions.Do a feedforward pass for the next state s’ and calculate maximum overall network outputs max a’ Q(s’, a’).Set Q-value target for action to r + γmax a’ Q(s’, a’) (use the max calculated in step 2). For all other actions, set the Q-value target to the same as originally returned from step 1, making the error 0 for those outputs.Update the weights using backpropagation.
The loss function equation here is this
where my reward is +1, maxQ(s',a') =0.8375 and Q(s,a)=0.6892
My L would be 1/2*(1+0.8375-0.6892)^2=0.659296445
Now how should I update my model neural network weights using the above loss function value if my model structure is this
model = Sequential()model.add(Dense(150, input_dim=150))model.add(Dense(10))model.add(Dense(1,activation='sigmoid'))model.compile(loss='mse', optimizer='adam')