Trjtdtk

(Soft question) does light intensity oscillate really fast since it is a wave?

Typesetting a double Over Dot on top of a symbol

Filling an area between two curves

Extreme, but not acceptable situation and I can't start the work tomorrow morning

How to manage monthly salary

How could a lack of term limits lead to a "dictatorship?"

Can Pesach Mitzvot be performed before Tzet Kochavim?

What causes the sudden spool-up sound from an F-16 when enabling afterburner?

How to move the player while also allowing forces to affect it

Shall I use personal or official e-mail account when registering to external websites for work purpose?

Does it makes sense to buy a new cycle to learn riding?

Landlord wants to switch my lease to a "Land contract" to "get back at the city"

I see my dog run

Uplifted animals have parts of their "brain" in various locations of their body. Where?

Map list to bin numbers

If a centaur druid Wild Shapes into a Giant Elk, do their Charge features stack?

Prime joint compound before latex paint?

Is there a familial term for apples and pears?

What is GPS' 19 year rollover and does it present a cybersecurity issue?

aging parents with no investments

Should the British be getting ready for a no-deal Brexit?

What is the command to reset a PC without deleting any files

Was there ever an axiom rendered a theorem?

Are white and non-white police officers equally likely to kill black suspects?

Is reward accumulated during a play iteration when performing SARSA?

0

$begingroup$

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going around in circles. I have not tried exploring more states yet because I am confused about how to properly assign reward.

Currently, I am using a 2D-Gaussian distribution to assign reward where $f(x=x_food,y=y_food) = 1$. Terminal states like hitting the wall or itself result in a reward value of -1.

My reason for using the Gaussian was because of the relatively sparse rewards in this game and the ability to easily clip rewards between [1,-1] in meaning full way.

I have two questions.

1. Is this an appropriate way to define the reward function?


2. Currently during training, I do no accumulate the reward during each play iteration. So each reward for transitions are independent of the reward values before it. Right now I am doing $big[S_1,A_1,R_1,S_2,A_2,R_2,S_3big]$. I've looked at other code where people have accumulate the reward like $big[S_1,A_1,R_1,S_2,A_2,R_2 = R_1+r,S_3big]$. Where $r$ is given by the reward function. The thing is, I can't find a paper that defines why you should do this. So my question is, which way is the appropriate way to assign reward?

machine-learning deep-learning reinforcement-learning q-learning dqn

share|improve this question

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

$endgroup$

add a comment | 

0

$begingroup$

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going around in circles. I have not tried exploring more states yet because I am confused about how to properly assign reward.

Currently, I am using a 2D-Gaussian distribution to assign reward where $f(x=x_food,y=y_food) = 1$. Terminal states like hitting the wall or itself result in a reward value of -1.

My reason for using the Gaussian was because of the relatively sparse rewards in this game and the ability to easily clip rewards between [1,-1] in meaning full way.

I have two questions.

1. Is this an appropriate way to define the reward function?


2. Currently during training, I do no accumulate the reward during each play iteration. So each reward for transitions are independent of the reward values before it. Right now I am doing $big[S_1,A_1,R_1,S_2,A_2,R_2,S_3big]$. I've looked at other code where people have accumulate the reward like $big[S_1,A_1,R_1,S_2,A_2,R_2 = R_1+r,S_3big]$. Where $r$ is given by the reward function. The thing is, I can't find a paper that defines why you should do this. So my question is, which way is the appropriate way to assign reward?

machine-learning deep-learning reinforcement-learning q-learning dqn

share|improve this question

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

$endgroup$

add a comment | 

$begingroup$

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going around in circles. I have not tried exploring more states yet because I am confused about how to properly assign reward.

Currently, I am using a 2D-Gaussian distribution to assign reward where $f(x=x_food,y=y_food) = 1$. Terminal states like hitting the wall or itself result in a reward value of -1.

My reason for using the Gaussian was because of the relatively sparse rewards in this game and the ability to easily clip rewards between [1,-1] in meaning full way.

I have two questions.

1. Is this an appropriate way to define the reward function?


2. Currently during training, I do no accumulate the reward during each play iteration. So each reward for transitions are independent of the reward values before it. Right now I am doing $big[S_1,A_1,R_1,S_2,A_2,R_2,S_3big]$. I've looked at other code where people have accumulate the reward like $big[S_1,A_1,R_1,S_2,A_2,R_2 = R_1+r,S_3big]$. Where $r$ is given by the reward function. The thing is, I can't find a paper that defines why you should do this. So my question is, which way is the appropriate way to assign reward?

machine-learning deep-learning reinforcement-learning q-learning dqn

share|improve this question

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

$endgroup$

share|improve this question

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

share|improve this question

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164

asked Mar 29 at 1:44

DevarakondaVDevarakondaV

164