IMPALA¶
-
class
IMPALA(model, sample_batch_steps=None, gamma=None, vf_loss_coeff=None, clip_rho_threshold=None, clip_pg_rho_threshold=None)[source]¶ Bases:
parl.core.fluid.algorithm.Algorithm-
__init__(model, sample_batch_steps=None, gamma=None, vf_loss_coeff=None, clip_rho_threshold=None, clip_pg_rho_threshold=None)[source]¶ IMPALA algorithm
Parameters: - model (parl.Model) – forward network of policy and value
- sample_batch_steps (int) – steps of each environment sampling.
- gamma (float) – discounted factor for reward computation.
- vf_loss_coeff (float) – coefficient of the value function loss.
- clip_rho_threshold (float) – clipping threshold for importance weights (rho).
- clip_pg_rho_threshold (float) – clipping threshold on rho_s in rho_s delta log pi(a|x) (r + gamma v_{s+1} - V(x_s)).
-
learn(obs, actions, behaviour_logits, rewards, dones, learning_rate, entropy_coeff)[source]¶ Parameters: - obs – An float32 tensor of shape ([B] + observation_space). E.g. [B, C, H, W] in atari.
- actions – An int64 tensor of shape [B].
- behaviour_logits – A float32 tensor of shape [B, NUM_ACTIONS].
- rewards – A float32 tensor of shape [B].
- dones – A float32 tensor of shape [B].
- learning_rate – float scalar of learning rate.
- entropy_coeff – float scalar of entropy coefficient.
-