parl.Model

class Model(name_scope=None, dtype='float32')

alias: parl.Model

alias: parl.core.paddle.agent.Model

Model is a base class of PARL for the neural network.

A Model is usually a policy or Q-value function, which predicts an action or an estimate according to the environmental observation.

To use the PaddlePaddle2.0 backend model, user needs to call

super(Model, self).__init__() at the beginning of __init__ function.

Model supports duplicating a Model instance in a pythonic way:

copied_model = copy.deepcopy(model)

Example:

import parl
import paddle.nn as nn

class Policy(parl.Model):
    def __init__(self):
        super(Policy, self).__init__()
        self.fc = nn.Linear(input_dim=100, output_dim=32)

    def policy(self, obs):
        out = self.fc(obs)
        return out

policy = Policy()
copied_policy = copy.deepcopy(policy)

Variables:model_id (str) – each model instance has its unique model_id.

Public Functions:

• sync_weights_to: synchronize parameters of the current model

to another model. - get_weights: return a list containing all the parameters of the current model. - set_weights: copy parameters from set_weights() to the model. - forward: define the computations of a neural network. Should be overridden by all subclasses.

get_weights()

Returns a Python dict containing parameters of current model.

Returns:a Python dict containing the parameters of current model.

set_weights(weights)

Copy parameters from set_weights() to the model.

Parameters:weights (dict) – a Python dict containing the parameters.

sync_weights_to(target_model, decay=0.0)

Synchronize parameters of current model to another model.

target_model_weights = decay * target_model_weights

• (1 - decay) * current_model_weights

Parameters:

• target_model (parl.Model) – an instance of Model that has the same neural network architecture as the current model.
• decay (float) – the rate of decline in copying parameters. 0 if no parameters decay when synchronizing the parameters.

Example:

import copy
# create a model that has the same neural network structures.
target_model = copy.deepcopy(model)

# after initilizing the parameters ...
model.sync_weights_to(target_mdodel)

Note

Before calling sync_weights_to, parameters of the model must have been initialized.