parl.Model

class Model(model_id=None)[source]
alias: parl.Model
alias: parl.core.fluid.agent.Model
Model is a base class of PARL for the neural network. A Model is usually a policy or Q-value function, which predicts an action or an estimate according to the environmental observation.
To track all the layers , users are required to implement neural networks with the layers from parl.layers (e.g., parl.layers.fc). These layers has the same APIs as fluid.layers.
Model supports duplicating a Model instance in a pythonic way:
copied_model = copy.deepcopy(model)

Example:

import parl

class Policy(parl.Model):
    def __init__(self):
        self.fc = parl.layers.fc(size=12, act='softmax')

    def policy(self, obs):
        out = self.fc(obs)
        return out

policy = Policy()
copied_policy = copy.deepcopy(model)
Variables:model_id (str) – each model instance has its uniqe model_id.
Public Functions:
  • sync_weights_to: synchronize parameters of the current model to another model.
  • get_weights: return a list containing all the parameters of the current model.
  • set_weights: copy parameters from set_weights() to the model.
  • forward: define the computations of a neural network. Should be overridden by all subclasses.
  • parameters: return a list containting names of parameters of the model.
  • set_model_id: set model_id of current model explicitly.
  • get_model_id: return the model_id of current model.
get_params()[source]

Return a Python list containing parameters of current model.

Deprecated since version 1.2: This will be removed in 1.3, please use get_weights instead.

Returns:a Python list containing parameters of the current model.
Return type:parameters
get_weights()[source]

Returns a Python list containing parameters of current model.

Returns: a Python list containing the parameters of current model.

parameter_names

Get names of all parameters in this Model.

Deprecated since version 1.2: This will be removed in 1.3, please use parameters instead.

Only parameters created by parl.layers are included. The order of parameter names is consistent among different instances of the same Model.

Returns:list of string containing parameter names of all parameters.
Return type:param_names(list)
parameters()[source]

Get names of all parameters in this Model.

Only parameters created by parl.layers are included. The order of parameter names is consistent among different instances of the same Model.

Returns:list of string containing parameter names of all parameters
Return type:param_names(list)

Example:

model = Model()
model.parameters()

# output:
['fc0.w0', 'fc0.bias0']
set_params(params, gpu_id=None)[source]

Set parameters in the model with params.

Deprecated since version 1.2: This will be removed in 1.3, please use set_weights instead.

Parameters:params (List) – List of numpy array .
set_weights(weights)[source]

Copy parameters from set_weights() to the model.

Parameters:weights (list) – a Python list containing the parameters.
sync_params_to(target_net, gpu_id=None, decay=0.0, share_vars_parallel_executor=None)[source]

Synchronize parameters in the model to another model (target_net).

Deprecated since version 1.2: This will be removed in 1.3, please use sync_weights_to instead.

target_net_weights = decay * target_net_weights + (1 - decay) * source_net_weights

Parameters:
  • target_model (parl.Model) – an instance of Model that has the same neural network architecture as the current model.
  • decay (float) – the rate of decline in copying parameters. 0 if no parameters decay when synchronizing the parameters.
  • share_vars_parallel_executor (fluid.ParallelExecutor) – Optional. If not None, will use fluid.ParallelExecutor to run program instead of fluid.Executor
sync_weights_to(target_model, decay=0.0, share_vars_parallel_executor=None)[source]

Synchronize parameters of current model to another model.

To speed up the synchronizing process, it will create a program implicitly to finish the process. It also stores a program as the cache to avoid creating program repeatedly.

target_model_weights = decay * target_model_weights + (1 - decay) * current_model_weights

Parameters:
  • target_model (parl.Model) – an instance of Model that has the same neural network architecture as the current model.
  • decay (float) – the rate of decline in copying parameters. 0 if no parameters decay when synchronizing the parameters.
  • share_vars_parallel_executor (fluid.ParallelExecutor) – Optional. If not None, will use fluid.ParallelExecutor to run program instead of fluid.Executor.

Example:

import copy
# create a model that has the same neural network structures.
target_model = copy.deepcopy(model)

# after initilizing the parameters ...
model.sync_weights_to(target_mdodel)

Note

Before calling sync_weights_to, parameters of the model must have been initialized.