PARL is a flexible, distributed and object-oriented programming reinforcement learning framework.

Features

Object Oriented Programming Distributed Training
class MLPModel(parl.Model):
  def __init__(self, act_dim):
    self.fc1 = layers.fc(size=10)
    self.fc2 = layers.fc(size=act_dim)

  def forward(self, obs):
    out = self.fc1(obs)
    out = self.fc2(out)
    return out

model = MLPModel()
target_model = copy.deepcopy(model)
# Absolute multi-thread programming
# witout the GIL limitation

@parl.remote_class
class HelloWorld(object):
    def sum(self, a, b):
        return a + b

parl.connect('localhost:8003')
obj = HelloWorld()
ans = obj.sum(a, b)

Abstractions

_images/abstractions.png
PARL aims to build an agent for training algorithms to perform complex tasks.
The main abstractions introduced by PARL that are used to build an agent recursively are the following:
  • Model is abstracted to construct the forward network which defines a policy network or critic network given state as input.
  • Algorithm describes the mechanism to update parameters in the model and often contains at least one model.
  • Agent, a data bridge between the environment and the algorithm, is responsible for data I/O with the outside environment and describes data preprocessing before feeding data into the training process.

Installation

Features

High-quality Implementations