MAML

MAML(model, lr, first_order=False)

[Source]

Description

High-level implementation of Model-Agnostic Meta-Learning.

This class wraps an arbitrary nn.Module and augments it with clone() and adapt methods.

For the first-order version of MAML (i.e. FOMAML), set the first_order flag to True upon initialization.

Arguments

References

  1. Finn et al. 2017. “Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks.”

Example

linear = l2l.algorithms.MAML(nn.Linear(20, 10), lr=0.01)
clone = linear.clone()
error = loss(clone(X), y)
clone.adapt(error)
error = loss(clone(X), y)
error.backward()

adapt

MAML.adapt(loss, first_order=None)

Description

Updates the clone parameters in place using the MAML update.

Arguments

clone

MAML.clone(first_order=None)

Description

Returns a MAML-wrapped copy of the module whose parameters and buffers are torch.cloned from the original module.

This implies that back-propagating losses on the cloned module will populate the buffers of the original module. For more information, refer to learn2learn.clone_module().

Arguments

maml_update

maml_update(model, lr, grads=None)

Description

Performs a MAML update on model using grads and lr. The function re-routes the Python object, thus avoiding in-place operations.

NOTE: The model itself is updated in-place (no deepcopy), but the parameters' tensors are not.

Arguments

Example

maml = l2l.algorithms.MAML(Model(), lr=0.1)
model = maml.clone() # The next two lines essentially implement model.adapt(loss)
grads = autograd.grad(loss, model.parameters(), create_graph=True)
maml_update(model, lr=0.1, grads)

MetaSGD

MetaSGD(model, lr=1.0, first_order=False, lrs=None)

[Source]

Description

High-level implementation of Meta-SGD.

This class wraps an arbitrary nn.Module and augments it with clone() and adapt methods. It behaves similarly to MAML, but in addition a set of per-parameters learning rates are learned for fast-adaptation.

Arguments

References

  1. Li et al. 2017. “Meta-SGD: Learning to Learn Quickly for Few-Shot Learning.” arXiv.

Example

linear = l2l.algorithms.MetaSGD(nn.Linear(20, 10), lr=0.01)
clone = linear.clone()
error = loss(clone(X), y)
clone.adapt(error)
error = loss(clone(X), y)
error.backward()

clone

MetaSGD.clone()

Descritpion

Akin to MAML.clone() but for MetaSGD: it includes a set of learnable fast-adaptation learning rates.

adapt

MetaSGD.adapt(loss, first_order=None)

Descritpion

Akin to MAML.adapt() but for MetaSGD: it updates the model with the learnable per-parameter learning rates.

meta_sgd_update

meta_sgd_update(model, lrs=None, grads=None)

Description

Performs a MetaSGD update on model using grads and lrs. The function re-routes the Python object, thus avoiding in-place operations.

NOTE: The model itself is updated in-place (no deepcopy), but the parameters' tensors are not.

Arguments

Example

meta = l2l.algorithms.MetaSGD(Model(), lr=1.0)
lrs = [th.ones_like(p) for p in meta.model.parameters()]
model = meta.clone() # The next two lines essentially implement model.adapt(loss)
grads = autograd.grad(loss, model.parameters(), create_graph=True)
meta_sgd_update(model, lrs=lrs, grads)