pyFDN.Sparsity

class pyFDN.Sparsity(ref)

Density penalty on a square mixing matrix, after Optimizing Tiny Colorless FDNs (Dal Santo et al.).

Rewards a dense matrix – 0 when \(|A|\) is maximally dense (its entries all \(1/\sqrt{N}\), i.e. best mixing) and 1 when fully sparse. Registered with a positive weight it therefore pushes the feedback matrix away from the sparse, poorly-mixing corners of SO(N) that a magnitude-only objective is otherwise happy to sit in.

Parameters:

ref (ParamRef) – The matrix to penalize, e.g. pyFDN.param(model, "feedback"). Must be square.

__init__(ref)

Methods

__init__(ref)
check(model)	Preflight against the model it will train, before the first step.
penalty(value)	The penalty on the parameter's mapped value.
terms()	Flatten into (weight, loss) leaves.

Attributes

name	Short label, used as the key in pyFDN.TrainLog.loss_log.

penalty(value)

The penalty on the parameter’s mapped value.

Return type:

Tensor