Sparse transition matrices

SparseTransitions represents a row-stochastic transition matrix with an L1-like shrinkage on off-diagonal probabilities. During Baum-Welch updates, each off-diagonal entry is reduced by λ and clipped at zero before rows are renormalised. Larger values of λ therefore encourage self-transitions and sparsity away from the diagonal.

julia> trans = HMMRBM.SparseTransitions(4, 0.2);

julia> size(trans)
(4, 4)

julia> trans[1, 4]
0.25

Sparse transitions can be passed directly to the transitions field of MultiSeqHMM and remain stochastic through the EM loop thanks to HMMRBM.baum_welch_transition_update!. If a row’s expected counts are fully zeroed, it is reset to a uniform distribution.

Constructors

HMMRBM.SparseTransitions — Type

struct SparseTransitions{T, M} where {T, M<:AbstractMatrix{T}} <: AbstractMatrix{T}

SparseTransitions(transition, λ)
SparseTransitions(::Type{<:AbstractMatrix}, state_count, λ)
SparseTransitions(::Type{<:Real}, state_count, λ)
SparseTransitions(state_count, λ)

A transition matrix with a penalty on non diagonal elements. During Baum-Welch updates the off-diagonals are shrunk by λ before rows are renormalised, promoting self-transitions and sparsity away from the diagonal while keeping the matrix row-stochastic.

source

Accessors

HMMRBM.state_count — Method

state_count(A::SparseTransitions)

Get the number of states represented by the sparse transition matrix.

source

Training support

HMMRBM.baum_welch_transition_update! — Method

baum_welch_transition_update!(trans, logtrans, expected)

Update the transition matrix trans and logtrans in place using the expected transition counts expected.

source

Annealed transition matrices

AnnealedTransitions represents a row-stochastic transition matrix with a parameter β that controls the inverse temperature of the model during training. Lower values lead to more uniform transitions; higher values to sparser transitions.

Constructors

HMMRBM.AnnealedTransitions — Type

struct AnnealedTransitions{T, M} where {T, M<:AbstractMatrix{T}} <: SparseTransitions{T,M}

AnnealedTransitions(transition, β)
AnnealedTransitions(::Type{<:AbstractMatrix}, state_count, β)
AnnealedTransitions(::Type{<:Real}, state_count, β)
AnnealedTransitions(state_count, β)

A variant of SparseTransitions where the penalty is applied as a power to the usual Baum-Welch update rather than a subtraction. In other words, this changes the temperature of the transition update, with β > 1 promoting sparsity and β < 1 promoting more uniform transitions.

source

Accessors

HMMRBM.state_count — Method

state_count(A::AnnealedTransitions)

Get the number of states represented by the annealed transition matrix.

source

Training support

HMMRBM.baum_welch_transition_update! — Method

baum_welch_transition_update!(trans, logtrans, expected)

Update the transition matrix trans and logtrans in place using the expected transition counts expected.

source