Sitelet https://github.com/skaae/rbm_toolbox_lua/issues/8
Skip to content

Initialisation of W and U from a uniform distribution is incorrect #8

Description

@elezar

In the paper "Learning Algorithms for the Classification Restricted Boltzmann Machine", a uniform distribution in the interval [-m**-0.5, m**-0.5] is used for the initialisation of the weight matrices (rbm.W and rbm.U).

However, in the initcrbm the following code is used:

M = math.max(m,n);
interval_max = math.pow(M,-0.5);
interval_min = -interval_max;
weights = torch.rand(m,n):mul( interval_min + (interval_max-interval_min) )

which initialises the weights using the interval [0, m**-0.5].

Should the calculation of the weights not be?

weights = torch.rand(m,n):mul(interval_max-interval_min):add(interval_min)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions