Gumbel-(Soft)max sampling

This demo illustrates the Gumbel-(Soft)max tricks and shows the influence of the Gumbel-Softmax temperature λ and the scale parameter of the added Gumbel noise β.
Details can be found in the corresponding paper.
Here you can also find a Jupyter Notebook that accompanies Figure 4 of the paper, and provides insight in the different routes that can be taken to draw a soft or hard sample from a categorical distribution using the Gumbel-(Soft)max trick.

Distribution settings

Number of classes in categorical distribution Classes:

or fill yourself:

Unnormalized log-probabilities of the categorical distribution. Format: [unnormalized logit1, unnormalized logit2, ...] Unnormalized log-probabilities :

Unnormalized probabilities of the categorical distribution. Format: [unnormalized prob1, unnormalized prob2, ...] Unnormalized probabilities :

Normalized probabilities of the categorical distribution. Format: [normalized prob1, normalized prob2, ...] Normalized probabilities :

Sampling settings

Number of Monte-Carlo samples drawn using Gumbel-(Soft)max sampling Number of Monte Carlo draws (sample size):

Scale of the added Gumbel noise in the Gumbel-(Soft)max trick Gumbel noise scale β:

Temperature parameter used in the Gumbel-Softmax trick Gumbel-Softmax temperature λ:

This demo belongs to the paper: "A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning" from Huijben, I. A. M., Kool, W., Paulus, M. B., & van Sloun, R. J. G. (2022).
Please cite this paper if you find this demo useful in your work:

    
        @article{
            huijben2022,
	    author={Iris A.M. Huijben and Wouter Kool and Max B. Paulus and Ruud J.G. van Sloun},
	    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
	    title={A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning}, 
	    year={2022},
	    doi={10.1109/TPAMI.2022.3157042},
        }