|| hsfzxjy || | 言えない言葉は、ここに隠される

Read More

## Statement

Assume that $\alpha_1, \alpha_2, \ldots, \alpha_n$ satisify $\sum_k\alpha_k=1$. Define

$$Z=\arg\max_k\{\log\alpha_k+G_k\}$$where $G_1,\ldots,G_n \text{ i.i.d.}\sim Gumbel(0,1)$, whose PDF and CDF are defined as

\begin{align} f(x)&=e^{-(x+e^{-x})} \\ F(x)&=e^{-e^{-x}}\end{align}. Then $\mathbb{P}(Z=k)=\alpha_k$.

## Proof

Set $u_k=\log{\alpha_k}+G_k$. We prove by direct calculations.

\begin{align} \mathbb{P}(Z=k)&=\mathbb{P}(u_k \geq u_j,\forall j \neq k) \\ &=\int_{-\infty}^\infty \mathbb{P}(u_k \geq u_j, \forall j \neq k|u_k)\mathbb{P}(u_k) du_k \\ &=\int_{-\infty}^\infty \prod_{j\neq k}\mathbb{P}(u_k \geq u_j|u_k)\mathbb{P}(u_k) du_k \\ &=\int_{-\infty}^\infty \prod_{j\neq k}e^{-e^{-u_k+\log \alpha_j}} e^{-(u_k-\log\alpha_k+e^{-(u_k-\log\alpha_k)})} du_k \\ &=\int_{-\infty}^\infty e^{-\sum_{j\neq k}\alpha_je^{-u_k}} \alpha_k e^{-(u_k+\alpha_k e^{-u_k})} du_k \\ &=\alpha_k \int_{-\infty}^\infty e^{-u_k-(\alpha_k+\sum_{j\neq k}\alpha_j)e^{-u_k}} du_k \\ &= \alpha_k \end{align}.

## Application

The trick is commonly used in DL to make sampling over a discrete distribution differentiable.

ひぐらしの声が聞きった

もう雪が積もる

あなたのいない世界
きっと寂しいんだ

## Option::as_ref

Let’s consider the following function:

The function transform takes in an Option<NonNull<T>> as input, and convert the inner pointer to immutable reference &T if available. The method NonNull::as_ref() is marked unsafe so we need an unsafe block. The snippet didn’t compile well however:

Read More

## Rc and RefCell tricks

Say we need a Cursor<T> type, which should contain a mutable reference to T. The type has a method .dup() that will create a new instance of Cursor<T> while sharing the reference between new and old instances. This is common in database driver designing, where users might expect to have multiple cursors alive simultaneously, owning references to the same connection object. A possible implementation might look like:

Perfect and neat, and luckily Rust compiler did not complain. As a Rust freshman, like me, it’s not easy to keep compiler silent all the time, especially when playing with references. But once I try to chain up the constructor and .dup(), the compiler screams:

Read More

Read More

## Visualizing Correlation

Say we have a matrix A of shape N x M, which can be viewed as a collection of N vectors of shape 1 x M. The code below gives us the correlation matrix of A:

To visualize it, just use plt.matshow(A_corr).

If N is so large that the figure could not provide a clear insight, we might alternatively use histograms like this: