Let $A$ be a complex $n\times n$ matrix. We call the set of matrices $X$ such that $X^*AX=A$ the congruence centralizer of $A$. This is an analog of the classical centralizer of $A$ in the case where the group $\mathrm{GL}_n(\mathbb C)$ acts on the matrix space $M_n(\mathbb C)$ by congruence rather than similarity. We find the congruence centralizer of the matrix $$ \Delta_n=\left(\begin{array}{cccc} &&&1\\ &&\cdots&i\\ &1&\cdots&\\ 1&i&& \end{array}\right). $$ This matrix represents one of the three types of building blocks for the canonical form of square complex matrices with respect to congruences found by R. Horn and V. Sergeichuk.