Exercise 6

from Example Sheet 3

Exercise 6.

Let \(\rho=\sum_ip_i\rho_i\) where \(\rho_i\) are density matrices which have support on orthogonal subspaces. Then prove that \[ S\bigg(\sum_ip_i\rho_i\bigg)=H(p)+\sum_ip_iS(\rho_i) \] where \(p=\{p_i\}\) is a probability distribution, and \(H(p)\) is the corresponding Shannon entropy.

Let us write the total number of states as \(n\). Recall \(S(\rho) := - \operatorname{tr}[\rho \log \rho]\). In fact, we don’t need to restrict \(S\) to act on states; for any \(X\geq 0\), we can define \[ S(X) = - \operatorname{tr}[X\log X]. \] We notice for \(p> 0\) and a state \(\rho\), we have \[\begin{aligned} S(p\rho) &= - \operatorname{tr}[ p \rho \log (p \rho)] = -p \operatorname{tr}[ \rho (\log p I + \log \rho)] = - p \operatorname{tr}(\rho) \log p - p \operatorname{tr}[ \rho \log \rho]\\ &= -p \log p + p S(\rho). \end{aligned}\](1) Now, if the supports of the \(\rho_i\) are all orthogonal, then in particular their eigenvectors are too. We can thus make an orthonormal basis of the Hilbert space by the eigenvectors of all the \(\rho_i\), so that each \(\rho_i\) is diagonal in this basis. We see that \[ \sum_i p_i \rho_i = \begin{pmatrix} p_1 \rho_1 \\ &&p_2 \rho_2 \\ &&& \ddots \\ &&&& p_n \rho_n \end{pmatrix} \] where here we write \(\rho_1 \equiv \rho_1|_{\operatorname{supp}\rho_1}\) to mean the block matrix of \(\rho_1\) restricted to its support. Then \[ -\big(\sum_i p_i \rho_i \big) \log \big(\sum_i p_i \rho_i \big) = \begin{pmatrix} -p_1 \rho_1 \log (p_1 \rho_1) \\ && -p_2 \rho_2\log(p_2 \rho_2) \\ &&& \ddots \\ &&&& -p_n \rho_n \log(p_n \rho_n) \end{pmatrix}, \] and taking the trace, we see \[ S\left(\sum_i p_i \rho_i\right) = \sum_i S(p_i \rho_i) = -\sum_i p_i \log p_i + p_i S(\rho_i) \] by (1).