Under which condition regarding the underlying distribution will the sample mean *not* converge to a single finite value according to the LLN?

Answer

If the distribution is heavy-tailed, like the Cauchy distribution.

The law requires the expected value ($\mu$) to exist and be finite. If the distribution is heavy-tailed, such as the Cauchy distribution, extreme values occur often enough that the sample mean keeps getting pulled around indefinitely, preventing convergence to a single finite value.

Under which condition regarding the underlying distribution will the sample mean *not* converge to a single finite value according to the LLN?
Lawprobabilitystatisticnumber