Many fast renewing tissues are characterized by a hierarchical cellular architecture, with tissue specific stem cells at the root of the cellular hierarchy, differentiating into a whole range of specialized cells. There is increasing evidence that tumors are structured in a very similar way, mirroring the hierarchical structure of the host tissue. In some tissues, differentiated cells can also revert to the stem cell phenotype, which increases the risk that mutant cells lead to long lasting clones in the tissue. However, it is unclear under which circumstances de-differentiating cells will invade a tissue. To address this, we developed mathematical models to investigate how de-differentiation is selected as an adaptive mechanism in the context of cellular hierarchies. We derive thresholds for which de-differentiation is expected to emerge, and it is shown that the selection of de-differentiation is a result of the combination of the properties of cellular hierarchy and de-differentiation patterns. Our results suggest that de-differentiation is most likely to be favored provided stem cells having the largest effective self-renewal rate. Moreover, jumpwise de-differentiation provides a wider range of favorable conditions than stepwise de-differentiation. Finally, the effect of de-differentiation on the redistribution of self-renewal and differentiation probabilities also greatly influences the selection for de-differentiation.