Researchers have tested a variety of personal informatics systems to encourage diversity in the political leaning, geography, and demographics of information sources, often with a belief in the normative value of exposure to diverse information sources. Methods attempted have included information labeling of media sources, personalized metrics of reading behavior, personalized visualization of social media behavior, recommendation systems, and social introductions. Although some of these systems demonstrate positive results for the metrics they define, substantial questions remain on the interpretation of these results and their implications for future design. We identify challenges in defining normative values of diversity, potential algorithmic exclusion for some groups, and the role of personal tracking as surveillance. Furthermore, we outline challenges for evaluating systems and defining the meaningful social impact for information diversity systems operating at scale.