Diversity for NLP: how to measure it and how it may help

Whenever working with data, one may argue that a resource’s diversity conditions its quality. But how can diversity be measured? A substantial amount of the literature comes from ecology; their diversity measures may be ported to linguistics, but some differences between the fields may render them suboptimal. We discuss such cases here, and follow with some actual experiments on systems and the impact of diversity on them.