Visualization literacy, previously called graphicacy, relies on an analogy to textual literacy to communicate its intended meaning as the ability to use visualizations. However, there are two issues with this analogy: a mismatch where analysis contexts do not have an analogous place in textual literacy and an inherited dichotomy of literate vs. illiterate which does not accurately represent the spectrum of visualization skill. In this paper, we discuss existing literature and how it typically describes visualization literacy and the connection to textual literacy. We then propose new terminology, the visualization skillset, to provide more flexibility when discussing this ability and its spectrum of proficiency, and to provoke discussion on how to improve on the issues we identified. Finally, we discuss how to move forward as a field and propose directions for future work.