LLMs are trained via “future token prediction”: They are supplied a sizable corpus of text collected from various sources, for instance Wikipedia, information Web sites, and GitHub. The textual content is then damaged down into “tokens,” which can be basically parts of words (“text” is a single token, “fundamentally” is two tokens).