The various token types are used to syntax highlight particular words in a language. This makes code easier to read. There is a wide latitude in the usage of the token types, and really it depends on the specifics of the language as to which token represents which type. Some examples are given below, but these are just guidelines, not hard and fast rules.
Parser rules can highlight tokens using any of the following token types:
NULL - no special highlighting is
performed on tokens of type
jEdit supports four different types of comment tokens. Generally, comments are programmer-readable constructs that are ignored by compilers and interpreters. As an example, the lisp mode defines four comment types:
The function token is intended to identify functions, methods, procedures, routines, or named subprograms.
The digit token is to identify numbers.
The invalid type is to indicate that particular words are not to be used, for example, the java mode defined both "goto" and "const" as invalid words. These are words that are defined by the language, but are not to be used.
Keywords are used to identify well-defined words within a language. Some languages naturally divide keywords into groups, for example, the pascal mode identifies "for" as a KEYWORD1, "private" as a KEYWORD2, and "int" as a KEYWORD3.
A label is generally a named position within a source, for example, the ada mode defined a label as <<foo>>.
Literals are usually, but not always, uninterpreted strings, for example, "foo" or 'bar'. There are a wide variety of usages of literals in the mode files.
The markup token is generally used in the various "markup" languages, such as xml and html. Markup is used for those elements that are not specified as words belonging to the language. For example, in html, <body> would be considered a keyword, where <foo> would be considered markup.
Common examples of operators are the math symbols, such as '+', '-', and so on.