Java Tokens

In Java, the program contains classes and methods. Further, the methods contain the expressions and statements required to perform a specific operation. These statements and expressions are made up of tokens. In other words, we can say that the expression and statement is a set of tokens. The tokens are the small building blocks of a Java program that are meaningful to the Java compiler. Further, these two components contain variables, constants, and operators. In this section, we will discuss what is tokens in Java.

What is token in Java?

The Java compiler breaks the line of code into text (words) is called Java tokens. These are the smallest element of the Java program. The Java compiler identified these words as tokens. These tokens are separated by the delimiters. It is useful for compilers to detect errors. Remember that the delimiters are not part of the Java tokens.

For example, consider the following code.

In the above code snippet, public, class, Demo, {, static, void, main, (, String, args, [, ], ), System, ., out, println, javatpoint, etc. are the Java tokens.

The Java compiler translates these tokens into Java bytecode. Further, these bytecodes are executed inside the interpreted Java environment.

Types of Tokens

Java token includes the following:

• Keywords
• Identifiers
• Literals
• Operators
• Separators
• Comments

Keywords: These are the pre-defined reserved words of any programming language. Each keyword has a special meaning. It is always written in lower case. Java provides the following keywords:

Identifier: Identifiers are used to name a variable, constant, function, class, and array. It usually defined by the user. It uses letters, underscores, or a dollar sign as the first character. The label is also known as a special kind of identifier that is used in the goto statement. Remember that the identifier name must be different from the reserved keywords. There are some rules to declare identifiers are:

• The first letter of an identifier must be a letter, underscore or a dollar sign. It cannot start with digits but may contain digits.
• The whitespace cannot be included in the identifier.
• Identifiers are case sensitive.

Some valid identifiers are:

Literals: In programming literal is a notation that represents a fixed value (constant) in the source code. It can be categorized as an integer literal, string literal, Boolean literal, etc. It is defined by the programmer. Once it has been defined cannot be changed. Java provides five types of literals are as follows:

• Integer
• Floating Point
• Character
• String
• Boolean

Operators: In programming, operators are the special symbol that tells the compiler to perform a special operation. Java provides different types of operators that can be classified according to the functionality they provide. There are eight types of operators in Java, are as follows:

• Arithmetic Operators
• Assignment Operators
• Relational Operators
• Unary Operators
• Logical Operators
• Ternary Operators
• Bitwise Operators
• Shift Operators

Separators: The separators in Java is also known as punctuators. There are nine separators in Java, are as follows:

Note that the first three separators (; , and .) are tokens that separate other tokens, and the last six (3 pairs of braces) separators are also known as delimiters. For example, Math.pow(9, 3); contains nine tokens.

• Square Brackets []: It is used to define array elements. A pair of square brackets represents the single-dimensional array, two pairs of square brackets represent the two-dimensional array.
• Parentheses (): It is used to call the functions and parsing the parameters.
• Curly Braces {}: The curly braces denote the starting and ending of a code block.
• Comma (,): It is used to separate two values, statements, and parameters.
• Assignment Operator (=): It is used to assign a variable and constant.
• Semicolon (;): It is the symbol that can be found at end of the statements. It separates the two statements.
• Period (.): It separates the package name form the sub-packages and class. It also separates a variable or method from a reference variable.

Comments: Comments allow us to specify information about the program inside our Java code. Java compiler recognizes these comments as tokens but excludes it form further processing. The Java compiler treats comments as whitespaces. Java provides the following two types of comments:

• Line Oriented: It begins with a pair of forwarding slashes (//).
• Block-Oriented: It begins with /* and continues until it founds */.