Lexical Elements

Before a Go program can be understood, the compiler must break the source file into the smallest meaningful pieces. These pieces are called lexical elements — the raw vocabulary of the language. They include comments (which are discarded during compilation), identifiers, keywords, operators, punctuation, and the invisible insertion of semicolons that Go performs to make the syntax cleaner.

Overview

Go source text is a sequence of Unicode characters. The compiler groups those characters into tokens, the atomic units of the language. Tokens fall into four categories:

• Identifiers — names you define for variables, functions, types, and packages.
• Keywords — reserved words with fixed meaning, like func or return.
• Operators and punctuation — symbols that perform operations or structure the code, like +, =, (, and {.
• Literals — constant values such as 42, "hello", or 3.14.

Whitespace (spaces, tabs, carriage returns, newlines) separates tokens but is otherwise ignored — with one critical exception: a newline can trigger the automatic insertion of a semicolon. Comments are also treated as whitespace, so they exist only for humans reading the code.

These lexical rules are the foundation every other Go concept builds on. If a token is malformed or a semicolon gets inserted where you didn't intend, the parser won't understand the rest of the program.

Comments

Comments are text in source code that the compiler completely ignores. They exist solely to explain intent, document APIs, or temporarily disable code. Go provides two comment forms, both inherited from C.

• Line comments start with // and stop at the end of the current line. Everything after // on that line is a comment.
• General comments (also called block comments) start with /* and end with */. They can span multiple lines.

package main
import "fmt"
// main is the entry point of the program.
func main() {
    fmt.Println("Hello") // print greeting
    /*
        The following line is temporarily disabled:
        fmt.Println("Debug info")
    */
}

A general comment that does not contain a newline behaves like a space. If it does contain a newline, it behaves like a newline for the purposes of the next important rule: automatic semicolon insertion.

Comments cannot appear inside a rune or string literal. Inside those literals, the sequences //, /*, and */ are just ordinary characters, not comment delimiters.

For documenting packages, functions, and types, Go uses comments written in a specific format that the go doc tool can extract. The convention is to start the comment with the name of the element being documented, with no blank line between the comment and the declaration. That convention belongs to the domain of documentation, not lexical rules, but it explains why Go developers write comments the way they do.

Identifiers and Keywords

An identifier is a name you give to something in a program — a variable, function, type, constant, or package. An identifier is a sequence of one or more letters (including _) and digits, with the first character required to be a letter.

x           // valid: starts with a lowercase letter
_x9         // valid: underscore counts as a letter
myFunction  // valid: mixed case
αβ          // valid: Unicode letters from any language are allowed

The blank identifier _ is a special case. It can appear as a variable name, but you can never read its value. Its purpose is to hold values you must receive but don't intend to use. In a language where unused variables cause compilation errors, the blank identifier prevents you from being forced to use a value you don't need.

result, _ := someFunction()  // ignore the second return value

Go reserves 25 keywords that cannot be used as identifiers. They are the language's built-in vocabulary.

break    default     func   interface  select
case     defer       go     map        struct
chan     else        goto   package    switch
const    fallthrough if     range      type
continue for         import return     var

Some identifiers are predeclared — they are not keywords, so you can technically redefine them (though doing so is almost always a bad idea). They include the names of built-in types (int, string, bool, error), constants (true, false, nil), and built-in functions (make, len, append, panic, recover).

Exported vs. unexported identifiers is a rule that cuts across all of Go's package system: an identifier is exported (visible outside its package) if its first character is an uppercase Unicode letter. This is not a lexical rule in the strictest sense — it's a semantic rule — but it is so tightly tied to identifier naming that every Go programmer learns it early.

var Name string   // exported (starts with uppercase)
var count int     // unexported (starts with lowercase)

Tokens and Automatic Semicolon Insertion

How tokens are formed

The Go lexer scans source text left to right, greedily consuming the longest sequence of characters that forms a valid token. Whitespace, including newlines, serves as a separator between tokens but is not itself a token.

For example, x+=y is tokenized as x, +=, y — not as x, +, =, y — because += is a single valid token and is longer than + alone. Similarly, 123abc is a single identifier, not the number 123 followed by the identifier abc, because the longest valid sequence starting from the first character is an identifier that happens to start with digits (which is illegal, so the compiler will report an error).

Semicolons

The formal Go grammar uses semicolons as statement terminators, but you almost never write them. Instead, the compiler inserts them automatically. The rules:

1. A semicolon is inserted immediately after a line's final token if that token is:
   • An identifier (e.g., a variable name)
   • A literal (integer, floating-point, imaginary, rune, or string)
   • One of the keywords break, continue, fallthrough, or return
   • One of the operators/punctuation ++, --, ), ], or }
2. To allow a complex statement to span multiple lines, a semicolon is not inserted immediately before a closing ) or }.

These two rules together explain why Go code looks the way it does. The canonical formatting places the opening brace { on the same line as the if, for, or func keyword — because if you put it on the next line, a semicolon gets inserted after the keyword or closing parenthesis, breaking the statement.

// Correct: opening brace on the same line
if x > 0 {
    fmt.Println("positive")
}
// Incorrect: semicolon inserted after the ) — compilation error
if x > 0
{
    fmt.Println("positive")
}

Semicolons can also be omitted before a final ) or } in multi-line expressions, which is why you can break function call arguments or struct literals across lines without adding semicolons.

Operators and Punctuation

Go recognizes a fixed set of character sequences that serve as operators and punctuation. These tokens do everything from arithmetic to control flow delimiting.

The full set, organized by function:

The tilde ~ was added in Go 1.18 as part of the generics implementation, where it is used in type constraints to denote an underlying type set.

These operator and punctuation tokens appear in almost every line of Go code. Their precise meaning depends on context — the same & can be the bitwise AND operator or the address-of operator — but from the lexer's perspective, they are simply one of the four token classes, recognized character by character before the parser ever sees them.

The lexical layer is where every Go program begins — and where small missteps cause the most immediately visible compilation errors. Once you internalize the four token classes, the two comment forms, the semicolon insertion rules, and the naming rules for identifiers, you can read and write Go source text with full confidence in what the compiler is seeing.