Handling Errors When Reading Files in Go
How to handle missing files, permission errors, read errors and scanner errors when reading files in Go, with complete runnable examples.
Reading a file looks like one step but can fail in several different ways. The file might not exist, you might not have permission, a line might be too long, or the disk might fail halfway through. Each of these returns a different error, and good code handles the ones it can and reports the rest clearly.
Reading a Whole File With os.ReadFile
For small files, os.ReadFile reads everything into memory in one call:
package main
import (
"fmt"
"os"
)
func main() {
data, err := os.ReadFile("notes.txt")
if err != nil {
fmt.Println("error:", err)
return
}
fmt.Print(string(data))
}
If notes.txt is missing, the program prints:
error: open notes.txt: no such file or directory
On Windows the text is open notes.txt: The system cannot find the file specified. The wording differs between systems, which is one more reason never to match on message text.
Telling Missing Files Apart From Other Failures
A missing file is often fine, for example an optional config file. A permission error is not. Use errors.Is with the io/fs sentinels to tell them apart, on every operating system:
package main
import (
"errors"
"fmt"
"io/fs"
"os"
)
func readOptional(path string) (string, error) {
data, err := os.ReadFile(path)
switch {
case errors.Is(err, fs.ErrNotExist):
return "", nil // optional file, no content
case errors.Is(err, fs.ErrPermission):
return "", fmt.Errorf("no permission to read %s, check the file owner: %w", path, err)
case err != nil:
return "", fmt.Errorf("read %s: %w", path, err)
}
return string(data), nil
}
func main() {
text, err := readOptional("optional.conf")
fmt.Printf("text=%q err=%v\n", text, err)
}
When the file does not exist this prints text="" err=<nil>. Permission problems and other failures are passed up with context.
os.ErrNotExist and fs.ErrNotExist are the same:
os.ErrNotExist is defined as fs.ErrNotExist, so either works with errors.Is. Newer code tends to use the io/fs names.
Checking if a File Exists
People often search for a way to check if a file exists. Use os.Stat and errors.Is:
package main
import (
"errors"
"fmt"
"io/fs"
"os"
)
func exists(path string) (bool, error) {
_, err := os.Stat(path)
if err == nil {
return true, nil
}
if errors.Is(err, fs.ErrNotExist) {
return false, nil
}
return false, err // could be permission denied: we do not know
}
func main() {
fmt.Println(exists("/etc/hostname"))
fmt.Println(exists("/nope/missing"))
}
On Linux this prints true <nil> and false <nil>. The third case matters: if Stat fails for another reason, the honest answer is "unknown", so the error is returned.
Check-then-open is a race:
The file can be deleted between your exists check and the actual open. When you are going to open the file anyway, skip the check, just open it and handle fs.ErrNotExist from the open itself.
Reading Line by Line With bufio.Scanner
For large files or line-based processing, open the file and use bufio.Scanner. There are now three places an error can appear: opening, each line you process, and the scanner itself.
package main
import (
"bufio"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
)
func sumFile(path string) (int, error) {
f, err := os.Open(path)
if err != nil {
return 0, fmt.Errorf("sum file: %w", err)
}
defer f.Close() // read-only, ignoring the close error is fine
total := 0
line := 0
sc := bufio.NewScanner(f)
for sc.Scan() {
line++
text := strings.TrimSpace(sc.Text())
if text == "" {
continue
}
n, err := strconv.Atoi(text)
if err != nil {
return 0, fmt.Errorf("%s line %d: %w", path, line, err)
}
total += n
}
if err := sc.Err(); err != nil {
return 0, fmt.Errorf("read %s: %w", path, err)
}
return total, nil
}
func main() {
dir := os.TempDir()
good := filepath.Join(dir, "good.txt")
bad := filepath.Join(dir, "bad.txt")
os.WriteFile(good, []byte("10\n20\n\n12\n"), 0o644)
os.WriteFile(bad, []byte("1\n2\nthree\n"), 0o644)
fmt.Println(sumFile(good))
_, err := sumFile(bad)
fmt.Println(err)
}
Output (the folder part of the path depends on your system):
42 <nil>
/tmp/bad.txt line 3: strconv.Atoi: parsing "three": invalid syntax
The line number in the error is what makes it useful. Without it, someone with a ten-thousand-line file has to search for the bad value by hand.
Forgetting sc.Err():
sc.Scan() returns false both at the normal end of the file and when reading fails. If you never call sc.Err() after the loop, a read failure looks exactly like a short file, and your program silently works with half the data.
The token too long Error
bufio.Scanner refuses lines longer than 64 KB by default. When it meets one, Scan returns false and sc.Err() returns:
bufio.Scanner: token too long
This often shows up with minified JSON or log files that have huge lines. Give the scanner a bigger buffer before the loop:
package main
import (
"bufio"
"errors"
"fmt"
"strings"
)
func main() {
long := strings.Repeat("x", 100_000)
sc := bufio.NewScanner(strings.NewReader(long))
for sc.Scan() {
}
fmt.Println("default:", sc.Err(), errors.Is(sc.Err(), bufio.ErrTooLong))
sc = bufio.NewScanner(strings.NewReader(long))
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for sc.Scan() {
fmt.Println("bigger buffer: read", len(sc.Text()), "bytes")
}
fmt.Println("error:", sc.Err())
}
Output:
default: bufio.Scanner: token too long true
bigger buffer: read 100000 bytes
error: <nil>
The second argument to Buffer is the maximum line size, here 1 MB. For data without a sensible limit, use bufio.Reader and its ReadString('\n') method instead.
io.EOF Is Not a Failure
Lower-level reads such as bufio.Reader.ReadString return io.EOF when the data ends. It is a signal, not an error to report:
package main
import (
"bufio"
"errors"
"fmt"
"io"
"strings"
)
func main() {
r := bufio.NewReader(strings.NewReader("alpha\nbeta"))
for {
line, err := r.ReadString('\n')
if line != "" {
fmt.Printf("got %q\n", strings.TrimSuffix(line, "\n"))
}
if errors.Is(err, io.EOF) {
break
}
if err != nil {
fmt.Println("read failed:", err)
return
}
}
}
Output:
got "alpha"
got "beta"
The last line, beta, has no newline, so it arrives together with io.EOF. Processing line before checking for io.EOF keeps it from being lost.
Your file reading is robust when:
Opening errors are checked and wrapped with the path, missing files are detected with errors.Is(err, fs.ErrNotExist), parse errors include line numbers, sc.Err() is checked after every scan loop, and io.EOF is treated as the normal end.
Summary
os.ReadFile suits small files, and os.Open with bufio.Scanner suits large or line-based ones. Use errors.Is with fs.ErrNotExist and fs.ErrPermission to react to specific failures on any operating system. Add the path and line number to errors, always check sc.Err() after the loop, raise the scanner buffer for long lines, and treat io.EOF as the normal end of input after processing the last chunk.