Reading and Writing Files in Go

Read and write files in Go with os.ReadFile, os.WriteFile, os.Open and os.Create, handle missing files, append, and close files correctly.

Most programs need to read a config file, write some output, or check whether a file exists. Go puts all of this in the os package. This page shows the small set of functions that cover nearly every case, and the mistakes that cause lost data or leaked file handles.

The Short Way: os.ReadFile and os.WriteFile

If the file is small enough to fit in memory, read or write it in one call:

package main

import (
	"fmt"
	"log"
	"os"
	"path/filepath"
)

func main() {
	path := filepath.Join(os.TempDir(), "notes.txt")

	err := os.WriteFile(path, []byte("first line\nsecond line\n"), 0o644)
	if err != nil {
		log.Fatal(err)
	}

	data, err := os.ReadFile(path)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("read %d bytes:\n%s", len(data), data)

	os.Remove(path)
}

read 23 bytes:
first line
second line

os.WriteFile creates the file if it does not exist and replaces its contents if it does. The 0o644 is the permission for a new file: the owner can read and write, everyone else can read. os.ReadFile returns a []byte, so convert with string(data) when you need text.

filepath.Join builds the path with the right separator for the operating system, so the same code works on Windows, macOS and Linux.

Checking if a File Exists

There is no os.Exists. You try to use the file and check the error with errors.Is:

package main

import (
	"errors"
	"fmt"
	"io/fs"
	"os"
)

func main() {
	_, err := os.ReadFile("does-not-exist.json")
	if errors.Is(err, fs.ErrNotExist) {
		fmt.Println("config missing, using defaults")
		return
	}
	if err != nil {
		fmt.Println("could not read config:", err)
		return
	}
}

config missing, using defaults

Check for fs.ErrNotExist first, then handle every other error separately. A permission problem is not the same as a missing file, and treating them the same hides real bugs.

Check, then open is a race:

Calling os.Stat to see if a file exists and then opening it leaves a gap where another process can delete or create it. Open the file directly and handle the error instead.

Opening a File for Streaming

For large files, or when you want to read piece by piece, use os.Open. It returns an *os.File, which you must close:

package main

import (
	"fmt"
	"io"
	"log"
	"os"
	"path/filepath"
)

func main() {
	path := filepath.Join(os.TempDir(), "big.log")
	os.WriteFile(path, []byte("0123456789abcdefghij"), 0o644)
	defer os.Remove(path)

	f, err := os.Open(path)
	if err != nil {
		log.Fatal(err)
	}
	defer f.Close()

	buf := make([]byte, 8)
	for {
		n, err := f.Read(buf)
		if n > 0 {
			fmt.Printf("chunk: %q\n", buf[:n])
		}
		if err == io.EOF {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
	}
}

chunk: "01234567"
chunk: "89abcdef"
chunk: "ghij"

defer f.Close() goes right after the error check, never before it. If os.Open failed, f is nil and there is nothing to close. You rarely call Read by hand like this. The next pages show io.Copy and bufio.Scanner, which do the loop for you.

Creating and Appending

os.Create makes a new file or empties an existing one. To add to the end of a file, use os.OpenFile with flags:

package main

import (
	"fmt"
	"log"
	"os"
	"path/filepath"
)

func main() {
	path := filepath.Join(os.TempDir(), "app.log")
	os.Remove(path)
	defer os.Remove(path)

	for _, msg := range []string{"started", "request handled"} {
		f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Fprintln(f, msg)
		if err := f.Close(); err != nil {
			log.Fatal(err)
		}
	}

	data, _ := os.ReadFile(path)
	fmt.Print(string(data))
}

started
request handled

The flags read as: append to the end, create the file if needed, open it for writing only. Without os.O_APPEND, each write would start at the beginning and overwrite the old text.

Check the error from Close when writing:

For files you write to, Close can return an error, for example when the disk is full and buffered data could not be flushed. defer f.Close() throws that error away. For writes that matter, call Close yourself and check the result, as above.

Directories

A few helpers cover most directory work:

package main

import (
	"fmt"
	"log"
	"os"
	"path/filepath"
)

func main() {
	root, err := os.MkdirTemp("", "demo")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(root)

	os.MkdirAll(filepath.Join(root, "a", "b"), 0o755)
	os.WriteFile(filepath.Join(root, "a", "one.txt"), nil, 0o644)
	os.WriteFile(filepath.Join(root, "a", "b", "two.txt"), nil, 0o644)

	entries, _ := os.ReadDir(filepath.Join(root, "a"))
	for _, e := range entries {
		fmt.Println("entry:", e.Name(), "dir:", e.IsDir())
	}

	filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
		if err != nil {
			return err
		}
		if !d.IsDir() {
			rel, _ := filepath.Rel(root, p)
			fmt.Println("file:", filepath.ToSlash(rel))
		}
		return nil
	})
}

entry: b dir: true
entry: one.txt dir: false
file: a/b/two.txt
file: a/one.txt
  • os.MkdirAll creates every missing folder in the path and does nothing if it already exists.
  • os.ReadDir lists one folder, sorted by name.
  • filepath.WalkDir visits every file and folder below a root.
  • os.RemoveAll deletes a folder and everything inside it, so be careful which path you pass.

You can handle files:

If you can read a config with os.ReadFile, treat a missing file with errors.Is(err, fs.ErrNotExist), append to a log, and close written files while checking the error, you have the file basics covered.

Summary

Use os.ReadFile and os.WriteFile for small files, and os.Open, os.Create or os.OpenFile when you need streaming or appending. Close every file you open, right after the error check, and check the Close error when writing. There is no Exists function, so open the file and check for fs.ErrNotExist. Build paths with filepath.Join so they work on every system.