Go io Package

Reader, Writer, and the small interfaces that make Go I/O composable โ€” plus bufio for efficient buffered I/O.

io.Reader io.Writer io.Copy io.ReadAll io.Pipe io.LimitReader bufio.Scanner bufio.Writer bytes.Buffer
๐Ÿ”Œ

Core Interfaces

โ„น๏ธ
The power of Go I/O is that everything is composed from tiny interfaces. A function that accepts io.Reader works with files, network connections, HTTP bodies, strings, and in-memory buffers โ€” all without change.
The four fundamental interfaces Interfaces
// Reader โ€” anything you can read bytes from
type Reader interface {
    Read(p []byte) (n int, err error)
}

// Writer โ€” anything you can write bytes to
type Writer interface {
    Write(p []byte) (n int, err error)
}

// Closer โ€” anything that can be closed
type Closer interface {
    Close() error
}

// Seeker โ€” random access within a stream
type Seeker interface {
    Seek(offset int64, whence int) (int64, error)
    // whence: io.SeekStart=0, io.SeekCurrent=1, io.SeekEnd=2
}

// Composed interfaces used throughout the stdlib
// io.ReadCloser   = Reader + Closer  (e.g. http.Response.Body)
// io.WriteCloser  = Writer + Closer
// io.ReadWriter   = Reader + Writer
// io.ReadWriteCloser = Reader + Writer + Closer
// io.ReadSeeker   = Reader + Seeker  (e.g. *os.File, *bytes.Reader)
Common types that satisfy io.Reader and io.Writer Implementations
// io.Reader implementations
var _ io.Reader = (*os.File)(nil)            // files
var _ io.Reader = (*bytes.Buffer)(nil)        // in-memory buffer
var _ io.Reader = (*bytes.Reader)(nil)        // read-only byte slice
var _ io.Reader = (*strings.Reader)(nil)      // read-only string
var _ io.Reader = (net.Conn)(nil)             // network connections
var _ io.Reader = (http.Response{}).Body       // HTTP response body

// io.Writer implementations
var _ io.Writer = (*os.File)(nil)            // files, os.Stdout, os.Stderr
var _ io.Writer = (*bytes.Buffer)(nil)        // in-memory buffer
var _ io.Writer = (http.ResponseWriter)(nil)  // HTTP response
var _ io.Writer = (*bufio.Writer)(nil)        // buffered writer
โš™๏ธ

io Package Functions

io.Copy โ€” stream from reader to writer Copy
// Copy(dst Writer, src Reader) โ†’ written int64, err error
// Copies until EOF or error; uses a 32 KB internal buffer

// Copy a file
src, _ := os.Open("input.txt")
dst, _ := os.Create("output.txt")
defer src.Close(); defer dst.Close()
n, err := io.Copy(dst, src)
// n = bytes written

// Serve a file over HTTP
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    f, _ := os.Open("index.html")
    defer f.Close()
    io.Copy(w, f)
})

// CopyN โ€” copy exactly n bytes
io.CopyN(dst, src, 1024)

// CopyBuffer โ€” use your own buffer (avoids alloc in tight loops)
buf := make([]byte, 64*1024)
io.CopyBuffer(dst, src, buf)
io.ReadAll โ€” read everything at once ReadAll
// ReadAll reads until EOF, returns []byte
body, err := io.ReadAll(resp.Body)
if err != nil {
    log.Fatal(err)
}

// Read a string from stdin
data, _ := io.ReadAll(os.Stdin)

// Read from a strings.Reader
r := strings.NewReader("hello world")
b, _ := io.ReadAll(r)
// b = []byte("hello world")
io.WriteString & io.Discard Misc
// WriteString โ€” write a string to any Writer
// (avoids converting to []byte manually)
io.WriteString(os.Stdout, "Hello\n")
io.WriteString(w, "HTTP/1.1 200 OK\r\n")

// io.Discard โ€” a Writer that discards all bytes
// Useful for draining a reader (e.g. HTTP body)
io.Copy(io.Discard, resp.Body)

// Benchmark without I/O overhead
n, _ := io.Copy(io.Discard, bigReader)
io.LimitReader, io.MultiReader, io.TeeReader Wrappers
// LimitReader โ€” read at most n bytes from r
limited := io.LimitReader(resp.Body, 10*1024*1024) // cap at 10 MB
data, _ := io.ReadAll(limited)

// MultiReader โ€” concatenate multiple readers into one
header := strings.NewReader("HEADER\n")
body   := strings.NewReader("BODY\n")
footer := strings.NewReader("FOOTER\n")
combined := io.MultiReader(header, body, footer)
io.Copy(os.Stdout, combined)
// HEADER
// BODY
// FOOTER

// TeeReader โ€” read and simultaneously write to w
var buf bytes.Buffer
tee := io.TeeReader(resp.Body, &buf) // taps the stream
json.NewDecoder(tee).Decode(&result) // decode AND keep raw bytes in buf
io.Pipe โ€” synchronous in-memory pipe Pipe
// Pipe creates a connected (PipeReader, PipeWriter) pair
// Write blocks until Read consumes the data (no internal buffer)
pr, pw := io.Pipe()

go func() {
    json.NewEncoder(pw).Encode(myStruct)
    pw.Close()  // signal EOF to reader
}()

var result MyStruct
json.NewDecoder(pr).Decode(&result)

// Common pattern: stream large data through an encoder
// without buffering the whole thing in memory
pr2, pw2 := io.Pipe()
go func() {
    gzip.NewWriter(pw2)  // compress as you go
    pw2.Close()
}()
http.Post(url, "application/gzip", pr2)
๐Ÿงฑ

bufio โ€” Buffered I/O

โ„น๏ธ
Raw Read calls can be expensive โ€” each call may be a syscall. bufio wraps any Reader or Writer and batches those calls, dramatically reducing overhead for line-at-a-time or small-write workloads.
bufio.Scanner โ€” line-by-line reading Scanner
// The idiomatic way to read lines from any Reader
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
    line := scanner.Text()  // string, no trailing \n
    fmt.Println(line)
}
if err := scanner.Err(); err != nil {
    log.Fatal(err)
}

// Read lines from a file
f, _ := os.Open("data.txt")
defer f.Close()
scanner = bufio.NewScanner(f)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}

// Bytes() instead of Text() avoids a string alloc
for scanner.Scan() {
    b := scanner.Bytes()  // valid only until next Scan()
    _ = b
}
Scanner split functions โ€” words, runes, custom SplitFunc
// Default split: ScanLines (split on \n)
scanner := bufio.NewScanner(r)

// Split on whitespace-delimited words
scanner.Split(bufio.ScanWords)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}

// Split on individual runes
scanner.Split(bufio.ScanRunes)

// Split on individual bytes
scanner.Split(bufio.ScanBytes)

// Increase buffer for long lines (default 64 KB)
scanner.Buffer(make([]byte, 1*1024*1024), 1*1024*1024)
bufio.Reader โ€” buffered reading Reader
br := bufio.NewReader(os.Stdin)

// ReadString โ€” read until delimiter
line, err := br.ReadString('\n')
// includes the delimiter in the result

// ReadLine โ€” no delimiter in result
line2, isPrefix, _ := br.ReadLine()
// isPrefix=true when line > buffer size

// Peek โ€” look ahead without consuming
b, _ := br.Peek(4)  // first 4 bytes, not consumed

// ReadByte / UnreadByte โ€” one byte at a time
c, _ := br.ReadByte()
br.UnreadByte()
bufio.Writer โ€” buffered writing Writer
bw := bufio.NewWriter(os.Stdout)

// Writes go to the buffer until Flush
fmt.Fprintln(bw, "line 1")
fmt.Fprintln(bw, "line 2")
bw.WriteString("line 3\n")

// Flush sends buffered data to underlying writer
if err := bw.Flush(); err != nil {
    log.Fatal(err)
}

// Custom buffer size (default 4 KB)
bw2 := bufio.NewWriterSize(f, 64*1024)
// Don't forget to Flush when done!
๐Ÿ—„๏ธ

In-Memory Buffers

bytes.Buffer โ€” read/write in-memory buffer bytes.Buffer
var buf bytes.Buffer  // zero value is ready to use

// Write to it
buf.WriteString("Hello")
buf.WriteByte(',')
fmt.Fprintf(&buf, " %s!", "World")

// Read from it
fmt.Println(buf.String()) // "Hello, World!"
fmt.Println(buf.Len())    // 13

// Use as io.Writer then io.Reader
json.NewEncoder(&buf).Encode(myData) // write JSON
json.NewDecoder(&buf).Decode(&out)   // read it back

buf.Reset()  // clear without reallocating
strings.NewReader & bytes.NewReader Readers
// strings.NewReader โ€” wrap a string as io.Reader
r := strings.NewReader("Hello, World!")
io.Copy(os.Stdout, r)

// Implements io.ReadSeeker โ€” can seek
r.Seek(0, io.SeekStart)  // rewind to beginning
r.Seek(-5, io.SeekEnd)   // 5 bytes before end

// bytes.NewReader โ€” wrap []byte as io.Reader
data := []byte{0x47, 0x6f}
br := bytes.NewReader(data)
json.NewDecoder(br).Decode(&v)

// bytes.NewBuffer โ€” writable starting content
buf2 := bytes.NewBuffer([]byte("seed"))
๐Ÿ“

Common Patterns

Implementing io.Reader on your own type Custom Reader
// A simple counter reader that wraps another reader
type CountReader struct {
    r     io.Reader
    total int64
}

func (c *CountReader) Read(p []byte) (int, error) {
    n, err := c.r.Read(p)
    c.total += int64(n)
    return n, err
}

cr := &CountReader{r: resp.Body}
io.Copy(io.Discard, cr)
fmt.Println("bytes read:", cr.total)

// io.EOF is the expected signal for "no more data"
// Return it (or wrap it) when your reader is exhausted
func (c *CountReader) Read(p []byte) (int, error) {
    if c.total >= c.limit {
        return 0, io.EOF
    }
    // ...
}
Testing with io โ€” use strings.NewReader and bytes.Buffer Testing
// Functions that accept io.Reader are trivially testable
func CountWords(r io.Reader) int {
    scanner := bufio.NewScanner(r)
    scanner.Split(bufio.ScanWords)
    count := 0
    for scanner.Scan() { count++ }
    return count
}

// In tests โ€” no need for real files
func TestCountWords(t *testing.T) {
    r := strings.NewReader("hello world foo")
    got := CountWords(r)
    if got != 3 {
        t.Errorf("got %d, want 3", got)
    }
}

// Functions that accept io.Writer use bytes.Buffer in tests
func TestWrite(t *testing.T) {
    var buf bytes.Buffer
    WriteReport(&buf, data)
    if !strings.Contains(buf.String(), "Total:") {
        t.Error("missing Total: in output")
    }
}
๐Ÿ“‹

Quick Reference

io function / typeReturnsDescription
io.ReaderinterfaceRead(p []byte) (n int, err error)
io.WriterinterfaceWrite(p []byte) (n int, err error)
io.CloserinterfaceClose() error
io.SeekerinterfaceSeek(offset int64, whence int) (int64, error)
io.ReadCloserinterfaceReader + Closer (e.g. http.Response.Body)
io.ReadWriterinterfaceReader + Writer
io.Copy(dst, src)int64, errorStream all bytes from src to dst
io.CopyN(dst, src, n)int64, errorCopy exactly n bytes
io.CopyBuffer(dst, src, buf)int64, errorCopy using a caller-supplied buffer
io.ReadAll(r)[]byte, errorRead until EOF, return all bytes
io.WriteString(w, s)int, errorWrite string to Writer without []byte cast
io.LimitReader(r, n)ReaderWrap r to read at most n bytes
io.MultiReader(rs...)ReaderConcatenate multiple readers
io.TeeReader(r, w)ReaderRead from r and copy to w simultaneously
io.Pipe()*PipeReader, *PipeWriterSynchronous in-memory pipe
io.DiscardWriterWriter that silently discards all bytes
io.EOFerrorSentinel: no more data to read
io.SeekStart / SeekCurrent / SeekEndintWhence values for Seek: 0 / 1 / 2
bufio / bytes / stringsReturnsDescription
bufio.NewScanner(r)*ScannerLine-by-line scanner over any Reader
scanner.Scan()boolAdvance to next token; false at EOF or error
scanner.Text()stringCurrent token as string
scanner.Bytes()[]byteCurrent token as bytes (no alloc)
scanner.Split(f)โ€”Set split func: ScanLines/Words/Runes/Bytes
scanner.Buffer(buf, max)โ€”Override internal buffer size
bufio.NewReader(r)*ReaderBuffered reader; ReadString, Peek, etc.
bufio.NewWriter(w)*WriterBuffered writer; must call Flush()
bytes.BufferstructRead/write in-memory buffer; zero value ready
bytes.NewReader(b)*ReaderRead-only Reader + Seeker over []byte
strings.NewReader(s)*ReaderRead-only Reader + Seeker over string