Core Interfaces
Reader ยท Writer ยท Closer ยท SeekerThe power of Go I/O is that everything is composed from tiny interfaces. A function that accepts
io.Reader works with files, network connections, HTTP bodies, strings, and in-memory buffers โ all without change.
The four fundamental interfaces
Interfaces
// Reader โ anything you can read bytes from type Reader interface { Read(p []byte) (n int, err error) } // Writer โ anything you can write bytes to type Writer interface { Write(p []byte) (n int, err error) } // Closer โ anything that can be closed type Closer interface { Close() error } // Seeker โ random access within a stream type Seeker interface { Seek(offset int64, whence int) (int64, error) // whence: io.SeekStart=0, io.SeekCurrent=1, io.SeekEnd=2 } // Composed interfaces used throughout the stdlib // io.ReadCloser = Reader + Closer (e.g. http.Response.Body) // io.WriteCloser = Writer + Closer // io.ReadWriter = Reader + Writer // io.ReadWriteCloser = Reader + Writer + Closer // io.ReadSeeker = Reader + Seeker (e.g. *os.File, *bytes.Reader)
Common types that satisfy io.Reader and io.Writer
Implementations
// io.Reader implementations var _ io.Reader = (*os.File)(nil) // files var _ io.Reader = (*bytes.Buffer)(nil) // in-memory buffer var _ io.Reader = (*bytes.Reader)(nil) // read-only byte slice var _ io.Reader = (*strings.Reader)(nil) // read-only string var _ io.Reader = (net.Conn)(nil) // network connections var _ io.Reader = (http.Response{}).Body // HTTP response body // io.Writer implementations var _ io.Writer = (*os.File)(nil) // files, os.Stdout, os.Stderr var _ io.Writer = (*bytes.Buffer)(nil) // in-memory buffer var _ io.Writer = (http.ResponseWriter)(nil) // HTTP response var _ io.Writer = (*bufio.Writer)(nil) // buffered writer
io Package Functions
Copy ยท ReadAll ยท WriteString ยท Discard
io.Copy โ stream from reader to writer
Copy
// Copy(dst Writer, src Reader) โ written int64, err error // Copies until EOF or error; uses a 32 KB internal buffer // Copy a file src, _ := os.Open("input.txt") dst, _ := os.Create("output.txt") defer src.Close(); defer dst.Close() n, err := io.Copy(dst, src) // n = bytes written // Serve a file over HTTP http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { f, _ := os.Open("index.html") defer f.Close() io.Copy(w, f) }) // CopyN โ copy exactly n bytes io.CopyN(dst, src, 1024) // CopyBuffer โ use your own buffer (avoids alloc in tight loops) buf := make([]byte, 64*1024) io.CopyBuffer(dst, src, buf)
io.ReadAll โ read everything at once
ReadAll
// ReadAll reads until EOF, returns []byte body, err := io.ReadAll(resp.Body) if err != nil { log.Fatal(err) } // Read a string from stdin data, _ := io.ReadAll(os.Stdin) // Read from a strings.Reader r := strings.NewReader("hello world") b, _ := io.ReadAll(r) // b = []byte("hello world")
io.WriteString & io.Discard
Misc
// WriteString โ write a string to any Writer // (avoids converting to []byte manually) io.WriteString(os.Stdout, "Hello\n") io.WriteString(w, "HTTP/1.1 200 OK\r\n") // io.Discard โ a Writer that discards all bytes // Useful for draining a reader (e.g. HTTP body) io.Copy(io.Discard, resp.Body) // Benchmark without I/O overhead n, _ := io.Copy(io.Discard, bigReader)
io.LimitReader, io.MultiReader, io.TeeReader
Wrappers
// LimitReader โ read at most n bytes from r limited := io.LimitReader(resp.Body, 10*1024*1024) // cap at 10 MB data, _ := io.ReadAll(limited) // MultiReader โ concatenate multiple readers into one header := strings.NewReader("HEADER\n") body := strings.NewReader("BODY\n") footer := strings.NewReader("FOOTER\n") combined := io.MultiReader(header, body, footer) io.Copy(os.Stdout, combined) // HEADER // BODY // FOOTER // TeeReader โ read and simultaneously write to w var buf bytes.Buffer tee := io.TeeReader(resp.Body, &buf) // taps the stream json.NewDecoder(tee).Decode(&result) // decode AND keep raw bytes in buf
io.Pipe โ synchronous in-memory pipe
Pipe
// Pipe creates a connected (PipeReader, PipeWriter) pair // Write blocks until Read consumes the data (no internal buffer) pr, pw := io.Pipe() go func() { json.NewEncoder(pw).Encode(myStruct) pw.Close() // signal EOF to reader }() var result MyStruct json.NewDecoder(pr).Decode(&result) // Common pattern: stream large data through an encoder // without buffering the whole thing in memory pr2, pw2 := io.Pipe() go func() { gzip.NewWriter(pw2) // compress as you go pw2.Close() }() http.Post(url, "application/gzip", pr2)
bufio โ Buffered I/O
Scanner ยท Reader ยท WriterRaw
Read calls can be expensive โ each call may be a syscall. bufio wraps any Reader or Writer and batches those calls, dramatically reducing overhead for line-at-a-time or small-write workloads.
bufio.Scanner โ line-by-line reading
Scanner
// The idiomatic way to read lines from any Reader scanner := bufio.NewScanner(os.Stdin) for scanner.Scan() { line := scanner.Text() // string, no trailing \n fmt.Println(line) } if err := scanner.Err(); err != nil { log.Fatal(err) } // Read lines from a file f, _ := os.Open("data.txt") defer f.Close() scanner = bufio.NewScanner(f) for scanner.Scan() { fmt.Println(scanner.Text()) } // Bytes() instead of Text() avoids a string alloc for scanner.Scan() { b := scanner.Bytes() // valid only until next Scan() _ = b }
Scanner split functions โ words, runes, custom
SplitFunc
// Default split: ScanLines (split on \n) scanner := bufio.NewScanner(r) // Split on whitespace-delimited words scanner.Split(bufio.ScanWords) for scanner.Scan() { fmt.Println(scanner.Text()) } // Split on individual runes scanner.Split(bufio.ScanRunes) // Split on individual bytes scanner.Split(bufio.ScanBytes) // Increase buffer for long lines (default 64 KB) scanner.Buffer(make([]byte, 1*1024*1024), 1*1024*1024)
bufio.Reader โ buffered reading
Reader
br := bufio.NewReader(os.Stdin) // ReadString โ read until delimiter line, err := br.ReadString('\n') // includes the delimiter in the result // ReadLine โ no delimiter in result line2, isPrefix, _ := br.ReadLine() // isPrefix=true when line > buffer size // Peek โ look ahead without consuming b, _ := br.Peek(4) // first 4 bytes, not consumed // ReadByte / UnreadByte โ one byte at a time c, _ := br.ReadByte() br.UnreadByte()
bufio.Writer โ buffered writing
Writer
bw := bufio.NewWriter(os.Stdout) // Writes go to the buffer until Flush fmt.Fprintln(bw, "line 1") fmt.Fprintln(bw, "line 2") bw.WriteString("line 3\n") // Flush sends buffered data to underlying writer if err := bw.Flush(); err != nil { log.Fatal(err) } // Custom buffer size (default 4 KB) bw2 := bufio.NewWriterSize(f, 64*1024) // Don't forget to Flush when done!
In-Memory Buffers
bytes.Buffer ยท strings.Reader ยท bytes.Reader
bytes.Buffer โ read/write in-memory buffer
bytes.Buffer
var buf bytes.Buffer // zero value is ready to use // Write to it buf.WriteString("Hello") buf.WriteByte(',') fmt.Fprintf(&buf, " %s!", "World") // Read from it fmt.Println(buf.String()) // "Hello, World!" fmt.Println(buf.Len()) // 13 // Use as io.Writer then io.Reader json.NewEncoder(&buf).Encode(myData) // write JSON json.NewDecoder(&buf).Decode(&out) // read it back buf.Reset() // clear without reallocating
strings.NewReader & bytes.NewReader
Readers
// strings.NewReader โ wrap a string as io.Reader r := strings.NewReader("Hello, World!") io.Copy(os.Stdout, r) // Implements io.ReadSeeker โ can seek r.Seek(0, io.SeekStart) // rewind to beginning r.Seek(-5, io.SeekEnd) // 5 bytes before end // bytes.NewReader โ wrap []byte as io.Reader data := []byte{0x47, 0x6f} br := bytes.NewReader(data) json.NewDecoder(br).Decode(&v) // bytes.NewBuffer โ writable starting content buf2 := bytes.NewBuffer([]byte("seed"))
Common Patterns
Implement Reader ยท Chain ยท Test
Implementing io.Reader on your own type
Custom Reader
// A simple counter reader that wraps another reader type CountReader struct { r io.Reader total int64 } func (c *CountReader) Read(p []byte) (int, error) { n, err := c.r.Read(p) c.total += int64(n) return n, err } cr := &CountReader{r: resp.Body} io.Copy(io.Discard, cr) fmt.Println("bytes read:", cr.total) // io.EOF is the expected signal for "no more data" // Return it (or wrap it) when your reader is exhausted func (c *CountReader) Read(p []byte) (int, error) { if c.total >= c.limit { return 0, io.EOF } // ... }
Testing with io โ use strings.NewReader and bytes.Buffer
Testing
// Functions that accept io.Reader are trivially testable func CountWords(r io.Reader) int { scanner := bufio.NewScanner(r) scanner.Split(bufio.ScanWords) count := 0 for scanner.Scan() { count++ } return count } // In tests โ no need for real files func TestCountWords(t *testing.T) { r := strings.NewReader("hello world foo") got := CountWords(r) if got != 3 { t.Errorf("got %d, want 3", got) } } // Functions that accept io.Writer use bytes.Buffer in tests func TestWrite(t *testing.T) { var buf bytes.Buffer WriteReport(&buf, data) if !strings.Contains(buf.String(), "Total:") { t.Error("missing Total: in output") } }
Quick Reference
All key functions & types| io function / type | Returns | Description |
|---|---|---|
| io.Reader | interface | Read(p []byte) (n int, err error) |
| io.Writer | interface | Write(p []byte) (n int, err error) |
| io.Closer | interface | Close() error |
| io.Seeker | interface | Seek(offset int64, whence int) (int64, error) |
| io.ReadCloser | interface | Reader + Closer (e.g. http.Response.Body) |
| io.ReadWriter | interface | Reader + Writer |
| io.Copy(dst, src) | int64, error | Stream all bytes from src to dst |
| io.CopyN(dst, src, n) | int64, error | Copy exactly n bytes |
| io.CopyBuffer(dst, src, buf) | int64, error | Copy using a caller-supplied buffer |
| io.ReadAll(r) | []byte, error | Read until EOF, return all bytes |
| io.WriteString(w, s) | int, error | Write string to Writer without []byte cast |
| io.LimitReader(r, n) | Reader | Wrap r to read at most n bytes |
| io.MultiReader(rs...) | Reader | Concatenate multiple readers |
| io.TeeReader(r, w) | Reader | Read from r and copy to w simultaneously |
| io.Pipe() | *PipeReader, *PipeWriter | Synchronous in-memory pipe |
| io.Discard | Writer | Writer that silently discards all bytes |
| io.EOF | error | Sentinel: no more data to read |
| io.SeekStart / SeekCurrent / SeekEnd | int | Whence values for Seek: 0 / 1 / 2 |
| bufio / bytes / strings | Returns | Description |
|---|---|---|
| bufio.NewScanner(r) | *Scanner | Line-by-line scanner over any Reader |
| scanner.Scan() | bool | Advance to next token; false at EOF or error |
| scanner.Text() | string | Current token as string |
| scanner.Bytes() | []byte | Current token as bytes (no alloc) |
| scanner.Split(f) | โ | Set split func: ScanLines/Words/Runes/Bytes |
| scanner.Buffer(buf, max) | โ | Override internal buffer size |
| bufio.NewReader(r) | *Reader | Buffered reader; ReadString, Peek, etc. |
| bufio.NewWriter(w) | *Writer | Buffered writer; must call Flush() |
| bytes.Buffer | struct | Read/write in-memory buffer; zero value ready |
| bytes.NewReader(b) | *Reader | Read-only Reader + Seeker over []byte |
| strings.NewReader(s) | *Reader | Read-only Reader + Seeker over string |