The Go Programming Language

Pedigree

The main designers of the Go Programming Language are Rob Pike, Ken Thompson, Rob Griesemer, Russ Cox, and Ian Taylor.

Ken Thompson was involved in design of Unix and Plan 9 operating systems at Bell Labs. Ken Thompson and Dennis Ritchie jointly received Turing Award in 1993 for their work on Unix and Computer Security.

Rob Pike in collaboration with Brian Kernighan wrote The Unix Programming Environment and The Practice of Programming. Rob wrote the first window system for Unix in 1981, he implemented the "backing store" for the X windows which is now a common paradigm to improve performance of graphical user interfaces. Rob was the architect of Plan 9 operating system and the Acme IDE windowing system.

Russ Cox is the implementor of Google Code Search which allows you to use regular expressions and grep over all of the public source code.

Rob Griesemer implemented the distributed consensus algorithm Paxos for the Google File System.

Ian Taylor rewrote the GNU GCC linker from scratch to improve it's performance. It has been measured to run five times faster linking large C++ applications as compared to the previous versions of the linker.

Description

Here is how Go Language designer's describe Go:

Design Principles

Programming today involves too much bookkeeping, repetition, and clerical work.

Go attempts to reduce the amount of typing in both senses of the word. Throughout its design, Go tries to reduce clutter and complexity.

There are no forward declarations and no header files; everything is declared exactly once. Initialization is expressive, automatic, and easy to use.

Syntax is clean and light on keywords.

Stuttering

(foo.Foo* myFoo = new(foo.Foo))
is reduced by simple type derivation using the := declare-and-initialize construct.

And perhaps most radically, there is no type hierarchy: types just are, they don't have to announce their relationships.

These simplifications allow Go to be expressive yet comprehensible without sacrificing sophistication.

Another important principle is to keep the concepts orthogonal.

Methods can be implemented for any type; structures represent data while interfaces represent abstraction; and so on.

Orthogonality makes it easier to understand what happens when things combine.

Let's write some code

Fast Compilation

Go compiler is five times faster compared to gcc.

Striving for million times improvement for very large code bases, and in order to do so, they have to tackle the dependency problem.

Go programs compile into packages and each compiled package file imports transitive dependency info.

If A.go depends on B.go which depends on C.go
Go compiler compiles C.go, B.go and finally A.go
To recompile A.go, compiler reads B.o but not C.o

This small change has a massive impact on very large scale and provides huge speedup

Large C++ programs such as Firefox, OpenOffice, Chromium have huge build times.

Using gcc 4.0.1 on Mac OS X 10.5.7:

C: #include stdio.h reads 360 lines from 9 files
C++: #include iostream reads 25,536 lines from 131 files
Objective C: #include Cocoa/Cocoa.h reads 112,047 lines from 689 files
Go: import "fmt" reads 195 lines from only 1 file

Applying this over thousands of files, the improvement becomes exponential

No Type Hierarchy

Object-oriented programming, at least in the best-known languages, involves too much discussion of the relationships between types, relationships that often could be derived automatically.

Rather than requiring the programmer to declare ahead of time that two types are related, in Go a type automatically satisfies any interface that specifies a subset of its methods.

Any object that implements Handler can serve HTTP requests.

type Handler interface {
    ServeHTTP(*Conn, *Request)
}

For brevity, let's ignore POSTs and assume HTTP requests are always GETs; that simplification does not affect the way the handlers are set up.

Here's a trivial but complete implementation of a handler to count the number of times the page is visited.

// Simple counter server.
type Counter struct {
    n int
}

func (ctr *Counter) ServeHTTP(c *http.Conn, req *http.Request) {
    ctr.n++
    fmt.Fprintf(c, "counter = %d\n", ctr.n)
}

Here's how to attach such a server to a node on the URL tree.

import "http"
...
ctr := new(Counter)
http.Handle("/counter", ctr)

But why make Counter a struct? An integer is all that's needed.

// Simpler counter server.
type Counter int

func (ctr *Counter) ServeHTTP(c *http.Conn, req *http.Request) {
    *ctr++
    fmt.Fprintf(c, "counter = %d\n", *ctr)
}

Besides reducing the bookkeeping, this approach has real advantages.

Types can satisfy many interfaces at once, without the complexities of traditional multiple inheritance.

Interfaces can be very lightweight - having one or even zero methods in an interface can express useful concepts. An interface is simply a set of methods.

Interfaces can be added after the fact if a new idea comes along or for testing - without annotating the original types.

Because there are no explicit relationships between types and interfaces, there is no type hierarchy to manage or discuss.

It's possible to use these ideas to construct something analogous to type-safe Unix pipes.

For instance, fmt.Fprintf enables formatted printing to any output, not just a file, or the bufio package can be completely separate from file I/O, or the crypto packages stitch together block and stream ciphers.

All these ideas stem from a single interface (io.Writer) representing a single method (Write).

It takes some getting used to but this implicit style of type dependency is one of the most exciting things about Go.

There is no such thing as a class. There is no sublcassing. Any types, even basic types such as integers and strings can have methods.

No method overloading, for a given type there is only one method with that name.

There are no public or private labels on methods or fields. Instead items with UpperCaseNames are visible to clients, lowerCaseNames are hidden.

Instead of subclassing, Go supports the notion of embedding.

type Reader interface {
    Read(p []byte) (n int, err os.Error)
}

type Writer interface {
    Write(p []byte) (n int, err os.Error)
}

You can embed the Reader and Writer interface into a new interface

type ReadWriter interface {
    Reader
    Writer
}

Similar concept of embedding is applicable to structs with some minor differences.

Embedding can also be a simple convenience. This example shows an embedded field alongside a regular, named field.

type Job struct {
    Command string
    *log.Logger
}

The Job type now has the Log, Logf and other methods of *log.Logger. We could have given the Logger a field name, of course, but it's not necessary to do so.

job.Log("starting now...")

The Logger is a regular field of the struct and we can initialize it in the usual way with a constructor,

func NewJob(command string, logger *log.Logger) *Job {
    return &Job{command, logger}
}

or with a composite literal,

job := &Job{command, log.New(os.Stderr, nil, "Job: ", log.Ldate)}

If we need to refer to an embedded field directly, the type name of the field, ignoring the package qualifier, serves as a field name.

If we needed to access the *log.Logger of a Job variable job, we would write job.Logger. This would be useful if we wanted to refine the methods of Logger.

func (job *Job) Logf(format string, args ...) {
    job.Logger.Logf("%q: %s", job.Command, fmt.Sprintf(format, args))
}

Embedding types introduces the problem of name conflicts but the rules to resolve them are simple. First, a field or method X hides any other item X in a more deeply nested part of the type.

If log.Logger contained a field or method called Command, the Command field of Job would dominate it.

Goroutines

You can start a new flow of control with the go keyword.

func main() {
   go someExpensiveComputation(x, y, z)
   anotherComputation(a, b, c)
   for {
     rw := socket.Accept()
     conn := newConn(rw, handler)
     go conn.serve()
   }
}

A goroutine is a lightweight thread, implemented using stacks.

Stacks are small, segmented and sized on demand.

Goroutines are multiplexed by demand into true threads.

Requires support from language, compiler, runtime.

How would we retrieve the results from someExpensiveComputation?

Go is deeply influenced by C.A.R. Hoare's Communicating Sequential Processes.

Do not communicate by sharing memory, instead, share memory by communicating.

A channel is a typed synchronous communications mechanism.

Channels do not require mutexes (locks) to protect shared data structures.

Channels allow you to pass references to data structures between goroutines. If you consider this as passing around ownership of the data (the ability to read and write it), they become a powerful and expressive synchronization mechanism.

Receivers always block until there is data to receive.

If the channel is unbuffered, the sender blocks until the receiver has received the value.

If the channel has a buffer, the sender blocks only until the value has been copied to the buffer; if the buffer is full, this means waiting until some receiver has retrieved a value.

c := make(chan int)  // Allocate a channel.
// Start the sort in a goroutine; when it completes, signal on the channel.
go func() {
    list.Sort()
    c <- 1       // Send a signal; value does not matter. 
}()
doSomethingForAWhile()
<-c              // Wait for sort to finish; discard sent value.

Implement semaphores using channels

A buffered channel can be used like a semaphore, for instance to limit throughput. In this example, incoming requests are passed to handle, which sends a value into the channel, processes the request, and then receives a value from the channel. The capacity of the channel buffer limits the number of simultaneous calls to process.

var sem = make(chan int, MaxOutstanding)

func handle(r *Request) {
    sem <- 1    // Wait for active queue to drain.
    process(r)     // May take a long time.
    <-sem       // Done; enable next request to run.
}

func Serve(queue chan *Request) {
    for {
        req := <-queue
        go handle(req)  // Don't wait for handle to finish.
    }
}

Codewalk: Share Memory by Communicating

Defer

A defer statement pushes a function call onto a list.

The list of saved calls is executed after the surrounding function returns.

Defer is commonly used to simplify functions that perform various clean-up actions.

For example, let's look at a function that opens two files and copies the contents of one file to the other:

func CopyFile(dstName, srcName string) (written int64, err os.Error) {
    src, err := os.Open(srcName, os.O_RDONLY, 0)
    if err != nil {
        return
    }
    defer src.Close()
 
    dst, err := os.Open(dstName, os.O_WRONLY|os.O_CREATE, 0644)
    if err != nil {
        return
    }
    defer dst.Close()
 
    return io.Copy(dst, src)
}

The behavior of defer statements is straightforward and predictable. There are three simple rules:

A deferred function's arguments are evaluated when the defer statement is evaluated.

Deferred function calls are executed in Last In First Out order after the surrounding function returns.

Deferred functions may read and assign to the returning function's named return values.

Panic and Recover

Panic is a built-in function that stops the ordinary flow of control and begins panicking.

When the function F calls panic, execution of F stops, any deferred functions in F are executed normally, and then F returns to its caller.

Recover is a built-in function that regains control of a panicking goroutine.

Recover is only useful inside deferred functions. During normal execution, a call to recover will return nil and have no other effect.

If the current goroutine is panicking, a call to recover will capture the value given to panic and resume normal execution.

The convention in the Go libraries is that even when a package uses panic internally, its external API still presents explicit error return values.

Other uses of defer include releasing a mutex:

mu.Lock()
defer mu.Unlock()

printing a footer:

printHeader()
defer printFooter()

Init

A package with no imports is initialized by assigning initial values to all its package-level variables and then calling any package-level function with the name and signature of

func init()

defined in its source. A package may contain multiple init() functions, even within a single source file; they execute in unspecified order.

If a package has imports, the imported packages are initialized before initializing the package itself. If multiple packages import a package P, P will be initialized only once.

The importing of packages, by construction, guarantees that there can be no cyclic dependencies in initialization.

Other notable features

Slice types

A slice is reference to a contiguous segment of an array, it shares storage with its underlying array and with other slices of the same array.

Most array programming in Go is done with slices rther than simple arrays.

func (file *File) Read(buf []byte) (n int, err os.Error)
To read first 32 bytes of a arger buffer b, slice the buffer
n, rr := f.Read(b[0:32])

Templates

Data-driven templates for generating textual output such as HTML or XML or JSON etc.

Templates are executed by applying them to a data structure.

Annotations in the template refer to elements of the data structure (typically a field of a struct or a key in a map) to control execution and derive values to be displayed.

The template walks the structure as it executes and the "cursor" @ represents the value at the current location in the structure.

package main

import (
    "flag"
    "http"
    "io"
    "log"
    "template"
)

var addr = flag.String("addr", ":1718", "http service address") // Q=17, R=18
var fmap = template.FormatterMap{
    "html": template.HTMLFormatter,
    "url+html": UrlHtmlFormatter,
}
var templ = template.MustParse(templateStr, fmap)

func main() {
    flag.Parse()
    http.Handle("/", http.HandlerFunc(QR))
    err := http.ListenAndServe(*addr, nil)
    if err != nil {
        log.Exit("ListenAndServe:", err)
    }
}

func QR(c *http.Conn, req *http.Request) {
    templ.Execute(req.FormValue("s"), c)
}

func UrlHtmlFormatter(w io.Writer, v interface{}, fmt string) {
    template.HTMLEscape(w, []byte(http.URLEscape(v.(string))))
}


const templateStr = `
<html>
<head>
<title>QR Link Generator</title>
</head>
<body>
{.section @}
<img src="http://chart.apis.google.com/chart?chs=300x300&cht=qr&choe=UTF-8&chl={@|url+html}"
/>
<br>
{@|html}
<br>
<br>
{.end}
<form action="/" name=f method="GET">
<input maxLength=1024 size=70 name=s value="" title="Text to QR Encode">
<input type=submit value="Show QR" name=qr>
</form>
</body>
</html>
`

Systems Language

Go allows developers to control memory layout but does not support pointer arithmetic.

No dangling pointers, locals move to heap as need

No pointer to integer conversions

All variables are zero initialized

All indexing is bounds checked

Designed to do speed-critical things well such as crypto.

Google is using Go to write web servers, web crawlers, search indexers, databases, compilers, file system, etc

Awards

Go was the 2009 TIOBE "Language of the year" two months after it was released.

From InfoWorld's Bossie Award citation:

Google's Go language tries to restore simplicity to programming. It does away with numerous constructs that have crept into all OO languages by re-imagining how to simplify and improve the conversation between a developer and the code.

Go provides garbage collection, type safety, memory safety, and built-in support for concurrency and for Unicode characters.

In addition, it compiles (fast!) to binaries for multiple platforms.

Go is still in development (new features are being added) and it has limitations, notably poor support for Windows.

But it shows a new, exciting direction in programming languages.

Interested in following up with more details, please contact me at shakeel.mahate@us.abb.com.