Go (Golang) is often used to implement microservices and system software. However, it can also be used to implement back-ends for full-stack web applications. When used for web applications, Go is part of a technology stack which also includes Javascript, CSS, and HTML. The development environment will also include tools for building, testing, integrating and deploying the application.
On a software project for a web application, the software engineering tools, processes and workflows must consider and encompass every element of the technology stack, not just the Go back-end. In this regard, it is important to consider how the software engineering for Go development can be effectively incorporated. For example, some relevant concerns include:
- Agile software development with Go.
- A strategy for organizing the Go implementation into packages.
- Leveraging Go’s strengths for effective system architecture and design.
- Building and integrating a complete web application with a Go back-end.
The software engineering which supported Resultra’s development addressed many of the concerns listed above. Therefore, Resultra can serve as a representative example for the software engineering which needs to accompany Go development.
Agile Software Development with Go
To some extent, almost every contemporary software development project is based around the principles and values for agile software development. With respect to Go development, several of these principles and values are relevant, including:
- The project should focus more on creating software than documentation. Working software is more important than comprehensive documentation.
- The project should accommodate changing requirements, even late in development.
Go is a statically typed language with memory safety and garbage collection. Given these language features, Go inherently supports a type of development process like the following:
- Implement a basic working version of a software feature.
- Implement automated tests for the functionality and/or performance of the feature.
- As the requirements change, a better design is found or performance bottlenecks are encountered, incrementally improve and refactor the code.
Since Go is statically typed, many source code conflicts introduced during refactoring will be immediately identified by the compiler. Existing automated tests ensure there is not a regression in functionality or performance. In other words, Go supports a software development process which focuses on working software, but also gives developers the confidence to refactor and change the code as needed.
Even with continuous code improvements and refactoring, my own experience with Go development is the resulting software remained very stable and robust. Resultra’s development included unit tests and API tests, and the use of a profiler to identify performance bottlenecks. Resultra’s development notably did not include the use of a debugger; the types of serious defects which are typically found with a debugger did not occur or were more readily found with test cases or simple logging output.
Modular Package Organization
The code base for any non-trivial software project will expand beyond a single package. Go uses packages to identify code from a specific directory. Software engineering discipline and package principles help to determine what constitutes a package and how the software code and accompanying files should be organized and split into multiple packages.
One lesson learned from Resultra’s development is to orient towards smaller packages. This means packages will have a manageable number of files and a well-defined purpose. In fact, Resultra’s Go code base currently has a total of 97 different packages. These packages are roughly categorized as follows:
- Generic — These packages implement generic and reusable functionality which can be used throughout the project. Examples include packages for string manipulation and helper methods for handling JSON data passed through APIs.
- Common — These packages implement reusable functionality for a specific sub-system. There are multiple common packages throughout the code base. For example, in Resultra, there is a common package with helper functions to store and retrieve form data.
- Data Model — These packages implement the persistence for individual parts of Resultra’s data model. There are many data model packages. These packages also expose REST APIs for manipulating the data.
- Controller — These packages implement the application’s business logic and build upon the data model. Where appropriate, this business logic exposed through REST APIs.
- Web pages — Using Go’s HTML template library, these packages implement end-points for specific web pages. To the extent parts of a web page have stand-alone functionality, different packages may also be implemented for different parts of the page (e.g., a menu bar, search box).
The rationale for smaller packages includes the following:
- In terms of object-oriented design, Go encapsulates functions, types and names at a package level. Names which begin with a lowercase letters are only visible within the package. Therefore, when encapsulation is needed, it makes sense to isolate private functions and types within packages.
- In most cases, data model and controller packages can have a 1:1 correspondence with API endpoints. For example all the APIs for “packageA” would have a base URL like the following: “http://mydomain.com/packageA”.
- A unit testing strategy with Go is to place unit tests in the same directory as the code itself, but with a “_test.go” suffix added to the file names. To the extent directories for individual packages have files for both the code and unit tests, keeping package sizes manageable is important.
- For Go packages corresponding to web pages, other types of assets are associated with the web page, such as HTML templates, Javascript and CSS files. Keeping these mixed file types together ensures changes to the web page are constrained to one directory (i.e., “what changes together, stays together”). However, to keep package and directory sizes manageable, it is helpful to keep the code and web assets for different pages separate.
Modular APIs
Consistent with the modular package organization described above, there are several approaches to registering the APIs for individual packages. For each API endpoint, what this entails in to register a callback function which is invoked when the API is called.
One approach is a “top down” registration scheme. During program initialization, this involves a top-level package calling registration functions within each of the different packages.
A second “bottom up” approach is for each package to independently register its APIs at startup. Resultra uses the Gorilla mux library for this purpose. Within an individual package, an init() function is used to register the APIs, as shown in the following example:
func init() {
checkBoxRouter := mux.NewRouter()
checkBoxRouter.HandleFunc(“/api/frm/checkBox/new”, newCheckBox)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/resize”,
resizeCheckBox)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setColorScheme”,
setColorScheme)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setStrikethrough”,
setStrikethrough)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setLabelFormat”,
setLabelFormat)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setVisibility”,
setVisibility)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setPermissions”,
setPermissions)
checkBoxRouter.HandleFunc(
“/api/frm/checkBox/setClearValueSupported”,
setClearValueSupported)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setHelpPopupMsg”,
setHelpPopupMsg)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/setValidation”,
setValidation)
checkBoxRouter.HandleFunc(“/api/frm/checkBox/validateInput”,
validateInputAPI)
http.Handle(“/api/frm/checkBox/”, checkBoxRouter)
}
There are several advantages of a “bottom up” or self-contained API registration scheme. First, a top-level package doesn’t need to establish a package dependency upon another package just to complete the API registrations. Secondly, when a package is included in another package, the registrations happen automatically. Finally, as a system grows and changes over time, it is conceivable for different packages to be integrated into different micro-services, tested independently, etc.; to the extent both the functionality and APIs for a package are fully self-contained with a package, “bottom up” API registration ensures that a package’s APIs are always registered at runtime.
System Design Considerations
For a full-stack web application, there are many high level system design considerations; for example:
- How will the application’s performance scale to support many users?
- Can the application’s workload be split amongst several processing nodes?
- How will the application serve up static web assets, such as HTML and CSS files?
- Is there a way to partition (shard) the data for different users or companies using the application?
Using Go as a back-end has some implications on system design.
First, Go’s runtime model is such that the back-end executable will startup then run continuously. This is different from other languages such as PHP, where the lifetime of the script’s execution is only for a single API request or web page load. Since Golang runs continuously, it can potentially cache information between API requests or maintain persistent connections with clients.
Go has built-in concurrent programming primitives, such as goroutines and channels. This allows the performance of a single Go executable to seamlessly scale (vertically scale) with the available CPU and memory resources on the server. Unless an application is expected to serve millions of users, running a single back-end Go executable may be the best system design alternative.
In full-stack web applications, there is a separation of concerns between the front-end (web browser) and back-end (server). Go is arguably a good choice for the back-end. For numerically intensive computations and complex business logic, compiled Go executables can have optimum performance. The front-end is then focused on viewing or presenting the information.
Integrating Go with a System Build Environment
Nominally, only the `go build` command is needed to build a Golang executable. However, in the broader context of building and integrating a complete web application, many other steps are typically required to build and integrate the entire system. For example:
- Download and install 3rd party libraries needed to compile the Go back-end. For this purpose, Resultra uses Go’s dependency management tool.
- Run other commands from Go’s toolchain as necessary. For example, Resultra uses Go’s implementation of the YACC parser generator for its calculated field formulas.
- Package front-end web assets, such as HTML templates, CSS files and Javascript files. Resultra uses Gulp for this purpose.
- Run automated unit and integration tests.
Any number of build tools could be used to accomplish the above. Resultra uses Makefiles to invoke the individual commands, and a Python build script to perform a phased build.
Summary
Go is a suitable language for full-stack web applications. Choosing Go for a technology stack has implications on the software engineering which accompanies and provides discipline for Go development.
During the development of Resultra, numerous software engineering decisions were made to effectively incorporate a Go back-end implementation into a web application technology stack. Notably, consideration was given to aligning the Go development with an agile software development process, modular package organization, and automatically building and testing the Go back-end within a system build environment. The software engineering used for Resultra’s development is representative of the tools, processes and disciplines needed to develop any full-stack web application with a Go backend.