
Manage Go application configuration using go-aws-ssm and Viper
Scenario
We are building a Go application that stores all the application secrets in the AWS Parameter Store and all the non sensitive configuration to environment variables. We are gonna use our favourite config tool Viper to deal with configuration management and our go-aws-ssm package to manage parameter secrets. Secrets will be loaded during runtime in application startup.
Tools used
go-aws-ssm https://github.com/PaddleHQ/go-aws-ssm
What we will do
Set up Secure Strings to AWS Parameter Store for our application
Load environment variables and defaults using Viper
Fetch secrets from AWS Parameter Store using go-aws-ssm and load them into Viper in order to create a Database connection (a fake one).
1. Set secure strings in AWS Parameter Store
Navigate to AWS Systems Manager -> Parameter Store in the AWS Console
Select Create Parameter and fill out the form like the following picture.
We will create all parameters of our application under /demo-service/development/
Select Type as Secure String that will get encrypted using one of the KMS keys
We will create the following parameters
/demo-service/development/DB_USERNAME -> Value: my-database-username
/demo-service/development/DB_PASSWORD -> Value: my-database-password

2. Load environment variables and defaults
The following environment variables are set:
export PARAMETER_STORE_PATH=/demo-service/development/
export DB_HOST=mysql.host.com
Let's load our non sensitive config to our application. We create a new viper instance, enable environment check for all keys and set a default for the DB_PORT key.
func main(){
viperInstance := viper.New()
viperInstance.AutomaticEnv()
viperInstance.SetDefault("DB_PORT", "3306")
}
Viper should able to load environment variables and defaults now:
viperInstance.GetString("PARAMETER_STORE_PATH") -> "/demo-app/development/"
viperInstance.GetString("DB_HOST") -> "mysql.host.com"
viperInstance.GetInt("DB_PORT") -> 3306
3. Fetch secrets from AWS Parameter Store
Create a new Parameter Store client, this will create an AWS Session and SSM Client in the background. If you want more control use the awsssm.NewParameterStoreWithClient() that accepts AWS SSM Client and you can set all the AWS config that is needed for your application. We fetch the parameters using the path that is set on PARAMETER_STORE_PATH environment variable and decrypt the values.
...
paramstore, err := awsssm.NewParameterStore()
if err != nil {
log.Fatal(err)
}
params, err := paramstore.GetAllParametersByPath
(viperInstance.GetString("PARAMETER_STORE_PATH"), true)
if err != nil {
log.Fatal(err)
}
...
Next step is to load our parameters to the Viper, if you don't use Viper you can simply get the values from the awssm.Parameters object:
params.GetValueByName("DB_USERNAME") -> "my-database-username"
params.GetValueByName("DB_PASSWORD") -> "my-database-password"
In our case we will load them to Viper by configuring it to expect a json document. Then we are all set; we use Viper as normal to create a Database connection.
...
viperInstance.SetConfigType(`json`)
err = viperInstance.ReadConfig(params)
if err != nil {
log.Fatal(err)
}
database := Database{
Username: viperInstance.GetString("DB_USERNAME"),
Password: viperInstance.GetString("DB_PASSWORD"),
Host: viperInstance.GetString("DB_HOST"),
Port: viperInstance.GetInt("DB_PORT"),
}
// do something with the database connection
_ := database
...
The completed code should look like this now:
func main(){
viperInstance := viper.New()
viperInstance.AutomaticEnv()
viperInstance.SetDefault("DB_PORT", "3306")
paramstore, err := awsssm.NewParameterStore()
if err != nil {
log.Fatal(err)
}
params, err := paramstore.GetAllParametersByPath
(viperInstance.GetString("PARAMETER_STORE_PATH"), true)
if err != nil {
log.Fatal(err)
}
viperInstance.SetConfigType(`json`)
err = viperInstance.ReadConfig(params)
if err != nil {
log.Fatal(err)
}
database := Database{
Username: viperInstance.GetString("DB_USERNAME"),
Password: viperInstance.GetString("DB_PASSWORD"),
Host: viperInstance.GetString("DB_HOST"),
Port: viperInstance.GetInt("DB_PORT"),
}
// do something with the database connection
_ := database
}
Summary
Summary
Storing your application secrets to AWS Parameter Store and using go-aws-ssm to manage them in your Go application enables easy integration with existing configuration tools. We are currently using this approach for Go lambdas and containerized Go applications.
Happy coding!