Manage Go application configuration using go-aws-ssm and Viper

Manage Go application configuration using go-aws-ssm and Viper

Scenario

We are building a Go application that stores all the application secrets in the AWS Parameter Store and all the non sensitive configuration to environment variables. We are gonna use our favourite config tool Viper to deal with configuration management and our go-aws-ssm package to manage parameter secrets. Secrets will be loaded during runtime in application startup.

Tools used

• go-aws-ssm https://github.com/PaddleHQ/go-aws-ssm

• viper https://github.com/spf13/viper

What we will do

1. Set up Secure Strings to AWS Parameter Store for our application

2. Load environment variables and defaults using Viper

3. Fetch secrets from AWS Parameter Store using go-aws-ssm and load them into Viper in order to create a Database connection (a fake one).

1. Set secure strings in AWS Parameter Store

• Navigate to AWS Systems Manager -> Parameter Store in the AWS Console

• Select Create Parameter and fill out the form like the following picture.

  • We will create all parameters of our application under /demo-service/development/

  • Select Type as Secure String that will get encrypted using one of the KMS keys

  • We will create the following parameters

    • /demo-service/development/DB_USERNAME -> Value: my-database-username

    • /demo-service/development/DB_PASSWORD -> Value: my-database-password

2. Load environment variables and defaults

The following environment variables are set:

export PARAMETER_STORE_PATH=/demo-service/development/

export DB_HOST=mysql.host.com

Let's load our non sensitive config to our application. We create a new viper instance, enable environment check for all keys and set a default for the DB_PORT key.

func main(){

  viperInstance := viper.New()

  viperInstance.AutomaticEnv()

  viperInstance.SetDefault("DB_PORT", "3306")

}

Viper should able to load environment variables and defaults now:

• viperInstance.GetString("PARAMETER_STORE_PATH") -> "/demo-app/development/"

• viperInstance.GetString("DB_HOST") -> "mysql.host.com"

• viperInstance.GetInt("DB_PORT") -> 3306

3. Fetch secrets from AWS Parameter Store

Create a new Parameter Store client, this will create an AWS Session and SSM Client in the background. If you want more control use the awsssm.NewParameterStoreWithClient() that accepts AWS SSM Client and you can set all the AWS config that is needed for your application. We fetch the parameters using the path that is set on PARAMETER_STORE_PATH environment variable and decrypt the values.

...
paramstore, err := awsssm.NewParameterStore()

if err != nil {

  log.Fatal(err)

}

params, err := paramstore.GetAllParametersByPath

(viperInstance.GetString("PARAMETER_STORE_PATH"), true)

if err != nil {

  log.Fatal(err)

}
...

Next step is to load our parameters to the Viper, if you don't use Viper you can simply get the values from the awssm.Parameters object:

• params.GetValueByName("DB_USERNAME") -> "my-database-username"

• params.GetValueByName("DB_PASSWORD") -> "my-database-password"

In our case we will load them to Viper by configuring it to expect a json document. Then we are all set; we use Viper as normal to create a Database connection.

...
viperInstance.SetConfigType(`json`)

err = viperInstance.ReadConfig(params)

if err != nil {

  log.Fatal(err)

}

database := Database{

  Username: viperInstance.GetString("DB_USERNAME"),

  Password: viperInstance.GetString("DB_PASSWORD"),

  Host: viperInstance.GetString("DB_HOST"),

  Port: viperInstance.GetInt("DB_PORT"),

}
// do something with the database connection

_ := database
...

The completed code should look like this now:

func main(){

  viperInstance := viper.New()

  viperInstance.AutomaticEnv()

  viperInstance.SetDefault("DB_PORT", "3306")

  paramstore, err := awsssm.NewParameterStore()

  if err != nil {

    log.Fatal(err)

  }

  params, err := paramstore.GetAllParametersByPath

  (viperInstance.GetString("PARAMETER_STORE_PATH"), true)

  if err != nil {

    log.Fatal(err)

  }
  

  viperInstance.SetConfigType(`json`)

  err = viperInstance.ReadConfig(params)

  if err != nil {

    log.Fatal(err)

  }

  database := Database{

    Username: viperInstance.GetString("DB_USERNAME"),

    Password: viperInstance.GetString("DB_PASSWORD"),

    Host: viperInstance.GetString("DB_HOST"),

    Port: viperInstance.GetInt("DB_PORT"),

  }
  // do something with the database connection

  _ := database

}

Summary

Storing your application secrets to AWS Parameter Store and using go-aws-ssm to manage them in your Go application enables easy integration with existing configuration tools. We are currently using this approach for Go lambdas and containerized Go applications. Happy coding!