Secret Management in Helm - Kubernetes

,

Secret management in Helm

Helm is a Kubernetes package manager, Helm helps developer deploy their application to Kubernetes.

Helm also provide chart as dependencies for your application at https://hub.helm.sh/

The problem with Helm is the secret variables (saved in values.yaml file) and will be passing to templates (e.g: deployment.yaml, service.yaml). And in the production environment, there are variables that you don’t want to expose, you want to encrypt these variables.

I did some research and come up with Helm Secret - A plugin for Secret management in Helm.

Helm Secret

Basically Helm Secret will integrate with SOPS to encrypt/decrypt variables file. The encryption key is created and managed either by AWS KMS, GCP KWS, Azure Key Vault or PGP.

Helm Secrets provide a wrapper in the shell that runs Helm within but wrapping secret decryption and cleaning on-the-fly, before and after Helm run.

In this example, I’ll use AWS KMS to create the encryption key.

Getting started

  • Assume I have the encryption key created from AWS KMS. The KMS ARN looks like:
1
arn:aws:kms:ap-southeast-1:xxx:key/xxx

And it will be saved to .sops.yaml file.

1
2
3
---
creation_rules:
  - kms: 'arn:aws:kms:ap-southeast-1:xxx:key/xxx'
  • Init the secrets.yaml file, this file will contain all secrets variables that we want to encrypt.
1
2
3
4
5
6
7
8
9
# secrets.yaml

postgresql:
  enabled: true
  replication:
    enabled: false
  postgresqlUsername: postgres
  postgresqlPassword: 2RpYdiDg1lv1Vu9LTm3kXD
  postgresqlDatabase: example
  • Now, encrypt time! Run the following command:
1
2
3
╰─$ helm secrets enc secrets.yaml
Encrypting secrets.yaml
Encrypted secrets.yaml
  • Look the secrets.yaml file, all variables are encrypted, and now you can commit this file
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
postgresql:
    enabled: ENC[AES256_GCM,data:Kxntww==,iv:OBlc/XxAaxvuwM22V5fxfxNyDxEqTv1RnU/pgd2kLZs=,tag:c24wK29Yg08wbe04pOLDZA==,type:bool]
    replication:
        enabled: ENC[AES256_GCM,data:OeJ04II=,iv:bB0+q0efphJZr4k2mMjIoRJFMpURk5U63LHb2/X99EE=,tag:OCimlajGFvZDOjf4+xkRqQ==,type:bool]
    postgresqlUsername: ENC[AES256_GCM,data:D14/CcA3WjY=,iv:GNEtQrDD9C7UDmiu0cluL1oauWsHKGB863044AZ+jhQ=,tag:4WQCJHZAIea9ixAHtfK5Gw==,type:str]
    postgresqlPassword: ENC[AES256_GCM,data:Wd7VEKSoqVDbLm8iLK7nO7yLKQulow==,iv:obbgZ0y9UdwxWEyy6RibPEg2IUNuiV+dYo2e+nuf0kg=,tag:nqX+OmxpZeXA+2aF3Q3M5A==,type:str]
    postgresqlDatabase: ENC[AES256_GCM,data:8ur9pqDxUA==,iv:RawVg4QWUgYrfoNgDaqzLcIl2GOQ1H6R8At+MBMIXqE=,tag:1L+THewSe4DD1CGtdUMftg==,type:str]
sops:
    kms:
    -   arn: arn:aws:kms:ap-southeast-1:xxx:key/xxx
        created_at: '2019-06-28T10:14:43Z'
        enc: AQICAHjfqxSiTEzQ0JrEoBFESmBwcr121212312TlQ8RtxsZjT+q4kdwBhYiqRdgJVuMjg0I7n0N1q0jCwnUbtSkKeGWXzo/xcl2JuICqsS5NdQhw==
    gcp_kms: []
    azure_kv: []
    lastmodified: '2019-06-28T10:14:43Z'
    mac: ENC[AES256_GCM,data:fRcbPlfZRRFcU1231231238VjVcZMOBr9K8B1k=,iv:X4X06G56xvL+PvvkVQNCDQAXzTqAYa1ro9gbQerxjNQ=,tag:IQQabaFHVcrANuHK0PyH/Q==,type:str]
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.2.0
  • Helm Secrets provides list of commands to decrypt, view, or edit the secret file
1
2
3
4
5
6
Basic commands:
  enc           Encrypt secrets file
  dec           Decrypt secrets file
  view          Print decrypted secrets file
  edit          Edit secrets file (decrypt before and encrypt after)
  clean         Delete *.yaml-dec files in the directory (recursively)

For example, to view decrypted variables, the command will be:

1
2
3
4
5
6
7
8
╰─$ helm secrets view secrets.yaml
postgresql:
    enabled: true
    replication:
        enabled: false
    postgresqlUsername: postgres
    postgresqlPassword: 2RpYdiDg1lv1Vu9LTm3kXD
    postgresqlDatabase: example

Deploy the application to Kubernetes

Use helm secrets as wrapper command when you want to deploy the application to Kubernetes

1
helm secrets upgrade "app_name" chart/path/ -f deployment/example/values.yaml -f deployment/example/secrets.yaml

In this command, I loaded normal variables in values.yaml file and encrypted variables in encrypted secrets.yaml file, helm secrets upgrade means will init/update the chart app_name.

Conclusion

Helm Secrets is really a good approach for people want to manage secret variables in Kubernetes environment. If you guys have any feedback, feel free to contact me.