# 使用 cfssl 来提高 NATS 集群的安全性

## 使用NATS Operator 来加固 Kubernetes 中的 NATS 集群

### 功能：

* Clients TLS setup
* 客户端 TLS 配置
* 通过密钥来授权基于 TLS 的证书
  * 只有更新密钥才能重载证书
* 路由TLS设置
* 为每个NATS service 发布公共IP以供外部访问

### 证书创建

### 生成CA根证书

```javascript
{
    "CN": "nats.io",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "OU": "nats.io"
        }
    ]
}
```

```bash
(
  cd certs

  # CA certs
  cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
)
```

Setup the profiles for the Root CA, we will have 3 main profiles: one for the clients connecting, one for the servers, and another one for the full mesh routing connections between the servers.

CA根证书有三个主要的配置文件:一个用于客户端的连接，一个用于服务端，一个用于服务间的网格路由连接。

```bash
{
    "signing": {
        "default": {
            "expiry": "43800h"
        },
        "profiles": {
            "server": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            },
            "client": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "route": {
                "expiry": "43800h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}
```

### 生成 NATS service 证书

首先我们生成server的证书。

```
{
    "CN": "nats.io",
    "hosts": [
        "localhost",
        "*.nats-cluster.default.svc",
        "*.nats-cluster-mgmt.default.svc",
        "nats-cluster",
        "nats-cluster-mgmt",
        "nats-cluster.default.svc",
        "nats-cluster-mgmt.default.svc",
        "nats-cluster.default.svc.cluster.local",
        "nats-cluster-mgmt.default.svc.cluster.local",
        "*.nats-cluster.default.svc.cluster.local",
        "*.nats-cluster-mgmt.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "OU": "Operator"
        }
    ]
}
```

```bash
(
  # Generating the peer certificates
  cd certs
  cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server.json | cfssljson -bare server
)
```

### Generating the NATS server routes certs 生成NATS server 路由证书

我们当然也需要为网格路由配置TLS。

```javascript
{
    "CN": "nats.io",
    "hosts": [
        "localhost",
        "*.nats-cluster.default.svc",
        "*.nats-cluster-mgmt.default.svc",
        "nats-cluster",
        "nats-cluster-mgmt",
        "nats-cluster.default.svc",
        "nats-cluster-mgmt.default.svc",
        "nats-cluster.default.svc.cluster.local",
        "nats-cluster-mgmt.default.svc.cluster.local",
        "*.nats-cluster.default.svc.cluster.local",
        "*.nats-cluster-mgmt.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "OU": "Operator"
        }
    ]
}
```

```bash
# Generating the peer certificates
(
  cd certs
  cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=route route.json | cfssljson -bare route
)
```

### 为客户端生成证书(CNCF && ACME)

```javascript
{
    "CN": "nats.io",
    "hosts": [""],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "OU": "CNCF"
        }
    ]
}
```

```bash
(
  cd certs
  # Generating NATS client certs
  cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
)
```

### Kubectl Create

```
cd certs kubectl create secret generic nats-tls-example --from-file=ca.pem --from-file=server-key.pem --from-file=server.pem kubectl create secret generic nats-tls-routes-example --from-file=ca.pem --from-file=route-key.pem --from-file=route.pem kubectl create secret generic nats-tls-client-example --from-file=ca.pem --from-file=client-key.pem --from-file=client.pem
```

### 创建授权密钥

```javascript
{
  "users": [
    { "username": "CN=nats.io,OU=ACME" },
    { "username": "CN=nats.io,OU=CNCF",
      "permissions": {
    "publish": ["hello.*"],
    "subscribe": ["hello.world"]
      }
    }
  ],
  "default_permissions": {
    "publish": ["SANDBOX.*"],
    "subscribe": ["PUBLIC.>"]
  }
}
```

```bash
kubectl create secret generic nats-tls-users --from-file=users.json
```

#### 创建支持TLS的集群

```bash
echo '
apiVersion: "nats.io/v1alpha2"
kind: "NatsCluster"
metadata:
  name: "nats-cluster"
spec:
  size: 3

  # Using custom edge nats server image for TLS verify and map support.
  serverImage: "wallyqs/nats-server"
  version: "edge-2.0.0-RC5"

  tls:
    enableHttps: true

    # Certificates to secure the NATS client connections:
    serverSecret: "nats-tls-example"

    # Certificates to secure the routes.
    routesSecret: "nats-tls-routes-example"

  auth:
    tlsVerifyAndMap: true
    clientsAuthSecret: "nats-tls-users"

    # How long to wait for authentication
    clientsAuthTimeout: 5

  pod:
    # To be able to reload the secret changes
    enableConfigReload: true
    reloaderImage: connecteverything/nats-server-config-reloader

    # Bind the port 4222 as the host port to allow external access.
    enableClientsHostPort: true

    # Initializer container that resolves the external IP from the
    # container where it is running.
    advertiseExternalIP: true

    # Image of container that resolves external IP from K8S API
    bootconfigImage: "wallyqs/nats-boot-config"
    bootconfigImageTag: "0.5.0"

  # Service account required to be able to find the external IP
  template:
    spec:
      serviceAccountName: "nats-server"
' | kubectl apply -f -
```

### 使用证书创建应用

####  添加一个使用证书的pod

**Development**

```go
package main

import (
    "encoding/json"
    "flag"
    "fmt"
    "log"
    "time"

    "github.com/nats-io/go-nats"
    "github.com/nats-io/nuid"
)

func main() {
    var (
        serverList     string
        rootCACertFile string
        clientCertFile string
        clientKeyFile  string
    )
    flag.StringVar(&serverList, "s", "tls://nats-1.nats-cluster.default.svc:4222", "List of NATS of servers available")
    flag.StringVar(&rootCACertFile, "cacert", "./certs/ca.pem", "Root CA Certificate File")
    flag.StringVar(&clientCertFile, "cert", "./certs/client.pem", "Client Certificate File")
    flag.StringVar(&clientKeyFile, "key", "./certs/client-key.pem", "Client Private key")
    flag.Parse()

    log.Println("NATS endpoint:", serverList)
    log.Println("Root CA:", rootCACertFile)
    log.Println("Client Cert:", clientCertFile)
    log.Println("Client Key:", clientKeyFile)

    // Connect options
    rootCA := nats.RootCAs(rootCACertFile)
    clientCert := nats.ClientCert(clientCertFile, clientKeyFile)
    alwaysReconnect := nats.MaxReconnects(-1)

    var nc *nats.Conn
    var err error
    for {
        nc, err = nats.Connect(serverList, rootCA, clientCert, alwaysReconnect)
        if err != nil {
            log.Printf("Error while connecting to NATS, backing off for a sec... (error: %s)", err)
            time.Sleep(1 * time.Second)
            continue
        }
        break
    }

    nc.Subscribe("discovery.*.status", func(m *nats.Msg) {
        log.Printf("[Received on %q] %s", m.Subject, string(m.Data))
    })

    discoverySubject := fmt.Sprintf("discovery.%s.status", nuid.Next())
    info := struct {
        InMsgs        uint64   `json:"in_msgs"`
        OutMsgs       uint64   `json:"out_msgs"`
        Reconnects    uint64   `json:"reconnects"`
        CurrentServer string   `json:"current_server"`
        Servers       []string `json:"servers"`
    }{}

    for range time.NewTicker(1 * time.Second).C {
        stats := nc.Stats()
        info.InMsgs = stats.InMsgs
        info.OutMsgs = stats.OutMsgs
        info.Reconnects = stats.Reconnects
        info.CurrentServer = nc.ConnectedUrl()
        info.Servers = nc.Servers()
        payload, err := json.Marshal(info)
        if err != nil {
            log.Printf("Error marshalling data: %s", err)
        }
        err = nc.Publish(discoverySubject, payload)
        if err != nil {
            log.Printf("Error during publishing: %s", err)
        }
        nc.Flush()
    }
}
```

```
FROM golang:1.11.0-alpine3.8 AS builder
COPY . /go/src/github.com/nats-io/nats-kubernetes/examples/nats-cluster-routes-tls/app
WORKDIR /go/src/github.com/nats-io/nats-kubernetes/examples/nats-cluster-routes-tls/app
RUN apk add --update git
RUN go get -u github.com/nats-io/go-nats
RUN go get -u github.com/nats-io/nuid
RUN CGO_ENABLED=0 go build -o nats-client-app -v -a ./client.go

FROM scratch
COPY --from=builder /go/src/github.com/nats-io/nats-kubernetes/examples/nats-cluster-routes-tls/app/nats-client-app /nats-client-app
ENTRYPOINT ["/nats-client-app"]
```

```bash
docker build . -t wallyqs/nats-client-app
docker run wallyqs/nats-client-app
docker push wallyqs/nats-client-app
```

#### Pod spec

```
echo ' apiVersion: apps/v1beta2 kind: Deployment

## The name of the deployment

metadata: name: nats-client-app

spec:

## This selector has to match the template.metadata.labels section

## which is below in the PodSpec

selector: matchLabels: name: nats-client-app

## Number of instances

replicas: 1

## PodSpec

template: metadata: labels: name: nats-client-app spec: volumes:

* name: "client-tls-certs"

  secret:

    secretName: "nats-tls-client-example"

  containers:

* name: nats-client-app

  command: \["/nats-client-app", "-s", "tls://nats-cluster.default.svc:4222", "-cacert", '/etc/nats-client-tls-certs/ca.pem', '-cert', '/etc/nats-client-tls-certs/client.pem', '-key', '/etc/nats-client-tls-certs/client-key.pem'\]

  image: wallyqs/nats-client-app:latest

  imagePullPolicy: Always

  volumeMounts:

  * name: "client-tls-certs"

    mountPath: "/etc/nats-client-tls-certs/"

    ' \| kubectl apply -f -
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://nats.golang.im/nats-on-kubernetes/operator-tls-setup-with-cfssl.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.