Talk: Container Performance Analysis

Today, I watched DockerCon 2017 talk on Container Performance Analysis. Talk is given by Brendan Gregg, Senior Performance Architect at Netflix. In his talk, he shares various linux tools that can help you understand performance of your container platform. It is a great talk for anyone trying to do performance analysis of containers. In one of his slides, he shared 10 tools that he will use to start the investigation.

  1. uptime to check load averages
  2. dmesg | tail  to check kernel errors
  3. vmstat 1 to see overall stats by time
  4. mpstat -P ALL 1 to check CPU balance
  5. pidstat 1 to check process usage
  6. iostat -xz 1 to disk I/O
  7. free -m to check memory usage
  8. sar -n DEV 1 to check network I/O
  9. sar -n TCP, ETCP 1 to view TCP stats
  10. top for overview

The talk goes deeper into how to use different tools to understand performance characteristics of the container platforms.

Below are some of the main points that I wrote down:

  1. Netflix has their container platform Titus. It does scheduling and container execution. Titus talks to AWS EC2.
  2. Netflix has more than million containers, running on 25 large EC2 instance
  3. They uses containers for services, batch, queued worked model
  4. Namespace limits visibility
  5. Control group limits what you can use — cpuset, device, memory, block io
  6. Combination of namespace and cgroup is called container
  7. Container’s CPU limit = 100% x container’s share/total busy shares .This let’s container use other tenant’s idle CPU (aka bursting) when available
  8. Container’s minimum CPU limit = 100% x container’s shares/total allocated shares. This is minimum a container will have when all containers are busy doing their job.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: