Software Architecture Diagrams as Code


I love the idea of everything as code. It enables use of version control for all aspects of software development. These days we use everything as code approach for documentation, infrastructure provisioning, deployment automation, continuous integration build jobs, pull request and issue templates, and many other things. The beauty of this approach is that it enables us to use the same tools as we use use for software development.

One aspect of the software development where we are lacking the as code approach is in creating architecture diagrams.

As software engineers and architects we all have to create architecture diagrams to convey our ideas with other fellow engineers. For this discussion it does not matter whether you create architecture diagrams before or after or during development. The fact of the matter is that we all have to create diagrams in our software engineering life.

Since the last couple of years I was on the look out for a free and open-source diagram tool that allow me to save the diagram source in the version control. One tool that I used heavily in the last couple of years draw.io. We can use either the web or desktop editor to create diagrams. The diagram source in XML format can be backed to Google drive or Git repository. This tool has been great help for me and I really love using it. But to me draw.io is still not the pure as code tool. You use it like just any other diagram tool. It does not have code first approach. Code is the second class citizen in draw.io.

Recently I discovered a pure diagram as code tool called Diagrams. With Diagrams, you create diagrams by writing Python code. You can use it to build cloud and on-premise architecture diagrams. It supports AWS, Azure, GCP, Kubernetes, Alibaba Cloud, and Oracle Cloud.

Prerequisites

To get started with Diagrams, you need to run following commands. The below are instructions for running it on Mac. Create a directory where you want to store diagrams and navigate to it.

$ brew install graphviz
$ virtualenv -p python3 venv
$ source venv/bin/activate
$ pip install diagrams

You don’t have to type $. It signifies command prompt.

Most tools stand on the shoulders of other powerful tools. Diagrams uses Graphviz its backbone. In the commands shown above, we first installed Graphviz. It is an open source graph visualisation software that makes use of abstract graphs and network to represent structural information.

Next, we installed diagrams library. You will notice that I am using Python Virtualenv to avoid polluting my global Python installation.

Creating Diagrams

Now that we have met our prerequisites. Let’s learn how to use Diagrams by creating a few diagrams.

I will use examples mentioned in the talk: Scaling Up to Your First 10 Million Users.

It is one of the best conference talk on building scalable systems I have seen. Since this talk is about AWS we will be building diagrams that use AWS services.

Please note that I will be deviating a bit from the architecture presented in the video. Also, I am still learning Diagrams library so the code shown below might not be the best.

Diagram 1: Application built for 1 user

In the talk author starts by defining architecture for a single user application. For a single user application, we can use following:

  1. AWS Lightsail: Virtual private server
  2. VPC
  3. Database and application server running on single machine
  4. Elastic IP
  5. Amazon Route 53

The Python code for the diagram is shown below.

from diagrams import Cluster, Diagram, Edge

from diagrams.onprem.client import User
from diagrams.aws.compute import Lightsail
from diagrams.aws.network import Route53

with Diagram("Single User Architecture", show=True, direction="TB"):
    with Cluster(""):
        route_53 = Route53("dns")
        with Cluster("AWS"):
            route_53 - Edge(label="Elastic IP", color="orange",
                            forward=True) >> Lightsail("Lightsail Server")
        User("User") >> route_53

The above code will produce following diagram.

single_user_architecture

Diagram 2: Users between 1 and 100

Now, we will install database on its own server. We will use AWS RDS service.

from diagrams import Cluster, Diagram

from diagrams.onprem.client import User
from diagrams.aws.compute import EC2
from diagrams.aws.network import Route53
from diagrams.aws.database import RDS

with Diagram("100 User Architecture", show=True):
    route_53 = Route53("dns")
    with Cluster("AWS"):
        route_53 >> EC2("Web Server") >> RDS("Database")
    User("User") >> route_53

The above code will produce following diagram.

100_user_architecture

Diagram 3: Between 100 users and 1000

Add a cache and use bigger machines.

from diagrams import Cluster, Diagram

from diagrams.onprem.client import User
from diagrams.aws.compute import EC2
from diagrams.aws.network import Route53, ElasticLoadBalancing
from diagrams.aws.database import RDS, ElastiCache

with Diagram("1000 User Architecture", show=True):
    with Cluster("AWS"):
        route_53 = Route53("dns")
        lb = ElasticLoadBalancing("lb")
        vm_group = [EC2("Web1"), EC2("Web2")]
        cache = ElastiCache("Redis Cache")
        route_53 >> lb >> vm_group >> cache >> RDS("Database")

    User("User") >> route_53

The above will create following diagram.

1000_user_architecture

Diagram 4: Between 1000 and 10000 users

Now, we will use running application instances in multiple AZs. Also, we will use horizontally scalable database Aurora in multiple AZ. We are rendering static assess using Cloud Front and S3

The Python code is shown below.

from diagrams import Cluster, Diagram

from diagrams.onprem.client import User
from diagrams.aws.compute import EC2
from diagrams.aws.network import Route53, ElasticLoadBalancing, Cloudfront
from diagrams.aws.database import Aurora, ElastiCache
from diagrams.aws.storage import S3

with Diagram("10000 User Architecture", show=True):
    with Cluster("AWS"):
        route_53 = Route53("DNS")
        cloud_front = Cloudfront("CDN")
        s3 = S3("S3 Object Storage")
        lb = ElasticLoadBalancing("LB")
        cache = ElastiCache("Redis Cache")
        aurora_db = Aurora("Aurora DB")

        with Cluster("Availablity Zone 1"):
            vm_group1 = [EC2("Web1"), EC2("Web2")]
            lb >> vm_group1 >> cache >> aurora_db
        with Cluster("Availablity Zone 2"):
            vm_group2 = [EC2("Web3"), EC2("Web4")]
            lb >> vm_group2  >> cache >> aurora_db

        route_53 >> lb
        route_53 >> cloud_front >> s3

    User("User") >> route_53

10000_user_architecture

Conclusion

In this blog we covered how to use Diagrams library to create diagrams by writing Python code. I think Diagrams is a good start in the right direction. It still does not give all the freedom and you will be constrained by it. I will keep following this library and hopefully contribute back.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s