Useful Stuff I Read This Week

  Here are 10 posts I thought were worth sharing this week.

This post covers how and why Hasura switched their CI service from CircleCI to Buildkite. They started by defining the requirements from their CI, then they evaluated different solutions, and finally introduced it in their ecosystem. Their main reason to switch CI service was cost. They reduced the cost by 50%. This required them to own some of the aspects of the CI operations. A couple of interesting things I learnt from this post:

  • Use of labels to trigger build. They used them to save costs.
  • The use of dynamic configuration. They wrote their build code in a Go program. This saved them from YAML hell. Interestingly, they use shellcheck   t   static analysis of shell script

It is all about perspective. Tech Debt brings negative emotions in people and it becomes difficult to sell it to higher management. In this post, the author suggests we reframe tech debt as tech wealth while communicating with stakeholders. Building tech wealth means getting more value out of the software we’re creating, as well as our efforts to develop and maintain it.  Author suggests two ways we can plan for tech wealth:

  • Allocate time within each planning cycle
  • Dedicate the last few cycles in a quarter

In one of the products I worked on we used to schedule 1 day per sprint (2 weeks) for paying tech debt. We had sprint demo every alternate Thursday and the next day i.e. Friday was scheduled for working on tech debt items. One problem with 1 day every sprint is that bigger items can’t be handled. We used to create stories for them and pick them as part of the sprint backlog.

Continue reading “Useful Stuff I Read This Week”

When to use JSON data type in database schema design?

Today, I was doing solution design for a system when I started to think when we should use JSON data type for columns. Coming up with the right schema design takes multiple iterations. I consider it more as an art than science. All the mainstream RDBMS have JSON data type support.

  • Postgres has JSON data type since version 9.2. The 9.2 version was released in September 2012
  • MySQL has JSON data type since version 5.7.8. The 5.7.8 version was released in August 2015
  • SQL Server has JSON data type since version SQL Server 2016. The 2016 version was released in June 2016
  • Oracle has JSON data type since version 19c. The 19c version was released in February 2019

They all support efficient insertion and querying of JSON data type. I will not compare their JSON capabilities today. Today, I want to answer a design question – when should a column have a JSON data type?

I use the JSON data type in design situations mentioned below. There could be other places as well where JSON is a suitable data type. 

  1. Dump request data that will be processed later
  2. Support extra fields
  3. One To Many Relationship where many side will not have to its own identity
  4. Key Value use case
  5. Simpler EAV design

Let’s talk about each of these use cases in more detail.

Continue reading “When to use JSON data type in database schema design?”