Month: September 2019

MemSQL Introduction: A Hybrid transactional/analytical processing database

main-how-it-works-ecosystem-diagram

MemSQL is s fast, commercial, ANSI SQL compliant, highly scalable HTAP database. HTAP databases are those that support both OLTP and OLAP workloads. It supports ACID transactions just like a regular relational database .It also supports document and geospatial data types.

MemSQL is fast because it stores data in-memory. But, it does not mean it is not durable. It maintains a copy of data on disk as well. Transactions are committed to the transaction log on disk and later compressed into full-database snapshots. One of the main reason new databases are designed as in-memory first is because memory is getting cheaper every year. It is estimated memory is becoming cheaper 40% every year.

MemSQL has tuneable durability. You can make it fully durable or completely ephemeral. It can be sync or async.

MemSQL simplifies your architecture as you don’t have to write ETL jobs to move data from one data store to another data store. This is the biggest selling point of any HTAP database.

Continue reading “MemSQL Introduction: A Hybrid transactional/analytical processing database”

System Design: Design the Amazon Recently Viewed Items Page API

I enjoy working through system design problems. It helps me think how I will design interesting features of various systems. I will post design solutions to interesting problems.

Today, I will share how I will design Amazon recently viewed items page. You can view this page by going to https://www.amazon.com/gp/history/

To me it showed last 73 items I viewed on Amazon.com. I don’t think they are showing last N items rather they are showing items that I viewed in last X days(or months) with some max limit.

Let’s redefine problem now that we better understand it.

Design the Amazon recently viewed items page API. The recently viewed items are all the items that you viewed in the last 6 months. The max count of items could be 100.

Continue reading “System Design: Design the Amazon Recently Viewed Items Page API”

Issue #35: 10 Reads, A Handcrafted Weekly Newsletter For Software Developers

The time to read this newsletter is 130 minutes.

A busy calendar and a busy mind will destroy your ability to create anything great. – Naval Ravikant

  1. GitHub stars won’t pay your rent: 20 mins read. The key point in the post is that you should not feel bad about charging money for your work. I think we software developers have taken it too far. Most of us feel that by making our work open source we are making the world better. But, the reality is that if you loose your job and need financial support then no user of your open source project will come to help. We need to become practical and keep financial reality in mind.
  2. Building a Kubernetes platform at Pinterest: 15 mins read. A lot of things you can learn about Kubernetes from this post by Pinterest engineering team. The key points for me are:
    1. You can use CRD to define your organisation specific service. Look at PinterestService.
    2. CRD can be used as an alternative to Helm
    3. Infrastructure team has three main priorities: 1) Service Reliability 2) Developer Productivity 3) Infra Efficiency
  3. Six Shades of Coupling: 15 mins read.
  4. When Redundancy Actually Helps: 10 mins read.
  5. The (not so) hidden cost of sharing code between iOS and Android: 10 mins read. So, we have come back the full circle. Organisations are moving away from code sharing approach when building same application for different mobile platforms. I have seen multiple organisations using C++ to write share code. The use of C++ limits number of developers you can find in the market and overall slows you down. You have to build tools to support your custom journey.
  6. 3 Strategies for implementing a microservices architecture: 5 mins read. The three strategies
    1. The Strangler method
    2. The Lego strategy
    3. The nuclear option
  7. Microservices, Apache Kafka, and Domain-Driven Design: 20 mins read.
  8. Habits vs. Goals: A Look at the Benefits of a Systematic Approach to Life: 10 mins read.
  9. Building an analytics stack from scratch: 15 mins read.
  10. Cutting Through Indecision & Overthinking: 10 mins read. Take action. Half the battle is won if you get started.

Video of the week

Mental Models for Software Engineers: Improve Your Productivity by Using the Eisenhower matrix

One of the mental model that I find useful for prioritising my todo list is Eisenhower matrix named after US president Dwight D. Eisenhower. Dwight D. Eisenhower once said

I have two kinds of problems, the urgent and the important. The urgent are not important, and the important are never urgent.

Eisenhower matrix is a simple decision making tool for organising your tasks. You can use it to find the task you should act on first.

Continue reading “Mental Models for Software Engineers: Improve Your Productivity by Using the Eisenhower matrix”