Month: June 2019

Issue #32: 10 Reads, A Handcrafted Weekly Newsletter For Software Developers

The time to read this newsletter is 150 minutes.

Writing may be the skill with the highest return of all – Seth Godin

  1. Undervalued Software Engineering Skills: Writing Well: 5 mins read. I echo with the author. Being a senior engineer in my organization this is one advice I usually end up giving to people. The key points from the post are:
    1. In a large engineering organization writing is the only medium that will help you propagate your message forward.
    2. You can learn to improve your writing skills. It is a learnable skill.
    3. Writing code is not the only activity in software development.
    4. When you write things down, you build better understanding of the topic. I personally find that writing helps me think clearly about a problem.
  2. Why Github used Haskell for Semantic? – 20 mins read. We need more such posts from the community. These type of posts can help developers understand how organizations take technical choices. The key lessons for me in this post are:
    1. The problem Github is trying to solve with Semantic is related to domain of programming language theory. This domain is an active research area and most of the researchers in this domain use Haskell for its brevity, power, and focus on correctness. Writing in Haskell allows us to build on top of the work of others rather than getting stuck in a cycle of reading, porting, and bug-fixing
    2. Haskell makes it nigh-impossible to build programs that contain such bugs
    3. Haskell used in industry at scale. Facebook open sourced a project called Haxi that is written in Haskell.
  3. Let’s build a SQL parser in Go!: 20 mins read. I enjoyed reading this post. It shows in a step by step manner how to write a SQL parser. The author implemented it in Go.
  4. How I decimated Postgres response times for my SaaS: 15 mins read. There are two key points in this post:
    1. You can only fix a problem if you can successfully reproduce in your local environment. I know this sound common sense but ask yourself honestly how many times you have tried solutions without reproducing the problem in a local environment only to discover that your proposed solution does not work. This happened to me this week.
    2. Composite index in PostgreSQL can help you avoid heap sort if you do order by in your query.
  5. 13 Tips for Writing a Technical Book: 15 mins read. A lot of useful advice. I wrote a similar post when I published my first book. The key points for me in this post were:
    1. Pad the timeline
    2. Schedule regular time to write: every morning, every weekend, etc
    3. Use lots of TODOs to keep track of what’s left
    4. Getting good technical editors is hard
    5. Writing is lonely
  6. Remote working – Bringing sanity to mind & lessons worth learning: 20 mins read. This post covers the other side of remote working — anxiety and depression. The article shared tips that can help.
    1. First thing you need to do is to get out of the denial mode. Mental issues can happen with anyone. Setup a weekly wellbeing check-up with yourself.
    2. Create a schedule and stick to it. Know when to stop and detach from work.
    3. Setup a separate remote working space.
    4. Limit your digital life. Talk to people
  7. Amdahl’s law: 10 mins read. Amdahl’s Law is a formula which gives the theoretical speedup in latency of the execution of a task at fixed workload that can be expected of a system whose resources are improved. In essence, it says you can’t speed up beyond the sequential part of your program irrespective of how many cores you add. Gene Amdahl’s said, If 50% of the execution time is sequential, the maximum speed up is 2, no matter how many cores you use. Good video that you can watch on Amdahl’w Law is by Professor Ben H. Juurlink.
  8. Blameless PostMortems and a Just Culture: 15 mins read. Having a “blameless” Post-Mortem process means that engineers whose actions have contributed to an accident can give a detailed account of:
    1. what actions they took at what time,
    2. what effects they observed,
    3. expectations they had,
    4. assumptions they had made,
    5. and their understanding of timeline of events as they occurred.
  9. Love DevOps? Wait until you meet SRE: 10 mins read. If you have not heard about SRE then this post will help you get started. SRE as defined by its mastermind Ben Treynor is “SRE is what happens when a software engineer is tasked with what used to be called operations”.
  10. 3 Mindfulness Rituals That Will Make You Happy – 20 mins read. You are not your thoughts.

Video of the week

Linux Tip: How to create a list of directories inside a directory and place an empty file in each of them

Today, I wanted to create a list of directories inside a directory and place an empty file in each of them. In this quick post, I will share two commands that can do that for you.

$ mkdir $(printf "%02i-day " $(seq 1 50))
$ find . -type d -exec touch {}/.gitkeep \;

You don’t have to type $. It is only to signify command-prompt.

Tools Software Engineers Should Know: 6 Things You Can Do with NetCat

In the second post of the tools series, I am going to cover netcat utility. It comes prepackaged on most of the linux distributions. The netcat utility is often referred to as a Swiss army knife of networking tools. It has a long list of features. Few of them are mentioned below.

  1. Outbound or inbound connections, TCP or UDP, to or from any ports
  2. Full DNS forward/reverse checking, with appropriate warnings
  3. Ability to use any local source port
  4. Built-in port-scanning capabilities

Continue reading “Tools Software Engineers Should Know: 6 Things You Can Do with NetCat”

Mental Models for Software Engineers: Regret Minimization Framework

I became aware of Regret Minimization Framework quite recently while reading a random post on the web. With in that timeframe it has helped me take a couple of big decisions with clarity. Had I known this earlier I could have avoided taking few bad decisions.

Regret Minimization Framework is a mental model that wants you to project into the future and look back at your decision so that you can quantify which of the options could lead to minimal regret. Then, you should go with the option that will lead to minimum regret.

Continue reading “Mental Models for Software Engineers: Regret Minimization Framework”

Issue #31: 10 Reads, A Handcrafted Weekly Newsletter For Software Developers

The time to read this newsletter is 200 minutes.

A liar will not be believed, even when he speaks the truth – Aesop

  1. How to remove duplicate lines from files keeping the original order: 15 mins read. Finally learnt something about awk. The post explains how you can remove duplicate lines in a file while preserving their order. This deduplication on steroids. It is in my todo list to learn awk one day.
  2. Google’s Chrome Becomes Web ‘Gatekeeper’ and Rivals Complain: 15 mins read. I have read this multiple times. Chrome is at the core of Google’s digital strategy. Google needs to track us to show ads and make money. This is the reason they are coming up with updated Chrome Extension API that will limit what ad blockers can do. In my view, the big problem is not Chrome or Google. We have ads because people want to earn money from their content. Google does not put ads magically; site owners add Google ad tracking scripts that share information with Google. Till the time, we don’t create a better financial model for content creators. This problem can’t be solved. Brave browser by Brendan Eich, co-founder of Mozilla and the current CEO of Brave Software Inc. is trying to do some work on it but it is still early days for it.
  3. Tests that sometimes fail: 30 mins read. Author makes following valid points:
    1. Flaky tests are useful at finding underlying flaws in our application. In some cases when fixing a flaky test, the fix is in the app, not in the test
    2. Common patterns of flaky tests
      1. Flaky tests caused by hard coded ids because they rely on database sequences
      2. Making bad assumptions about DB ordering. Result returned by SQL query is unordered.
      3. Incorrect assumptions about time
      4. Bad assumptions about the environment
    3. Mitigation patterns
      1. Run test suite in a tight loop, over and over again on a cloud server. Each time tests fail we flag them and at the end of a week of continuous running we mark flaky specs as “skipped” pending repair.
      2. One big issue with flaky tests is that quite often they are very hard to reproduce. To accelerate a repro I tend to try running a flaky test in a loop.
      3. Invest in fast test suite
      4. Add purpose built diagnostic code to debug flaky tests you can not reproduce
  4. You need neither PWA nor AMP to make your website load fast: 10 mins read. Author writes, “why was AMP needed? Well, basically Google needed to lock content providers to be served through Google Search. But they needed a good cover story for that. And they chose to promote it as a performance solution”. I kind of agree with author that AMP hurts the web community more than it helps. I have disabled AMP in my blog.
  5. Fast key-value stores: An idea whose time has come and gone: 30 mins read. Interesting paper by Google on building stateful services instead of stateless. I also went with stateful service architecture in my last application. It has its own challenges but in some cases it is the only viable option.
  6. 6 new ways to reduce your AWS bill with little effort: 10 mins read. This post can help you save some $$$ in your monthly AWS bill. The author suggests 6 ways we can reduce AWS bill. Out of the 6, I found following two ways worth a try:
    1. Use EC2 AMD instances
    2. Use VPC endpoints instead of NAT gateways
  7. Disaster Tolerance Patterns Using AWS Serverless Services: 30 mins read. Just read it if you are using AWS.
  8. How Far Out is AWS Fargate?: 15 mins read. This is a good post comparing AWS Fargate and AWS Lambda.
    1. With Lambda you pay per invocation and the price is based on the memory you allocate for your function (up to 30GB) and its execution time. The amount of compute available to your Lambda function is based on it’s memory allocation. This pricing model is ideal for workloads that have spikes and/or long periods of downtime.
    2. Fargate, on the other hand, lets you configure how many VCPUs (up to 8) and GBs of memory (up to 3GB) you want your Fargate tasks to have independently, priced by the secondrounded up to one minute.
  9. Learning to Listen to one’s own Boredom: 15 mins read. All of us need to learn to develop a ‘late style’ – ideally as early on in our lives as possible: a way of being wherein we shake off the dead hand of habit and social fear and relearn to listen to what entertains us
  10. How We Built a Content-Based Filtering Recommender System For Music with Python: 30 mins read. I love these kind of tutorial that help you learn by building an application in step by step manner. Give it a try and you will learn something about building a content-based recommender system for music.

Video of the week

Tools Software Engineers Should Know: MTR: A network diagnostic tool

We software developers are good at debugging code related issues but when it comes to issues that require fighting with infrastructure or network then we find ourselves in a difficult position. We can solve these issues if we know the right tool to use. I faced a similar position this week. I am starting a new series where every week or two I will write about a new tool that can help us debug these kind of issues.

This week I was debugging an issue where few requests to the destination server were timing out. These types of issues typically fall under networking errors and require you to use a network diagnostic tool. Most developers start diagnosing the issue using ping and traceroute tool. Both these tools are useful but you have to run them both together to debug the issue. Recently, I discovered MTR which combines ping and traceroute tools in a single tool. I found that most developers that I work with are not unaware of this tool so I decided to document it for future me and others.

mtr stands for My traceRoute. It is useful when you need to figure out number of hops to the destination server or latency at each hop. It also help you see packet loss at each hop so that you can narrow down the place where you might be facing issue. MTR collects information regarding the state, connection, and latency of the intermediate hosts. Thus giving a complete overview of the connection between two hosts on the network.

Continue reading “Tools Software Engineers Should Know: MTR: A network diagnostic tool”