The time to read this newsletter is 160 minutes.
We change our behavior when the pain of staying the same becomes greater than the pain of changing. Consequences give us the pain that motivates us to change. — Henry Cloud
- Advertising is a cancer on society: 20 mins read. This is a long read. Author makes many valid points on why Advertising should be consider cancer. Advertising is a cancer because it has symptoms mentioned below.
- Privacy violations
- Outrage-inducing news reporting
- Decaying and ephemeral Internet services
- Some items from my “reliability list”: 15 mins read. This post make thoughtful points that software architects should keep in mind while designing or reviewing systems.
- Can you handle rollbacks?
Are new states forward compatible? This is related to Postel’s law
- > Be conservative in what you do, be liberal in what you accept from others
Do you use a strong data exchange format like Protobuf or Thrift?
Why should use JSON as data exchange format between systems?
How I built a spreadsheet app with Python to make data science easier – 15 mins read. One of the cool open source projects that I have discovered in recent times.
Announcing PartiQL: One query language for all your data – 20 mins read. Looks like finally we have found a way to standardise on SQL for working across different data storage solutions be it RDBMS or NoSQL or File based. PartiQL extends SQL by adding minimal extensions required for working with different data models. SQL won! Like it or not SQL is still the best and most powerful query language.
Parallelism in PostgreSQL – 15 mins read. The post covers how modern Postgres implements parallelism for sequential scans, aggregations, and B-tree scans.
Who Actually Feels Satisfied About Money? – 20 mins read. The post makes a good point on anxiety people have regarding money. More money does not always translate to more happiness. It’s not just how much you have — it’s what you do with it.
Top Seven Myths of Robust Systems – 15 mins read. The number one myth we hear out in the field is that if a system is unreliable, we can fix that with redundancy. In some cases, redundant systems do happen to be more robust. In others, this is demonstrably false. It turns out that redundancy is often orthogonal to robustness, and in many cases it is absolutely a contributing factor to catastrophic failure. The problem is, you can’t really tell which of those it is until after an incident definitively proves it’s the latter.
Safely Rewriting Mixpanel’s Highest Throughput Service in Golang – 15 mins read. This post covers how Mixpanel made use of Diffy to safely migrate high throughput service from Python to Golang. Diffy is a service that accepts HTTP requests, and forwards them to two copies of an existing HTTP service and one copy of a candidate HTTP service.
The Business Executive’s Guide to Kubernetes – 10 mins read. A lot of useful advice on Kubernetes. The key points for me are:
- Stateful data is hard. Don’t try to reinvent AWS RDS. Stateful sets have limitations.
- Upgrading Kubernetes is hard. The advice is to run more than Kubernetes cluster in production.
- Managed Kubernetes does not take away all the problems.
- When a rewrite isn’t: rebuilding Slack on the desktop – 15 mins read. The approach used was at once incremental and all-encompassing, rewriting a piece at a time into a gradually growing “modern” section of the application that utilized React and Redux. And the results? 50% reduction of memory use and 33% improvement in load time
Video of the week
The time to read this newsletter is 150 minutes.
Writing may be the skill with the highest return of all – Seth Godin
- Undervalued Software Engineering Skills: Writing Well: 5 mins read. I echo with the author. Being a senior engineer in my organization this is one advice I usually end up giving to people. The key points from the post are:
- In a large engineering organization writing is the only medium that will help you propagate your message forward.
- You can learn to improve your writing skills. It is a learnable skill.
- Writing code is not the only activity in software development.
- When you write things down, you build better understanding of the topic. I personally find that writing helps me think clearly about a problem.
- Why Github used Haskell for Semantic? – 20 mins read. We need more such posts from the community. These type of posts can help developers understand how organizations take technical choices. The key lessons for me in this post are:
- The problem Github is trying to solve with Semantic is related to domain of programming language theory. This domain is an active research area and most of the researchers in this domain use Haskell for its brevity, power, and focus on correctness. Writing in Haskell allows us to build on top of the work of others rather than getting stuck in a cycle of reading, porting, and bug-fixing
- Haskell makes it nigh-impossible to build programs that contain such bugs
- Haskell used in industry at scale. Facebook open sourced a project called Haxi that is written in Haskell.
- Let’s build a SQL parser in Go!: 20 mins read. I enjoyed reading this post. It shows in a step by step manner how to write a SQL parser. The author implemented it in Go.
- How I decimated Postgres response times for my SaaS: 15 mins read. There are two key points in this post:
- You can only fix a problem if you can successfully reproduce in your local environment. I know this sound common sense but ask yourself honestly how many times you have tried solutions without reproducing the problem in a local environment only to discover that your proposed solution does not work. This happened to me this week.
- Composite index in PostgreSQL can help you avoid heap sort if you do order by in your query.
- 13 Tips for Writing a Technical Book: 15 mins read. A lot of useful advice. I wrote a similar post when I published my first book. The key points for me in this post were:
- Pad the timeline
- Schedule regular time to write: every morning, every weekend, etc
- Use lots of TODOs to keep track of what’s left
- Getting good technical editors is hard
- Writing is lonely
- Remote working – Bringing sanity to mind & lessons worth learning: 20 mins read. This post covers the other side of remote working — anxiety and depression. The article shared tips that can help.
- First thing you need to do is to get out of the denial mode. Mental issues can happen with anyone. Setup a weekly wellbeing check-up with yourself.
- Create a schedule and stick to it. Know when to stop and detach from work.
- Setup a separate remote working space.
- Limit your digital life. Talk to people
- Amdahl’s law: 10 mins read. Amdahl’s Law is a formula which gives the theoretical speedup in latency of the execution of a task at fixed workload that can be expected of a system whose resources are improved. In essence, it says you can’t speed up beyond the sequential part of your program irrespective of how many cores you add. Gene Amdahl’s said, If 50% of the execution time is sequential, the maximum speed up is 2, no matter how many cores you use. Good video that you can watch on Amdahl’w Law is by Professor Ben H. Juurlink.
- Blameless PostMortems and a Just Culture: 15 mins read. Having a “blameless” Post-Mortem process means that engineers whose actions have contributed to an accident can give a detailed account of:
- what actions they took at what time,
- what effects they observed,
- expectations they had,
- assumptions they had made,
- and their understanding of timeline of events as they occurred.
- Love DevOps? Wait until you meet SRE: 10 mins read. If you have not heard about SRE then this post will help you get started. SRE as defined by its mastermind Ben Treynor is “SRE is what happens when a software engineer is tasked with what used to be called operations”.
- 3 Mindfulness Rituals That Will Make You Happy – 20 mins read. You are not your thoughts.
Video of the week
The time to read this newsletter is 200 minutes.
A liar will not be believed, even when he speaks the truth – Aesop
- How to remove duplicate lines from files keeping the original order: 15 mins read. Finally learnt something about awk. The post explains how you can remove duplicate lines in a file while preserving their order. This deduplication on steroids. It is in my todo list to learn awk one day.
- Google’s Chrome Becomes Web ‘Gatekeeper’ and Rivals Complain: 15 mins read. I have read this multiple times. Chrome is at the core of Google’s digital strategy. Google needs to track us to show ads and make money. This is the reason they are coming up with updated Chrome Extension API that will limit what ad blockers can do. In my view, the big problem is not Chrome or Google. We have ads because people want to earn money from their content. Google does not put ads magically; site owners add Google ad tracking scripts that share information with Google. Till the time, we don’t create a better financial model for content creators. This problem can’t be solved. Brave browser by Brendan Eich, co-founder of Mozilla and the current CEO of Brave Software Inc. is trying to do some work on it but it is still early days for it.
- Tests that sometimes fail: 30 mins read. Author makes following valid points:
- Flaky tests are useful at finding underlying flaws in our application. In some cases when fixing a flaky test, the fix is in the app, not in the test
- Common patterns of flaky tests
- Flaky tests caused by hard coded ids because they rely on database sequences
- Making bad assumptions about DB ordering. Result returned by SQL query is unordered.
- Incorrect assumptions about time
- Bad assumptions about the environment
- Mitigation patterns
- Run test suite in a tight loop, over and over again on a cloud server. Each time tests fail we flag them and at the end of a week of continuous running we mark flaky specs as “skipped” pending repair.
- One big issue with flaky tests is that quite often they are very hard to reproduce. To accelerate a repro I tend to try running a flaky test in a loop.
- Invest in fast test suite
- Add purpose built diagnostic code to debug flaky tests you can not reproduce
- You need neither PWA nor AMP to make your website load fast: 10 mins read. Author writes, “why was AMP needed? Well, basically Google needed to lock content providers to be served through Google Search. But they needed a good cover story for that. And they chose to promote it as a performance solution”. I kind of agree with author that AMP hurts the web community more than it helps. I have disabled AMP in my blog.
- Fast key-value stores: An idea whose time has come and gone: 30 mins read. Interesting paper by Google on building stateful services instead of stateless. I also went with stateful service architecture in my last application. It has its own challenges but in some cases it is the only viable option.
- 6 new ways to reduce your AWS bill with little effort: 10 mins read. This post can help you save some $$$ in your monthly AWS bill. The author suggests 6 ways we can reduce AWS bill. Out of the 6, I found following two ways worth a try:
- Use EC2 AMD instances
- Use VPC endpoints instead of NAT gateways
- Disaster Tolerance Patterns Using AWS Serverless Services: 30 mins read. Just read it if you are using AWS.
- How Far Out is AWS Fargate?: 15 mins read. This is a good post comparing AWS Fargate and AWS Lambda.
- With Lambda you pay per invocation and the price is based on the memory you allocate for your function (up to 30GB) and its execution time. The amount of compute available to your Lambda function is based on it’s memory allocation. This pricing model is ideal for workloads that have spikes and/or long periods of downtime.
- Fargate, on the other hand, lets you configure how many VCPUs (up to 8) and GBs of memory (up to 3GB) you want your Fargate tasks to have independently, priced by the secondrounded up to one minute.
- Learning to Listen to one’s own Boredom: 15 mins read. All of us need to learn to develop a ‘late style’ – ideally as early on in our lives as possible: a way of being wherein we shake off the dead hand of habit and social fear and relearn to listen to what entertains us
- How We Built a Content-Based Filtering Recommender System For Music with Python: 30 mins read. I love these kind of tutorial that help you learn by building an application in step by step manner. Give it a try and you will learn something about building a content-based recommender system for music.
Video of the week
The time to read this newsletter is 175 minutes.
Do every act of your life as though it were the very last act of your life – Marcus Aurelius
- Learn more programming languages, even if you won’t use them: 10 mins read. I first got this advice few years back when I watched a two minute video by Bjarne Stroustrup, creator of C++. He recommended you should not call yourself programmer if you only know one programming language. The magic number he mentioned in the video was 5. This post also makes the same point. Different programming languages are good at different things. Every programming language makes a tradeoff. They help you think about a problem in different way. I got hang of functional programming once I learnt Scheme basics. I try to learn a new programming language every couple of years. I need to start using them in my side projects.
Announcing AMP Real URL: 20 mins read. In case you are not aware, AMP stands for Accelerated Mobile Pages. AMP is an open source standard led by Google that helps speed up access to websites by caching the content near to user. This is good for readers but for content producers there were few issues. The biggest issue with AMP is that rendered webpage has a URL starting with
https://google.com/amp/. Users have become used to looking at the navigation bar in a web browser to see what web site they are visiting. The AMP cache breaks that experience. In this post by Cloudflare folks authors talk about how they fixed the real origin URL problem with AMP using web packaging and Cloudflare workers.
Infrastructure as Code, Part One: 15 mins read. This is an introductory read on infrastructure as code. If you are not aware of it then you should give it a read. It is a nicely written introduction to IaC.
When rules don’t apply: 30 mins read. This is a 30 mins video that talks about how executives at Apple, Google, eBay, Intuit, and other big tech companies conspire against their own employees by secretly agreeing among themselves not to hire each other employees. Tech companies treat their employees as their assets and cheat them.
Designing a modern serverless application with AWS Lambda and AWS Fargate: 20 mins read. A lot of good ideas in this post on how to build modern applications. The key points for me in this post are:
- You should different compute services based on the use case. The post talks about why author used both AWS Lambda and AWS Fargate. For short computation jobs use lambda and for long compute jobs that have no designated end use AWS Fargate.
- Give a thought on isolation model when deciding which compute service to use. AWS Lambda compute instances are isolated from each other so if one rogue your application will not suffer.
- When you are building a side project or building MVP for your startup your goal should be to minimise maintenance and operation tasks. Serverless services help you do that.
- AWS CDK is a service that allows you to write IaC in your own preferred language.
- Thundering Herds & Promises: 10 mins read. I love this kind of posts which share how team solved a real-world technical problem. This post covers how Instagram solved the thundering herd problem with their cache using the promises. The below explains explains what thundering herd problem means
> If your cache is hit with 100 concurrent requests, then, since the cache is empty, all of them will get a cache-miss at the one moment, resulting in 100 individual requests to the backend. If the backend is unable to handle this surge of concurrent requests (ex: capacity constraints), additional problems arise. This is what’s sometimes called a thundering herd.
The Good and the Bad of Google Cloud Run: 10 mins read. The key point made in this post is that Google Cloud Run is not FaaS. Google Cloud Run allows developers to push container images with HTTP server to GCP and GCP takes care of running them at scale. If you have build pure serverless application you will know that pure serverless apps architecture is event-driven service-full architecture. This forces developers to think about applications in a different way. According to author, Cloud Run is providing a safety blanket for developers intimidated by the paradigm shift of FaaS and service-full architecture.
Azure Cosmos DB: Microsoft’s Cloud-Born Globally Distributed Database: 20 mins read.This is a detailed explanation of Azure’s Cosmos DB internals. This article was too technical and detailed for me. I will try to re-read it again to better grasp the underlying details of Cosmos DB.
How to Improve Your Memory (Even if You Can’t Find Your Car Keys): 10 mins read. This post by Adam Grant talks about how to improve your retention power. The key points are:
- Take rest after learning a new concept.
- Don’t re-read stuff
- Try to do a small quiz on what you have learnt or try to explain it to someone
- I also apply similar technique in my newsletter by summarising what I have learnt from a post in my own words.
- An Overview of Go’s Tooling: 30 mins read. This is the post that you should bookmark if you are a Go developer. The post covers most the Go tools a developers need to interact with. I wish more such posts should be written for other languages as well.
Video for this week:
The time to read this newsletter is 175 minutes.
Good friends, good books, and a sleepy conscience: this is the ideal life – Mark Twain
Continue reading “Issue #28: 10 Reads, A Handcrafted Weekly Newsletter For Software Developers”
The time to read this newsletter is 180 minutes.
Wealth is the ability to fully experience life. — Henry David Thoreau
- Don’t get clever with login forms: 10 mins read. This post points to a valid concern related to cleverness of login forms. Author through a set of examples explain why clever login forms end up confusing users. Another example of clever login experience that author does not cover is https://login.microsoftonline.com . I agree with author recommendations for login page:
- Have a dedicated page for login
- Expose all required fields
- Keep all fields on one page
- Don’t get fancy.
- Why Google Needed a Graph Serving System: 30 mins read. In this post, author shares his story of building a distributed graph database that can answer queries with relationship. The post goes over various Graph based systems developed at Google and why Google failed to build a distributed Graph database that does not suffer from depth join problem. This post highlights an interesting point related to Google’s struggle to build innovative solution because of their internal politics. Building a distribued graph database that does not suffer from depth join problem is a herculean task. Dgraph an open source database developed by the author along with others in community is trying to build such a system.
- You probably don’t need a single-page application: 10 mins read. I agree in entirety with the author that best solution to build web application is somewhere in middle i.e. building hybrid apps. Build SPA only for parts where you need rich interaction and keep most other pages server rendered.
- Google wants Cloud Services Platform to Borg your datacenter: 20 mins read. This post gives insight into why Google made the move to build and open source Kubernetes. Google knew they are going to have a hard time beating AWS and Azure. So, they built and released Kubernetes and hoped it becomes a successful project with big community. This means cloud just became an implementation detail and most big enterprises started considering Kubernetes as a choice of softwaere to build a modern hybrid datacenter. Google’s Cloud Service Platform(CSP) will give enterprises a hardened Kubernetes, Istio, Knative software distribution. CSP is going to be a game changer for Google Cloud. Also, many OpenShift users might consider going for CSP. Interesting time ahead!
- Four Techniques Serverless Platforms Use to Balance Performance and Cost: 30 mins read. This is the best article I have read on Serverless. It starts by helping reader understand architecture of Serverless platform and then it talks about elephant in the room — cold start problem associated with Serverless platforms. The article covers four techniques that is employed by different Serverless platforms to overcome cold start issue. The techniques mentioned in the post are following:
- Function resource sharing
- Function resource pooling
- Function prefetching
- Function prewarming.
- Lessons from 6 software rewrite stories: 20 mins read. Another amazing read for this week. This post through real examples explain when it is fine to rewrite software. If you are building software for long, you will have come across advice by Joel Spolsky that rewriting software is the single worst strategic mistake that a software company can make. The post author tells the other side of the story in this post. The key take away from the post is
- Once you’ve learned enough that there’s a certain distance between the current version of your product and the best version of that product you can imagine, then the right approach is not to replace your software with a new version, but to build something new next to it — without throwing away what you have.
- How to build a distributed throttling system with Nginx + Lua + Redis: 15 mins read. This post covers how to build API rate limiting system with Nginx, Lua, and Redis. Instructions mentioned in the post are clear and to the point.
- Monte Carlo Simulation with Python: 20 mins read. The post explains Monte Carlo simulation using a simple but realisitic example. As per wikipedia,
- Monte Carlo methods are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. Their essential idea is using randomness to solve problems that might be deterministic in principle. They are often used in physical and mathematical problems and are most useful when it is difficult or impossible to use other approaches. Monte Carlo methods are mainly used in three problem classes: optimization, numerical integration, and generating draws from a probability distribution.
- 5 Ways To Process Feedback At Work Without Triggering A Stress Response: 10 mins read. This post covers an important aspect of professional life — taking feedback. The author suggests following:
- Keep an open mind about receiving feedback. Focus on how your work can be improved with some extra perspective.
- Don’t respond right away, take a few seconds to really process the feedback. You can assess rationally and logically, without undue emotion.
- Make sure you understand the feedback. In cases where you don’t, ask questions! The feedback giver should be happy to discuss specific points deeper to help clarify their suggestions.
- Be humble and gracious! Let them know you appreciate that they gave their time and energy to help make you more successful.
- Don’t let constructive criticism go in one ear and out the other. Take what you hear, implement it, and follow-up.
- How to Organize your Monolith Before Breaking it into Services: 15 mins read. This post talks about an intermediary stage between monolithic and microservices – a monolithic organized by domain without the entanglement or fragility of our original codebase. I agree with author in entirety that we should start with monolithic and modularise applications based on sub domains by applying DDD principles. If required in future, we can easily make these subdomain functional modules to services. It is great to read post like this as they provide valuable information that is usually missing in most posts found on the web.