Issue #29: 10 Reads, A Handcrafted Weekly Newsletter For Software Developers

The time to read this newsletter is 180 minutes.

Quality is not an act, it is a habit – Aristotle

  1. Your boss is 90% of the ‘Employee Experience’. Nothing else comes close.10 mins read.
  2. How NOT to hire a software engineer10 mins read.
  3. How We Used WebAssembly To Speed Up Our Web App By 20X20 mins read.
  4. Why We Moved from Heroku to Google Kubernetes Engine15 mins read.
  5. We deployed Envoy Proxy to make Monzo faster20 mins read.
  6. Achieving consistency where distributed transactions have failed: 30 mins read.
  7. Scaling Elasticsearch Part 1: How to Speed Up Indexing: 15 mins read.
  8. Five Super Helpful Rust Things That Nobody Told You About: 10 mins read.
  9. Linkerd v2: How Lessons from Production Adoption Resulted in a Rewrite of the Service Mesh: 30 mins read.
  10. 5 Things to Stop Doing When You’re Struggling and Feeling Drained: 20 mins read.

Introducing Chaos Engineering to an Organization

This post explains my learning on how to introduce Chaos Engineering to an organisation. This is based on my experience of re-architecting monolithic application to Microservices based architecture. Microservices architecture style structures an application as a collection of loosely-coupled services. Microservices architecture has many benefits like independent development and deployments of services, eliminate long-term commitment to a technology stack, specialized services built by small teams, and many others. One of the drawbacks of Microservices is that it increases the surface area of failures. You now have to deal with failures related to the interaction between services and system boundaries. Our client was facing issues running their distributed application in a steady state. The issues that we faced were:

  1. Communication failure between services. There was no clear strategy on how to handle network failure between services and how to give proper feedback to the customers of the application.
  2. Difficulty in understanding why the whole application became unavailable when only a single service was down. Is there any single point of failure? These types of issues were not visible with usual testing.
  3. System becoming partially unavailable when the network gets choked.
  4. Unwanted local state leading to system unavailability when one instance of the service dies.
  5. Out of memory errors in production services leading to complete or partial unavailability of the system.
  6. Possible data loss issues as data replication and backup strategies were never tested in real workloads.

Continue reading

A minimalistic guide to distributed tracing with OpenTracing and Jaeger

If you have ever worked on a distributed application you will know that it is difficult to debug when things go wrong. The two common tools to figure out root cause of the problem are logging and metrics. But the fact of the matter is that logs and metrics fail to give us complete picture of a situation in a distributed system. They fail to tell us the complete story.

If you are building a system using Microservices / Serverless architecture then you are building a distributed system.

Logs fail to give us the complete picture of a request because they are scattered across a number log files and it is difficult to link them together to form a shared context. Metrics can tell you that your service is having high response time but it will not be able to help you easily identify the root cause.

Logging and Metrics are not enough to build observable systems.

Observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs. It helps bring visibility into systems. – Wikipedia

Logs, metrics, and traces are the three pillars of observability. While most software teams use logging and monitoring few of them use traces. Before we look at distributed tracing in depth, let’s define logs, metrics, and traces.

Continue reading

Machine Learning using TensorFlow and PyCharm Series: Part 1: Setting up environment and writing our first TensorFlow program

TensorFlow is an open source computation framework for building machine learning models. Its design make use of lessons learnt from earlier machine learning frameworks — Torch, Theano, Caffe, and Keras. Torch is the earliest machine learning framework that made of the term Tensor. Theano makes use of Graph data structure to store operations and compile them to high-performance code. Caffe is a high performance framework written in C++ and makes feasible to execute applications on different devices. Keras provides an easy to use API to interface with various machine learning frameworks like Theano.

Continue reading

Eclipse Memory Analyzer: Solution to Use the -data parameter Exception

If you launch Eclipse MAT and get following error

java.lang.IllegalStateException: The platform metadata area could not be written: /private/var/folders/9q/zhpkyd3s4y9d5t1nv_5hszww0000gp/T/AppTranslocation/DF264CA5-4EEF-4916-A3FA-881B111294E5/d/mat.app/Contents/MacOS/work/.metadata.  By default the platform writes its content
under the current working directory when the platform is launched.  Use the -data parameter to
specify a different content area for the platform.

Then you should open the MemoryAnalyzer.ini in your favorite editor and add the -data argument as shown below.

The location of MemoryAnalyzer.ini depends on your platform. If you are using Mac, then it is inside the app folder like mat.app/Contents/Eclipse/MemoryAnalyzer.ini

-startup
../Eclipse/plugins/org.eclipse.equinox.launcher_1.5.0.v20180512-1130.jar
-data
/Users/shekhargulati/dev/tmp/mat
--launcher.library
../Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.700.v20180518-1200
-vmargs
-Xmx4096m
-Dorg.eclipse.swt.internal.carbon.smallFonts
-XstartOnFirstThread

Markov chains in Java: Suggest what Narendra Modi will say using Markov chains

Recently, I read an article[1] on Markov chains. In the post, author showed how we can build autocomplete functionality using them. The article piqued my interest to learn more about Markov chain and I started looking for an example application that I can build using it. I decided to build a web application that will suggest me what Indian prime minister Narendra Modi[2] will say after a word/pair of words/triplet for words.

I am not a supporter of Narendra Modi style of leadership. The reason I chose him is because I could easily find text of all his speeches on the web [3].

This post is divided into three sections:

  1. What is Markov chain?
  2. Create dataset for the application
  3. Build the application that uses Markov chain

Continue reading