Category: 52-technologies-in-2016

Sentiment Analysis in Python with TextBlob

Welcome to the eleventh blog of 52 Technologies in 2016 blog series. If you are following this series then you would have probably noticed that I already wrote week 11 blog on tweet deduplication. I was not happy with the content so I decide to write another blog for week 11.

In week 11, I decided to spend time to learn about text processing using the Python programming language. We will only focus on Sentiment Analysis in this blog. I have written about sentiment analysis multiple times in last few years. We learnt how to do sentiment analysis in Scala using Stanford CoreNLP in week 3 blog. Sentiment analysis gives you the power to mine emotions in text. This can help you build awesome applications that understand human behavior. Few years back, I built an application that helped me decide if I should watch a movie or not by doing sentiment analysis on social media data for a movie. There are many possible applications of Sentiment analysis like understanding customer sentiment for a product by analysis of reviews.

You can read full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/11-textblob/README.md

Realtime People Counter with Google’s Cloud Vision API and RxJava

Welcome to the ninth blog of 52 Technologies in 2016 blog series. Recently, Google released Cloud Vision API that enables developers to incorporate image recognition in their applications. Image Recognition allow developers to build applications that can understand content of images. Google’s Cloud Vision API is very powerful and support following features:

  1. Image categorization: The API can help classify images into categories. You can build powerful applications like Google Photos that do automatic categorization.
  2. Inappropriate content detection: The API can detect inappropriate content in an image like nudity, violence, etc. It uses Google Safe search capabilities underneath.
  3. Emotion detection: This allows you to detect happy, sad or moderate emotions in an image.
  4. Retrieve text from the image: This allows you to extract text in multiple languages from the images.
  5. Logo detection: It can help you identify product logos within an image.

There are many possible applications that you can build using this powerful API. In this tutorial, we will learn how to build a realtime people counter. The application will subscribe to a twitter stream for a topic and would return number of people found in each image. We can then use this data to get advanced statistic like number of people in a time frame using RxJava buffer capabilities.

You can read full blog https://github.com/shekhargulati/52-technologies-in-2016/blob/master/09-cloudvision/README.md

CoreOS for Application Developers

Welcome to eighth week of 52 Technologies in 2016 blog series. This week we will learn about CoreOS, an Open source Linux distribution built to run and manage highly scalable and fault tolerant systems. It is designed to docker and rocket containers. When I started learning about CoreOS, I was overwhelmed by its complexity and different components that you have to know and interact with like etcd, systemd, fleet, Flannel. I am not an Ops guy so CoreOS documentation and many tutorials that I found on the web didn’t clicked with me. The goal of this tutorial is to help application developers understand why they should care about CoreOS and show them how to work with CoreOS cluster running on top of Amazon EC2.

What is CoreOS?

According to CoreOS website, CoreOS is a Linux for Massive Server Deployments. This means it is not a general purpose Linux distro that you can use as your development workspace instead, you will use it to run and your applications at scale.

Built on Chrome OS, CoreOS is a lean and mean operating system that runs minimal Linux. When you limit your OS to the bare minimal i.e. just openssl, ssh, linux kernel, gcc then you need a mechanism to run package and run applications that you want to use. CoreOS does not even has a package manager like yum or Apt. CoreOS is very different from other Linux distributions as it is centered around containers. Linux Containers is an operating-system-level virtualization environment for running multiple isolated Linux systems (containers) on a single Linux control host. CoreOS uses containers to run and manage applications services. You package application along with its dependencies within a container that can be run on a single or multiple CoreOS machines. CoreOS supports both Docker and Rocket containers.

Docker is the poster child of containers. In November 2013, I first learnt and wrote about Docker. Docker is a set of toolset geared around containers. Docker clicked with everyone and overnight became the tool that everyone wanted to learn and introduce in their organization. One reason Docker became popular very quickly is its approachability to an average developer. To use Docker, you don’t have to know Linux internals and work with complicated tools.

CoreOS developers claim that it is 40% more efficient in RAM usage than an average linux installation.

Read the full blog at https://github.com/shekhargulati/52-technologies-in-2016/blob/master/08-coreos/README.md

Hugo: A Modern WebSite Engine That Just Works

This week I decided to take a break from Scala and scratch my own itch my building an online bookshelf using Hugo. Hugo is a static site generator written in Go programming language. You can use it for building modern static websites. Static site generator takes your content files written in a markup language like Markdown, apply layouts you have defined, and generate static HTML files that can be delivered to the user. Static websites are nothing new, they date back to the first ever website in human history. We started with static websites, then moved to dynamic websites, and finally we are moving back to static websites for use-cases where it make sense. Most common use-cases for static websites are blogs, product documentation, help guides, tutorials, online portfolio or resume.

You can read full blog https://github.com/shekhargulati/52-technologies-in-2016/blob/master/07-hugo/README.md

Building A Lightweight Scala REST API Client with OkHttp

Welcome to the sixth blog of 52-technologies-in-2016 blog series. In this blog, we will learn how to write Scala REST API client for Medium’s REST API using OkHttp library. REST APIs have become a standard method of communication between two devices over a network. Most applications expose their REST API that developers can use to get work with an application programmatically. For example, if I have to build a realtime opinion mining application then I can use Twitter or Facebook REST APIs to get hold of their data and build my application. To work with an application REST APIs, you either can write your own client or you can use one of the language specific client provided by the application. Last few weeks, I have started using Medium for posting non-technical blogs. Medium is a blog publishing platform created by Twitter co-founder Evan Williams. Evan Williams is the same guy who earlier created Blogger, which was bought by Google in 2003.

Medium exposed their REST API to the external world last year. The API is simple and allows you to do operations like submitting a post, getting details of the authenticated user, getting publications for a user, etc. You can read about Medium API documentation in their Github repository. Medium officially provides REST API clients for Node.js, Python, and Go programming languages. I couldn’t find Scala client for Medium REST API so I decided to write my own client using OkHttp.

You can read the full blog here.

Slick 3: Functional Relational Mapping for Mere Mortals Part 2: Querying data

Last week we learnt the basics of Slick library. We started with a general introduction of Slick, then covered how to define a table definition, custom mappers, and perform insert queries. Today, we will learn how to perform select queries with Slick. Slick allows you to work with database tables in the same way as you work with Scala collections. This means that you can use methods like map, filter, sort, etc. to process data in your table.

You can read the full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/05-slick/README.md

Slick: Functional Relational Mapping for Mere Mortals Part 1

Welcome to the fourth blog of 52-technologies-in-2016 blog series. Today, we will get started with Slick. Slick(Scala Language-Integrated Connection Kit) is a powerful Scala library to work with relational databases. Slick is not an ORM library. It bases its implementation on functional programming and does not hide database behind an ORM layer giving you full control over when a database access should happen. It allows you to work with database just like you are working with Scala collections. Slick API is asynchronous in nature making it suitable for building reactive applications. Although Slick itself is asynchronous in nature, internally it uses JDBC which is a synchronous API. Slick is a big topic so today we will only cover basics. I will write couple more parts to this blog.

The core idea behind Slick is that as a developer you don’t have to write SQL queries. Instead, library will create SQL for you if you build the query using the constructs provided by the library.

You can read full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/04-slick/README.md

Sentiment Analysis in Scala with Stanford CoreNLP

So far in this series, we have looked at finatra and sbt open-source Scala projects. This week I decided to learn Stanford CoreNLP library for performing sentiment analysis of unstructured text in Scala.

Sentiment analysis or opinion mining is a field that uses natural language processing to analyze sentiments in a given text. It has applications in many domains ranging from marketing to customer service. Few years back, I wrote a simple Java application using Naive Bayes classifier to determine whether people liked a movie or not based on sentiment analysis of tweets about a movie.

From the Stanford CoreNLP website,

Stanford CoreNLP provides a set of natural language analysis tools. It can give the base forms of words, their parts of speech, whether they are names of companies, people, etc., normalize dates, times, and numeric quantities, and mark up the structure of sentences in terms of phrases and word dependencies, indicate which noun phrases refer to the same entities, indicate sentiment, extract open-class relations between mentions, etc.

You can read full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/03-stanford-corenlp/README.md

SBT: The Missing Tutorial

Welcome to the second blog of 52-technologies-in-2016 blog series. From last year, I have started using Scala as my main programming language. One of the tools that you have to get used to while working with a programming language is a build tool. In my office projects, we use Gradle for all our projects be it Scala or Java. In most of my personal Scala projects, I have started using sbt as my preferred build tool. sbt is a general purpose build tool written in Scala. Most of the time we try to hack our way while using a build tool never learning it properly. As Scala will be the language that I will cover most in this series, I decided to thoroughly learn sbt this week. We(developers) often underestimate the importance of learning a build tool thoroughly and end up not using build tool in the most effective way. Good working knowledge of a build tool can make us more productive so we should take it seriously.

You can read full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/02-sbt/README.md

Finatra Tutorial — Build Beautiful REST API The Twitter Way

Finatra is an open-source project by Twitter that can be used to build REST APIs in Scala programming language. Finatra builds on top of Twitter’s Scala stack — twitter-server, finagle, and twitter-util.

  1. Finagle: It can be used to construct high performance servers.
  2. Twitter Server: It defines a template from which servers at Twitter are built. It uses finagle underneath.
  3. Twitter-Util: A bunch of idiomatic, small, general purpose tools for Scala.

In this step-by-step tutorial, we will cover how to build a Scala REST API using Finatra version 2. Finatra version 2 is a complete rewrite of finatra and is significantly faster(50 times according to documentation) than version 1.x.

This blog is part of my year long blog series 52 Technologies in 2016

You can read full blog here https://github.com/shekhargulati/52-technologies-in-2016/blob/master/01-finatra/README.md