TIL #8: Installing and Managing PostgreSQL with Brew

This post documents how to install and manage PostgreSQL using brew package manager for macOS.

To install PostgreSQL using brew, run the following command.

$ brew install postgresql

If you wish to start and stop PostgreSQL, then you do that using following brew commands.

$ brew services stop postgresql
$ brew services start postgresql

Create a database for your username

createdb `whoami`

To fix the error role "postgres" does not exist, run the following command

createuser -s postgres

Now, you will be able to log into psql using both your username and postgres

psql
psql -U postgres

To get information about your PostgreSQL installation, you can run following brew command.

brew info postgresql

There are time you might forget if you installed PostgreSQL using brew. You can check if PostgreSQL was installed by brew by running following command.

brew list |grep postgres

The brew package manager installs PostgreSQL under following location.

/usr/local/Cellar/postgresql/

The configuration files as inside following directory.

/usr/local/var/postgres

TIL #7: Java Lambda Puzzler

Today, a colleague asked me how they can pass a java.util.Stream to a function that accept an java.lang.Iterable.

Let’s suppose we have a following function that accepts an Iterable.

public static void doSth(Iterable<String> iterable){
        iterable.forEach(System.out::println);
 }

The calling function has a Stream that it want to pass to the doSth function.

public static void main(String[] args) throws IOException {
        Stream<String> lines = Files.lines(Paths.get("src", "main", "resources", "names.txt"));
}

One way we could easily achieve this is by collecting the Stream into a Collection like List. As List is an Iterable so we can pass it. This is should below

Stream<String> lines = Files.lines(Paths.get("src", "main", "resources", "names.txt"));
doSth(lines.collect(Collectors.toList()));

This works but what if Stream is big and collecting into an in-memory data structure like List leads to java.lang.OutOfMemoryError: Java heap space.

I googled around and found a nice Lambda hack.

Stream<String> lines = Files.lines(Paths.get("src", "main", "resources", "names.txt"));
doSth(lines::iterator);

I have worked a lot on Java 8 but first time I looked at it I couldn’t figure out how it works. If you know Java 8, give it a shot.

The magic here is that Iterable interface has a single abstract method iterator.

public interface Iterable<T> {
    Iterator<T> iterator();
}

This means we can write it as following Lambda function.

Iterable<T> iterable = () -> iterator();

In our case, Stream has an iterator method. So, we can convert Stream to Iterable by defining a lambda function as shown below.

Stream<String> lines = Files.lines(Paths.get("src", "main", "resources", "names.txt"));
doSth(() -> lines.iterator());

You can refactor the Lambda to a method reference.

Stream<String> lines = Files.lines(Paths.get("src", "main", "resources", "names.txt"));
doSth(lines::iterator);

Can we use () -> iterator() in the for-each loop

I wondered if we can use the lambda expression in the for-each loop

for (String str : () -> lines.iterator()) {
    System.out.println(str);
}

This looks like a valid use case. As for-each loop works with types that implement Iterable interface. But, it turns out the code does not compile. It gives Lambda expression not implemented here error.

The answer for this is mentioned in the JSR335

Deciding what contexts are allowed to support poly expressions 
is driven in large part by the practical 
need for such features:

The expression in an enhanced for loop is not in a 
poly context because, as the construct is currently defined, 
it is as if the expression were a receiver: 
exp.iterator() (or, in the array case, exp[i]). 
It is plausible that an Iterator could be wrapped 
as an Iterable in a for loop via a 
lambda expression (for (String s : () -> stringIterator)), 
but this doesn't mesh very well with the semantics of Iterable.

Another Interesting thing to note is that Iterable is not marked as @FunctionalInterface. I don’t know the exact reason why Iterable is not marked as @FunctionalInterface. My guess is that because Iterable has special semantics in Java so they didn’t explicitly marked it @FunctionalInterface.

TIL #6: Top 10 Commands That You Use On Your Command-line Terminal

I found a command that you can use to list down top commands that you use in your terminal

$ history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl |  head -n10

Following is my list

     1  1244  12.4412%   cd
     2  1056  10.5611%   git
     3  827   8.27083%   docker
     4  575   5.75058%   ls
     5  523   5.23052%   gdw
     6  447   4.47045%   docker-compose
     7  378   3.78038%   pass
     8  221   2.21022%   gs
     9  187   1.87019%   open
    10  184   1.84018%   rm

gdw is shortcut for Gradle wrapper.

TIL #5: Representing Response Object in REST API

I was thinking what is the best way to represent response object in REST API. So, I started thinking about all the things response object should provide to work effectively. Below is the list of things I want my response object to have:

  1. A single object structure to handle both successful and error scenarios
  2. Ability to handle multiple types of objects
  3. Ability to easily figure out if request failed or succeeded
  4. Store the error code and error message

The code in this post uses Java as the language of choice. As I use Spring Boot to build REST APIs so Jackson is the default JSON serialisation library.

The code below shows the ApiResponse object that captures both the success and error payload.

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonValue;

public final class ApiResponse<T> {

    private ErrorResponse error;
    private T data;
    private Status status;

    private ApiResponse(Status status, ErrorResponse error) {
        this.error = error;
        this.status = status;
    }

    private ApiResponse(Status status, T data) {
        this.data = data;
        this.status = status;
    }

    @JsonCreator
    public static <T> ApiResponse<T> success(
            @JsonProperty("status") Status status,
            @JsonProperty("data") T data) {
        return new ApiResponse<>(status, data);
    }

    @JsonCreator
    public static <T> ApiResponse<T> error(
            @JsonProperty("status") Status status,
            @JsonProperty("error") ErrorResponse error) {
        return new ApiResponse<>(status, error);
    }

    public enum Status {
        SUCCESS("success"), ERROR("error");

        private final String status;

        Status(String status) {
            this.status = status;
        }

        @JsonValue
        public String getStatus() {
            return this.status;
        }
    }

    public ErrorResponse getError() {
        return this.error;
    }

    public T getData() {
        return this.data;
    }

    public Status getStatus() {
        return this.status;
    }
}

The key points in the class shown above are:

  1. Status enum will hold the status of the request. User can just look at this request to figure out whether request succeeded or failed.
  2. The class is generic so it can be used to store any type of object in the data.
  3. User can get more details about the error from the error object.

The ErrorResponse class is shown below.

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;

public final class ErrorResponse {
    public final String code;
    public final String message;

    @JsonCreator
    public ErrorResponse(
            @JsonProperty("code") String code,
            @JsonProperty("message") String message) {
        this.code = code;
        this.message = message;
    }
}

The ErrorResponse class has two fields code and message. The code can be HTTP code or any business specific code. The message variable gives details about the error.

I found the above response classes work great for the REST APIs that I recently built.

Example of successful response is shown below.

{
    "status": "success",
    "data": {
        "task": "Write a post",
        "taskStatus":"in_progress",
        "tags":["writing"]
    }
}

Example of failure response is show below.

{
    "status": "error",
    "error": {
        "code" "409",
         "message" : "User with username xyz already exists"
    }
}

One thing that you should ensure is that your JSON serialisation library exclude null values. For example, in case of error scenario you don’t want data field to be null. If you are using Spring Boot, you can instruct Jackson to exclude null fields by specifying a property shown below.

spring.jackson.default-property-inclusion=NON_NULL

If you are using Java 8 or Google’s Gauva and want to exclude Optional type as well then you should use NON_ABSENT value.

spring.jackson.default-property-inclusion=NON_ABSENT

TIL #4: Downloading a zip from S3 to local directory using AWS CLI

Today, I had a need to download a zip file from S3 . I quickly learnt that AWS CLI can do the job. The AWS CLI has aws s3 cp command that can be used to download a zip file from Amazon S3 to local directory as shown below.

 $ aws s3 cp s3://my_bucket/myzip.zip ./

If you want to download all files from a S3 bucket recursively then you can use the following command

$ aws s3 cp s3://my_bucket/  ./ -- recursive

You can specify your AWS profile using the profile option shown below.

$ aws s3 cp s3://my_bucket/myzip.zip ./ -- profile test

To download all the files from a folder you can use following command:

$ aws s3 cp s3://my_bucket/my_folder  ./ -- recursive

You can also use include and exclude options to filter files based on wildcards. For example, let’s suppose you only want to download files with zip extension from a S3 bucket my_bucket then you can use the following command.

$ aws s3 cp s3://pcl-caps ./ --recursive --exclude "*" --include "*.zip"

There is another great option dryrun that you can use to see the actions that will be performed without running the command. In the above command, if we add —dryrun flag then we can see which all files will be downloaded to local directory.

$ aws s3 cp s3://pcl-caps ./ --recursive --exclude "*" --include "*.zip" --dryrun

TIL #2 | IntelliJ Copy Constructor Refactoring

I wanted to learn what I wanted to know, but they wanted me to learn for the exam — Albert Einstein

Today, I spent most of my day in meetings, thinking, and preparing backlog so I couldn’t do much code.

Learning 1: IntelliJ Copy Constructor

Today, I was reviewing a piece of code where we had to create an object with 30 fields. The code had a constructor with 30 fields and then in the calling code we were passing those 30 fields. How error prone and tiring is this? A simple example is shown below.
Continue reading “TIL #2 | IntelliJ Copy Constructor Refactoring”

TIL #1 | PostgreSQL Gzip Dump, Docker-Compose, Change owner of PostgreSQL tables, and finding non-ascii characters in IntelliJ

Each day I learn something new while doing software development. From today, I have started documenting them. I will be posting/sharing tips and tricks that I discover.

Continue reading “TIL #1 | PostgreSQL Gzip Dump, Docker-Compose, Change owner of PostgreSQL tables, and finding non-ascii characters in IntelliJ”