Bash history for each project


Previously, I tried to use zsh when I am working, so that my default bash history will not be filled with project related commands, especially git commands. Because some of my bash history are useful like ffmpeg related commands. So, I tried to distinct project work and non-project work with zsh and bash, so that they have different shell histories.

However, there are two problems when I use zsh,

  1. All my projects use the same zsh history. The old history may be lost if I have too many commands (depends on HISTSIZE variable).
  2. zsh is slow with git auto-completion.

However, zsh has one thing better than bash, that is, auto-completion navigation. Let’s say your directory has various sub-directories, double tab will bring to directory navigation that can be controlled with arrow keys. This is useful for me to navigate to non-alphabetic directories, like the directory with CJK characters.

Due to the slowness of git auto-completion as mentioned above, I decided to find an alternative solution, so that I can use bash instead of zsh, and separate the shell history from non-project work. At the end, I come out with this,

#!/bin/bash

# source any shell script, like rvm or nvm

touch "$PWD/.bash_history"
export HISTFILE="$PWD/.bash_history"
history -r
export PS1='\[\033[1m\]\u@\h \[\033[0m\]\w$(__git_ps1 " (%s)")\$ ' # custom prompt to differentiate from default prompt

So, add the above content to a file like init.sh in the project directory. Whenever starting with the project, run

source ./init.sh

Hence, the .bash_history will be saved and used in the project directory.

Advertisements

Rename files according to date


I recently wrote a Perl script, that renames the files in a directory according to the date, in the format “YYYYMMDD ##” where “##” is the running number.

Rationale

Because I used to download the photos using the mobile apps like Weibo or Twitter, however the file names are almost random. This made me hard to organize these photos on my computer.

The artists (or celebrities) usually share a set of their photos, so when I download these photos, the files should have mtime (modified time) in the correct order.

Yet, I don’t need to rename the file to the time precision like “HH:MM:SS”. I just need the date and followed by the running number, because it looks shorter.

Though we can just use the file browsers to sort the files according to the time, it is still inconvenient to browse the images by changing the sorting condition. Furthermore, mtime can be changed, and this will void the purpose of the sorting.

Lastly, the randomized filename is just meaningless to me. Rename them according to the date is much more useful, in my opinion.

 

Script

 

Why Perl?

In my opinion, Perl is less famous like Python in the present day. But I prefer to use Perl, due to the popularity in most Linux distribution. For example Perl is the base package of Arch Linux. Once I installed Arch Linux, I can run Perl script immediately.

Though Python is great, backward incompatibility sometimes causes issue, which I may need to maintain the script. If I write with Perl, I can pay less effort to maintain the script.

 

WARNING! And usage

As the script mentioned,

Rename is irreversible. Recommend to make a backup.

The usage is,

./rename_files_to_date.pl ./target_dir

Where target_dir is the directory that contains the files you want to rename. It will not rename the files recursively.

PLEASE USE THE SCRIPT AT YOUR OWN RISK!!!

After renaming the files, a log file will be created. It is used just in case you want to revert the file name. (But you have to do this manually.)

MySQL and PostgreSQL GUI tools


I am using Arch Linux, and recently there are some issues on MySQL Workbench and pgAdmin 4.

MySQL Workbench with the latest version 8.0.12 has issue to make connection through the SSH tunnelling. As a result, I have to downgrade to version 6.3.10. However, with the recent upgrade of icu package to 62.1, it breaks the MySQL Workbench 6.3.10. In order to solve this, I duplicate the following files,

cp libicudata.so.62 libicudata.so.61
cp libicui18n.so.62 libicui18n.so.61
cp libicuuc.so.62 libicuuc.so.61

So, it is a temporary solution until the package is fixed.

Then, for the PostgreSQL, I use pgAdmin. But again, with the recent upgrade of Python to version 3.7, which causes pgAdmin fails to start. I tried to run pgAdmin through Python 3.6 using virtualenv , but it doesn’t work. So, I have tried other solutions.

I tried Adminer. It depends on PHP and Apache server, similar to phpMyAdmin. However, it is less intuitive, compared to phpMyAdmin. Because Adminer doesn’t allow inline editing value.

So, I tried SQL Workbench/J. It requires Java and PostgreSQL JDBC. Once installed required package, it can work perfectly. Similar to Adminer, SQL Workbench/J can work with database other than PostgreSQL, such as MySQL.

 

Catechism and the Pope


I am neither theologian nor bible scholar. But just a Catholic layperson and would like to share my opinions.

The most recent hot issue is the changing of the Catechism about death penalty by Pope Francis. Then throughout the Twitter, I can see there are a lot of debates on this issue, including some people criticising the Pope.

Therefore, in this post, I will share some of my understandings.

Regarding the Catechism of the Catholic Church (CCC), according to USCCB Question 17,

Because the Catechism presents Catholic doctrine in a complete yet summary way, it naturally contains the infallible doctrinal definitions of the popes and ecumenical councils in the history of the Church.

However, there were interesting articles about authority of the CCC and non-infallible teachings, which should be read as well.

Next, Pope is not a saint. Throughout the Church history, there were some bad popes.

There are some Twitter users share about Summa Theologiae article whether it is lawful to kill sinners. However, we must bear in mind that Summa Theologiae is a great theological work, but it is not infallible.

Since I am not an expert, I cannot discuss more about the death penalty.

However, I would like to share some of the related chapters and verses from the Bible.

St John Gospel chapter 8 mentioned about an adulterous woman, which according to the Law of Moses, she should be stoned to death. However, Jesus responded to the crowd, “Let the one among you who is without sin be the first to throw a stone at her”.

Therefore, I am asking myself, if I support death penalty, what is my actual intention? Because of my hatred and my anger towards someone? Or is it because, if my beloved is murdered, I can revenge in the name of justice? And also, if I oppose death penalty, what is my actual intention? Do I just simply follow whatever Pope says?

In Gal 2:11, it mentions that Apostle Paul “opposed [Peter] to his face because he clearly was wrong”. Therefore, I don’t think it is wrong to oppose the Pope, if the Pope is wrong. However, we must be careful to ourselves, because the way we oppose may lead to more confusion, especially when we are using social network and we have some followers. And one should pray that he or she is not opposing the Pope because of pride.

In 1 Cor 1:12-13, it says that Corinthians were dividing themselves, some say they belong to Paul, some belong to Apollos, some belong to Cephas, and some belong to Christ. Similarly, this kind of behaviour continues until today. Therefore, as Catholics, we should continually pray for the Pope and the unity of the Church.

Firefox batch download extension: DownloadSelected


In my previous post, I mentioned about DownThemAll on Firefox 56. Now the latest release of Firefox is version 61, but there is no update of DownThemAll for Firefox Quantum support. Using older unsupported version like Firefox 56 is not a good idea, because there will have no security update. Firefox ESR 52 is slower than Firefox 56 as I have mentioned.

Since I have spare time recently, I spent some days to write a Firefox extension, DownloadSelected, to solve my problem.

DownloadSelected demo

The screenshot above shows a list of URLs, which I used Greasemonkey script to generate. Then highlight, right-click, and DownloadSelected. I don’t write any UI elements to indicate the download progress, but I put the progress to the console. Once the files are downloaded, they will be archived into a zip file and a Save As dialog will be shown. The source code can be found here.

This is not a replacement or alternative extension to DownThemAll, but this extension solve my fundamental problem. Main features are:

  1. Bulk download
  2. Download only selected text
  3. Downloaded filenames are based on HTML text instead of URLs
  4. Quantum and Google Chrome compatible

 

Inheritance and composition


The modern JavaScript with the ES6 syntax and the rise of the popularity like ReactJS, functional programming becomes more and more common. When using React, one of the common practice is to use composition instead of inheritance.

Because I started learning programming when the OOP was the most prevailing paradigm, I was trained to solve the problem by using OOP concepts like polymorphoism, inheritance, encapsulation, etc.

I think JS is the most interesting programming language in the modern technology. It supports server-side and client-side development. With the ES6, it supports OOP keywords like class and also using FP (functional programming) syntax like fat arrow (=>).

In OOP, the most common usage is the inheritance and polymorphoism. The following is an example of inheritance in JS,

class Shape {
  constructor(w, h) {
    this.width = w;
    this.height = h;
  }
  area() {
    return this.width * this.height;
  }
}

class Rectangle extends Shape {
  constructor(w, h) {
    super(w, h);
  }
}

class Triangle extends Shape {
  constructor(w, h) {
    super(w, h);
  }
  area() {
    return super.area() / 2;
  }
}

function main() {
  const rectangle = new Rectangle(4, 5);
  const triangle = new Triangle(4, 5);
  console.log('Rectangle area: ', rectangle.area());
  console.log('Triangle area: ', triangle.area());
}

main();

The shape area calculation can be re-written to composition instead of inheritance as followings,

class Rectangle {
  constructor(w, h) {
    this.width = w;
    this.height = h;
  }
  area() {
    return this.width * this.height;
  }
}

class Triangle {
  constructor(w, h) {
    this.width = w;
    this.height = h;
  }
  area() {
    const rect = new Rectangle(this.width, this.height);
    return rect.area() / 2;
  }
}

Therefore, Rectangle and Triangle do not inherit from Shape. In fact, Triangle uses Rectangle to calculate the area. This is the object composition, and it is same as the way of composition in React. Furthermore, one of the greatest features of JS is closure. This allows React to pass a function with specific logic as a parameter to a generic component. Thus, the generic component can be designed without the prior knowledge of the business/application logic. This will produce a result similar to method override in OOP.

Moreover, the object composition can be re-written to function composition as FP.

const rectangleArea = (w, h) => w * h; // In math, f(x,y) = x * y
const halving = (area) => area / 2; // In math, g(x) = x / 2
const triangleArea = (w, h) => halving(rectangleArea(w, h)); // In math, h(x,y) = g(f(x,y)) = f(x,y) / 2

function main() {
  console.log('Rectangle area: ', rectangleArea(4, 5));
  console.log('Triangle area: ', triangleArea(4, 5));
}

main();

Scientific Learning


In my previous post, I wrote about Lean Startup. And I found that it is very interesting that I have come through these things.

  • S.M.A.R.T (specific, measurable, actionable, realistic, time-bound) (post)
  • Machine Learning – Deep Learning – Convolutional Neural Network (CNN)
  • Statistics
  • Kaizen (改善法) (post)
  • Agile development. Test development development (TDD)
  • Research study

When I read The Lean Startup (Ries, 2011), I learn about lean manufacturing. Then only I discovered that kaizen is also part of the lean manufacturing. Wow! I realise that, all of these things are related. And the core of the concept is, “scientific learning”.

In Lean Startup, in my opinion, the fundamental concept is Build-Measure-Learn loop. According to Ries (2011), the “Entrepreneurship is a management, learning, and involves experiments”. Throughout the whole entrepreneurship, it is about the learning. It is a scientific learning, because entrepreneurs need to form hypotheses and then test the hypotheses with experiments. Furthermore, the method can be replicated,  The results can be measured and validated. It is a validated learning. The main purpose of Lean Startup is to make the Build-Measure-Learn loop faster, then for each iteration, the product will be optimized and improved through the learning.

This is exactly how Machine Learning works. In Machine Learning, the core concept is the gradient descent. For each calculation of output (≈ build), the machine get the errors (difference of actual output and the desired output) (≈ measure). Then using the errors, machine will update the weight so that it will produce the actual output that is closer to the desired output (≈ learn). It involves small batches and many iterations throughout the training.

Agile development shares many characteristics with Lean Startup. It involves iterative development and daily stand-up meetings (scrum) which improves the communication. Besides that, TDD is also part of the agile development. Why is TDD important? I have wrote about it previously. But after reading The Lean Startup, I found that TDD is more than just making the product stable. As we use TDD, we can test the features or functions with command-line. It is extremely faster than testing the functionality by running the application from beginning to the end manually. Imagine that you want to test a webapp functionality, you have to open the web browser, login, click-and-click-and-click, refresh-refresh-and-refresh. If using TDD, as lean manufacturing, this can minimize the waste drastically. Similarly, you write the code (≈ build), you test it (≈ measure), and you make changes to optimize your code (≈ learn).

Unluckily, when I was doing PhD research study, I didn’t know about Build-Measure-Learn. Research study also shares a characteristic of entrepreneurship, i.e. uncertainty. However, getting supervisors’ feedback is not an easy job, as most of the time our supervisors are hardly to give instant response. As a result, unlike entrepreneurship, research study has difficulty to accelerate the Build-Measure-Learn loop. And most of the time, research study requires enormous prior knowledge of the research area.  Without fulfilling this requirement, supervisors’ feedback doesn’t help much at all.

When I was a lecturer, my institution adopts OBE (Outcome-Based Education). It shares the characteristics as Lean Startup and S.M.A.R.T, as the outcomes have to be measurable and observable. However, I personally don’t agree the ways of implementation of OBE, because it makes the teaching become very rigid, especially setting up the exam papers have to fulfil the criteria strictly. I agree that the outcome should be measurable, yet the ways of implementation is exhaustive and not really effective. It is muda and muri. But I believe that, if students are able to implement Build-Measure-Learn loop in learning, the learning will become effective. For example, “build” the knowledge through lecture class, group discussion, and other activities. “Measure” their gained knowledge through various tests. “Learn” from the results of the tests. And, repeat. Each iteration should be small. By each iteration, the students should able to learn through the failures they made and optimize their solution to get the desired result. (Nevertheless, I believe that this implementation is impractical, due to the cultural pressure and stereotype that the students results should be a bell-curve. W*F!)

As a conclusion, by using the scientific learning method like Lean Startup, kaizen, or even Machine Learning in our daily life (small batches, iterative, and measurable), surely this will make our life better.