Technology that you must know

Linux, git, ssh, and Docker.

Linux is the most flexible OS kernel in the world right now. It is used in supercomputer, personal computer, server, embedded system like smart vacuum cleaner, mobile phone, smart TV, etc. Firstly because it is open source, everyone can study it, modify it, and redistribute it. Secondly, thanks to GPL (license). Because of GPL, anyone who use Linux need to open source their code. Those bossy people who knows Windows only and like to earn money without paying any efforts, and never understood programming and programmers, they will never understand the power of Linux and GPL.

git, like Linux, the main author is Mr. Torvalds. It is the most popular source code and revision management tool in the world right now. Those who only know GUI will never understand the power of text files. Those who don’t know what is open source are even more out of their topics.

SSH, with the advancement of cloud computing, you need to access to remote computer or server, SSH is your need!

[B]locking an SSH port for a Linux user is like taking a mouse away from a Windows user! (Powers, 2015)

Then lastly is Docker. It is an OS level virtualisation. The development environment and production environment always has a gap. Therefore, to develop a system in an environment as similar as the production is always a need. For the software level, we can use rbenv or rvm for Ruby, virtualenv for Python. However, if the production environment uses stable version of PHP and Apache, and your working computer is using the latest version of PHP and Apache, are you going to uninstall the latest version and install the stable version? If you have multiple projects, are you going to buy many computers for each of them? One of the solution is virtualisation using virtual machine like VirtualBox. But when we are using virtual machines, we need to reserve the memory and hard disk space to the virtual machines. And some of the hardware cannot be shared, but virtualised, like the graphic card and network card. However, if using container like Docker, it can access the hardware and share the OS libraries. You can also run the GUI application as it is running on the host computer. For those Windows users they will not understand, because Docker is based on Linux technology. However, you can still using Docker on Windows and Mac OS X.

Gaming on Linux

Recently, I admit that Chromium is faster and smoother than Firefox. And now, for me, Firefox is my primary bookmark manager, reference manger with Zotero, and best ever download manager; yet Chromium is my primary web browser.

This is just like Windows is no more an Operating System for me, but a game console. I have no choice to admit this because playing games on Linux, is still far behind from Windows. The main reason is the developers usually target on Windows instead of Linux. However, it is truly worth for me to optimise the functionality of a computer by running Linux and sacrificing the gaming potentiality of Windows.

Native, Emulators, and Steam

There are several methods to play games on Linux. Steam is one of the methods, but it is not my favourite. Other than that, there are native games in Linux, one of my favourite is Cave Story. Another solution is using emulators, such as DOXBox, VBA-M, MAME, etc. However, not all emulators work perfectly, especially PS2 emulators.

WINE

WINE (WINE Is Not Emulator) is my another favourite. It allows to run Windows applications including some games. Yet, it does not work perfectly, not all games work. I have successfully used it to play KOEI Sangokushi (Romance of the Three Kingdoms) 12 and The Sims 4. WINE works fine with these games. However, some 3D games such as Assassin’s Creed II and Dead or Alive 5 Last Round do not work for me. Moreover, latest version of WINE does not guarantee better compatibility of all games. Some games work better with a certain version of WINE; some games requires certain patched version of WINE.

Virtualisation

So, the native Windows is still an attractive solution for playing games. Therefore, virtualisation is the possible solution. (LXC and Docker do not work, because they are containers.)

VirtualBox

VirtualBox is my favourite hypervisor because it is very easy to use. VirtualBox allows Direct3D yet it is under experimental. It has to be installed through the Guest Addition ISO image. However, I found that the Direct3D driver installed is wined3d.dll. As a result, the 3D rendering produces the same problem as in using WINE for certain games. As a conclusion, using VirtualBox to play 3D games does not work as expected.

QEMU/KVM

Since VirtualBox does not work with 3D games, I recently tried QEMU with KVM. Because I read some articles (this and this) regarding QEMU/KVM, it can virtualise the 3D graphics card. However, this requires 2 graphics cards: one for the host machine, another for guest machine. And this method uses VFIO, which is very troublesome.

Furthermore, there is a Virgil 3D project which targets to produce virtual 3D GPU. I studied a little, yet I didn’t try to compile it, because it seems like very troublesome. As a conclusion, QEMU/KVM does not work either.

VMware Player

So, the final solution I tried recently was VMware Player, which is free for non-commercial use. Installation of the VMware Player version 7 requires GCC 5.1.0 in Arch Linux. In this case, I have to install GCC 5.1.0 and compile the modules, then downgrade to GCC 4.9.2 again.

VMware Player is very similar to VirtualBox, but the VM management less useful than VirtualBox. Besides that, some VM configurations have to be done manually using text editor.

After some configurations and testing, I found that it works as the best among the virtualisation software. It is better than VirtualBox.

VMware Player 3D configuration

To enable VMware 3D graphics for Direct3D, firstly we have to make sure our host machine (Linux) allows “direct rendering” by entering the following command,

glxinfo | grep direct

Then, to make sure it works,

glxgears

Next, install WMware Player, then download and install VMware Tools for the guest OS. This can be done when running the Windows in the VM, by clicking “Virtual Machine” > “Install VMware Tools…”. Alternatively, we can download all VMware Tools for all the supported OSes from Player Preferences menu item. After finish downloading, we can run the setup in the Windows just like VirtualBox. The installation of VMware Tools allows seamless integration of the host and the guest like VirtualBox: such as resizing the window will resize the Windows resolution directly, and also the seamless mouse integration.

After installation, make sure 3D acceleration is enabled. This can be done from VMX file or from the Virtual Machine Settings,

mks.enable3d = "TRUE"
svga.graphicsMemoryKB = "1000000"

If 3D acceleration still does not work, then in the ~/.vmware/preferences, add

mks.gl.allowBlacklistedDrivers = "TRUE"

Next, in the ~/.vmware/preferences, add followings

pref.motionUngrab = "FALSE"
pref.motionGrab = "FALSE"

This disable auto grabbing or ungrabbing our mouse pointer when it moves into the VM window.

However, in my case, my 3D game is very sensitive to the mouse movement. Thus, I have to disable the drivers installed by the VMware Tools. This can be done through Device Manager in Windows. After disabling the drivers, my mouse cannot click anything in VM. I have to use keyboard to restart the Windows.

 

Finally, I think I can enjoy more games in Linux now, though playing the games virtually in Windows.

Web, cloud, virtualization, Docker, and Linux

From time to time, I always feel that I have to choose the “best” Linux distro. And I feel that, various Linux distros are somehow annoying, why not just combine all the best features into one powerful OS? (That is why I always struggle for the distro like Arch Linux and Debian.) With the recent trend of the technology such as LXC and Docker, I found that the varieties of the Linux distros is really a good way as it is diverging and exploring the new solutions for our daily problem.

Web is a cross-platform solution, because whatever OS you are using, as long as you have a web browser which is compatible to the web standard, then you can use the web browser to browse the web and use the service properly. That is why, the Internet is so important and so popular in our daily life. Then the cloud storage solution allows us to store our files in the cloud, and the client software will synchronise the files among multiple devices. The synchronisation solves the problem not only the files, but the contacts, calendar, task lists, email, notes, photos, etc.

OpenStack is a term that connects cloud computing with the virtualisation. Previously, when talking about “virtualisation”, my knowledge is something like virtual machine such as Qemu, VirtualBox or VMware. In the Linux world, other than cross-platform virtual machine like VirtualBox, there are some other terms related to virtualisation, and I personally not very familiar with all of them: KVM, libvirt, vagrant, Xen, etc. And recently, LXC and Docker grab my attention, because they are OS-level virtualisation. Instead of virtual machine, what we created is called “container”. Furthermore, the container uses the same kernel as the host OS in spite of the container image is based on other OS.

As DistroWatch.com mentioned, rolling release is difficult to target, because it keeps changing. That is why some users prefer fixed release instead of rolling release, as they want something which can work tomorrow as it works today, namely, consistent.

With the solution like Docker and LXC, rolling release may not be any problem, because we can use rolling release distro to create container based on the fixed release. For instance, running Debian in Arch Linux, or vice versa. That means, developing a solution by targeting on a fixed release, will also work on any type of distros as long as LXC or Docker can work.

Similar to cloud which solves the cross-platform problem, in my opinion, container is a good solution to solve different distros problem.

Linux containers (LXC) on Arch Linux

Due to a recent Q&A from DistroWatch.com, I am considering the advantages of using the scheduled-release or fixed-release distros, especially Debian which is famed of its stability. The following is the quote from DistroWatch.com,

Rolling releases do tend to have their downsides though. For instance, it is difficult for third-party developers to create software for rolling-releases as a rolling-release distribution is a moving target. It is difficult to target and support an operating system which is changing on a regular basis.

I agree with this point, because I have to change the source code on my old project which uses CEGUI. Moreover, latest PHP source code may not work on the web hosting server, due to the version differences. Though some people argue that rolling release like Arch Linux is not stable, yet I am very satisfied with its stability. The only thing is the compatibility problem to our own source code and other distros or OSes.

Because of this problem, the Debian is always my interested distro. Yet Debian packages are normally not up-to-date, and there is no Mozilla Firefox in its official repository due to the strong philosophy in Debian.

Running the different OSes in the virtual machines such as VirtualBox and Qemu with KVM is a very good solution. However, both solutions show a strong feeling that an OS (guest) within another OS (host). VirtualBox is really useful because of the configuration of the bridged network is very easy. Moreover, the USB devices sharing allows to use the Windows in the VirtualBox to do the printing and using the interactive projector (which beyond my expectation). On the other hand, Qemu is difficult to setup the bridged network. (But Android Emulator is using Qemu.)

There are some disadvantages on VirtualBox. The 3D graphics do not work properly in VirtualBox (I didn’t try on Qemu about 3D graphics). However, we can still install the Direct3D drivers through the VirtualBox Guest Addition. Furthermore, the solutions of using VirtualBox and Qemu require to create virtual hard drives. That means, transferring the files requires some networking solutions such as FTP or SSH. Yet, VirtualBox allows easier solution that mounts the shared folder from the host.

Interestingly, when I come across to the Linux Containers (LXC), this shows an interesting alternative to the VirtualBox and Qemu. Unlike the VirtualBox or Qemu, the guest OS can be run almost side-by-side with the host OS.

Running GVim in LXC on Arch Linux
Running GVim in LXC on Arch Linux

Setup LXC network

There are some useful tutorials/documents regarding how to setup LXC on the Linux distribution on the Internet. However, I found some difficulties when setting up the LXC on the Arch Linux. In Arch Linux, the kernel does not support User Namespace for the LXC. Thus, the LXC has to run in root privilege.

(I will not explain some of the basic steps such as using the commands lxc-create, lxc-start, lxc-stop, etc.)

After creating the container, the network support is also very important, because by default, the container cannot access the network. The easiest way is using bridge.

To setup the bridge, we can use the netctl.

Create a static IP for the netctl,

#In the /etc/ctl/lxbridge
Description="LXC Bridge"
Interface=br0
Connection=bridge
BindsToInterfaces=(wlan0) #Depends which connection we want to bind, do not use multiple interface
IP=static
Address=10.0.2.1/24 #Any
FwdDelay=0

Then,

sudo netctl start lxcbridge

This will produce a bridge interface br0.

Then, similar to Qemu, iptables and IP fowarding are required.

sudo sysctl net.ipv4.ip_foward=1
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE #where wlan0 can be others

Note: iptables is a must, despising whether the iptables service is started or not.

Then, in the “config” file of the created container, we have to setup the networking

lxc.network.type=veth
lxc.network.link=br0
lxc.network.flags=up
lxc.network.name=eth0
lxc.network.ipv4=10.0.2.2/24
lxc.network.ipv4.gateway=10.0.2.1 #Based on the Bridge address

Because of using the root, the lxc-usernet file is not required to be configured.

In order to run the GUI application, in the config file, we can add this

lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir
lxc.mount.entry = /dev/video0 dev/video0 none bind,optional,create=file

After installing GUI application such as GVim in the guest, to run and use the display,

DISPLAY=:0 gvim

This will run the GUI application as on the host.

Setup LXC sound

In order to play the sound, this can be done through PulseAudio. (I mostly refers to this page.)

The easier way is using the paprefs on the host OS and “Enable network access to local sound devices” in the “Network Server” tab. However, it can also be done in the command-line,

pactl load-module module-native-protocol-tcp #no root required

(If referring to other tutorials, there are other parameters. But in my case, the above command is sufficient.)

Note: The above command is run on host OS.

Once the module loaded, in the guest (container), we can set the environment variable,

export PULSE_SERVER=192.168.1.2 #where the IP address is the host IP address

Then, using the mplayer on any audio file can play the sound successfully.

 

Please note that LXC does not work with non-Linux OS such as Windows. There are some available OS templates for the container, such as Debian, CentOS, Fedora, Gentoo,OpenMandriva, OpenSUSE, and Ubuntu. This is very interesting that we can use different package manager in one computer almost seamlessly working on the host OS.

So far I didn’t test the OpenGL in container. But running glxinfo, it shows the same output as the host computer. I expect that it is using the host display, so the OpenGL should have no problem.

 

What next?

My next intention is to make the container accessible through the LAN of the host OS. Needs time to figure it out.