Learn to root Android using emulator

Introduction

What is rooting actually? How does it work? I tried to figure this out these days. Now, I know how to at least root an Android emulator including creating ROM image. Since I am using emulator only, the following explanation is only applied on emulator. I have not tested on real mobile phone.

Before I begin, the followings are my working environment:

OS: Arch Linux
Packages:
android-2.2 r03-2 – required for the Android platform
android-sdk r16-1 – required for emulator
android-sdk-platform-tools r10-2 – required for adb command

I was trying on Android Froyo (2.2) only.

Superuser (root)

So, to root the Android, first we need to obtain “su” binary for Android. We can get it easily from the internet. Besides that, busybox binary for Android is very useful, this can also be obtained from the internet.

Then, we need to start the emulator by providing extra partition size to /system. This can be done only through command-line, eg

emulator -avd MyAndroid -partition-size 128 -no-snapshot-load

Make sure the AVD “MyAndroid” is already created. The “-no-snapshot-load” option is used if we enabled the snapshot. As a result, we will start the emulator with extra disk space for /system. By this, we can adding extra files to /system later.

Then, we need to use “adb shell” to remount the /system so that we have write access to the /system.

adb shell mount -o rw,remount -t yaffs2 /dev/block/mtdblock0 /system

Then, we can push the su and busybox to the /system/xbin.

adb push su /system/xbin/
adb push busybox /system/xbin/

Now, to actually make the su work properly, we need to chmod it,

adb shell chmod 06755 /system/xbin/su
adb shell chmod 06755 /system/xbin/busybox

Now, we need to install Superuser.apk (which you can get together with “su”). We can install it with “adb install” command. It is required when the other apps request for “su”.

We can try our “su” with Root Checker Basic or Stericson Busybox. We should see the following image.

Now, that is rooting. You might feel happy with it. However, if we restart the emulator, without snapshot, all the “su” does not work any more. So, we are going to customise the ROM image. But before this I just want to go through about Android Market first.

Anroid Market

To make the emulator work with Android Market, we need two APKs, Google Services Framework and Market. They are available in internet. The Market package name is com.android.vending and Google Services Framework is com.google.process.gapps

After obtaining the files, then we can push them into /system/app.

adb push GoogleServicesFramework.apk /system/app/
adb push Market.apk /system/app/

Wait a while, then you can see Market app in the emulator.

Besides that, according to this page, we need to remove the /system/app/SdkSetup.apk.

adb shell rm /system/app/SdkSetup.apk

Then, the emulator can run the Android Market, we can sign in with Google Account.

Customise ROM image for emulator

Failures

Now, if we restart the emulator, everything will return to default. No more “su” and Android Market. Why? The reason is because whatever we done to the /system, it will not save to the ROM image.

The /system is actually from the system.img, installed with android-2.2 in my case. This file is mounted in emulator as yaffs2 file system format. In order to customise this ROM image, I have tried several ways.

Using unyaffs, it can only extract the image, but I cannot continue. I can simply pull the /system using adb command without it.

Using yaffs2utils, unyaffs2 produced nothing. And the image built by mkyaffs2 does not allow the emulator to boot.

Using mkyaffsimage, also not work as above.

Some resources mentioned that, we can get the mkyaffs2image utility when building the Android source. But the problem is, to get the source, it requires a lot of disk space and need a long time to download.

So, what is the best way to modify the system.img? I even tried the low level way using dd to extract the /dev/block/mtdblock0, but failed.

Solution

Finally, I tried to solve it with userdata-qemu.img. This is the /data folder that will always modified when we install apps in the emulator. Restarting the emulator does not reset /data. Therefore, we can simply create the ROM image from /data. We can done it simply clean all the /data.

adb shell busybox rm -R /data/*

This will remove everything is /data, except “lost+found” folder.

adb shell busybox ls /data

Check with ls command, make sure only “lost+found” left.

Now, since the /system contains the “su”, “busybox”, Market and Google Services Framework we have done earlier, we need to copy the whole /system to the /data.

adb shell busybox cp -a /system/* /data/
adb shell busybox ls /data

As a result, the /data is identical to the /system.

Now, we have the userdata-qemu.img file in the AVD folder, which is modified. Close the emulator. Then, we can use the userdata-qemu.img file as the system image. We can rename it to system.img, or calling it as the system with “-system” option from the emulator command-line.

Start the emulator with this customised system.img, now, we have busybox by default, and also Android Market.

Missing Market apps problem (added 2012-03-14)

But if we look into the Android Market, we will discover that, a lot of apps are not available, only a few apps are available. To show most of the apps (not all the apps), we need to customise the boot image, namely ramdisk.img. We can get this file from android-2.2 package. It is together with default system.img.

Unlike system.img, ramdisk.img is actually a cpio gzip file. So, we can extract it with,

mkdir temp #make a folder
cd temp #change to the folder "temp"
gunzip -c ../ramdisk.img | cpio -i #extract the ramdisk.img, 
# where it is located at the parent directory of temp

This will extract all the files to the temp/ folder.

Now, edit the default.prop, modify lines as following,

ro.secure=1
ro.allow.mock.location=0

Optionally, we can also modify ro.build.fingerprint key in /system/build.prop, to bypass regional restriction. But I am not sure what are the value should be used.

After customisation, we need to re-compile the ramdisk.img. In the temp/,

find . | cpio -o -H newc | gzip > ../new-ramdisk.img

This will make a new-ramdisk.img. We can rename it as “ramdisk.img” and put it in AVD folder, so that the emulator will use it be default.

Now, if we find out the apps are still missing after customisation, then we need to stop and clear both cache and data for Google Services Framework and Market. Then restart the emulator (several times), then it can work.

The Arch way of syncing script

I like FreeFileSync, a syncing utility. There are several reasons for me to consider it is a good syncing utility,

Allows preview
Mirror, destination exactly as source
Update, update destination based on source, no deletion
Sync, allows propagate deletion, detects conflicts.
Delete to Trash can (or Recycle bin)
Multiple pairs of folders
Swap directions
Allows to edit the direction of specific files, or ignore the files.

All the bold features are which I prefer the most, especially the last one. That means, after the preview (or comparing), I can decide to change the direction of the files to be copied, left-to-right or right-to-left.

The next thing I like about FreeFileSync is the cross-platform. It is working on Windows and Linux. However, there is a drawback, it does not work in Mac OS X. Then, I look for other alternative syncing tools that work on Mac OS X with these features. None of them works as FreeFileSync.

The best among them is Synkron. However, the Synkron’s interface is not as intuitive as FreeFileSync. It uses multiple tabs for the syncing pairs. I cannot edit the syncing direction of specific files as FreeFileSync.

As a result, I tried to write a shell script to solve the syncing problem on Mac OS X. The script does not have all the features as FreeFileSync, but it provides an easier way for me to do the syncing. Since I like Arch Linux, I tried to use the Arch Linux way to write the script.

So, my shell script uses “rsync” for synchronisation. The script will read a configuration file, written in shell script form. The pairs of folders are written in an array form.

#example, sync.conf
pairs=("temp/" "temp2/" #first pair, always use variable called "pairs"
"foo/" "bar/") #next pair
# Always trailing with "/" for each directory, it is based on rsync

The script actually works with rsync for each pair of items in “pairs” array. To use the script,

./sync.sh -f sync.conf

Then, there are 2 prompts, first let you make sure the then pairs and the pairs and the synchronisation mode, update, mirror, or merge. Default mode is “mirror”.

The following is your synchronization setting:
temp/    =>     temp2/
foo/     =>     bar/
Mode: mirror
Confirm this setting? [y,n]

If “y” (yes), then the script will use rsync to do a dry-run (meaning without any changes), this is something like preview. Then, the second prompt asks for confirmation whether you want to continue. If “yes”, then start syncing.

One of the features I like over the script is the backup. The script will backup the deleted files into the folder “syncYYYYMMDD”. This is because I cannot use the Trash Can from command-line. So, I am using this script to sync in Mac OS X.

The script is available here.

My rules of thumb for using multiple programming languages

Since I know multiple programming languages: C, C++, Python, PHP, Perl, Java, C#, I would like to share about my rules of thumb to choose which language during the development. So, when do I use these languages?

When doing the web development, I will definitely use PHP. It is easy to configure with Apache and MySQL.

When doing the text processing, cleaning the unwanted text, search and replace, I will definitely use Perl, because of the power of regular expression it provides. However, if the search and replace pattern is simple, then I will just use “sed” command or “awk” command.

Then, if I want to develop an application rapidly, Python is my first choice. The GUI can be used with PyGTK.

If I want to develop an end user application, then I will use C++. Because it will produce compiled executable binary. No interpreter needed.

When the application I want to build needs to be cross-platform, from the desktop to the web, then I have no choice, but choosing Java. I don’t choose over Java in developing general application because Java uses Swing, but I prefer GTK (in the sense of development, not the Look and Feel).

For the C#, I learnt it because of curiosity. I will not use it if possible.

Then, for the shell script, which I prefer bash. I will write a shell script when processing a bunch of files with other command-line tools, such as perl, sed, rsync, etc.

As a conclusion, my most favourite programming languages are C++ and Python for desktop application development, then PHP for the web development.

CJK fonts in Wine and also works with IBus input

CJK (Chinese, Japanese, Korean) fonts in Wine was really troublesome for me. Today, I have solved it totally. Before explaining the solution, just to say about the system environment I am using.

Arch Linux
wine 1.3.37
winetricks 20111115
ibus 1.4.0
ibus-pinyin 1.4.0
ibus-qt 1.3.1
ibus-table 1.3.9.20110827
ibus-table-extraphrase 1.3.9.20110827
ibus-table-wubi-git 20110410

Problem

I have installed all the fonts using winetricks with allfonts. I have also used winetricks to set fakechinese, fakejapanese, fakekorean. But none of these tricks work fine on my Wine.

Changing locale works fine, “env LANG=zh_CN.gbk wine notepad”. The menu is okay, but in some dialogs, some characters are still missing.

Solution

To solve it, make sure you have installed several fonts, from winetricks or from the package manager. I have installed allfonts from winetricks and other fonts using pacman. Wine is able to use external fonts.

In order to make the CJK fonts display properly, firstly, we need to know what type of system font is used in the different locale,

env LANG=zh_CN.gbk wine regedit

Then, we look into the “HKLM\Software\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink” to check what is the system font that is used in this locale.

(I just found that, for the newer version of Wine, mine is 1.5.6, there is no registry key as mentioned above. But we can still get the information in “HKLM\Software\Microsoft\Windows NT\CurrentVersion\FontSubstitutes“, look at the MS Shell Dlg. It will show the font that used in the Windows Dialog.)

For my experience, I can see the following fonts in different locale

SimSun in zh_CN.gbk
PMingLiu in zh_TW.big5
MS UI Gothic in ja_JP.utf8
Lucida Sans Unicode, Tahoma, Microsoft Sans Serif.

Now, to make them work properly, we need to add the key “HKCU\Software\Wine\Fonts\Replacements” as mentioned here.

Then, add the string value with those system font name, such as SimSun, PMingLiu, MS UI Gothic.

For the data, we need to use the font name. The font name we can get from “HKCU\Software\Wine\Fonts\Cache”, see all the keys. We need to use the font name based on those key, not the “English Name”.

After setting this registry, we should able to read the fonts correctly.

My registry is

Lucida Sans Unicode = Lucida Console (external, get from ttf-mac-fonts)
Microsoft Sans Serif = MS Sans Serif (installed with wine)
MS UI Gothic = TakaoGothic (installed through winetricks)
PMingLiu = AR PL Ukai TW (external, get from ttf-arphic-ukai)
SimSun = AR PL Ukai CN (external, get from ttf-arphic-ukai)
Tahoma = Tahoma (installed through winetricks)

I am not sure does Tahoma setting is useless or not. But this setting works fine for me. Now, start the notepad with different locale, everything should work fine.

Now, after this setting, even the tray icon with CJK tooltip also works fine.

IBus input method

I tried to play a game with traditional Chinese characters, namely Big5. In the game, I cannot input the Chinese characters with IBus input method. I asked in Wine Forum (here), but no one can answer me how to solve it. But today, after solving the CJK problem, I also manage to solve this IBus input problem.

Firstly, we must enable IBus to run properly, we must set the environment variables,

export GTK_IM_MODULE=ibus
export XMODIFIERS=@im=ibus
export QT_IM_MODULE=ibus

Some of the people mentioned that these environment variables can be set in .xinitrc, .bashrc, or some other place. But in my case, they can only work in .xprofile.

The next thing, make sure ibus-daemon is started with “-x” or “–xim”.

ibus-daemon --xim

In my case, I just use the file “/etc/xdg/autostart/ibus.desktop”.

Now, the crucial part is Wine. If the locale is non-UTF8, then IBus will not work. Simply to say, IBus will work on UTF8 locale in Wine. Try it

env LANG=zh_TW.utf8 wine notepad

Now, we should able to enter any characters in notepad with IBus. Howevere, if it is zh_TW.big5 or zh_CN.gbk, IBus will not work.

Yes, this is the solution.

P/S (2020-03-31): With the recent Wine 5.4 and using Fcitx 4.2.9, the above method doesn’t work on games. No solution found yet.

P/S (2020-04-05): Tried to install Wine 1.3.37 using PlayOnLinux and run the game Taikou Risshiden 5, CJK input with Fcitx 4.2.9 has no issue. Meaning it is the bug from latest version of Wine.