Transcript Android Below Java - FSU Computer Science

Android Below Java
Understanding the Underlying System
Architecture, and Build Process
Michael Mitchell
[email protected]
Department of Computer Science
Florida State University
Tallahassee, Florida 32306, USA
Overview of Topics
•
•
•
•
•
•
•
Terminology
Android System Architecture
Android Tools and Utilities
Compiling the Android System
Flashing firmware images
Compiling the Android kernel
Links to additional resources
Terminology (ROM)
• Technically, ROM stands for Read Only Memory, which - as it says means that you cannot write to it, it is read only - like a DVD.
• Confusing, because manufacturers refer to the system partition as
ROM because they do not intend for the user to write to it.
– So, even though it is technically writable, it is sometimes referred to as
ROM because of the way it is designed to be used.
• Furthermore, ROM is often used as a shorthand for "ROM image".
– In other words, the files that you put in the system partition are also
referred to as ROM sometimes.
– Thus, you'll hear people say "flashing a ROM".
• Example: You can flash a ROM onto the ROM, which isn't really
ROM, actually.
Terminology (Root)
• Like ROM, root can mean several different things,
depending on the context:
– 'root' is the name for administrator in Android and other
Linux based systems.
– 'root' also means having administrator powers.
– 'root' is the lowest level one can go in a file system. In
other words, it is / in Linux or C:/ in Windows.
– 'rooting' is the process of acquiring root (or administrator)
privileges on the device.
• Example: You need to root the device so that you can
have root access to the root of the file system.
Terminology cont.
• Firmware - Programs stored in the ROM, EPROM, or
flash memory that usually control various internal
electronic devices (Hard Drives, Keyboards, Displays,
etc). Firmware is typically 'fixed' software that is not
updated in consumer devices, however it is often
updated (or 'flashed') by advanced users to fix bugs or
add features to the device.
• Kernel - The central or core software component of
most operating systems. Its responsibilities include
managing the system's resources (the communication
between hardware and software components) and can
provide the lowest-level abstraction layer for resources
(especially memory, processors, and I/O devices).
Terminology cont.
• Radio - The cellular radio on the device which needs
firmware to control it.
• SPL - Second Program Loader, in conjunction with the
IPL comprise a device's bootloader. Aside from
bootstrapping Android, the bootloader also fulfills
various diagnostic functions. One of these functions is
the manipulation of data in the device's internal flash
ram. Depending on the SPL installed, the user might be
able to flash signed NBH files, flash nand images, and
more.
– Note that the SPL is installed and operates independently
of the Android build that runs atop it!
Terminology cont.
• Flashing - The process of applying a firmware image (or
ROM) to a device. It generally entails a very specific order
of steps. Failing to complete any one of these steps
properly may result in bricking the device.
• Unlock - Most GSM devices are locked to only work with
the sim cards of a particular carrier. Obtaining root access
on the device usually does nothing to the sim-lock. The
usual way to unlock the device to all SIM cards is to obtain
a subsidy unlock code from a carrier.
• Brick - a device that no longer functions, generally caused
by a failed firmware or SPL update. Since the device no
longer works as intended, it is often referred to as a "brick"
or "paper-weight", since that is all it is good for.
Android Tools
• ADB - Android Development Bridge. Command line
tool used to communicate with & control the device
over a USB link from a computer.
– Daemon runs as root on emulator/root’d phone, otherwise
very limited privileges
– Can send shell commands (dd, ls, mount, cat, ps, date,
uptime,uname -a, mount, etc.)
– Can recursively push/pull files (logical)adp pull|push <src>
<dest>
• Fastboot - a protocol used to directly update the flash
filesystem in Android devices from a host over USB. It
allows flashing of unsigned partition images.
Android Tools
• flash_image - A small utility to be used on the device
that allows you to re-write system partitions with
image files.
• NANDroid - A set of tools that will enable anyone who
has root on their Android device to make FULL system
backups. NANDroid will backup (and restore) /system,
/data, /cache, and /boot partitions.
• Zipalign - Reduces the amount of RAM consumed
when running the application by allowing data to be
mmap'd in; which causes all uncompressed data within
the .apk, such as images or raw files, to be aligned on
4-byte boundaries.
Android System Architecture
• Android is a multiprocessing system.
• Each application runs on top of the Android
Dalvik machine.
• Each Dalvik machine runs on top of a Linux
process.
• Each process runs in its own sandbox, which
means it can access only the resources it creates.
• To access resources outside of the application
sandbox, the application needs to request
permission from the Android system.
– android.Manifest.permission.
System Architecture Overview
Linux Kernel
• Android relies on Linux version 2.6.x for core
– Approximately 115 patches from mainline
• Provides system services, including:
– security, memory management, process management,
network stack, and device driver model.
• The kernel also acts as an abstraction layer
between the hardware and the rest of the
software stack.(easier access to peripheral
hardware devices)
System Libraries
• On top of Linux is a set of libraries including
bionic (the Google libc), media support for
audio and video, graphics and a lightweight
database, which is a useful repository for
storage and sharing of application data.
• Provides most of the functionality available in
the core libraries of the Java programming
language.
Android Run-time
• Provides a consistent environment on which every
Android application runs
• Each Android application runs in its own process, with
its own instance of the Dalvik Virtual Machine.
• Relies on the Linux Kernel for underlying functionality
such as:
– Multi-Threading
– Low-level memory
management
Dalvik Virtual Machine
• Dalvik VM != Java VM
• Designed specifically for Android and is
optimized in two key ways.
– multiple instantiation (each application has its
own private copy running in a Linux process)
– register based (instead of being stack based like
most Java VMs)
• Executes the Dalvik Executable (.dex)
– optimized for minimal memory footprint.
Processes & Lifecycles
• Components share a common Linux process:
– by default, one process per .apk file.
• .apk files are isolated and communicate with each other via
Intents or AIDL.
• When the first of an application's components needs to be
run, Android starts a Linux process for it with a single
thread of execution (Main Thread).
– Android tries to maintain a process for as long as possible,
– eventually will need to remove old processes when memory
runs low.
• To determine candidates to be killed, Android places each
process into an "importance hierarchy" based on the
components running in it and the state of those
components.
Android Device Partitions
• Android devices have their internal flash memory divided into partitions.
• Common Partitions
– splash1: This partition contains the very first image you see when you turn on
the device.
– boot: This partition stores the kernel. It also controls other aspects of the
operating system on the device, such as what file formats the device is able to
support.
– recovery: This partition holds the device's recovery software, which gives the
user access to special functions, such as upgrading the operating system, or
making a backup of the device's software.
– system: This is the main partition where the device's operating system is
stored. Generally speaking, this partition is write-protected, and only able to
be modified under special circumstances.
– cache: This partition stores temporary files. Generally, files on this partition
are not critical to the operation of the device.
– data: This partition is where all of your data is stored. If you download an app
from the Android Market, it goes here. If you change a setting, that change is
stored here. If you receive a SMS, it is stored here.
Android file systems
michael@xt2lin:~$ adb shell
# mount
rootfs on / type rootfs (ro,relatime)
tmpfs on /dev type tmpfs (rw,relatime,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
none on /acct type cgroup (rw,relatime,cpuacct)
tmpfs on /mnt/asec type tmpfs (rw,relatime,mode=755,gid=1000)
tmpfs on /mnt/obb type tmpfs (rw,relatime,mode=755,gid=1000)
none on /dev/cpuctl type cgroup (rw,relatime,cpu)
/dev/block/mtdblock3 on /system type yaffs2 (ro,relatime)
/dev/block/mmcblk0p1 on /data type ext3
(rw,nosuid,noatime,nodiratime,barrier=1,nodelalloc,data=ordered)
/dev/block/mtdblock6 on /data/data type yaffs2 (rw,nosuid,nodev,relatime)
/dev/block/mmcblk0p2 on /cache type ext3
(rw,nosuid,nodev,noatime,nodiratime,barrier=1,nodelalloc,data=ordered)
Android MTD blocks
michael@xt2lin:~$ adb shell
# cat /proc/mtd
dev:
size
erasesize name
mtd0: 000a0000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 0f800000 00020000 "system"
mtd4: 000a0000 00020000 "local"
mtd5: 02800000 00020000 "cache"
mtd6: 09500000 00020000 "datadata"
Android boot.img
+-----------------+
| boot header
|
+-----------------+
| kernel
|
+-----------------+
| ramdisk
|
+-----------------+
| second stage
|
+-----------------+
1 page
n pages
m pages
o pages
n = (kernel_size + page_size - 1) / page_size
m = (ramdisk_size + page_size - 1) / page_size
o = (second_size + page_size - 1) / page_size
Android boot.img
•
•
•
A ramdisk is a small filesystem containing the core files needed to initialize the system.
It includes the critical init process, as well as init.rc, which is where you can set many systemwide properties.
Files on a typical ramdisk:
./init.trout.rc
./default.prop
./proc
./dev
./init.rc
./init
./init.goldfish.rc
./sbin
./sbin/adbd
./system
./data
Compile Android System
• Android system is complete compilation contains
the following:
– Host tools; Target system, boot, and data image files;
Target Linux kernel (compiled separately); Installable
zip file to flash in Recovery mode
• Basic Process:
–
–
–
–
Install Dependencies
Acquire the Source
Copy proprietary files
Configure & Compile
Install Dependencies
• For 32/64-bit systems:
sudo add-apt-repository "deb http://archive.canonical.com/
maverick partner“
sudo apt-get update
sudo apt-get install git-core gnupg flex bison gperf libsdl1.2dev libesd0-dev libwxgtk2.6-dev squashfs-tools buildessential zip curl libncurses5-dev zlib1g-dev sun-java6-jdk
pngcrush schedtool
• For 64-bit systems only :
sudo apt-get install g++-multilib lib32z1-dev lib32ncurses5-dev
lib32readline5-dev gcc-4.3-multilib g++-4.3-multilib
• Get the repo tool
curl http://android.git.kernel.org/repo > ~/bin/repo
chmod a+x ~/bin/repo
Acquire the Source Tree
• Make a directory for the source tree
mkdir ~/android
cd ~/android
• Use the repo tool to initialize and sync the remote/local
repositories
repo init -u
git://github.com/CyanogenMod/android.git -b
gingerbread
repo sync -j16
• ~123,694 items, requires about 3.0 GB storage
Copy Proprietary Files
• While android is almost entirely open-source certain components remain
proprietary & closed
– Mostly hardware related
– Must be copied from current system builds
• Navigate to your specific device
cd ~/android/device/[vendor]/[device]/
./extract-files.sh
• Get Rom-manager and Google App files
cd ~/android/vendor/cyanogen
./get-rommanager
./get-google-files
• NOTE: If some hardware isn't working, like camcorder or FM radio, you will
need to find the updated prop blobs.
Configure & Compile
• Setup the build environment
. build/envsetup.sh
• Use the ‘brunch’ utility to select build target
brunch [ target ]
• Takes ~1 - 3 hours to complete
• After compiled:
• ~172,562 items, requires about 6.6 GB storage
Compilation Results
• Compiled host utilities:
~/android/out/Host/Linux-x86/bin
• Compiled target files:
~/android/out/target/Product/[target]
–
–
–
–
–
–
ramdisk . img
[root file system image]
root
[root file system directory]
System
[master file system directory]
System . img
[main file system image]
UserData - qemu . img [ QEMU data image]
UserData . img
[image data]
Flash the system image
• Make a NANDroid backup
• Copy your .zip file from:
~/android/out/target/product/[target]/update.cmXXXXX-signed.zip
• to the root of the SD card.
• Download Google Apps and place at root of the SD card. (optional,
but recommended)
• Reboot into device Recovery mode
• Seriously, make a NANDroid backup
• Flash one/both of these .zip files from recovery.
• NOTE: Be patient, Sometimes this may take a VERY long time!
Loading source in eclipse
• Make sure you have source and your build completed
without errors!
• Copy the .classpath file to the root folder of your source
cp ~/bin/eclipse/.classpath ~/android/
• Open Eclipse
–
–
–
–
–
Click the File menu » new » java project
Give a project name like Cyanogen
Uncheck use default location
Point the location to ~/android
Keep all other defaults & Click finish
Download the kernel source
• Make a directory to store the kernel tree
git clone git://github.com/CyanogenMod/cmkernel.git
• 38,463 files, requires about 409.4 MB
• View other available branches:
git branch
• Checkout a branch:
git checkout –b ‘branch_name’
Configure the Build
• Pull the .config from the newest boot.img
scripts/extract-ikconfig boot.img > .config
• Alternatively, retrieve a working kernel config from the device, and
unzip it:
adb pull /proc/config.gz
/home/michael/android/kernel/cm-kernel/config.gz
cat config.gz | gunzip > .config
• Just enter to accept defaults, or customize as needed:
make ARCH=arm CROSS_COMPILE=$CCOMPILER oldconfig
make ARCH=arm CROSS_COMPILE=$CCOMPILER menuconfig
Compile the kernel
• Make the build
make ARCH=arm CROSS_COMPILE=$CCOMPILER –j5
• This step may take a while (but not nearly as long as
the system compilation
• Finally, you should have a kernel stored in
~/android/kernel/cm-kernel/arch/arm/boot/zImage
Compiling kernel modules
• After installing a custom kernel, the wifi module may be unstable or unusable.
• The solution is to recompile the module, linking it to the new kernel build.
cd ~/android/system/system/wlan/ti/sta_dk_4_0_4_32
KERNEL_DIR=~/android/kernel CROSS_COMPILE=$CCOMPILER ARCH=arm
make –j5
• A file named wlan.ko will be produced in the current directory.
• You must install the new module to the device.
adb shell mount -o remount,rw /system
adb shell cp /system/lib/modules/wlan.ko
/system/lib/modules/wlan.ko.backup
adb push wlan.ko /system/lib/modules/wlan.ko
•
•
Reboot the device.
If all goes well, you should be using the newly compiled 'wlan.ko'.
Additional Info & References
• http://android.git.kernel.org/
• http://wiki.cyanogenmod.com/
• http://wiki.cyanogenmod.com/index.php?title=B
uilding_Kernel_from_source
• http://android-dls.com/wiki/
• http://androiddls.com/wiki/index.php?title=HOWTO:_Unpack,_
Edit,_and_Re-Pack_Boot_Images
• http://source.android.com/source/download.ht
ml