幻灯片 1 - system
Download
Report
Transcript 幻灯片 1 - system
introduction to CTS
Sep 2, 2010
Jingtao
agenda
back ground
CTS history
CTS work flow
download, install and run CTS
check CTS report
typical fail cases
background
Google define an “Android compatibility” device as one that can run any application written by
third-party developers using the Android SDK and NDK.
Device that are properly compatible can seek approval to use Android trademark and Android
Market client software.
5
how to verify compatibility
the Android Compatibility Definition Document(CDD) spells out the specific device
configuration that will be considered compatible.
CTS(an automatically compatibility test tool) test
Third party application test
6
the process to get certification
Google release CTS according to each Android platform publicly.
manufacturer run CTS on their device.
manufacture fix any bug found by CTS.
if manufacture think any CTS case is wrong, he should upload the
CTS report and detail description to Google's database.
if Google accept manufacture's description, he will grant wavier to
that device.
Google will think the device is Android compatible if it pass all the
CTS case or all the fail cases have waiver.
7
requirement in CDD
• software API/UI compatibility
• reference app test
• application packaging compatibility
• multimedia compatibility
• developer tool compatibility
• hardware compatibility
• performance compatibility
• security model compatibility
• system update
hardware compatibility
•
display
•
keyboard
•
Non-touch navigation
•
screen orientation
•
touch-screen input: must have a touch screen
•
USB: must support adb and mass storage mode
•
navigation keys: function of home, menu and back must be available to user at any time.
•
wireless data networking: must support for at least one wireless data standard capable of 200kbit/sec or greater.
•
camera: must include a rear-facing camera, at least 2 megapixels
•
accelerometer: must include a 3-axis accelerometer and must be able to deliver events at 50 Hz or greater.
•
compass: must include a 3-axis compass and must be able to deliver events 10 Hz or greater.
•
GPS: must include a GPS receiver
•
telephony
•
memory and storage: /data partition must be at least 150MB and must have at least 92MB memory.
•
application shared storage: must be at least 2GB in size.
•
Bluetooth: must include a Bluetooth transceiver and enable the RFCOMM-based Bluetooth API.
What is CTS
Compatibility Test System
open source
commercial-grade(20000+ cases)
Automation compatibility test tool
what does it test
• Signature
• Platform API
• Dalvik VM
• Platform Data Model
• Intents
• Permissions
• Resources
CTS History
• Android1.0 - 16784 cases
• Cupcake - 21383
• Donut - 22756
• Eclair r3 - 23138
• Froyo r2 - 23830
CTS work flow
download CTS
http://source.android.com/compatibility/downloads.html
1, git ls-remote --heads git://192.168.1.250/android-mirror/platform/manifest.git
...
refs/heads/android-cts-2.1_r3
refs/heads/android-cts-2.1_r4
refs/heads/android-cts-2.2_r1
refs/heads/android-cts-2.2_r2
...
2, repo init -u git://192.168.1.250/android-mirror/platform/manifest.git -b android-cts-2.2_r2
3, repo sync cts
build CTS
• svn://192.168.1.250/oms/branches/horse
make BUILD_CTS=true cts
install CTS
install Android SDK
unzip android-cts-2.1_r3-x86.zip
android-cts
|-- docs
|-- repository
`-- tools
|-- cts.jar
`-- startcts
edit android-cts/tools/startcts, set
SDK_ROOT=<absolute fold of Android SDK>
run CTS
connect phone to PC
android-cts/tools/startcts to enter CTS shell
3 mode to start test:
plan mode
package mode
case mode
Sample of CTS commands
• start --plan CTS
• start --plan Android -p
android.app.cts.ActivityManagerTest
• start --plan Android -t
android.app.cts.ActivityManagerTest#testGe
tRunningServices
what does CTS do
install test case apk
(adb -s deviceID install -r ...apk)
execute test
(am instrument -w -e [para] <component>...)
uninstall test case apk
(adb -s deviceID uninstall ...)
run CTS under Windows
1. unzip CTS package and android SDK package to your windows PC.
2. Modified the SDK_ROOT in cts\android-cts\tools\startcts :
SDK_ROOT=D:/CTS/sdk/android-sdk_r05-windows/android-sdk-windows/tools
3. use below command to enter CTS shell:
java -Xmx512M -cp D:\CTS\cts\android-cts\tools\cts.jar; D:\CTS\cts\androidcts\tools\hosttestlib.jar;D:\\lib\ddCTS\cts\android-cts\tools\junit.jar;D:\CTS\sdk\android-sdk_r05windows\android-sdk-windows\tools\lib\ddmlib.jar com.android.cts.TestHost D:\CTS\cts\androidcts\repository\host_config.xml
Please change red marked to you current file path.
known issue:
The file (cts_result.css, cts_result.xsl, logo.gif, newrule-green.png ) under CTS result can not
automatically copy in new session folder.
Attention:
If you need to clear the data in addon emulator, Please use: emulator @addon -wipe-data
config phone before run CTS
empty SD card
never Screen Timeout
English locale
install and set input method to LatinIME
open WIFI and GPRS
correct date/time
/opl/etc/mountd.conf, system.prop
CtsDelegatingAccessibilityService.apk
test report
firefox repository/results/20xx.../testResult.xml
Fail Cases
Case Study: CTS on FBW1.4
Waivered
Internal resource ID: 38 cases
Locale related: 11 cases
Others: 4 cases
Fixed certification time: 3 cases
Fixed camera parameter: 1 case
UI related( For example, check box layout): 6 cases
Mock Locations: 22 cases (just test in eng build)
Waiting for waiver
Google Search dependant: 2 cases
no popup window when application force close
Public Resource ID
• http://developer.android.com/reference/andr
oid/R.html
• Frameworks/base/core/res/res/values/public
.xml
public static final int
accelerate_decelerate_interpolator
Constant Value: 17432580 (0x010a0004)
Internal Resource ID
out/target/common/R/com/android/internal/R.java
com.android.internal.R.id.alertTitle
cupcake: 0x0102014e
borqs:
0x01020167
168
String expected = dateFormat.format(mCalendar.getTime());
169
TextView tv = (TextView)
d.getWindow().findViewById(com.android.internal.R.id.alertTitle);
170
assertEquals(expected, tv.getText());
resource ID
• R.id.contentPanel
• com.android.internal.R.id.topPanel
• R.id.alertTitle
• R.id.custom
Mock Location
• 84
// test that mock locations are allowed so a more descriptive
error message can be logged
• 85
• 86
if (Settings.Secure.getInt(mContext.getContentResolver(),
Settings.Secure.ALLOW_MOCK_LOCATION, 0) == 0) {
• 87
fail("Mock locations are currently disabled in Settings - this
test requires "
• 88
+ "mock locations");
• http://androidappdocs.appspot.com/guide/topics/location/index.html
• during initial development in the emulator, you may not have access to
real data from a real location provider (Network or GPS). In that case,
it may be necessary to spoof some data for your application using a
mock location provider.
Settings-> Applications-> Development
check box
camera
• android.hardware.cts.CameraTest#testAccessParameters
junit.framework.AssertionFailedError: expected:<2048> but was:<2560> at
android.hardware.cts.CameraTest.assertParameters(CameraTest.java:392)
381
private void assertParameters(Parameters parameters) {
...
387
final int ORIGINALPICWIDTH = 2048;
388
final int ORIGINALPICHEIGHT = 1536;
389
390
// Before Set Parameters
391
assertEquals(PixelFormat.JPEG,
parameters.getPictureFormat());
392
assertEquals(ORIGINALPICWIDTH,
parameters.getPictureSize().width);
Case Study: CTS on OPhone 2.0
Case number: 23107 cases
Nexus One
Fail cases:
3
Execut time: 6 h
Oscar-r42369
S1-r49137
34
5
6h
4h
fail case summary on Oscar
• internal resource ID: 17
• UI: 8. (check box, grid layout, list padding, pixel
buffer)
• phone number utils: 4
• network type: 1
• SD card write permission: 1
• CTS case issue: 3
phone number util cases(92585)
• android phone do not have sys.min.match.digits but can pass
• set sys.min.match.digits to 7 can pass the case but phone will not
match incoming call correctly.
• e.g. create user A(tel: 01150539) and user B(mobile:13701150539) in
sequence in Contacts, when mobile 13701150539 incoming call,
phone will always show user A incorrectly on screen.
Test Package
android.telephony.cts.PhoneNumberUtilsTest
Test Cases
testCallMethods
android.telephony.cts.PhoneNumberUtilsTest
testCompareLoosely
android.telephony.cts.PhoneNumberUtilsTest
testCompareStrictly
android.telephony.cts.PhoneNumberUtilsTest
testJudgeMethods
comments
According to CTA, we need
to set sys.min.match.digits
to 11. These 4 cases will
pass if set the value to 7.
Sdcard write permission case
• android.permission.cts.NoSdCardWritePermissionTest
• android.permission.WRITE_EXTERNAL_STORAGE
uid=1000: sysetm user
gid=1015: Sdcard write group
SDcard: android
075
oms(before)oms(fixed)
777
775
Ipv6 related fail
• tests.api.java.net.SocketTest#test_getLocalAddre
ss
junit.framework.AssertionFailedError:
ANY address not returned correctly (getLocalAddress) with
preferIPv6Addresses=true,
preferIPv4Stack=false
0.0.0.0/0.0.0.0:49380 at
tests.api.java.net.SocketTest.test_getLocalAddress(Socket
Test.java:645)
known failure actually passed
• android.provider.cts.ContactsTest#testGroupMembershipTable
bug 2258907, needs investigation
will pass after install gms. the gms package link:
smb://192.168.0.146/share/gms
user: b184
pass: zhanghui
rule to pass CTS
Do not modify framework.
review with SDK team before modify
run related CTS cases after modify
summary
• CTS can block SA
• manufacture will submit ticket relate to CTS
• we can help you setup CTS env
41