Android, low level shell click on screen

Android, low level shell click on screen

To send touch event you need to do:

1 Set coordinates:

adb shell sendevent /dev/input/event2 3 0 x

adb shell sendevent /dev/input/event2 3 1 y

2 Send touch event (must have 0 0 0 pair):

adb shell sendevent /dev/input/event2 1 330 1

adb shell sendevent /dev/input/event2 0 0 0

3 Send release finger event (must have 0 0 0 pair):

adb shell sendevent /dev/input/event2 1 330 0

adb shell sendevent /dev/input/event2 0 0 0

Please note:

1 You can record events:

adb shell getevent

2 if you use getevent all event values are in hex.

CONFIRMED, THIS WAY IS NOT WORKING ANYMORE, at least for Android 2.2.

I have rechecked, Android low level events signature was changed starting from Android 2.2 Froyo.

I just dumped single click:

/dev/input/event2: 0003 0030 00000068
/dev/input/event2: 0003 0032 0000000a
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 0000002f
/dev/input/event2: 0003 0014 00000001
/dev/input/event2: 0000 0000 00000000
/dev/input/event2: 0003 0030 00000000
/dev/input/event2: 0003 0032 0000000a
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 00000020
/dev/input/event2: 0003 0014 00000000
/dev/input/event2: 0000 0000 00000000

It has obviously another structure.. I'm investigating right now this situation,

solution will be here tomorrow ;)

After a quick investigation I have reproduced the click for Froyo Android:

Perl script (click on screen at 558, 57 point):

system("adb shell sendevent /dev/input/event2 0003 48 104");
system("adb shell sendevent /dev/input/event2 0003 50 10");
system("adb shell sendevent /dev/input/event2 0003 53 558");
system("adb shell sendevent /dev/input/event2 0003 54 57");
system("adb shell sendevent /dev/input/event2 0000 2 00000000");
system("adb shell sendevent /dev/input/event2 0003 18 47");
system("adb shell sendevent /dev/input/event2 0003 20 00000001");
system("adb shell sendevent /dev/input/event2 0000 0000 00000000");
system("adb shell sendevent /dev/input/event2 0003 48 00000000");
system("adb shell sendevent /dev/input/event2 0003 50 10");
system("adb shell sendevent /dev/input/event2 0003 53 558");
system("adb shell sendevent /dev/input/event2 0003 54 57");
system("adb shell sendevent /dev/input/event2 0000 0002 00000000");
system("adb shell sendevent /dev/input/event2 0003 18 32");
system("adb shell sendevent /dev/input/event2 0003 20 00000000");
system("adb shell sendevent /dev/input/event2 0000 0000 00000000");

How I did it:

1. Start dump motion event you need to reproduce:

~$ adb shell getevent | grep event2

grep is very useful to filter output.

2. Do motion event you want to reproduce;

3. Than just convert all values from hex in dump to decimal values! :)

To find what eventX is working for you do following:

1. Start terminal and type

~$ adb shell getevent

You will see quickly moving traces with for example /dev/input/event4 ......

2. Touch screen once

You must see between event4 few eventX and these eventX right in the moment of the touch

will be yours input interface for reproducing motion events! :)

Please check my tutorial for low level interaction with device

Happy motion events reproducing! (:

Best regards,

Yahor

Posted by Yahor Paulavets at 7



----------------------------------------------------------------

Android, writing events low level touch screen automated shell test, part #1

Hello everybody,

Some information I have already covered in the article:
How to click screen using shell script (Froyo)
- In the beginning of this article you will find how to reproduce click for before Froyo Android

So lets start from the beginning, for implementation of automated test you need only 2 simple and essential shell commands:

~$ adb shell getevent
~$ adb shell sendevent $device $command $argument $value

First command is dumping all events from all interfaces (usually /dev/input/event0-4) in real time into the terminal output. For high usability you can use grep with regular expression to set a filter for output (it is so useful because of there are a lot of events out of our interest)

(리눅스에서 입력 디바이스에 대한 장치파일은 /dev/input 에서 관리)



for example, we need to trace only touch interface events:
-- Assume /dev/input/event2 is a touch screen interface



~$ adb shell getevent | grep event2

or we need to get only click coordinates:

~$ adb shell getevent | grep event2:\ 0003\ 003[56]

By this way we can very simply determinate UI elements position at the screen! Sure, after we will conver hexa values of getevent to the decimal :) For this you can use a pretty nice online tool with various converters on the board: easy calculation converter
For checking consistency of your regular expressions, you are using with grep command, you are welcome to use: my favourite regexpal, sure you can use your favourite tools :)

Okay, lets try to determinate what roadblocks you may have firing this commands:
1. "adb is not a command" or something like this
- Check in what folder you are, and change it to the .../android_sdk_location/tools
- adb (Android debug bridge) is situated here

2. "adb server is not running" or something like this
- try to start adb daemon using command adb start-server

3. You have fired an event, all was properly set, but nothing happens!
- Make sure you are not using HEXA args/values, when you are using sendevent command
- you must specify all data in the decimal format (0-9)

Let me provide you with some examples:

~$ adb shell getevent | grep event2

-> I'm doing single tap on the screen

Following events sequence will be dumped:

/dev/input/event2: 0003 0030 00000068
/dev/input/event2: 0003 0032 0000000a
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 0000002f
/dev/input/event2: 0003 0014 00000001
/dev/input/event2: 0000 0000 00000000
/dev/input/event2: 0003 0030 00000000
/dev/input/event2: 0003 0032 0000000a
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 00000020
/dev/input/event2: 0003 0014 00000000
/dev/input/event2: 0000 0000 00000000

Okay, now we can analyse it and determinate what these numbers mean:

1. Finger is touching the screen:
/dev/input/event2: 0003 0030 00000068
/dev/input/event2: 0003 0032 0000000a

2. At screen coordinates:
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039

3. Still not recognized part, but I assume it describes number of pointers and its ID:
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 0000002f
/dev/input/event2: 0003 0014 00000001

4. Pure event separator:
/dev/input/event2: 0000 0000 00000000

5. Finger is releasing the point at the screen:
/dev/input/event2: 0003 0030 00000000
/dev/input/event2: 0003 0032 0000000a

6. At the screen position:
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039

7. Not recognized series, possibly describing number of touches and ID of the current touch:
/dev/input/event2: 0000 0002 00000000
/dev/input/event2: 0003 0012 00000020
/dev/input/event2: 0003 0014 00000000

8. Pure event separator, always should close events sequence
/dev/input/event2: 0000 0000 00000000

So what information can we extract from these dump?
A lot of information! :)

1. How to set X,Y coordinate of the click:
/dev/input/event2: 0003 0035 0000022e
/dev/input/event2: 0003 0036 00000039

Convert it to decimal and you will get the valuable ids:
Where 0003 53 and 0003 54 set x and set y commands
accordignaly. 558 is X, 57 is Y, to reproduce it you can
send these events:

~$ adb shell sendevent /dev/input/event2 0003 53 558
~$ adb shell sendevent /dev/input/event2 0003 54 57

2. I think it will be interesting for you to reflect other events
by yourself :) Please let me know if you do not succeed to do
something yourself.

INFO: Click all params decoded. Shell script sample.

PART 2: implementing simple shell script with ClickOnScreen(x,y) function.

Happy clicking (:

Best regards,
Yahor
-------------------------------------------------------------------------

리눅스에서 입력 디바이스에 대한 장치파일은 /dev/input 에서 관리한다.

기본적인 입력 장치만 있는 경우 다음과 같은 장치들이 보인다.(환경에 따라 다름)

event1, event0, mice, event2, mouse0

만약 USB mouse와 keyboard와 같은 장치를 추가하면 2개의 장치 파일이 추가된다. 예를 들어 mouse를 연결하면 위의 파일에 다음 파일이 추가된다.

event3, mouse1

keyboard를 추가하면 다음과 같다.

event4, keyboard

실제 안드로이드에서는 /dev/input/의 파일들을 이용해서 입력 장치의 핸들을 관리한다. 

리눅스 디바이스 드라이버의 장치를 검색하고 이들 장치을 열어서 핸들을 관리하는 것이 EventHub이다. 

frameworks/base/libs/ui/EventHub.cpp

처음 사용 가능한 장치를 검색하여 등록하는 과정은 다음과 같이 이루어진다.

openPlatformInput()

-> scan_dir()                             

-> open_device()          

scan_dir()은 "/dev/input" 디렉토리를 오픈하고 모든 장치들을 검색하게 한다.

open_devie()는 다음과 같은 동작을 한다.

1) mFDs에 장치를 등록

2) ioctl의 EVIOGBIT 명령을 이용하여 얻어진 속성으로 장치 종류 구별

3) keyboard 장치의 경우 /system/usr/keylayout/ 디렉토리의 kl 파일을 검토하고 로드한 후, property 설정

I/SystemServer(  686): Starting Window Manager.

I/EventHub(  855): New device: path=/dev/input/event3 name=ARROW STRONG USB Mouse id=0x10000 (of 0x1) index=1 fd=46 classes=0x10

E/EventHub(  855): could not get driver version for /dev/input/mouse1, Not a typewriter

I/EventHub(  855): New device: path=/dev/input/event1 name=telechips power key id=0x10001 (of 0x2) index=2 fd=48 classes=0x1

I/EventHub(  855): New keyboard: publicID=65537 device->id=65537 devname='telechips power key'

propName='hw.keyboards.65537.devname' keylayout='/system/usr/keylayout/qwerty.kl'

I/EventHub(  855): New device: path=/dev/input/event0 name=telechips keypad id=0x10002 (of 0x3) index=3 fd=49 classes=0x1

I/EventHub(  855): New keyboard: publicID=0 device->id=65538 devname='telechips keypad'

propName='hw.keyboards.0.devname' keylayout='/system/usr/keylayout/telechips_keypad.kl'

E/EventHub(  855): could not get driver version for /dev/input/mice, Not a typewriter

I/EventHub(  855): New device: path=/dev/input/event2 name=tcc-tsc2003-ts id=0x10003 (of 0x4) index=4 fd=51 classes=0x4

E/EventHub(  855): could not get driver version for /dev/input/mouse0, Not a typewriter

I/KeyInputQueue(  686): Device added: id=0x10002, name=null, classes=4

I/KeyInputQueue(  686):   X: min=10 max=2000 flat=0 fuzz=0

I/KeyInputQueue(  686):   Y: min=140 max=1887 flat=0 fuzz=0

I/KeyInputQueue(  686):   Pressure: unknown values

I/KeyInputQueue(  686):   Size: unknown values

I/KeyInputQueue(  686): Device added: id=0x0, name=telechips keypad, classes=1

I/KeyInputQueue(  686): Device added: id=0x10000, name=telechips power key, classes=1

빨간색으로 표시된 로그는 USB mouse를 연결한 경우 추가되는 장치이다. 파란색으로 된 것은 기본적으로 등록된 장치에 대한 로그이다.

실제 USB mouse나 keyboard의 경우 동작 중에 삽입/제거될 수 있는데 이를 처리하는 과정은 다음과 같다.

getEvent()

-> read_notify()

-> open_device()

-> close_device()

close_device()의 경우 mFDs에 등록된 장치를 제거한다. 

장치를 새로 연결하는 경우의 로그를 보면 다음과 같다. 

usb 1-1.3: new low speed USB device using tcc-ohci and address 3

D/vold    (  786): process_uevent_message

usb 1-1.3: configuration #1 chosen from 1 choice

D/vold    (  786): dispatch_uevent

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

input: ARROW STRONG USB Mouse as /devices/platform/tcc-ohci/usb1/1-1/1-1.3/1-1.3:1.0/input/input3

generic-usb 0003:0AB0:0001.0001: input: USB HID v1.00 Mouse [ARROW STRONG USB Mouse] on usb-tcc-1.3/input0

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

E/EventHub(  847): could not get driver version for /dev/input/mouse1, Not a typewriter

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

I/EventHub(  847): New device: path=/dev/input/event3 name=ARROW STRONG USB Mouse id=0x10003 (of 0x4) index=4 fd=31 classes=0x10

I/KeyInputQueue(  847): Device added: id=0x10003, name=null, classes=10

I/WindowManager(  847): Config changed: { scale=1.0 imsi=0/0 loc=en_US touch=3 keys=1/1/2 nav=1 orien=2 layout=35}

빨간색은 커널에서 USB mouse 연결시의 로그이며 연결이 이루어지면 vold 데몬에서 uevent_message를 먼저 발생시키고 이를 EventHub에서 처리하는 것을 알 수 있다.

EvnetHub의 처리 과정이 끝나면 KeyInputQueue에서 다시 이벤트를 처리하고 최종적으로 WindowManager 서비스로 전달되어 최종적으로 처리된다. WindowManager 서비스는 현재 Activity에 다시 이벤트를 전달하며 여기에서 이를 다시 View로 전달하는 과정으로 처리된다.

장치 연결을 해제하는 경우의 로그를 보면 다음과 같다.

usb 1-1.3: USB disconnect, address 3

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

E/EventHub(  847): remote device: /dev/input/mouse1 not found

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent

I/KeyInputQueue(  847): Device removed: id=0x10003

I/WindowManager(  847): Config changed: { scale=1.0 imsi=0/0 loc=en_US touch=3 keys=1/1/2 nav=1 orien=2 layout=35}

D/vold    (  786): process_uevent_message

D/vold    (  786): dispatch_uevent