Appearance
ui
The ui module reads the accessibility element tree of the frontmost app. It is useful for locating controls by text or role instead of relying only on screenshots and color matching.
TIP
All coordinates returned by this module are screen pixels, so they can be passed directly to touch.click().
WARNING
The element tree is sampled from the frontmost app. It is not available when SpringBoard is in the foreground, and it may take a few seconds on complex screens.
ui.tree()
Get the UI element tree of the frontmost app.
python
ui.tree() -> dict[str, object]Arguments
None.
Return
python
{
"app": str, # Bundle identifier of the frontmost app.
"pid": int, # Process id of the frontmost app.
"screen": {
"w": int, # Screen width in pixels.
"h": int, # Screen height in pixels.
},
"elements": [
{
"role": str, # Display text of the element.
"label": str, # Optional.
"value": str, # Optional.
"frame": [int, int, int, int], # [x, y, width, height] in pixels.
"center": [int, int], # Center point in pixels.
"center_norm": [float, float], # Center point normalized to 0.0–1.0.
}
],
}label and value are omitted when they are empty or identical to role.
Raises RuntimeError if the accessibility tree is unavailable, the frontmost app is SpringBoard, or the IPC request fails.
Example
python
from catscript import sys, ui
if __name__ == "__main__":
tree = ui.tree()
sys.log(f"app={tree['app']} elements={len(tree['elements'])}")
for element in tree["elements"]:
sys.log(element.get("role"))ui.elements()
Get only the element list from
ui.tree().
python
ui.elements() -> list[dict[str, object]]Arguments
None.
Return
The elements list described in ui.tree().
Example
python
from catscript import sys, ui
if __name__ == "__main__":
for element in ui.elements():
sys.log(element.get("role"))ui.find()
Find the first matching element.
python
ui.find(
role: str | None = None,
label: str | None = None,
value: str | None = None,
contains: bool = False,
index: int = 0,
) -> dict[str, object] | NoneArguments
| Name | Type | Default | Description |
|---|---|---|---|
| role | str | None | None | Match the role field. |
| label | str | None | None | Match the label field. |
| value | str | None | None | Match the value field. |
| contains | bool | False | Use substring matching instead of exact matching. |
| index | int | 0 | Index of the match to return. |
Return
The matching element dictionary, or None if no element matches.
- Multiple criteria are combined with AND.
- If an element has no
labelfield,labelmatching falls back torole. - Raises
ValueErrorifindex < 0.
Example
python
from catscript import touch, ui
if __name__ == "__main__":
element = ui.find(label="Wi-Fi")
if element:
touch.click(*element["center"])ui.wait()
Wait until a matching element appears.
python
ui.wait(
role: str | None = None,
label: str | None = None,
value: str | None = None,
contains: bool = False,
timeout: float = 10.0,
interval: float = 0.5,
) -> dict[str, object] | NoneArguments
| Name | Type | Default | Description |
|---|---|---|---|
| role | str | None | None | Match the role field. |
| label | str | None | None | Match the label field. |
| value | str | None | None | Match the value field. |
| contains | bool | False | Use substring matching instead of exact matching. |
| timeout | float | 10.0 | Timeout in seconds. |
| interval | float | 0.5 | Polling interval in seconds. |
Return
The matching element dictionary, or None if the timeout expires.
Raises ValueError if interval <= 0.
Example
python
from catscript import touch, ui
if __name__ == "__main__":
element = ui.wait(label="Allow", timeout=5)
if element:
touch.click(*element["center"])ui.tap()
Tap an element.
python
ui.tap(
element: dict[str, object] | None = None,
role: str | None = None,
label: str | None = None,
value: str | None = None,
contains: bool = False,
) -> boolArguments
| Name | Type | Default | Description |
|---|---|---|---|
| element | dict[str, object] | None | None | Element to tap. Takes priority over the criteria. |
| role | str | None | None | Match the role field. |
| label | str | None | None | Match the label field. |
| value | str | None | None | Match the value field. |
| contains | bool | False | Use substring matching instead of exact matching. |
Return
| Type | Description |
|---|---|
| bool | True if an element was found and tapped, otherwise False. |
Raises ValueError if the element has no center field.
Example
python
from catscript import ui
if __name__ == "__main__":
ui.tap(label="Wi-Fi")