Class elm
Represents a UI element. Clicks, gets properties, etc.
public sealed class elm : IDisposable
Remarks
UI elements are user interface (UI) parts that are accessible through programming interfaces (API). For example buttons, links, list items. This class can find them, get properties, click, etc. Web pages and most other windows support UI elements.
An elm variable contains a COM interface pointer (IAccessible or other) and uses methods of that interface or/and related API.
elm functions that get properties don't throw exception when the COM etc method failed (returned an error code of HRESULT type). Then they return ""
(string properties), 0, false
, null
or empty collection, depending on return type. Applications implement UI elements differently, often with bugs, and their COM interface functions return a variety of error codes. It's impossible to reliably detect whether the error code means an error or the property is merely unavailable. These elm functions also set the last error code of this thread = the return value (HRESULT) of the COM function, and callers can use lastError to get it. If lastError.code returns 1 (S_FALSE), in most cases it's not an error, just the property is unavailable. On error it will probably be a negative error code.
You can dispose elm variables to release the COM object, but it is not necessary (GC will do it later).
An elm variable cannot be used in multiple threads. Only Dispose can be called in any thread.
UI elements are implemented and live in their applications. This class just communicates with them.
Many applications have various problems with their UI elements: bugs, incorrect/nonstandard/partial implementation, or initially disabled. This class implements workarounds for known problems, where possible.
Known problematic applications
Application | Problems |
---|---|
Chrome web browser. Also Edge, Opera and other apps that use Chromium. Window class name is like "Chrome_WidgetWin_1" . |
|
Firefox web browser. |
|
Applications (or just some windows) that don't have accessible objects but have UI Automation elements. |
|
Java applications that use AWT/Swing. Window class name starts with "SunAwt" . |
|
Some controls. |
|
Some controls with flag EFFlags.NotInProc. |
UI elements of many standard Windows controls have bugs when they are retrieved without loading dll into the target process (see EFFlags.NotInProc). Known bugs:
|
When cannot load dll into the target process. For example Windows Store apps. |
|
Processes of a different 32/64 bitness than this process. |
|
DPI-scaled windows (see Dpi.IsWindowVirtualized). |
|
Examples
Click link "Example"
in Chrome.
var w = wnd.find(0, "* Chrome");
var e = w.Elm["web:LINK", "Example"].Find(5);
e.Invoke();
Click a link, wait for new web page, click a link in it.
var w = wnd.find(0, "* Chrome");
var e = w.Elm["web:LINK", "Link 1"].Find(5);
e.WebInvoke();
w.Elm["web:LINK", "Link 2"].Find(5).WebInvoke();
Namespace: Au
Assembly: Au.dll
Properties
Name | Description |
---|---|
ChildCount | Gets the number of direct child UI elements. |
DefaultAction | Gets default action of elm.Invoke. |
Description | Gets description. |
Elm | Gets an elmFinder for finding UI elements in this UI element.
Example: |
Help | Gets help text. |
IsChecked | Calls elm.State and returns |
IsChecked2 | Calls elm.State and returns |
IsDisabled | Calls elm.State and returns |
IsFocused | Calls elm.State and returns |
IsInvisible | Calls elm.State and returns |
IsOffscreen | Calls elm.State and returns |
IsPassword | Calls elm.State and returns |
IsPressed | Calls elm.State and returns |
IsReadonly | Calls elm.State and returns |
IsSelected | Calls elm.State and returns |
Item | Gets or changes simple element id, also known as child id. |
KeyboardShortcut | Gets keyboard shortcut. |
Level | Gets or sets indentation level for elm.ToString. |
MiscFlags | Returns some additional info about this variable, such as how the UI element was retrieved (inproc, UIA, Java). |
Name | Gets name. |
Parent | Gets parent element. Same as elm.Navigate with argument |
Rect | Gets location of this UI element in screen. |
Role | Gets standard or custom role, as string. |
RoleInt | Gets role as enum ERole. |
SelectedChildren | Gets selected direct child items. |
State | Gets UI element state (flags). |
UiaCN | Gets the UI Automation ClassName property. |
UiaId | Gets the UI Automation AutomationId property. |
Value | Gets or sets value. |
WndContainer | Gets the container window or control of this UI element. |
WndTopLevel | Gets the top-level window that contains this UI element. |
path | Gets an elmFinder for finding UI elements in a window or UI element that can be set later with elmFinder.In.
Example: |
Methods
Name | Description |
---|---|
Check(bool, Action<elm>) | Checks or unchecks this checkbox or toggle-button, or selects this radio button. To check/uncheck calls callback function. |
Check(bool, string) | Checks or unchecks this checkbox or toggle-button, or selects this radio button. Uses elm.Invoke or elm.SendKeys. |
ComboSelect(string, string, double) | Finds and selects an item in the drop-down list of this combo box or drop-down button. |
Dispose() | Releases COM object and clears this variable. |
Expand(Strings, Action<elm>, double, bool) | Expands multiple treeview control items using a path string. |
Expand(Strings, string, double, bool) | Expands multiple treeview control items using a path string. |
Expand(bool, Action<elm>, double, bool) | Expands or collapse this expandable UI element (tree item, combo box, expander, dropdown button). |
Expand(bool, string, double, bool) | Expands or collapse this expandable UI element (tree item, combo box, expander, dropdown button). |
Focus(bool) | Makes this UI element focused for keyboard input. |
GetProperties(string, out EProperties) | Gets multiple properties. |
GetRect(out RECT, wnd, bool) | Gets location of this UI element in the client area of window w. |
GetRect(out RECT, bool) | Gets location of this UI element in screen. |
Html(bool) | Gets HTML. |
HtmlAttribute(string) | Gets a HTML attribute. |
HtmlAttributes() | Gets all HTML attributes. |
Invoke() | Performs the UI element's default action (see elm.DefaultAction). Usually it is "click", "press" or similar. |
JavaInvoke(string) | Performs one of actions supported by this Java UI element. |
MouseClick(Coord, Coord, MButton, int) | Clicks this UI element. |
MouseClickD(Coord, Coord, int) | Double-clicks this UI element. |
MouseClickR(Coord, Coord, int) | Right-clicks this UI element. |
MouseMove(Coord, Coord, int) | Moves the cursor (mouse pointer) to this UI element. |
Navigate(string, double) | Gets an adjacent or related UI element - next, child, parent, etc. |
PostClick(Coord, Coord, MButton, int) | Posts mouse-click messages to the container window, using coordinates in this UI element. |
PostClickD(Coord, Coord) | Posts mouse-double-click messages to the container window, using coordinates in this UI element. |
PostClickR(Coord, Coord) | Posts mouse-right-click messages to the container window, using coordinates in this UI element. |
ScrollTo() | Scrolls this UI element into view. |
Select(ESelect) | Selects or deselects. |
SendKeys(params KKeysEtc[]) | Makes this UI element focused (elm.Focus) and calls keys.send. |
ToString() | Formats string from main properties of this UI element. |
WaitFor<T>(Seconds, Func<elm, T>) | Waits for a user-defined state/condition of this UI element. For example enabled, checked, changed name. |
WebInvoke(Seconds?, Action<elm>) | Calls elm.Invoke or action and waits until changes the web page (window name and page name). |
focused(EFocusedFlags) | Gets the keyboard-focused UI element. |
fromEvent(wnd, EObjid, int) | Gets the UI element that generated the event that is currently being processed by the callback function used with API SetWinEventHook or WinEventHook. |
fromMouse(EXYFlags) | Gets UI element from mouse cursor (pointer) position. |
fromWindow(wnd, EObjid, EWFlags) | Gets UI element of window or control. Or some its standard part - client area, titlebar etc. |
fromXY(POINT, EXYFlags) | Gets UI element from point. |
fromXY(int, int, EXYFlags) | Gets UI element from point. |