Generate QM code for UIA using Grok3.

[Davider]

I sent the definition of the built-in member function Acc.__FromPointDPI from QM to Grok3 for it to learn, and then asked it to generate the equivalent QM code for the C# function.
The generated code should be logically correct, but there are two errors(Todo1,Todo2). This might be due to the unique aspects of QM. I searched for relevant information but couldn't find an answer.

PS: If the UIA components used by QM are too outdated and no longer valuable, please ignore this post. I was just curious and wanted to test Grok3's coding capabilities. Also, I would like to know, does QM use UIA2 or UIA3?

Macro UIA2
 

Code:

Copy      Help

UIA.CUIAutomation-- u._create
UIA.IUIAutomationElement e = u.GetFocusedElement
UIA.IUIAutomationElementArray children = e.FindAll(UIA.TreeScope_Children, u.CreateTrueCondition)

str draftName = "HomePageDraftTitle:3月8日"
int found = 0
UIA.IUIAutomationElement c
for _i 0 children.Length
,;Get the Name property of the control
,c=children[_i] ;;Todo
,str name = c.GetCurrentPropertyValue(UIA.UIA_NamePropertyId)
,if(name == draftName)
,,out name
,,found = 1

,,; Get the parent control
,,UIA.IUIAutomationElement parent = c.GetCurrentPattern(UIA.UIA_LegacyIAccessiblePatternId).GetIAccessible.Parent ;;Todo

,,if (parent)
,,,out parent.CurrentName
,,,; Simulate click
,,,parent.DoDefaultAction
,,break

if (!found)
,out "No draft named %s found" draftName

Member function Acc.__FromPointDPI

Code:

Copy      Help

function x y

;Same as FromXY, but with many objects works correctly on Win8.1+ in DPI-scaled windows.
;Still does not work correctly with many objects, eg almost all Windows controls and window parts, but works with IE, FF, Chrome, many other non-Windows objects.


#opt nowarnings 1
#if _winver>=0x603 ;;note: no UIA typelib on XP/Vista
if DpiIsWindowScaled(win(x y))
,;get IUIAutomationElement from point and convert to Acc. It fails with many objects, but works in web browsers.
,UIA.CUIAutomation-- u._create
,UIA.tagPOINT p.x=x; p.y=y
,UIA.IUIAutomationElement e=u.ElementFromPoint(p)
,UIA.IUIAutomationLegacyIAccessiblePattern k=e.GetCurrentPattern(UIA.UIA_LegacyIAccessiblePatternId)
,a=k.GetIAccessible; elem=0
,if(a) ret
,err+
#endif

this=acc(x y 0)
err end ERR_OBJECTGET

[Gintaras]

QM uses UIA3. Its API is defined in a type library that comes with Windows (and therefore includes everything supported on that Windows version). In QM its name is UIA.

[Davider]

Thanks for your explanation.

If QM also uses UIA3, as I understand it, the descriptions/properties generated in LA's "Find UI Element" dialog for locating controls can also be referenced in QM? I found that LA and QM have very similar ways of locating elements.

In QM's "Find Accessible Object" dialog, I can directly locate the object I am looking for (as shown in the demonstration from the link), but I am unable to retrieve the `FullDescription` property value. In many Qt-developed software applications, the FullDescription property value is very useful for locating controls.

I frequently encounter this situation when using QM. Currently, my solution is to embed general C# code, but if I could use QM code instead, the execution speed would likely be faster.

[Gintaras]

My previous answer was incomplete. Only code with UIA uses UIA3 (eg your posted code). QM accessible object functions (Acc, acc) and the tool use MSAA, not UIA3 or UIA2.

[Gintaras]

QM UIA examples.

Macro UIA example - find by name and click
 

Code:

Copy      Help

int w=win("Edge" "Chrome_WidgetWin_1")
str name="Extensions"
int nameFlags=0 ;;1 case-insensitive, 2 match substring

UIA.CUIAutomation8 uia._create
UIA.IUIAutomationElement ew=uia.ElementFromHandle(+w)
UIA.IUIAutomationCondition pc=uia.CreatePropertyConditionEx(UIA.UIA_NamePropertyId name nameFlags)
UIA.IUIAutomationElement ef=ew.FindFirst(UIA.TreeScope_Descendants pc)
out ef.CurrentName

sub.UiaClick ef


#sub UiaClick
function UIA.IUIAutomationElement&e

UIA.tagPOINT p
if 0!=e.GetClickablePoint(p)
,lef p.x p.y
else
,UIA.tagRECT r=e.CurrentBoundingRectangle
,lef r.left+r.right/2 r.top+r.bottom/2


#sub UiaClickAt
function UIA.IUIAutomationElement&e x y

UIA.tagRECT r=e.CurrentBoundingRectangle
lef r.left+x r.top+y

Macro UIA example - find by description and get parent
 

Code:

Copy      Help

int w=win("Edge" "Chrome_WidgetWin_1")
str desc="Refresh"
int descFlags=2 ;;1 case-insensitive, 2 match substring

UIA.CUIAutomation8 uia._create
UIA.IUIAutomationElement ew=uia.ElementFromHandle(+w)
UIA.IUIAutomationCondition pc=uia.CreatePropertyConditionEx(UIA.UIA_FullDescriptionPropertyId desc descFlags)
UIA.IUIAutomationElement6 ef=+ew.FindFirst(UIA.TreeScope_Descendants pc)
out ef.CurrentFullDescription

UIA.IUIAutomationTreeWalker walk=uia.RawViewWalker
UIA.IUIAutomationElement ep=walk.GetParentElement(+ef)
UIA.tagRECT r=ep.CurrentBoundingRectangle
out F"{r.left} {r.top}"

Macro UIA example - find by type and name

Code:

Copy      Help

int w=win("Edge" "Chrome_WidgetWin_1")
int role=UIA.UIA_ButtonControlTypeId
str name="Extensions"
int nameFlags=0 ;;1 case-insensitive, 2 match substring

UIA.CUIAutomation8 uia._create
UIA.IUIAutomationElement ew=uia.ElementFromHandle(+w)
UIA.IUIAutomationCondition pc=uia.CreateAndCondition(uia.CreatePropertyCondition(UIA.UIA_ControlTypePropertyId role) uia.CreatePropertyConditionEx(UIA.UIA_NamePropertyId name nameFlags))
UIA.IUIAutomationElement ef=ew.FindFirst(UIA.TreeScope_Descendants pc)
out ef.CurrentName

[Davider]

thank you so much!
These codes are very useful. After submitting them to Grok3, its understanding ability is very strong. For similar problems, the QM code can now be generated automatically.