tok explanations

[ldarrambide]

Gintaras,

in tok documentation you state:

"Function tok splits string and stores tokens (parts of the string) into array arr. It can be safe array (ARRAY(str) or ARRAY(lpstr)) or pointer-based array"

I understand and use ARRAY(st) easily. As lpstr type must be similar to LPSTR C++ (alias for char *) it's already a pointer, and can use it with using index to get tokens.

I don't see how you use pointer-based array of lpstr in tok function.

An example to help understand this?

Thanks

[Gintaras]

Usually as "pointer-based array" I use the address of the first of several local variables.
Macro Macro2738

Code:

Copy      Help

str s="zero one two"
str s1 s2 s3
int n=tok(s &s1 3)
out s1
out s2
out s3

Example with lpstr.
Macro Macro2738

Code:

Copy      Help

str s="zero one two"
lpstr s1 s2 s3
int n=tok(s &s1 3) ;;or int n=tok(s &s1 3 "" 1)
out s1
out s2
out s3

[ldarrambide]

hmmm, i've got problem to get this in my mind

pointers base array should be ARRAY(lpstr*) b no???

How does the magic happen that str s1 s2 s3 becomes a pointer base array?

You only pass reference (i suppose it's memory address) of s1 (which is not a pointer) to tok function, and it can fill s2 and s3 based on n=3???

and why this is supposed to be faster than using lpstr array,

ARRAY(lpstr) a
tok _s &a -1 " " 1


Wouaa Gintaras, it burns my brain!!!!

[Gintaras]

Pointer-based array = raw C/C++ pointer. QM local variables, when declared like this, in memory are like array.

Help topics:
tok and pointer-based arrays
Pointer, reference, array

Not much faster. Just don't need to allocate memory for ARRAY.

[ldarrambide]

yes, i read those, but kinda cryptic....

[Gintaras]

Pointer-based array = raw C/C++ pointer. QM local variables, when declared like this, in memory are like array.

Not much faster. Just don't need to allocate memory for ARRAY.

[ldarrambide]

ok, so QM internals treats declaration

str s1 s2 s3

as

ARRAY(str) s.create(3) because s1 s2 s3 have continuous memeory allocation, and &s2=&s1+sizeof(str)????

[ldarrambide]

Usually as "pointer-based array" I use the address of the first of several local variables.
Example with lpstr.
Macro Macro2738

Code:

Copy      Help

str s="zero one two"
lpstr s1 s2 s3
int n=tok(s &s1 3) ;;or int n=tok(s &s1 3 "" 1)
out s1
out s2
out s3

yes, but size of str to tokenize is unknown, i'm forced to use

ARRAY(lpstr) s
int n=tok(s &s 3)
for _i 0 s.len
out s[_i]

correct?

[Gintaras]

str s1 s2 s3 is not ARRAY, it is single block of thread stack memory of size = 3*sizeof(str) which is 3*16.
Usually it's easier to use str, but with lpstr faster because don't need to allocate stack memory for each str string.

[Gintaras]

The slowest part usually is heap memory allocation.
Assume there are 3 tokens. How many allocations of heap memory:

ARRAY(str): 5.
ARRAY(lpstr): 2.
str*: 3.
lpstr*: 0.

[ldarrambide]

ok, but i still don't get how tok function (assuming 3 tokens to find)

fills s1 s2 s3 str variables given only address of s1 by reference..must be stupid but really can't understand this. I can understand with array(str)
as it points to the first element of the array.

str s="zero one two"
str s1 s2 s3
int n=tok(s &s1 3)
out s1
out s2
out s3

this is a mystery of inner QM managment of the tok function...