GetFilesInFolder - exclude subfolders

[ssimop]

I understand that if flag 0x10000 is used with GetFilesInFolder, you may exclude filenames, using regexp. I also understand that if flag 4 is used as well, you cannot exclude - somehow - particular subfolders.

In this case you can scan ARRAY "a" to further exclude files in particular subfolders. I wonder whether there exists a fast and elegant way to do this, other than a classical "for int'i 0 a.len" loop checking one-by-one pathnames.

Many thanks in advance for any suggestion.

[Gintaras]

Try flag 0x20000. But regular expressions are good to INCLUDE, not to EXCLUDE. Maybe easier with "for int'i 0 a.len", like you say. Or use Dir d; foreach(d "*" FE_Dir)..., not GetFilesInFolder.

[ssimop]

Thank you for useful hints. Let me, please, extend my question regarding "GetFilesInFolder - exclude" by kindly ask you :
(a) That you explain the example you have given in topic Backup vs2010 project folder, simplified as

"(?i)(?<!\.txt)$'

or as applied by me

GetFilesInFolder a "C:\tmp\tmp1" "(?i)(?<!\.txt)$" 4|0x10000

I understand that it refers to lines ending with ".txt", but - although I have tested it and it is perfect - I cannot understand where is the "exclude" signaling.

(b) To help me in composing an exclusion statement, for filenames starting with "abcd".

Many thanks in advance. Best regards.

[ssimop]

My understanding regarding (a) above is :

case insensitive - not preceded by .txt at end-of line

I would please ask you to confirm.

[Gintaras]

Confirmed.

Regular expressions don't have "does not contain". As a workaround, sometimes can be used "not preceded by" or "not followed by".
https://www.google.com/search?q=regular ... ot+contain

[Gintaras]

Function GetFilesInFolderExcept

Code:

Copy      Help

;/
function ARRAY(str)&a $parentFolder [$filenamePattern] [$exceptPattern] [flags] [getProp] ;;flags: 0 files, 1 folders, 2 any, 4 +subfolders, 8 only subfolders, 32 skip symbolic-link subfolders, 64 skip hidden-system, 0x10000 regular expression, 0x20000 compare relative path, 0x100000 exceptPattern regular expression, 0x200000 exceptPattern compare relative path.  getProp: 1 size, 2 time modified, 3 time created, 4 time accessed, 0x100 (flag) don't need folder size

;Gets full paths of files in a folder.

;a - variable for results.
;parentFolder - full path of the folder. Can end with \.
;filenamePattern - if used and not "", gets only files that match filenamePattern. Else gets all files.
;;;Can be full filename, filename with <help #IDP_WILDCARD>wildcard characters</help> (*?), or regular expression (flag 0x10000).
;;;Can be relative path (flag 0x20000).
;;;Case insensitive.
;exceptPattern - if used and not "", skips files that match exceptPattern.
;;;Can be full filename, filename with <help #IDP_WILDCARD>wildcard characters</help> (*?), or regular expression (flag 0x100000).
;;;Can be relative path (flag 0x200000).
;;;Case insensitive.
;;;Note: flags 0x10000/0x20000 are for filenamePattern; flags 0x100000/0x200000 are for exceptPattern.
;flags:
;;;0, 1, 2 - file/folder type. If 0, gets only files. If 1 - only folders. If 2 - all.
;;;4 - also get files in subfolders.
;;;8 - get files only in subfolders, not in this folder.
;;;32 (QM 2.4.0) - don't get files in subfolders that have attribute FILE_ATTRIBUTE_REPARSE_POINT. It is used for symbolic links, junctions and volume mount points, that actually are not subfolders of that folder.
;;;64 (QM 2.4.0) - don't get hidden system files and folders (eg thumbnail cache files).
;;;0x10000 - filenamePattern is regular expression. Note: if need \ characters (if flag 0x20000), use \\.
;;;0x20000 - filenamePattern is path relative to parentFolder. Example: "subfolder\*\*.txt". Will be compared with whole relative paths of files, not just with filenames.
;getProp (QM 2.3.5) - prepend a file property to the path (in a), like "1000 c:\file.txt".
;;;File size is in bytes.
;;;File times are in FILETIME/DateTime format, UTC.
;;;To sort the array by a file property (or just by name, if getProp not used): a.sort(8)
;;;To get the number and path string: see example.
;;;File access time auto-updating on most computers is disabled, therefore it can be useful only if set explicitly.

;See also: <FE_Dir>.


a=0

str s.expandpath(parentFolder)
s.dospath(s 1)
s+iif((s.end("\")||!s.len) "*" "\*")
int isfp=!empty(filenamePattern)
int isep=!empty(exceptPattern)
int getPropFlags=getProp&~255; getProp&255

Dir d; lpstr sPath ss
foreach(d s FE_Dir flags&0xff)
,if isfp
,,if(flags&0x20000) ss=d.RelativePath; else ss=d.FileName
,,if(flags&0x10000) if(findrx(ss filenamePattern 0 1)<0) continue
,,else if(!matchw(ss filenamePattern 1)) continue
,if isep
,,if(flags&0x200000) ss=d.RelativePath; else ss=d.FileName
,,if(flags&0x100000) if(findrx(ss exceptPattern 0 1)>=0) continue
,,else if(matchw(ss exceptPattern 1)) continue
,
,sPath=d.FullPath
,if getProp
,,long x
,,sel getProp
,,,case 1 if(getPropFlags&0x100 and d.IsFolder) x=0; else x=d.FileSize
,,,case 2 x=d.TimeModifiedUTC
,,,case 3 x=d.TimeCreatedUTC
,,,case 4 x=d.TimeAccessedUTC
,,,case else end ERR_BADARG
,,a[].from(x " " sPath)
,else a[]=sPath

err+ end _error ;;error in rx

example - exclude folders "Debug" and "Release" in any depth
Macro Macro522

Code:

Copy      Help

out
ARRAY(str) a
GetFilesInFolderExcept a "$qm$" "" "(?<=^|\\)(Debug\\|Release\\)" 4|0x300000
out a
out a.len

[ssimop]

Excellent ! Many thanks indeed.

[ssimop]

This works :

GetFilesInFolderExcept a "C:\tmp\tmp1" files "*.pdf"

I have two further questions :

(a) I need to exclude ".eml" too (with same as above statement)

(b) I need to exclude all files starting with "abcd" too.

I would appreciate your advice.

[Gintaras]

Use regular expression.

[ssimop]

I had thought so, but trying to exclude .pdf files with the following statement, and although I tried various versions, I failed :

GetFilesInFolderExcept a "C:\tmp\tmp1" "\." "???" 0x10000

I mark with ??? the string failed. Please advise.

[Gintaras]

Wrong flags.

[ssimop]

I am sorry. This is working perfectly :

GetFilesInFolderExcept a "C:\tmp\tmp1" "" ".pdf|.dwd|.eml" 0x100000

Thanks again.

[Gintaras]

This version also supports lists. Then don't need regular expressions for simple cases like several file types. And it will be in next QM.

Function GetFilesInFolder

Code:

Copy      Help

;/
function ARRAY(str)&a $parentFolder [$filenamePattern] [flags] [getProp] [$exceptPattern] ;;flags: 0 files, 1 folders, 2 any, 4 +subfolders, 8 only subfolders, 32 skip symbolic-link subfolders, 64 skip hidden-system, 0x10000 filenamePattern is regexp, 0x20000 filenamePattern is relative path, 0x40000 exceptPattern is regexp, 0x80000 exceptPattern is relative path.  getProp: 1 size, 2 time modified, 3 time created, 4 time accessed, 0x100 (flag) don't need folder size

;Gets full paths of files in a folder.

;a - variable for results.
;parentFolder - full path of the folder. Can end with \.
;filenamePattern - if used and not "", gets only files that match filenamePattern. Else gets all files.
;;;Can be full filename, filename with <help #IDP_WILDCARD>wildcard characters</help> (*?), or regular expression (flag 0x10000).
;;;Can be relative path (flag 0x20000).
;;;Case insensitive.
;;;QM 2.4.4: can be list, like "*.png[]*.gif[]*.jpg".
;flags:
;;;0, 1, 2 - file/folder type. If 0, gets only files. If 1 - only folders. If 2 - all.
;;;4 - also get files in subfolders.
;;;8 - get files only in subfolders, not in this folder.
;;;32 (QM 2.4.0) - don't get files in subfolders that have attribute FILE_ATTRIBUTE_REPARSE_POINT. It is used for symbolic links, junctions and volume mount points, that actually are not subfolders of that folder.
;;;64 (QM 2.4.0) - don't get hidden system files and folders (eg thumbnail cache files).
;;;0x10000 - filenamePattern is regular expression. Note: if need \ characters (if flag 0x20000), use \\.
;;;0x20000 - filenamePattern is path relative to parentFolder. Example: "subfolder\*\*.txt". Will be compared with whole relative paths of files, not just with filenames.
;;;0x40000 - like 0x10000 but is used with exceptPattern.
;;;0x80000 - like 0x20000 but is used with exceptPattern.
;getProp (QM 2.3.5) - prepend a file property to the path (in a), like "1000 c:\file.txt".
;;;File size is in bytes.
;;;File times are in FILETIME/DateTime format, UTC.
;;;To sort the array by a file property (or just by name, if getProp not used): a.sort(8)
;;;To get the number and path string: see example.
;;;File access time auto-updating on most computers is disabled, therefore it can be useful only if set explicitly.
;exceptPattern (QM 2.4.4) - if used and not "", skips files that match exceptPattern.
;;;Everything like with filenamePattern, but are used other flags (0x40000/0x80000).

;See also: <FE_Dir>.

;EXAMPLES
;ARRAY(str) a; int i
;GetFilesInFolder a "$desktop$"
;a.sort(8)
;for i 0 a.len
,;out a[i]
,;Dir d.dir(a[i])
,;out d.FileSize

;GetFilesInFolder a "$system$" "*.exe"
;GetFilesInFolder a "$system$" "^.*\.exe$" 0x10000

;;get file time and path, sort by file time, and display all in readable format
;ARRAY(str) a; int i
;GetFilesInFolder a "$my qm$" "" 0 2
;a.sort(8)
;for i 0 a.len
,;DateTime t=val(a[i] 1 _i) ;;use DateTime for times. Use long for size.
,;t.UtcToLocal
,;str sPath=a[i]+_i+1
,;out F"{t.ToStr(4)}    {sPath}"


a=0

str s.expandpath(parentFolder)
s.dospath(s 1)
s+iif((s.end("\")||!s.len) "*" "\*")
ARRAY(str) afp aep
if(!empty(filenamePattern)) afp=filenamePattern
if(!empty(exceptPattern)) aep=exceptPattern
int getPropFlags=getProp&~255; getProp&255

Dir d; lpstr sPath ss; int i
foreach(d s FE_Dir flags&0xff)
,if(afp.len and !sub.IsMatch(iif(flags&0x20000 d.RelativePath d.FileName) afp flags&0x10000)) continue
,if(aep.len and sub.IsMatch(iif(flags&0x80000 d.RelativePath d.FileName) aep flags&0x40000)) continue
,
,sPath=d.FullPath
,if getProp
,,long x
,,sel getProp
,,,case 1 if(getPropFlags&0x100 and d.IsFolder) x=0; else x=d.FileSize
,,,case 2 x=d.TimeModifiedUTC
,,,case 3 x=d.TimeCreatedUTC
,,,case 4 x=d.TimeAccessedUTC
,,,case else end ERR_BADARG
,,a[].from(x " " sPath)
,else a[]=sPath

err+ end _error ;;error in rx


#sub IsMatch
function! $ss ARRAY(str)&a isRX

int i
for i 0 a.len
,if(isRX) if(findrx(ss a[i] 0 1)>=0) ret 1
,else if(matchw(ss a[i] 1)) ret 1

[ssimop]

I have already tried it. I guess that, on behalf of all QM-users, I should thank you for making our lives easier!

[AlexZ]

...thank you for making our lives easier!

+1

You're right, Ssimop, I can't imagine my life without QM. For that reason, thanks to Gintaras and to all the QMers.

Cheers!

[ssimop]

Dear Gintaras,

I am in need of your advice regarding a possible mistake I commit in the following case :

GetFilesInFolder3 a folder "" 4|0x40000 2 exclude

where

str exclude=
.tmp
.temp
.chk
.pls
.mpg
.flv
~~back*
.log

The problem :
The file with filename :

[syllogos_dep_emp] Ekthesi_f-Nikos_160916.eml

is excluded, although its extension among these in string "exclude", but it contains string "log" in its filename.

Is it likely that I should use \.tmp$ to discriminate between dot as part of an rx against dot between filename and extension?

Many thanks in advance.

[Gintaras]

Yes, . in regular expression means "any character except newline".