Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
iAppsMail - Memory Usage Problem
#1
I have developed an email application using the best macro software available on this planet.

The software connects to a PHP backend on my webserver which connects to any IMAP/POP server anywhere and retrieves your mail.

USE: Mainly for corporates who work behind a stringent proxy or firewall.

----------------
Question 1:

When I am downloading files (refer screenshot), the memory usage (refer screenshot , right side) increases to a level which is highly unacceptable to me.

Can you think of any reason that is causing this ??

----------------


Attached Files Image(s)
   
#2
What code is used to download file?
#3
Methodology:
---------------

DL - Dialog with parent window handle of the main window

ProcessDL - Performs Download using IntGetFile

Code:
Copy      Help
str+ DL_Data=""
             int+ DL_Complete=0
             mac "TrackDL" "" filename filesize
             IntGetFile(filepath DL_Data 0)
             DL_Complete=1

             err+
                          DL_Complete=-1

TrackDL - Calculates string size using .len and performs percentage calculations
Code:
Copy      Help
function $filename #filesize

str+ DL_Data
DL_Data.escape

FLOAT tsize = filesize - DL_Data.len
FLOAT fperc
int perc
str downloaded
REPEAT
rep
    if DL_Complete = 1
        ret
        
    DL_Data.escape
    int size = DL_Data.len
    FLOAT fsize = size
    if size > 1048576
        downloaded.format("%.02f MB" fsize/1048576)
    else if size > 1024
        downloaded.format("%.02f KB" fsize/1024)
    else
        downloaded.format("%d bytes" size)
    
    if fsize != 0
        fperc = (fsize*100)/tsize
        perc = fperc
        _s = "Downloading "
        _s + filename
        _s + "... ["
        _s + perc
        _s + "%]"
        _s.setwintext(id(5 dl_hwnd))
        _s = downloaded
        _s + " complete"
        _s.setwintext(id(6 dl_hwnd))
        SendMessage dl_hpb PBM_SETPOS perc 0
        
    0.5

err+
    out "Download Track Error! - Tracker Restarted ..."
    goto REPEAT



I would appreciate if you could let me know of a more efficient way.
#4
Why DL_Data.escape?
#5
These codes are not fully into production. Was just playing around with string functions.

I tried a download of 23 MB file and mem usage at the end of dl was about 37 MB !!!!!

And after end of the download the mem usage did not drop back to normal even though the functions had ended.
#6
I tested memory usage of this code in exe.

Code:
Copy      Help
10
str s
IntGetFile "http://www.quickmacros.com/quickm21.exe" s 0 0 1

When starts - 3 MB,
when begins downloading - 6 MB (loads dlls, etc),
when ends downloading - 8 MB (quickm21.exe is 2 MB, and all data is stored in variable).

I am going to create IntGetFile function that writes directly to file instead of variable.
#7
Quote:And after end of the download the mem usage did not drop back to normal even though the functions had ended.

All data is stored in GLOBAL variable. Clear it.

DL_Data.all
#8
Ok.. I am able to regain mem Space after download completes.

When will the IntGetFile function that writes to file be available ? (Next version or sooner)

Thanks for looking into it.
#9
Quote:When will the IntGetFile function that writes to file be available ? (Next version or sooner)

Probably tomorrow.
#10
Big Grin

YOU ARE THE BEST !!!!!!!!!!!!!!!!!!

I will be sending you the packed exe in mail for testing. Once the application is fully developed, we can put it up on the forum.

Let me know what you think.

:-))
#11
OK.
#12
Any update on the IntGetFile function that writes directly to file ???
#13
Already created but needs more testing. Will post here tomorrow.
#14
Wow. You Rock..

No hurry. Just wanted to check on the status.


Thanks
#15
I am attaching whole System.qml. Download and move to QM folder, replacing old System.qml (make backup!!). What is new in it:

IntGetFile:
flag 16 - download directly to file.
Use callback function to receive progress info.

New function Http.PostFormData:
Easier to use and more powerful than Http.Post and IntPost.
Post multiple simple fields and/or files in single call.
Don't worry about posted string format, encoding, headers, etc.
Use callback function to receive progress info.


Attached Files
.zip   System.zip (Size: 192.32 KB / Downloads: 706)
#16
EXAMPLE 1 - IntPostFile with custom progress dialog.

Code:
Copy      Help
int hwnd=ShowDialog("" 0 0 0 1)

str localfile="$desktop$\quickm20.gi_"
IntGetFile "http://www.quickmacros.com/images/quickm20.gif" localfile 16 0 0 &IntGetFile_progress_callback hwnd
ren- localfile "$desktop$\quickm20.gif"

DestroyWindow hwnd

;BEGIN DIALOG
;0 "" 0x90C80A44 0x100 0 0 222 42 "Form"
;2 Button 0x54030000 0x4 164 24 48 14 "Cancel"
;3 msctls_progress32 0x54000000 0x0 8 6 204 14 ""
;END DIALOG
;DIALOG EDITOR: "" 0x2010900 "" ""

Function IntGetFile_progress_callback
Code:
Copy      Help
function# nbAll nbRead str&s fparam

if(!IsWindow(fparam)) ret 1 ;;Cancel
def PBM_SETPOS (WM_USER+2)
SendMessage id(3 fparam) PBM_SETPOS nbRead*100/nbAll 0

___________________

EXAMPLE 2 - Http.PostFormData with progress dialog.

Code:
Copy      Help
out
ARRAY(POSTFIELD) a.create(6); int i
a[i].name="exe1"; a[i].value="$system$\notepad.exe"; a[i].isfile=1; i+1
a[i].name="txt2"; a[i].value="some text 2"; a[i].isfile=0; i+1
a[i].name="userfile"; a[i].value="$desktop$\test.txt"; a[i].isfile=1; i+1
a[i].name="qml"; a[i].value="$desktop$\Macro295.qml"; a[i].isfile=1; i+1
a[i].name="txt"; a[i].value="some text 1"; a[i].isfile=0; i+1
a[i].name="exe2"; a[i].value="$system$\calc.exe"; a[i].isfile=1; i+1

int hwnd=ShowDialog("" 0 0 0 1)
Http h.Connect("www.quickmacros.com"); str r
if(!h.PostFormData("form.php" a r "" &PostFormData_progress_dialog hwnd)) end "failed"
out r

DestroyWindow hwnd

;BEGIN DIALOG
;0 "" 0x90C80A44 0x100 0 0 222 80 "Form"
;2 Button 0x54030000 0x4 168 62 48 14 "Cancel"
;3 msctls_progress32 0x54000000 0x0 30 6 186 12 ""
;4 msctls_progress32 0x54000000 0x0 30 24 186 12 ""
;5 Static 0x54000000 0x0 4 6 24 12 "Total"
;6 Static 0x54000000 0x0 4 24 22 12 "File"
;7 Static 0x54000000 0x0 4 42 212 16 ""
;END DIALOG
;DIALOG EDITOR: "" 0x2010900 "" ""

Function PostFormData_progress_dialog
Code:
Copy      Help
function# action nbAll nbRead $_file nbAllF nbReadF fparam
;out "%i %i %s %i %i %i" nbAll nbRead _file nbAllF nbReadF fparam

if(!IsWindow(fparam)) ret 1

def PBM_SETPOS (WM_USER+2)
SendMessage id(3 fparam) PBM_SETPOS nbRead*100/nbAll 0
lpstr st
sel action
,case 0 st="Sending data..."
,case 1
,st=_file
,SendMessage id(4 fparam) PBM_SETPOS nbReadF*100/nbAllF 0
,case 2 st="All data sent"
,case 3 st="Receiving response..."
,case 4 st="Finished"
SetDlgItemText fparam 7 st
0
#17
amazing. thanks for this.
pi
#18
I am speechless..... Just the functionality that I was hoping for !!!!!!!!!!

Application update -

Able to list and open email messages, download attachments.

Next changes -
Compose, File upload/attach

Will post a working demo here in a few days. (Hoping for all kinds of suggestions)

Thanks once again.
#19
how can i get the total bytes uploaded into 'Http.PostFormData with progress dialog' ?
not into the dialog, as value in the macro.
pi
#20
In Http.PostFormData replace

ret 1

to

ret pp.wrtotal

This will return total number of uploaded bytes only if posting files or using callback function.
#21
unclear question it seems Smile
for my uploader i would like to have the real filesize on the server for
the link list which will be generated.

http://server.tld/someweiredCode (99 MB)

i could read the filesize on my disk, but there is a difference between online and offline size.

on disk 98.078 MB
on web 95.78 MB
pi
#22
PostFormData returns total content length (headers, files, etc). Do you need only total size of uploaded files, not including size of headers?
#23
yes only filesize.
pi
#24
Use GetFileInfo or Dir.FileSize. Or calculate in callback function, like in example.

Code:
Copy      Help
out
ARRAY(POSTFIELD) a.create(2); int i
a[i].name="exe1"; a[i].value="$system$\notepad.exe"; a[i].isfile=1; i+1
a[i].name="exe2"; a[i].value="$system$\calc.exe"; a[i].isfile=1; i+1

type PFD_CALLBACK hwnd nbFilesTotal
PFD_CALLBACK d

d.hwnd=ShowDialog("" 0 0 0 1)
Http h.Connect("www.quickmacros.com"); str r
if(!h.PostFormData("form.php" a r "" &PostFormData_progress_dialog2 &d)) end "failed"
out "Total size of uploaded files is %i KB" d.nbFilesTotal/1024
out r

DestroyWindow d.hwnd

;BEGIN DIALOG
;0 "" 0x90C80A44 0x100 0 0 222 80 "Form"
;2 Button 0x54030000 0x4 168 62 48 14 "Cancel"
;3 msctls_progress32 0x54000000 0x0 30 6 186 12 ""
;4 msctls_progress32 0x54000000 0x0 30 24 186 12 ""
;5 Static 0x54000000 0x0 4 6 24 12 "Total"
;6 Static 0x54000000 0x0 4 24 22 12 "File"
;7 Static 0x54000000 0x0 4 42 212 16 ""
;END DIALOG
;DIALOG EDITOR: "" 0x2010900 "" ""

Function PostFormData_progress_dialog2
Code:
Copy      Help
function# action nbAll nbRead $_file nbAllF nbReadF PFD_CALLBACK&d
;out "%i %i %s %i %i" nbAll nbRead _file nbAllF nbReadF

if(!IsWindow(d.hwnd)) ret 1

def PBM_SETPOS (WM_USER+2)
SendMessage id(3 d.hwnd) PBM_SETPOS nbRead*100L/nbAll 0
lpstr st
sel action
,case 0 st="Sending data..."
,case 1
,st=_file
,SendMessage id(4 d.hwnd) PBM_SETPOS nbReadF*100L/nbAllF 0
,if(nbReadF=nbAllF) d.nbFilesTotal+nbAllF
,case 2 st="All data sent"
,case 3 st="Receiving response..."
,case 4 st="Finished"
SetDlgItemText d.hwnd 7 st
0
#25
Uploading multiple files, single file at a time, single progress dialog for all.

Code:
Copy      Help
out
;files
ARRAY(str) af.create(2)
af[0]="$system$\notepad.exe"
af[1]="$system$\calc.exe"
;callback function data
type PFD_CALLBACK2 hwnd nbTotal nbSent
PFD_CALLBACK2 c
;get total size
Dir d; int i
for(i 0 af.len) if(d.dir(af[i])) c.nbTotal+d.FileSize; else end "file does not exist"
;show dialog, connect
c.hwnd=ShowDialog("" 0 0 0 1)
Http h.Connect("www.quickmacros.com"); str r
;post
ARRAY(POSTFIELD) a.create(1); a[0].name="userfile"; a[0].isfile=1
for i 0 af.len
,a[0].value=af[i]
,if(!h.PostFormData("form.php" a r "" &PostFormData_progress_dialog3 &c)) end "failed"
,out r
;results
out "Total size of uploaded files is %i KB" c.nbTotal/1024

DestroyWindow c.hwnd

;BEGIN DIALOG
;0 "" 0x90C80A44 0x100 0 0 222 80 "Form"
;2 Button 0x54030000 0x4 168 62 48 14 "Cancel"
;3 msctls_progress32 0x54000000 0x0 30 6 186 12 ""
;4 msctls_progress32 0x54000000 0x0 30 24 186 12 ""
;5 Static 0x54000000 0x0 4 6 24 12 "Total"
;6 Static 0x54000000 0x0 4 24 22 12 "File"
;7 Static 0x54000000 0x0 4 42 212 16 ""
;END DIALOG
;DIALOG EDITOR: "" 0x2010900 "" ""

Function PostFormData_progress_dialog3
Code:
Copy      Help
function# action nbAll nbSent $_file nbAllF nbSentF PFD_CALLBACK2&c

if(!IsWindow(c.hwnd)) ret 1

def PBM_SETPOS (WM_USER+2)
lpstr st
sel action
,case 0 st="Sending data..."
,case 1
,st=_file
,SendMessage id(4 c.hwnd) PBM_SETPOS nbSentF*100L/nbAllF 0 ;;file
,SendMessage id(3 c.hwnd) PBM_SETPOS c.nbSent+nbSentF*100L/c.nbTotal 0 ;;total
,if(nbSentF=nbAllF) c.nbSent+nbAllF
,case 2 st="All data sent"
,case 3 st="Receiving response..."
,case 4 st="Finished"
SetDlgItemText c.hwnd 7 st
0
#26
works perfect !
pi


Forum Jump:


Users browsing this thread: 4 Guest(s)