Posts: 858
Threads: 196
Joined: Apr 2005
When I tried
Macro
str s
rep 10
,Http h.Connect("www.xxx.yyy" "user" "password")
,h.Post("test/test.php" "name1=value1&name2=value2" s)
,out s
the time used is high.
I tried also to use several functions in parallel.
It can be done faster?
note: in wikipedia (Hypertext Transfer Protocol):
In HTTP/0.9 and 1.0, the connection is closed after a single request/response pair. In HTTP/1.1 a keep-alive-mechanism was introduced, where a connection could be reused for more than one request.
Such persistent connections reduce lag perceptibly, because the client does not need to re-negotiate the TCP connection after the first request has been sent.
Version 1.1 of the protocol made bandwidth optimization improvements to HTTP/1.0. For example, HTTP/1.1 introduced chunked transfer encoding to allow content on persistent connections to be streamed, rather than buffered. HTTP pipelining further reduces lag time, allowing clients to send multiple requests before a previous response has been received to the first one. Another improvement to the protocol was byte serving, which is when a server transmits just the portion of a resource explicitly requested by a client.
Posts: 12,095
Threads: 142
Joined: Dec 2002
I don't know much about HTTP authentication, speed, etc.
QM uses wininet functions and HTTP 1.1. I think they don't close connection after each request but not sure. Try to add INTERNET_FLAG_KEEP_CONNECTION as 4-th argument of Post.
You can read about wininet functions in MSDN library.
Posts: 858
Threads: 196
Joined: Apr 2005
Posts: 12,095
Threads: 142
Joined: Dec 2002
It is not related to internet connection speed. It means that the thread can do other things while wininet functions are connecting and transferring data.
Posts: 858
Threads: 196
Joined: Apr 2005
I also found an activex dll: http://www.coalesys.com/products/httpcl ... efault.asp
Do you know if uses different functions that QM?
In affirmative case, could you add an example of several Post in parallel to test if is faster?
Posts: 12,095
Threads: 142
Joined: Dec 2002
Slower. Every time about 400 ms. QM - first time 400 ms then 190.
GET example
Macro
out
typelib CSHttpClientLib {12CB8C40-6A6B-11D0-A74C-444553540000} 1.0
CSHttpClientLib.CSHttpClient c._create
c.RequestURL="http://www.google.com"
c.Execute("GET")
out c.ResponseBody(0)
POST example
Macro
out
typelib CSHttpClientLib {12CB8C40-6A6B-11D0-A74C-444553540000} 1.0
CSHttpClientLib.CSHttpClient c._create
c.RequestURL="http://www.quickmacros.com/form2.php"
str s="a=b&c=d"
VARIANT v=s
c.RequestBody=v
c.RequestHeaders=_s.format("Content-Type: application/x-www-form-urlencoded[]Content-Length: %i[]Cache-Control: no-cache[]" s.len)
rep 5
,int t1=timeGetTime
,c.Execute("POST")
,out timeGetTime-t1
,out c.ResponseBody(0)
Don't know how to set password. Maybe in URL, but did not test.
Posts: 858
Threads: 196
Joined: Apr 2005
How can I measure the time of the Function_test?
Function Function_test
rep 10
,mac("test_post" "" _s)
Posts: 12,095
Threads: 142
Joined: Dec 2002
Macro
ARRAY(int) a
int t1=timeGetTime
rep 10 ;;max 64
,a[]=mac("test_post" "" _s)
;wait until all threads ended
if(WaitForMultipleObjects(a.len &a[0] 1 INFINITE)=WAIT_FAILED) end _s.dllerror
out timeGetTime-t1
Posts: 858
Threads: 196
Joined: Apr 2005
Thank you very much.
Is not possible for >64 threads?
Posts: 12,095
Threads: 142
Joined: Dec 2002
WaitForMultipleObjects supports max 64 handles.
You should not use many threads because each thread gets 1 MB of memory for stack. For example, on 256 MB RAM PC you cannot have more than about 200 threads.
Posts: 858
Threads: 196
Joined: Apr 2005
Almost without differences...
64 Post threads
Without INTERNET_FLAG_KEEP_CONNECTION
4943 ms
5712 ms
5218 ms
5567 ms
With INTERNET_FLAG_KEEP_CONNECTION
5053 ms
5198 ms
4903 ms
5251 ms
Posts: 858
Threads: 196
Joined: Apr 2005
It seems that the solution is here: http://forums.techarena.in/tips-tweaks/990999.htm
with 100 Post (time of each individual post 5-15 seconds):
before edit registry ----> 12 minutes
after edit registry -----> 4 minutes
Questions:
1) Can I do it without edit registry?
2) Can I Know the number of simultaneous connections of server for optimal performance?
3) How can I add a timeout of 1 minute in Post.
Note: sometimes any post time is very high (4-6 minutes) and if make the same post in that moment with IE the time is 5-15 seconds. Do you know why?
Posts: 12,095
Threads: 142
Joined: Dec 2002
can answer only question 3.
Member function Http.PostWithTimeout
function# timeout_s $action $data [str&responsepage] [$headers] [inetflags] [str&responseheaders] [&unused]
;Posts web form data. Returns 1 on success, 0 on failure. Error on timeout.
;This function cannot post files. To post files use PostFormData.
;timeout_s - timeout in seconds.
;data - urlencoded string, eg "name=John+Smith&[email protected]".
;action, responsepage, headers, inetflags, responseheaders - the same as with PostFormData.
;unused - used internally.
;See also: <Http.PostFormData>.
if(!&unused)
,if(!m_hi) end ES_INIT
,__HInternet _hi; int th _ret
,th=mac("__Http_PostWithTimeout" "" &this action data &responsepage headers inetflags &responseheaders &_hi &_ret)
,wait timeout_s H th
,err
,,InternetCloseHandle _hi; _hi.handle=0 ;;ends thread
,,wait 0 H th ;;because the thread uses this. ~0.5ms
,,end _error
,ret _ret
lpstr sh="Content-Type: application/x-www-form-urlencoded"
if(empty(headers)) headers=sh
else if(findrx(headers "(?i)^Content-Type *:" 8)<0) headers=_s.from(sh "[]" headers)
unused=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!unused) ret Error
if(!HttpSendRequest(unused headers -1 data len(data))) ret Error
if(&responseheaders and !GetResponseHeaders(unused responseheaders)) ret Error
if(&responsepage) ret Read(unused responsepage)
ret 1
Function __Http_PostWithTimeout
;\
function Http&h $action $data str&responsepage $headers inetflags str&responseheaders &handle &_ret
_ret=h.PostWithTimeout(0 action data responsepage headers inetflags responseheaders handle)
Posts: 858
Threads: 196
Joined: Apr 2005
Can Winhttp be better than Wininet ( http://msdn.microsoft.com/en-us/magazine/cc716528.aspx) to do this?
Can you add an example of
rep 10
,mac("test_post" "" _s)
using winhttp (this API has Timeout, Asynchronous...) to test?
Thanks in advance.
Posts: 12,095
Threads: 142
Joined: Dec 2002
WinHTTP is available on XP SP1 and later. Is it OK?
Posts: 858
Threads: 196
Joined: Apr 2005
Posts: 12,095
Threads: 142
Joined: Dec 2002
Much to learn. I don't think it will be better than you already have in QM.
Posts: 858
Threads: 196
Joined: Apr 2005
Posts: 12,095
Threads: 142
Joined: Dec 2002
Need declarations?
More Windows API declarations for QM
example
Macro
WINAPI2.WinHttpSendRequest
Posts: 858
Threads: 196
Joined: Apr 2005
No, I wonder if Winhttp functions are quicker than Wininet
Http.PostWithTimeout uses 2 threads and I need run more than 100 Post...
Posts: 858
Threads: 196
Joined: Apr 2005
I'm trying understand the function InternetSetOption
How can I create a Http member function to set:
INTERNET_OPTION_MAX_CONNS_PER_SERVER
INTERNET_OPTION_MAX_CONNS_PER_1_0_SERVER
INTERNET_OPTION_CONNECT_TIMEOUT
INTERNET_OPTION_RECEIVE_TIMEOUT
INTERNET_OPTION_FROM_CACHE_TIMEOUT
another question:
if set INTERNET_OPTION_RECEIVE_TIMEOUT is necessary to use Http.PostWithTimeout?
Posts: 12,095
Threads: 142
Joined: Dec 2002
The timeout options don't work, that is why I used another thread.
The options probably should be set on the hi variable that is in Http.Post. It is not a member variable and you cannot set options on it with a member function. Copy Http.Post and edit.
After
__HInternet hi=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!hi) ret Error
add
Member function Http.Post2
int option(10) optionlen(4); if(!InternetSetOption(hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error
Not tested.
Posts: 858
Threads: 196
Joined: Apr 2005
Can I add it in Http.PostWithTimeout?
other question:
Which you believe that it is the better option to send several Posts and receive the answer?
Posts: 12,095
Threads: 142
Joined: Dec 2002
Yes, instead of hi there is unused. But maybe the option should be set on m_hi or m_hitop. Try all.
I don't know what is the best. test all options and you will see.
Posts: 858
Threads: 196
Joined: Apr 2005
Ok.
I will test all options.
Posts: 858
Threads: 196
Joined: Apr 2005
I tried it but:
unused, m_hi ------> Error: Failed to create thread __Http_PostWithTimeout. Max 1000 threads allowed.
m_hitop -----------> Error (RT) in Http.PostWithTimeout: Exception 0xC0000005. Access violation. Cannot read memory at 0x0. In at 0x0 (0x0+0x0).
Can you modify Http.PostWithTimeout?
Posts: 12,095
Threads: 142
Joined: Dec 2002
QM does not allow more than 1000 threads.
Don't know where is the exception. Maybe because the thread was not created.
Posts: 858
Threads: 196
Joined: Apr 2005
Http.PostWithTimeout without line
int option(5) optionlen(4); if(!InternetSetOption(m_hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error
works
note: I only run 5 Http.PostWithTimeout.
Posts: 12,095
Threads: 142
Joined: Dec 2002
I tested, works like without the line. But maybe my testing code is different.
Posts: 858
Threads: 196
Joined: Apr 2005
My Http.PostWithTimeout
Member function Http.PostWithTimeout
function# timeout_s $action $data [str&responsepage] [$headers] [inetflags] [str&responseheaders] [&unused]
;Posts web form data. Returns 1 on success, 0 on failure. Error on timeout.
;This function cannot post files. To post files use PostFormData.
;timeout_s - timeout in seconds.
;data - urlencoded string, eg "name=John+Smith&[email protected]".
;action, responsepage, headers, inetflags, responseheaders - the same as with PostFormData.
;unused - used internally.
;See also: <Http.PostFormData>.
if(!&unused)
,if(!m_hi) end ES_INIT
,__HInternet _hi; int th _ret
,th=mac("__Http_PostWithTimeout" "" &this action data &responsepage headers inetflags &responseheaders &_hi &_ret)
,wait timeout_s H th
,err
,,InternetCloseHandle _hi; _hi.handle=0 ;;ends thread
,,wait 0 H th ;;because the thread uses this. ~0.5ms
,,end _error
,ret _ret
lpstr sh="Content-Type: application/x-www-form-urlencoded"
if(empty(headers)) headers=sh
else if(findrx(headers "(?i)^Content-Type *:" 8)<0) headers=_s.from(sh "[]" headers)
unused=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!unused) ret Error
int option(5) optionlen(4); if(!InternetSetOption(m_hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error
if(!HttpSendRequest(unused headers -1 data len(data))) ret Error
if(&responseheaders and !GetResponseHeaders(unused responseheaders)) ret Error
if(&responsepage) ret Read(unused responsepage)
ret 1
|
