Max Dynamic Response Body Size

Discussion in 'General OpenLiteSpeed Discussion' started by Fearless, Jul 15, 2013.

  1. Fearless

    Fearless Member

    Anyone have any issues with downloading large files? I'm trying to download/export a sql backup file that is no more than 100M, but it fails every time with a network error (after downloading around 30-50M first) even though I set the Max Dynamic Response Body Size to 1000M. Is there other settings I need to alter as well?
     
    lsmichael likes this.
  2. Faizan Ullah

    Faizan Ullah New Member

  3. Fearless

    Fearless Member

    Not using wordpress to do the backup, but I did try using this line anyways:
    RewriteRule ^$ - [E=noabort:1]

    Didn't seem to make any difference though.
     
  4. lsmichael

    lsmichael Active Member

    Howdy,

    We're going to try testing on our end and see if we can recreate it. Anything in your error logs?

    m
     
  5. Fearless

    Fearless Member

    Oh that reminds me, I do get errors it says [STDERR] but the rest is blank. Does this mean I don't have something set up correctly? My log level is set to debug, debug level is none, enable stderr is yes.
     
  6. lsmichael

    lsmichael Active Member

    Howdy,

    Yup. That's normal and those settings are good for normal use.

    But when you're trying to figure out something abnormal like this, it often helps to turn debug logging on. The easiest way to do this is to use the Toggle Debug Logging feature. It's under the Actions menu. This will switch your Debug Level from None to High.

    Turn it on and try to do the download again. Then turn it off after you get the bug again. You don't want to leave debug logging on for too long, because it will start to take up a lot of space. Send us the debug output surrounding the bug.

    m
     
  7. Fearless

    Fearless Member

    See anything wrong here?
    Code:
    2013-07-17 13:44:50.904 [DEBUG] [*:80] New connection from 192.168.1.112:33482.
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] concurrent conn: 1
    2013-07-17 13:44:50.904 [DEBUG] [*:80] 1 connections accepted!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] NtwkIOLink::handleEvents() events=1!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] HttpConnection::onReadEx(), state: 1!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] readToHeaderBuf().
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] Read from client: 2044
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] read 2044 bytes to header buffer
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] processHeader() return 0, header state: 3.
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482] readToHeaderBuf() return 0.
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] New request:
            Method=[POST], URI=[/admin.php],
            QueryString=[username=root&db=nsale&dump=]
            Content Length=1032
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] Read Request Body!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] Read from client: 95
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] Finsh request body 1032/1032 bytes!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] Finished request body 1032 bytes!
    2013-07-17 13:44:50.904 [DEBUG] [192.168.1.112:33482#Example] NtwkIOLink::suspendRead()...
    2013-07-17 13:44:50.905 [DEBUG] [192.168.1.112:33482#Example] Find context with URI: [/], location: [/usr/local/lsws/DEFAULT/html/]
    2013-07-17 13:44:50.905 [DEBUG] [192.168.1.112:33482#Example] Find handler [lsphp5] for [.php]
    2013-07-17 13:44:50.905 [DEBUG] [192.168.1.112:33482#Example] processContextPath() return 0
    2013-07-17 13:44:50.906 [DEBUG] [192.168.1.112:33482#Example] run lsapi processor.
    2013-07-17 13:44:50.906 [DEBUG] [uds://tmp/lshttpd/lsphp5.sock] connection available!
    2013-07-17 13:44:50.906 [DEBUG] [uds://tmp/lshttpd/lsphp5.sock] request [192.168.1.112:33482#Example:lsapi] is assigned with connection!
    2013-07-17 13:44:50.906 [DEBUG] [192.168.1.112:33482#Example:lsapi] request header is done
    2013-07-17 13:44:50.906 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::continueRead()
    2013-07-17 13:44:50.907 [DEBUG] [192.168.1.112:33482#Example:lsapi] processor sent request body 1032 bytes, total sent: 1032
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] Request body done!
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::suspendWrite()
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] processNewReq() return 0.
    2013-07-17 13:44:50.908 [DEBUG] EdStream::handleEvent(), fd: 20, event: 1
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::onRead()
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] LsapiConn::doRead()
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] process packet header 8 bytes
    2013-07-17 13:44:50.908 [DEBUG] [192.168.1.112:33482#Example:lsapi] process packet header 0 bytes
    2013-07-17 13:44:51.764 [DEBUG] EdStream::handleEvent(), fd: 20, event: 1
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::onRead()
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] LsapiConn::doRead()
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] process packet header 8 bytes
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] process response header 517 bytes
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] response header finished!
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] process packet header 8 bytes
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] process response stream 8192 bytes, packet left: 16384
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] HttpExtConnector::processRespBodyData()
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] process response stream 7275 bytes, packet left: 8192
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] HttpExtConnector::processRespBodyData()
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] use CHUNKED encoding!
    2013-07-17 13:44:51.764 [DEBUG] [192.168.1.112:33482#Example:lsapi] Written to client: 4756
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] Written to client: 4104
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] writeRespBody() return 8192
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] Written to client: 4104
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] Written to client: 3186
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] writeRespBody() return 7275
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] response buffer is empty, suspend HttpConn write!
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] NtwkIOLink::suspendWrite()...
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::continueRead()
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] m_pHandler->onWrite() return 1
    2013-07-17 13:44:51.765 [DEBUG] EdStream::handleEvent(), fd: 20, event: 1
    2013-07-17 13:44:51.765 [DEBUG] [192.168.1.112:33482#Example:lsapi] ExtConn::onRead()
     
  8. lsmichael

    lsmichael Active Member

    It looks like you didn't keep debug logging on until the connection closed. There's nothing in here saying the connection closed. Could you try leaving it on a little bit longer?

    m
     
  9. Fearless

    Fearless Member

    Oops, I realized I only copied the first page. It will probably be tough to post the full log, it's almost 30K lines! Most of the lines are repeated many times like process response, process packet, write response and write to client. Attempting to post the relevant portion that may be more helpful which is the very last lines:
    Code:
    2013-07-17 17:45:47.861 [DEBUG] [192.168.1.112:39276#Example:lsapi] Written to client: 4104
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] Written to client: 4104
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] writeRespBody() return 8192
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] Written to client: 1629
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] writeRespBody() return 1622
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] response buffer is empty, suspend HttpConn write!
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::suspendWrite()...
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] ExtConn::continueRead()
    2013-07-17 17:45:47.864 [DEBUG] [192.168.1.112:39276#Example:lsapi] m_pHandler->onWrite() return 1
    2013-07-17 17:45:49.118 [DEBUG] EdStream::handleEvent(), fd: 19, event: 17
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] ExtConn::onRead()
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] LsapiConn::doRead()
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] process packet header -1 bytes
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, error: Connection reset by peer!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] [ExtConn] close()
    2013-07-17 17:45:49.118 [DEBUG] [uds://tmp/lshttpd/lsphp5.sock] No Request has been processed successfully through this connection, the maximum connections allowed will be reduced!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpExtConnector::tryRecover()...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::continueWrite()...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] write resumed!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::handleEvents() events=4!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::suspendWrite()...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpConnection::sendHttpError(),code=503 Service Unavailable
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Written to client: 672
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpConnection::flush()!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::flush...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpConnection::nextRequest()!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::flush...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpExtConnector::cleanUp() ...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] abort request...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] release ExtProcessor!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Non-KeepAlive, CLOSING!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Shutting down out-bound socket ...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Available Connections: 2000, concurrent conn: 0
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] HttpConnection::recycle()
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Shutting down out-bound socket ...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] NtwkIOLink::handleEvents() events=17!
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Close socket ...
    2013-07-17 17:45:49.118 [DEBUG] [192.168.1.112:39276#Example:lsapi] Recycle NtwkIoLink
     
  10. lsmichael

    lsmichael Active Member

    Hi,

    It seems like your external application isn't working right. Try raising the memory limit in the external application setting. Set the soft limit to 900M and hard limit to 1000M.

    m
     
  11. Fearless

    Fearless Member

    Hmm, I tried upping the memory_limit, but it still didn't work. They were using ini_set, so I just tried as well disabling the ini_set function and no luck. If you're not able to reproduce it, I think you are right and this is probably not a litespeed issue. I am probably not going to rely on this script and just export the db from a shell script instead.
     
  12. lsmichael

    lsmichael Active Member

    What application are you using? And you set the memory limit in the external application settings, right? (Not in the security tab...)

    m
     
  13. Fearless

    Fearless Member

    Lol I misunderstood, when you said, "It seems like your external application isn't working right" I thought you meant the php script I'm using to generate the file. I was wondering what you were referring to as soft and hard limit. I set the values as you suggested now in the external app section, still didn't work.

    Started playing around with a bunch of other values until I came across CPU Soft Limit (sec) and CPU Hard Limit (sec) under security and finally managed to get it to work. By default it was set to 10 and 50. I had to set it to as high as 180 for it to work. Although I'm assuming the bigger the file, I'd have to keep increasing this limit, which probably isn't a good thing?
     
  14. lsmichael

    lsmichael Active Member

    Ding, ding, ding! That sounds like a bug! :ROFLMAO:

    The CPU Limit settings shouldn't apply to LSPHP processes. LSPHP processes should be persistent. The CPU Limit settings are only meant for CGI processes. We're looking into it.

    And what was the application you're using (not that it should matter...)?
     
  15. Fearless

    Fearless Member

    I was using the "LSAPI App".
     
  16. lsmichael

    lsmichael Active Member

    :p Sorry. I didn't which external application. I meant what's the PHP application (phpmyadmin?) that you're using to download your SQL backups.
     
  17. Fearless

    Fearless Member

    LOL! At first I was going to say the script, but because last time I mistook what you meant when you said application, I changed it. I'm using "Adminer" which is similar to phpmyadmin, but better IMO. :whistle:
     
  18. lsmichael

    lsmichael Active Member

    :LOL:Alright. Updating the bug report. Hopefully we'll get back to you on Monday.
     
    Fearless likes this.
  19. lsmichael

    lsmichael Active Member

    Hey,

    The developers say they've figured it out. It'll be in 1.2.3, to be released tomorrow.

    It's not Monday, I know...:barefoot:

    m
     
  20. Fearless

    Fearless Member

    No worries, I'm in no rush. I'm not even sure if I would be using it much now, according to the descriptions, doesn't look like it's a good idea to set them too high.
     

Share This Page