Why not dynamically set the number of workers at install time ?

Discussion in 'OpenLiteSpeed Suggestions and Requests' started by eva2000, Aug 24, 2013.

  1. eva2000

    eva2000 Member

    For my custom CentOS based OpenLiteSpeed install tests, I am starting to tweaking a few settings in default httpd_config.xml and dynamically setting the number of workers according to my server's cpu count now.

    But why not do this by default instead of leaving it blank = 1 worker out of the box ?

    i.e. Tweak default OpenLiteSpeed settings /usr/local/lsws/conf/httpd_config.xml:

    Code:
    CPUS=$(grep "processor" /proc/cpuinfo |wc -l)
    WORKERCHECK=$(grep '<httpdWorkers>' /usr/local/lsws/conf/httpd_config.xml)
    
    if [[ -z "$WORKERCHECK" ]]; then
    sed -i "4 i\  <httpdWorkers>${CPUS}<\/httpdWorkers>" /usr/local/lsws/conf/httpd_config.xml
    fi
    
    I guess it's probably to leave some cpus for serving LSPHP processes ?

    Maybe give half to OpenLiteSpeed and half to LSPHP processes ?

    Code:
    CPUS=$(grep "processor" /proc/cpuinfo |wc -l)
    CPUS=$(echo $CPUS/2 | bc)
    WORKERCHECK=$(grep '<httpdWorkers>' /usr/local/lsws/conf/httpd_config.xml)
    
    if [[ -z "$WORKERCHECK" ]]; then
    sed -i "4 i\  <httpdWorkers>${CPUS}<\/httpdWorkers>" /usr/local/lsws/conf/httpd_config.xml
    fi
    
    Better than just 1 cpu worker ?
     
    Slavik likes this.
  2. eva2000

    eva2000 Member

    Reminds me, maybe also raise the max number of worker count from 16 to something higher ?

    i.e. 40 in anticipation for dual xeon cpus with 10 cpu core / 20 cpu threads each ? :D
     
  3. lsmichael

    lsmichael Active Member

    Nail on the head. (And database processes.) You almost certainly don't want OpenLiteSpeed running on all your processors. Unless you're just sending out static files, you're going to have other processes that are more taxing for your system. Thus it's better to save some of your processors for just those processes. What proportion of your processors should be running OpenLiteSpeed (or LSWS)? That's the million dollar question. It depends on what your site is doing. (Do you need more processors for HTTPS? Do you have a lot of static files? Do you have an intense number of connections?) I think, though, a good rule of thumb is to have 1/4 of your processors used for LSWS. Setting OpenLiteSpeed to automatically do that is a good idea. Let me see what the developers think.
     
  4. eva2000

    eva2000 Member

    Reminds me of another possible feature to bind specific cpu thread/cores to lshttp and php processes from web admin console (and be able to define them in httpd_config.xml). i.e. if I have dual Xeon E5-2650 with 2x 8 core/16 threads = 16 cores and 32 threads. Bind specific cpu cores to say first 8-16 threads for lsws processes ?
     
  5. lsmichael

    lsmichael Active Member

    Right now you can bind specific listeners to specific worker processes, but what you want is to define which worker process goes to which CPU?
     
  6. eva2000

    eva2000 Member

    yeah similar to Nginx's worker_cpu_affinity setting http://wiki.nginx.org/CoreModule#worker_cpu_affinity :)

    I know you can do it via command line and taskset but more convenient if folks using OLS can do it from web admin console too and well if setting is made in httpd_config.xml I can also do it via my simple bash twekaing script for out of box custom OLS installs :D
     
  7. lsmichael

    lsmichael Active Member

    Thanks for the link. An interesting idea. Again, I'll push it to the developers. If they don't have any objections, we'll try to put it in.
     
  8. eva2000

    eva2000 Member

    Thanks Michael
     
  9. eva2000

    eva2000 Member

    More thoughts on this.

    Maybe a formula on sliding scale

    <4 cpu threads worker count set to 1/2
    6-12 cpu threads worker count set to 1/3
    13-16 cpu threads worker count set to 1/4
    17-24 cpu threads worker count set to 1/6
    25-32+ cpu threads worker count set to 1/8

    or something
     

Share This Page