Spawning multiple .NET delegates really slow

I recently worked on a project that had a listener service running that took TCP requests and made external calls to a remote WebService. The listener service could have up to 300 concurrent requests.

In testing the service it was showing a significant delay in executing requests above 3-4 concurrent clients. The first bottleneck had to do with the number of allowed HTTP connections to a remote machine from our service. It turns out that by default only 2 concurrent HTTP connections are allowed by default.

Here is an MSDN article that explains the settings that need to be set to allow more connections to be made.

http://msdn.microsoft.com/en-us/library/fb6y0fyc.aspx

So setting something like this what required.

configuration>
  <system.net>
    <connectionManagement>
      <add address = "http://www.contoso.com" maxconnection = "10" />
      <add address = "*" maxconnection = "2" />
    </connectionManagement>
  </system.net>
</configuration>

While this showed a little improvement the problems were not over. It turns out that the ThreadPool by default only starts up with 4 idle threads. As you exceed that limit the ThreadPool checks every 500ms to see if more threads are needed and only spawns up one thread every 500ms. So you can see that it will take a long time to get spawned a large number of concurrent requests.

Solution to this is to set the minimum threads in the threadpool.

ThreadPool.SetMinThreads - http://msdn.microsoft.com/en-us/library/system.threading.threadpool.setminthreads.aspx

So we set the Min to the minimum number of threads we wanted sitting idle. Ok now performance was very good up to the number of threads we set as our minimum. But wait things are not over. We moved the code to another machine and the performance degraged. What could be wrong? It worked on my machine just fine!!

So we looked at what was different about the machines. Turned out my machine was running .NET 2.0 and the other machine was running .NET 2.0 SP1. I guess in .NET 2.0 SP1 a new bug was introduced that if threads are requested to quickly it will revert back to slowly invoking new threads.

Solution is to put a delay after each new call to a delegate for a few milliseconds using something like Thread.Sleep(50);. I had to put about 50ms delay on the machine in question. However I found that this bug was to be fixed in .NET 2.0 SP2. A bit of searching I found that .NET 2.0 SP2 had actually been released but not by itself. You have to download .NET 3.5 SP1 which also includes .NET 2.0 SP2. After installing this on the machine no delay was required any longer and performance was very good.

Hope this helps someone

BlueQuartz installing Mono

Mono allows you to run applications written in .NET programming language on Linux machines.

Step 1: Install the YUM repository by grabbing the file mono.repo from the mono website and placing in /etc/yum.repos.d/ folder.

--- Update 7/22/2011 ---

The above file is no longer available. Here is the contents of the mono.repo file that I grabbed previously when it was available. Just take the contents and create a file in /etc/yum.repos.d/ called mono.repo with the following contents.

 [mono]
name=Mono for rhel-4-i386 (stable)
baseurl=http://go-mono.com/download-stable/rhel-4-i386/
enabled=1
gpgcheck=0

Step 2: Install Mono by executing this command. yum install xsp

The above command should also install these required components

Installing:
xsp                     noarch     1.9.1-0.novell   mono              210 k
Installing for dependencies:
mono-core               i586       1.9.1-2.novell   mono               17 M
mono-data               i586       1.9.1-2.novell   mono              1.8 M
mono-data-sqlite        i586       1.9.1-2.novell   mono              169 k
mono-nunit              i586       1.9.1-2.novell   mono              115 k
mono-web                i586       1.9.1-2.novell   mono              3.0 M
mono-winforms           i586       1.9.1-2.novell   mono              3.9 M

Congratulations you have installed mono and can now execute programs written in .NET on your machine. However I had you install XSP as it is a small webserver that can run a website written in .NET without having to integrate into your existing web application such as apache. If you want to integrate into apache follow the instructions at this Mono website.

 

Setting up XSP

Step 1: Create a folder that will be the root folder for your website. Example: /home/monoweb/web

Step2: Setup the startup script /etc/rc.d/init.d/xsp for XSP that will host your website. Change permissions to allow execute chmod +x /etc/rc.d/init.d/xsp  . Below you will find the example startup script I found on the web that has worked for me.

Step 3: Setup /etc/xsp.conf that the startup script uses to set the port and folder the website exists under.  Below is the example contents.

Step 4: Setup XSP to automatically start on reboots. chkconfig xsp on

Step 5: You are now ready to host your website. Simply publish your website files to the root folder you defined and issue this command service xsp start

 

Contents of /etc/xsp.conf

Note: you should change the --port definition to the port you want to use. Make sure nothing else is using the same port you define.

--port 8082 --root /home/monoweb/web/

 

Contents of /etc/rc.d/init.d/xsp

#!/bin/sh
#
# Startup script for xsp server
#
# chkconfig: 3 84 16
# description: xsp is a asp.net server
#
ARGS=`cat /etc/xsp.conf | grep -v \# `

. /etc/init.d/functions

start() {
echo -n $"Starting xsp: "
# Check PID/existence
pid=""
if [ -f /var/run/xsp.pid ] ; then
         read pid < /var/run/xsp.pid
         if [ -n "$pid" ]; then
   rm /var/run/xsp.pid
  else
   echo -n $"xsp is already running."
   failure
   echo
   return 1
         fi
fi

/usr/bin/xsp2 --nonstop $ARGS > /dev/null &
RETVAL=$?
if [ $RETVAL != 0 ]; then
  failure
  echo
  return $RETVAL
  fi
PID=$!
echo $PID > /var/run/xsp.pid
success
echo
return 0
}

stop() {
echo -n $"Shutting down xsp: "

if [ ! -f /var/run/xsp.pid ]; then
  echo -n $"xsp not running"
  failure
  echo
  return 1
fi

kill -15 `cat /var/run/xsp.pid`
RETVAL=$?

if [ $RETVAL = 0 ]; then
  rm /var/run/xsp.pid
  success
  echo
  return 0
else
  failure
  echo
  return $RETVAL
fi
}

restart() {
stop
start
}

case "$1" in
  start)
   start
;;
  stop)
   stop
;;
  restart)
   restart
;;
  *)
echo $"Usage: $0 {start|stop|restart}"
exit 1
esac

exit $?

 

 

BlueQuartz add virus scanning to sendmail

This HowTo is designed to provide the simplest solution for virus scanning to a BlueQuartz box. However this will also work on any box running Redhat Enterprise Linux 4 or CentOS 4

 Note this uses a MILTER to scan for viruses and assumption is that you are not running any other MILTERS. If you are you will need to be smart enough to know how to modify the following instructions.

Assuming you dont have DAG or RPMFORGE installed which a basic box does not you will want to install the RPMFORGE repository so that you can easily install CLAMAV.

Install RPMFORGE

rpm -Uhv http://apt.sw.be/redhat/el4/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el4.rf.i386.rpm

Edit rpmforge.repo

This is a safety step to ensure that you do not accidently update any of the following modules. Note however when we are done we will be disabling RPMFORGE as a secondary measure. Otherwise you could end updating modules specific to BlueQuartz that could cause issues.

nano -w /etc/yum.repos.d/rpmforge.repo


Add the following line to your config

exclude=yum*,centos-yumconf*,httpd*,mod_ssl*,sendmail*,procmail*,imap*,nss_db*,pam*,pwdb*,webalizer*,sysklogd*,proftpd*


Should look something like the following. Save the file once completed.

# Name: RPMforge RPM Repository for Red Hat Enterprise 4 - dag
# URL: http://rpmforge.net/
[rpmforge]
name = Red Hat Enterprise $releasever - RPMforge.net - dag
#baseurl = http://apt.sw.be/redhat/el4/en/$basearch/dag
mirrorlist = http://apt.sw.be/redhat/el4/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1
exclude=yum*,centos-yumconf*,httpd*,mod_ssl*,sendmail*,procmail*,imap*,nss_db*,pam*,pwdb*,webalizer*,sysklogd*,proftpd*

Install ClamAV, ClamAV-Devel and ClamAV-Milter

yum install clamav clamav-devel clamav-milter

This will display that it wants to include other needed packages. Says y to the prompt and it will download and install the packages.

Disable rpmforge

mv /etc/yum.repos.d/rpmforge.repo /etc/yum.repos.d/rpmforge.repo.bak

This will prevent this repository from being used in regular system updates. If you ever want to use the repository to install other modules you can simply copy it back to .repo and remove the .bak from the name.

Backup sendmail configuration

cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf.bak
cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc.bak

Edit sendmail.mc to include milter

nano -w /etc/mail/sendmail.mc


Add mail filter lines to sendmail.mc just above confCACERT

INPUT_MAIL_FILTER(`clmilter',`S=local:/var/clamav/clmilter.socket,T=S:4m;R:4m')dnl
define(`confINPUT_MAIL_FILTERS',`clmilter')dnl

Example sendmail.mc modification

TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl #
dnl # Rudimentary information on creating certificates for sendmail TLS:
dnl #     make -C /usr/share/ssl/certs usage
dnl #
INPUT_MAIL_FILTER(`clmilter',`S=local:/var/clamav/clmilter.socket,T=S:4m;R:4m')dnl
define(`confINPUT_MAIL_FILTERS',`clmilter')dnl

dnl #
define(`confCACERT_PATH',`/usr/share/ssl/certs')
define(`confCACERT',`/usr/share/ssl/certs/ca-bundle.crt')
define(`confSERVER_CERT',`/usr/share/ssl/certs/sendmail.pem')
define(`confSERVER_KEY',`/usr/share/ssl/certs/sendmail.pem')
dnl #
dnl # This allows sendmail to use a keyfile that is shared with OpenLDAP's
dnl # slapd, which requires the file to be readble by group ldap
dnl #
dnl define(`confDONT_BLAME_SENDMAIL',`groupreadablekeyfile')dnl

 

Save changes to file and update sendmail.cf

cd /etc/mail/
make

The installation is now complete. You can either reboot your machine or issue these commands.

service clamd restart
service clamav-milter restart
service sendmail restart

Verify installation is working

tail -f /var/log/maillog


send an email to the system in question
You should see a message like the following

Jun  1 14:04:16 sandbox sendmail[4600]: m51L409D004600: mailto:from=jscott@domain.com, size=11, class=0, nrcpts=1, msgid=<mailto:200806012104.m51L409D004600@sandbox2.10tohost.com>, proto=SMTP, daemon=MTA, relay=192-168-105-5.domain.com [192.168.105.5] (may be forged)
Jun  1 14:04:17 sandbox sendmail[4600]: m51L409D004600: Milter add: header: X-Virus-Scanned: ClamAV version 0.93, clamav-milter version 0.93 on sandbox.domain.com
Jun  1 14:04:17 sandbox sendmail[4600]: m51L409D004600: Milter add: header: X-Virus-Status: Clean
Jun  1 14:04:17 sandbox sendmail[4604]: m51L409D004600: to=admin, delay=00:00:04, xdelay=00:00:00, mailer=local, pri=30479, dsn=2.0.0, stat=Sent

Other logs to take a look at:

Clamd Logs: /var/log/clamav/clamd.log
Freshclam Logs: /var/log/clamav/freshclam.log

 

BlueQuartz how to resync quota of system with GUI

Not sure how many others have had this issue but on occasion I get a user that calls because they are getting email rejected because they are over quota. I log into the administration interface only to find that the quota does not report that they are over quota. I go to a command prompt and checkquota and sure enough  they are over quota.

 So seems that I have a difference between what BQ reports and what the system actually reports. I have asked, googled and for over a year did not find a solution other than to simply increase the site quota so it allowed for the difference. Obviously not a very good solution.

Well I finally figured out how to fix it and it is really simple so I thought I would put it online for others to hopefully use in the future. 

/sbin/quotaoff -u /home/
/sbin/quotacheck /home/ -m
/sbin/quotaon -u /home/

That is it. Could not believe for how many questions I asked and the googling I did that it came down to this set of simple steps.

 

 

 

BlueQuartz enabling SSL/TLS FTP Support

This is my first attempt at putting online a step by step process in support of the BlueQuartz server appliance.

1) See if you are running a version of proftpd that has mod_tls.c module included

Execute Statement

 proftpd -i


Example Output: Note you are looking for mod_tls.c to be included otherwise these instructions will not work for you.

[root@ruth ~]# proftpd -l
Compiled-in modules:
  mod_core.c
  mod_xfer.c
  mod_auth_unix.c
  mod_auth_file.c
  mod_auth.c
  mod_ls.c
  mod_log.c
  mod_site.c
  mod_readme.c
  mod_auth_pam.c
  mod_tls.c
  mod_cap.c

2) Create SSL Certificate and directory to hold certificate

Create Directory:

 mkdir -p /etc/proftpd/ssl


Create SSL Certificate:

 openssl req -new -x509 \
 -days 365 \
 -nodes \
 -out /etc/proftpd/ssl/proftpd.cert.pem \
 -keyout /etc/proftpd/ssl/proftpd.key.pem


This will ask you a series of questions and generate the certificates

Country Name (2 letter code) [AU]: <-- Enter your Country Name (e.g., "US").
State or Province Name (full name) [Some-State]:
<-- Enter your State or Province Name (eg., Washington).
Locality Name (eg, city) []:
<-- Enter your City (e.g., "Seattle").
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
<-- Enter your Organization Name (e.g., the name of your company).
Organizational Unit Name (eg, section) []:
<-- Enter your Organizational Unit Name (e.g. "IT Department").
Common Name (eg, YOUR name) []:
<-- Enter the Fully Qualified Domain Name of the system (e.g. "server1.example.com").
Email Address []:
<-- Enter your Email Address.


3) Edit /etc/proftpd.conf and add the text highlighted. Make note of placement in reference to existing configuration.
 

 nano -w /etc/proftpd.conf

<IfModule mod_tls.c>
    TLSProtocol TLSv1
</IfModule>

# Restore file permissions capability to site administrator
 <Global>
   # Report localtime, not GMT
   TimesGMT                     off
   ServerIdent                  on "FTP Server"
   IdentLookups                 off

<IfModule mod_tls.c>
    TLSEngine on
    TLSLog /var/log/tls.log

    # Are clients required to use FTP over TLS when talking to this server?
    TLSRequired off

    # Server's certificate
    TLSRSACertificateFile /etc/proftpd/ssl/proftpd.cert.pem
    TLSRSACertificateKeyFile /etc/proftpd/ssl/proftpd.key.pem

    # Authenticate clients that want to use FTP over TLS?
    TLSVerifyClient off

    # Allow SSL/TLS renegotiations when the client requests them, but
    # do not force the renegotations.  Some clients do not support
    # SSL/TLS renegotiations; when mod_tls forces a renegotiation, these
    # clients will close the data connection, or there will be a timeout
    # on an idle data connection.
    TLSRenegotiate required off

</IfModule>

</Global>

4) Connect to server using a secure FTP client and choose FTP over Explicit SSL/TLS. If you dont have a secure FTP client try this free one FileZilla http://filezilla-project.org/