Openx Handbook Openx Server Optimization And Performance Tuning
Index
- Introduction
- Requirements
- Hardware Optimization
- Operating System and Service Hardening
- Apache, PHP Configuration And Performance Tuning
- MySQL Configuration and Performance Tuning
- Server Monitoring And Service Tweaking
- Introduction to Advanced Openx Servers
- Appendix-A About The Author
- Appendix-B License
Openx Handbook : Openx Server Optimization And Performance Tuning
Disclaimer : Please notices this document will help you for configuring an Openx add server. This article is licensed under Creative Commons Attribution-Noncommercial 2.5 India .Please don’t copy and paste the configurations to your server if it is not the hardware that mentioned in the hardware requirements.
1. Introduction
This document will help you to configure and build a completely optimized Openx add server. This is a completely tested configuration and the methods used in this article is fully for a Linux( some for unix) operating system. Please don’t copy and paste the configurations into you’r server if you don’t have the software and hardware requirements mentioned in this document. This will help you to build and optimize your openx server upto 99% stability and performance and the rest depends on who configuring it.
Please note don’t use these configuration for normal shared hosting server, because it is only for Openx add servers. As per these configurations our add servers having load between 0 and 1 , and serving 3.6 million requests per minute. This is they way how we providing 70% of add in Croatia ( Zagrab ). I would like to share the server optimization and performance tuning techniques to the Openx community.
Also not I am not telling about how to secure your server, but how to optimize it.
1.1 What is Openx?
I think you know what is Openx otherwise you no need to look into this article. Am I right? If not please check the website to know more about openx. In short Openx is a community project for adds management and it is a nice application.
2. Requirements
This the requirements of our servers. I think this is not a bad server. Please remember don’t copy and paste the settings if you are not using these type of settings.
2.1 Hardware Requirements
See how our box looks like.
- Processor – 2X Quad core Xeon E5320
- RAM – 16 GB ,
- SWAP – 16 GB /
- HDD – 8 x 73 SAS 10.000 rpm in RAID 10
2.2 Software Requirements
We are using centos 5.2 64bit Operating system with Cpanel. If you think why need for cpanel?, because cpanel have good scripts for building apache , php and mysql in the server. Also it is the best control panel for hosting servers. Please see the complete software requirements below,
- Operating System – Centos 5.2 64 bit
- Control panel – Cpanel
- Webserver – Apache 2.2.9
- PHP – 5.2.6
- Mysql – 5.0
In general any LAMP environment is good. Even you can use Lighttpd as webserver for single add servers. If you are not using the Cpanel please use Lighttpd webserver.
3.Hardware Optimization
It an important thing and the first step before building the server.
3.1 Port Connection Checking
First you can check the NIC speed. It is better to connect your server to a 1000 MBit Port. You can check the NIC as follows.
# ethtool eth0
Here eth0 is NIC . Please notice the following values from the above report
Speed: You can see the transfer speed ( eg : 1000Mb/s )
Duplex: Full ( It must be Full duplex connection )
Auto-negotiation: on
Now please check the current port speed as follows
# mii-tool -v
Here you can see some values as follows,
eth0: negotiated 100baseTx-FD, link ok .
This means your server is connected to a 100 Mbit Full Duplex Stream.
3.2 I/O wait optimization
For a high traffic server the I/O wait is a major problem. I used the 8 SAS disks in a RAID10 array for getting the performance. I done the HDD partitioning as follows,
/ - For installing the OS /home - This is the cpanel user home directory. It contains the files /usr - It contains the OS components /var - It is mainly for the mysql database /tmp - It is the temp folder, which is for creating temporary files /backup - This is my backup disk
Now open your file system table ( /etc/fstab ) add the following flags to /, /var and /home partitions.
noatime
“noatime” a file system has been mounted with this option, reading accesses to the file system will no longer result in an update to the atime information associated with the file. So it is a benefit. An example is given below
LABEL=/var /var ext3 defaults,usrquota,noatime 1 2
NOTE: After modifying the value you may either need to reboot the system or need remount the disk partition to do this changes. I remounted the /var partition as follows
# mount -o remount /var
If you want to check the partition is mounted properly do this check as follows
# mount
Now you can see the /var is showing “notime” option.
Now you need to check DMA parameter of the HDD , in normal disks you can use the command “hdparm” . But hdparm not giving SCSI or SATA disks information. So there is a utility called ” sdparm ” . You cam use it for checking the Direct Memory Access ( DMA ). It must be enabled.
3.3 understanding the Processor
You will get all processor information from the file /proc/cpuinfo . If you need to know how many processors are available in your server , please do the following command.
# grep processor /proc/cpuinfo | wc -l
It will show the number of physical/virtual processors. If you see a count of 8 then server load less than 8 is normal.
This is all about hardware optimization
4.Operating System and Service Hardening
Next is our software optimization part. This is a very important part. I am here giving an example of a Centos Operating system with Cpanel as a control panel.
4.1 Useless Package Removal / Os hardening
First you may need to remove the USELESS packages from the server. For a server operating system in Centos/RHEL arong 350 to 400 packages required. But by default it coming with 700+ rpms. You can find the rpm’s as follows ,
# rpm -qa
From the above list remove the useless rpm’s as ,
# rpm -e
Please not if you removing a System package it will corrupt the OS. So only do it if you know about the packages. Note , you are warned.
4.2 Useless Service Removal
So now we have removed a set of packages. So now we need to stop a lot useless services, which will use our server resources . You can stop the server as follows ,
# service stop < service name >
eg : service cups off
Please you need to disable it during to avoid it from starting during the reboot/booting time as follows ,
# chkconfig < Service name > off
eg : chkconfig cups off< br /> Also you can verify that service is off or not as follows ,
# chkconfig --list < service name >
eg : # chkconfig --list cups cups 0:off 1:off 2:off 3:off 4:off 5:off 6:off
So please remove the following services from the server as mentioned above.
cups , xfs , atd , nfslock, canna, FreeWnn, cups-config-daemon, iiim , mDNSResponder, nifd, rpcidmapd, bluetooth, anacron, gpm, saslauthd, avahi-daemon, avahi-dnsconfd , hidd, pcscd, sbadm , webmin and ossec
Note : If your server is not a normal production server, you may be using some of the above services.
Also please remove any firewalls like apf or csf from your server. Because those may simply block highly accessible ip’s .
4.3 Sysctl Configuration
So the next step is optimizing the sysctl. You can tune you kernel via sysctl. To know your current sysctl parameter please do the following,
# sysctl -a
If you want to know all parameters starting with vm please use the command as follows,
# sysctl vm
Also now you can change any particular sysctl parameter value as follows.
# sysctl -w variablename=value
But notice your changes only be affect current settings. If you wish to add those values as permanent please include it in /etc/sysctl.confi
You can see a tuned sysctl configuration as follows, it is using in our server having the above configuration.
# Disables packet forwarding net.ipv4.ip_forward=0 # Disables IP source routing net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.lo.accept_source_route = 0 net.ipv4.conf.eth0.accept_source_route = 0 net.ipv4.conf.default.accept_source_route = 0 # Enable IP spoofing protection, turn on source route verification net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.lo.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 # Disable ICMP Redirect Acceptance net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.lo.accept_redirects = 0 net.ipv4.conf.eth0.accept_redirects = 0 net.ipv4.conf.default.accept_redirects = 0 # Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets net.ipv4.conf.all.log_martians = 0 net.ipv4.conf.lo.log_martians = 0 net.ipv4.conf.eth0.log_martians = 0 # Disables IP source routing net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.lo.accept_source_route = 0 net.ipv4.conf.eth0.accept_source_route = 0 net.ipv4.conf.default.accept_source_route = 0 # Enable IP spoofing protection, turn on source route verification net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.lo.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 # Disable ICMP Redirect Acceptance net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.lo.accept_redirects = 0 net.ipv4.conf.eth0.accept_redirects = 0 net.ipv4.conf.default.accept_redirects = 0 # Disables the magic-sysrq key kernel.sysrq = 0 # Decrease the time default value for tcp_fin_timeout connection net.ipv4.tcp_fin_timeout = 15 # Decrease the time default value for tcp_keepalive_time connection net.ipv4.tcp_keepalive_time = 1800 # Turn off the tcp_window_scaling net.ipv4.tcp_window_scaling = 0 # Turn off the tcp_sack net.ipv4.tcp_sack = 0 # Turn off the tcp_timestamps net.ipv4.tcp_timestamps = 0 # Enable TCP SYN Cookie Protection net.ipv4.tcp_syncookies = 1 # Enable ignoring broadcasts request net.ipv4.icmp_echo_ignore_broadcasts = 1 # Enable bad error message Protection net.ipv4.icmp_ignore_bogus_error_responses = 1 # Log Spoofed Packets, Source Routed Packets, Redirect Packets net.ipv4.conf.all.log_martians = 1 # Increases the size of the socket queue (effectively, q0). net.ipv4.tcp_max_syn_backlog = 1024 # Increase the tcp-time-wait buckets pool size net.ipv4.tcp_max_tw_buckets = 1440000 # Allowed local port range net.ipv4.ip_local_port_range = 16384 65536
Note: If you don’t know nothing about sysctl please leave it.
5.Apache and PHP Configuration And Performance Tuning
In this section you can see how to get a highly configured apache and php for better performance. Please not I am here using some commands and scripts related with the cpanel. But the building flags are coming with the sources itself.
Please use the following script to start the build process.
# /scripts/easyapache
Now select the Option “Basic” from the menu and ” start customization based on the profile ” . Select the Apaceh version 2.2 and php 5 from the coming menus. Remember don’t use pache 1.3 or apache 2.0.
Now please proceed with the Apache configuration section as follows.
5.1 Apache configuration
Please go to the ” Exhaustive Options” selections menu and enabled the following Apache modules.
1) Asis
2) Auth Digest
3) AuthDefault
4) Cache
5)Casefilter
6) DBD
7)Deflate
8)Disk cache
9)Distcache
10)Env
11)Expires
12)Mod fcGID
13)File cache
14) FileProtect
15)Frontpage
16)Headers
17)MPM Prefork
18) Memcache
19) Proxy
20) UniqueID
21) usertrack
22) Vhost Alias
Now from the other Modules section please select the following,
1) EAccelerator
2) IoncubeLoader
3) Mod Security
4) Suhosin
5) Zend Optimizer
NOTE: Please don’t use SUPH configuration in Openx Add server. It will eat up your server resources. The apache mod_php is the best configuration.
Now please chose the following option from PHP or enable these options.
5.2 PHP configuration
1) Bcmath 2) calendar 3) Concurrent DSO Patch 4) Curl, Curlssl, Curl wrappers 5) Dbase 6) Exif , FTP, GD, Gettext , Iconv 7) Imap, MM, magicQuotes,mailHeaders , 8) Mregex, Mbstring , mcrypt , mhash , Mimme 9) Mysql of the System , Mysqli Openssl , PDO , PDO mysql 10) POSIX, Pspell, SOAP, Safemod 11) TTF, Tidy, Wddx, XSL , XMLRPC, Zend , Zlib
Now enable the option reset apache configuration to default and then proceed to next step , save and build. It will take some long time to complete the build process.
So after completing the build process, please make sure apache is up and running. Now proceed to step 5.3
5.3 Stripping and tuning
This our configuration section. First we need to strip our binaries for getting light foot print. You can use the strip utility as follows.
#strip -s < full path to binary >
eg : strip -s /usr/local/bin/php
Please strip the following binaries.
1) /usr/local/bin/php 2) /usr/bin/php 3) /usr/local/apache/bin/httpd 4) All binaries under the folder /usr/local/apache/modules/
Now it is the time to tune the apache. We have enabled a lot caching moduels and an MPM preforker module. Now we need to configure the MPM parameters. After finishing the build process the httpd-mpm configurations can be found under the extra folder in apache installations prefix ( /usr/local/apache/conf/extra/ ) . But it is not included in the httpd.conf . So open the file /usr/local/apache/conf/httpd.conf and add the following line.
Include "/usr/local/apache/conf/extra/httpd-mpm.conf"
Now open the file /usr/local/apache/conf/extra/httpd-mpm.conf and modify the preforker module sections as follows ,
StartServers 30
MinSpareServers 5
MaxSpareServers 50
ServerLimit 2000
MaxClients 1800
MaxRequestsPerChild 0
Now please reduce the “Timeout” value in /usr/local/apache/conf/httpd.conf . Buy default it have value 300. Please reduce it to 100. It will be enough for Openx request.
So we completed modification of apache configurations. Please run the following command if you are in a cpanel server for updating the cpanel apache templates.
# /usr/local/cpanel/bin/apache_conf_distiller --update
Now open the php.ini file, we need to tweak it. In general the php.ini will be /usr/local/lib/php.ini. Please open this file using your favorite editor.
# vi /usr/local/lib/php.ini
Now set the following values.
memory_limit = 128M max_execution_time= 30 register_globals = Off mysql.allow_persistent = On mysql.allow_persistent = On mysql.trace_mode = Off
This is all about apache and php tuning . Now restart apache as follows.
# /etc/init.d/httpd restart
Now we need to configure MySQL server. Please proceed with next step.
6. MySQL Configuration and Performance Tuning
The global MySQL configuration is /etc/my.cnf. You may need to configure the mysql variables. But its memory variable and threading variables depends on your server RAM and number of processors. So the mysql configurations vary from server to server. Please see our my.cnf for the above hardware. Please don’t copy and paste the values. Take advices from your experts. Only an experienced guy can help you for tuning your mysql server. Any way I hope the following my.cnf will help you.
[mysqld] safe-show-database old-passwords = 1 max_connections =2048 max_user_connections = 1024 key_buffer_size = 2048M myisam_sort_buffer_size = 64M join_buffer_size = 1M read_buffer_size = 1M sort_buffer_size = 2M table_cache = 4000 thread_cache_size = 384 wait_timeout = 20 connect_timeout = 10 tmp_table_size = 2048M max_heap_table_size = 512M max_allowed_packet = 64M net_buffer_length = 16384 max_connect_errors = 10 thread_concurrency = 16 concurrent_insert = 2 table_lock_wait_timeout = 30 read_rnd_buffer_size = 786432 bulk_insert_buffer_size = 8M query_cache_limit = 7M query_cache_size = 64M query_cache_type = 1 query_prealloc_size = 262144 query_alloc_block_size = 65536 transaction_alloc_block_size = 8192 transaction_prealloc_size = 4096 max_write_lock_count = 16 long_query_time = 5 skip-name-resolve skip-locking [mysqld_safe] open_files_limit = 8192 [mysqldump] quick max_allowed_packet = 16M [myisamchk] key_buffer = 384M sort_buffer = 384M read_buffer = 256M write_buffer = 256M
Now please proceed with the following section.
7.Server monitoring And Service Tweaking
This is an important section. you need to monitor your server at least 48 hours to know the mysql and apache access limits. After that you can change the apache MPM configurations and mysql /etc/my.cnf variables. A perfect tuning will help you to get 100% uptime ( in service not in hardware ). So this is all about an Openx server building procedure.
8.Introduction to Advanced Openx Servers
This is an advanced section. Here I can give you some solutions for setting up advanced mysql servers. You can setup cluster for hosting your add server. You can chose any of the following server configurations for setting up an openx cluster Group
1) Simple cluster
It need the following configurations.
Load balancer – 1
Web server – 2
Database Server – 1
So a total of three Servers and one load balancer.
2) Redundancy clusters
It need the following hardwares,
Load balancer – 2
Web servers – 3
Database Servers – 2
So a total of 5 servers and 2 load balancers
3) Large Scale HA Cluster
These type of clusters are using in our high traffic add servers. This is little expensive, but give you high stability and uptime. Please see the hardware requirements below.
Hardware Firewall – 2
Load balancer – 2
Web servers – 3
Database servers – 3 File Servers – 2
So total of 7 servers , 2 load balancers
and 2 hardware firewalls.
But the configurations of these type of openx clusters are out of the scope of this article. Some one is calling me now. So I will write those cluster configurations in a new article, if i get time.
Appendix-A About The Author
Name : Sherin A
This article is licensed under Creative Commons Attribution-Noncommercial 2.5 India It means you can share and redistribute it , but give a link back to this original document, because sometimes I will add more tips to this document.
