WebEQ - Web Farm Equaliser

How does it work?

WebEQ can be deployed as a separate hardware module or, as a 'bolt-on' to your current Unix based web server. It listens to incoming requests for web services (usually on port 80), and assesses each request for validity (see below for details). Valid requests are then passed onto a real web server which is listening on a restricted internal port, preferably not viewable to the outside world, or ideally on an internal network segment. WebEQ operates at the http request level allowing much more flexibility in defining what are valid transactions to be honoured than packet filtering products.

Here is the process (simplified) in which every incoming request is handled (this happens in parallel - WebEQ can handle many requests simultaneously):

• An incoming request is received. WebEQ opens the connection and waits for the request header. If this does not happen within a few (configurable) seconds, the request is discarded and the port closed and re-used immediately. This mechanism deals with denial of service attacks.
• The request's URL is matched against an internal access list. For best results, this list should be configured for 'closed access' (ie. disallow everything except known valid URL's) for this web server. Requests that are rejected are returned (or alternatively simply discarded) as 'unknown' to the request originator. This prevents malcious or accidental requests (eg. 'Script Kiddies', nimda, code red) from reaching the web server.
• The request is passed to the next available web server in the farm in a transparent failover 'chain'. If a server in the chain cannot honour the request, then the request is passed to the next web server. Failed web servers are removed from the chain and support staff are notified.
• WebEQ captures the response from the web server, and returns this to the request originator.

WebEQ is written in multi-threaded POSIX compliant C for speed and platform portability. Request handling for WebEQ is very economical in CPU terms compared to that of a web server. WebEQ can still pass through valid requests successfully with no noticeable overall performance hit during an attempted denial-of-service attack if configured appropriately.

Key Benefits

Protection:

• Request flow management

By denying and allowing requests based on the amount of time taken for the request to be received, WebEQ protects against denial of service type attacks. The characteristic of these types of attacks is for a would-be attacker to send forged SYN packets to web server, which responds by opening a new connection and awaiting the actual request. These are sent in rapid succession, causing the web server to spend resources in handling these bogus requests. WebEQ will protect against these attacks by timing the interval between the connection open and the request data. If the timeout is exceeded, the connection is dropped. The intervals are configurable to ensure site-specific requirements.

• Buffer size management

Requests are processed in small chunks to a maximum limit. This reduces the possibility of exploitation of web server buffer overflow bugs.

• URL access list

Similar in style to a router or firewall access list, WebEQ allows the administrator to set up a filter based on IP address and a URL pattern. These patterns are written as unix style regular expressions. WebEQ is shipped with some sample access list templates that can be used 'as-is' or customised by the administrator if required. We also keep up-to-date copies of the templates online, to cater for new methods of attack.

Performance and Fortification:

• Load balancing

In the case of multiple web servers, WebEQ can be configured to direct valid requests to each of these on a round robin basis. This allows the load of the site to be distributed, enabling increased overall performance, as each web server is only accepting a share of the total requests. Servers can also be weighted, so that some servers can also do more work than others where the server facilities are not matched.

• Transparent failover and recovery

During operation, if WebEQ is unable to reach one or more of its web servers, it will transparently redirect to the next available operational server and mark the failed server as unavailable. Following a time interval (configurable) WebEQ will attempt to re-instate a failed server back into the available server pool. If it succeeds, the server is re-instated, if it fails, the server remains marked as such. This interval can be set so that WebEQ will never attempt to re-instate if the administrator prefers. Failures and re-instatements may be notified via email to an administrator.

• Web server 'keepalive'

Modern web servers have the ability to keep client connections alive if requested to do so. However, many browsers in use today do not support this, which results in the web server needing to close and open new connections to the same client. WebEQ incorporates a mechanism where it will always ask the web server to remain open regardless, and will even re-use a currently open connection for a completely different client. The result of this technique is that the web server opens and closes connections far less often, and when the site starts to become loaded will result in a significant overall throughput increase. As WebEQ runs in a very small resource footprint, handling the task of connection turnover is much more efficient. Of course, WebEQ handles a browser's request to keepalive also.

• Scalability

Multiple WebEQ engines may be deployed for the same (or multiple) web server farm, by configuring the DNS in a 'round robin' fashion. This causes traffic to first of all be shared (approximately) between the available WebEQ engines which in turn balances load amongst web servers. This architecture scales well and adds increased availability as there is not a single point of failure (as there would be with just one WebEQ). Here, network architectures (eg. separate fabrics) can also be employed to increase availability even further.

Management:

• Web based dynamic configuration

WebEQ provides a built-in browser interface that can be used by an administrator to configure it (or via a text configuration file). Changes may be made while WebEQ is operating, as it includes a feature to apply changes in a controlled fashion without affecting current sessions. This also implies that it does not need to be restarted for changes to take effect. WebEQ also responds to a SIGHUP signal to reconfigure itself, which can be useful for configuring from a script or the command line.

• Centralised logging and notification

In the case of multiple web servers, managing the logs that these produce requires some effort. WebEQ logs centrally in standard apache format, which can be fed into third party statistics packages (eg. pwebstats/MRTG). When multiple WebEQ engines are deployed, an internal logging synchroniser ensures central integrity of the logs.

• Statistics gathering

WebEQ maintains a set of internal current and historical statistics on its own performance. These can be viewed and browsed via the administration browser interface, or imported into other packages (eg. spreadsheet or database). The statistics are stored in plain text files.

• Virtual server support

Virtual servers are supported, and the logs may be merged or separated via a configuration parameter.

• Secure server support

WebEQ may also be deployed to validate https (secure sockets layer) connections.

Pilot and partner program

We are always looking to establish reference sites and construct relationships with suitable sales partners. If your organisation is interested in participating in running a WebEQ pilot site, we would be happy to talk to you and provide a custom installation to you at a significantly reduced cost. In return we would benefit from beng able to refer potential customers to live reference sites.

We are also keen to make contact with consultancy firms or distributors who would be interested in promoting WebEQ to their own customers. If any of the above applies, please get in touch with us via the contact page.

System Requirements

WebEQ Hardware Engine

Supplied in an industrial strength 1RU rack mountable computer, pre-installed with WebEQ, this option would suit organisations running either non-unix web services or who wish to seggregate their architecture into 'black boxes'.

WebEQ Software 'bolt-on'

Suited to customers already running their own unix platforms who are comfortable in installing and maintaining their own platforms. The software only version is installed on the customer's own hardware, either on the same physical machine as the web server or on a separate host.

Minimum System Requirements
(for standalone unix bolt-on)

• POSIX compliant unix operating system (eg. Solaris, Linux, HP-UX, Mac OS X)
• WebEQ memory footprint is approximately 2Mb
• The faster the CPU, the better, however WebEQ will run on lower spec. machines. (eg. Sun Sparc 10, Intel Pentium II) for smaller installations.
• Approx. 500Mb disk space for software (only 100K) and log space (the rest)