freebsd.munk.me.uk - Apache

Modsecurity 2.0 Released

nospam@example.com (munk) — Fri, 20 Oct 2006 14:24:52 +0000

A new version of mod_security has just been released - 2.0 - complete with a total rewrite that includes a number of new features. El reg is running an article on the new release which includes an interview with ModSecurity's author Ivan Ristic.

mod_security is an apache module for monitoring requests made to a web server and acting on those requests according to rules - useful for blocking malicious bots, stopping web spammers and so on. I've been using it for a few years now and it handles blocking of weblog spammers and trojan worms/bots very well, though it has to be said the configuration isn't the simplest of all time.

Hopefully this configuration issue might be made easier with the also newly released modsecurity console, although reading through that page it doesn't seem to mention anything about using it to configure mod_security... Will have a look at it later and see what's what.

A list of the new features or improved features in ModSecurity 2.0 - taken from the article above:

Five processing phases (where there were only two in 1.9.x). These are: request headers, request body, response headers, response body, and logging. Those users who wanted to do things at the earliest possible moment can do them now.

Per-rule transformation options (previously normalisation was implicit and hard-coded). Many new transformation functions were added.

Transaction variables. This can be used to store pieces of data, create a transaction anomaly score, and so on.

Data persistence (can be configured any way you want although most people will want to use this feature to track IP addresses, application sessions, and application users).

Support for anomaly scoring and basic event correlation (counters can be automatically decreased over time; variables can be expired).

Support for web applications and session IDs.

Regular Expression back-references (allows one to create custom variables using transaction content).

There are now many functions that can be applied to the variables (where previously one could only use regular expressions).

XML support (parsing, validation, XPath).

The article is well worth reading if you already use ModSecurity - particularly if you're interested in moving from just simple blocking and logging of requests as in mod_security 1.0 to a more sophisticated web application firewalling system - mod_security 2.0. 2.0 includes a pseudo web app firewalling programming language making it easy to manipulate and process HTTP in a stateful manner - tracking HTTP sessions per IP in real time for example or perhaps watching for anomalous web activity and then flagging any IP that transgresses behaviour deemed as acceptable and watching for that IP in the future.

Solving permission problems with parsepath.pl

nospam@example.com (munk) — Mon, 04 Sep 2006 22:41:00 +0000

parsepath.pl is a brilliant perl script for fixing permissions problems on Unix based platforms by Jeremy Mates. Probably the most common type of permission problem from a sysadmin/webmaster's viewpoint is uploading a file to a directory in a website's document root folder and then trying to access the file or script in a web browser only to get the dreaded 403 error message:

Forbidden
You don't have permission to access /foo/bar/test.php on this server.

Most time the solution is very simple, just change the permissions on 'test.php' to make sure the user the webserver runs as can read the file correctly - the simplest and most common method being to change the mode of the file to '755':

CODE:

chmod 755 test.php

Unfortunately sometimes it's not that easy and many times you see users asking 'I'm getting 'access denied' errors even though I've changed the perms to 755'. The problem is that one of the subdirectories that the 'test.php' file lives in has permissions set so that the webserver can't read the file properly. Now that's where the headache comes in :)

However, parsepath.pl can take the headache out of fixing permissions problems.

Say you have a website document root directory tree /usr/local/www/web/www.munk.me.uk/foo/bar and you upload a web script 'test.php' into that directory. You try and access the file in a webbrowser but get the 403 permission denied error above. First off you check the permissions on the file itself:

CODE:

[23:58:17] root@users /usr/local/www/web/www.munk.me.uk/foo/bar# ; ls -l
total 0
-rwxr-xr-x 1 www www 0 Sep 4 23:39 test.php

That looks ok, with permissions 755 and the owner/group set to 'www' the webserver user 'www' should be able to read the file ok. So in this case the problem must be with the permissions on one of the parent subdirectories. The old method of working out the perms would be either to trawl one by one through each directory checking the perms on each subdirectory or to change the permissions recursively on the document root folder so all subfolders have the read bit set for the webserver user/group.

With parsepath.pl things are a lot simpler though - just run the following command:

CODE:

[0:03:21] root@users /usr/local/www/web/www.munk.me.uk/foo/bar# parsepath.pl user=www +r test.php
! group=www +rx fails: d 0700 root:www /usr/local/www/web/www.munk.me.uk/foo
! unix-other +rx fails: d 0750 root:wheel /usr/local/www/web/www.munk.me.uk/foo/bar

With this command parsepath.pl recurses through each subdirectory below the file/path you feed it on the commandline and tells you the permissions problems - if any - for the user 'www' (the user=www argument) to read (the +r argument) the file 'test.php'.

In the output, we're told that permissions to read the test.php by the user www fails on two counts:

CODE:

# the group bit on the folder 'foo' doesn't have the +rx flag set:
! group=www +rx fails: d 0700 root:www /usr/local/www/web/www.munk.me.uk/foo

# the other bit on the folder 'bar' doesn't have the +rx flag set:
! unix-other +rx fails: d 0750 root:wheel /usr/local/www/web/www.munk.me.uk/foo/bar

With this information it's easy enough to go in and make the changes necessary to fix the problem using 'chmod g+rx foo foo/bar'.

There are other ways of invoking parsepath.pl though. Running it just with a file/path as an argument it'll tell you the permissions on each subdirectory under it:

CODE:

[0:10:33] root@users /usr/local/www/web/www.munk.me.uk/foo/bar#
> parsepath.pl /usr/local/www/web/www.munk.me.uk/foo/bar/test.php
% /usr/local/www/web/www.munk.me.uk/foo/bar/test.php
d 0755 root:wheel /
d 0755 root:wheel /usr
d 0755 root:wheel /usr/local
d 0755 root:wheel /usr/local/www
d 0770 www:wheel /usr/local/www/web
d 0750 www:www /usr/local/www/web/www.munk.me.uk
d 0700 root:www /usr/local/www/web/www.munk.me.uk/foo
d 0750 root:wheel /usr/local/www/web/www.munk.me.uk/foo/bar
f 0755 root:www /usr/local/www/web/www.munk.me.uk/foo/bar/test.php

which can is better to see a whole tree in one go.

No permissions were harmed in the making of this article!

I'll include the parsepath.pl script in the extended article just in case the original ever gets lost - big credit of course goes to the author of the script, Jeremy Mates. His site is actually very interesting from a sysadmin's point of view containing lots of interesting admin scripts and thoughts on system administration in general - spent quite a while grazing through his stuff there - cheers Jeremy.

Continue reading "Solving permission problems with parsepath.pl"

Apache Configuration Strategy For Moving A Website

nospam@example.com (munk) — Tue, 29 Aug 2006 09:44:15 +0000

Well, today is the first day after my old domain munk.me.uk expired - I'm now using the domain munk.me.uk, rest in peace munk.me.uk!

This article discusses a strategy for migrating a domain name in the case when the new domain will host exactly the same content of the old but you want to ensure that search engines will register the move and users who search for content you host will still find you after the move.
Continue reading "Apache Configuration Strategy For Moving A Website"

Apachectl Leaks Sensitive in phpinfo() PHP Calls

nospam@example.com (munk) — Sat, 15 Jan 2005 10:27:36 +0000

Quite a while ago I posted a question to the freebsd-isp mailing list about a minor modification that could be added to the apachectl apache startup shell script to stop it from displaying a lot of sensitive information when PHP and other CGI applications display apache's environment information to users. A bit mouthful - the gist of it is that when you use apachectl to start/stop apache from a shell, the environment variables of the shell user that invokes the apachectl script become available to CGI applications by default.

For example in my case the following types of sensitive environmental information is available by default when someone runs phpinfo.php on my server:

CODE:

_ENV["USER"] root
_ENV["MAIL"] /var/mail/root
_ENV["qbertconf"] /home/munk/eggdrop/bots/qbert/qbert.conf
_ENV["webcpdevcvs"] /home/munk/cvs/ver2
_ENV["alllog"] /var/log/all.log
_ENV["apacheerrlog"] /var/log/httpd-error.log
_ENV["SHLVL"] 2
_ENV["VENDOR"] intel
_ENV["snortruledir"] /usr/local/share/snort
_ENV["PHP4_OPTFILE"] /root/php4_options
_ENV["IRCNICK"] munk
_ENV["HOME"] /root

and so on along with all the other environment variables I have set in my default root shell - not really what I want displayed to all and sundry :P To see this yourself on your own server, try running phpinfo() from a php script:

CODE:

<?php phpinfo(); ?>

Anyway the solution I found is to 'sanitize' the environment when the apachectl script starts up the apache httpd process by using the following simple line in apachectl:

CODE:

HTTPD=`echo /usr/bin/env -i $HTTPD`

A problem report I was considering sending to the FreeBSD ports team is included in the extended article below - I didn't bother sending it in the end because it's not really a FreeBSD issue. Not sure why I didn't send it to the Apache mailing list though, I thought I did but I can't find it now. Ho hum.

Continue reading "Apachectl Leaks Sensitive in phpinfo() PHP Calls"

Awstats Updates and Broken Icons

nospam@example.com (munk) — Sat, 01 Jan 2005 17:13:40 +0000

AWStats was recently updated from version 6.1 to 6.2. Of itself this was no problem, checking the changelog there is nothing significant that breaks anything. However unfortunately the AWStats FreeBSD port was modified by the port maintainer to correct a problem that already existed - namely that some of the tools that ship with AWStats don't work because they can't find the files that they need - icon files for example.

The port maintainer modified the port so that the icon files for AWStats are now placed in /usr/local/www/awstats/icons/ - the original default location was /usr/local/www/icons/. This took me a little while to figure out, perhaps it's the New Years hangover but a more detailed note in the /usr/ports/UPDATING file for current users of awstats would have been good. Ho hum I suppose it's at least something that anything was added at all to UPDATING. I'm all for tidying up directory structures but in this case it seems that a 'fix' for something that was trivially broken has actually resulted in breaking something that was working fine - as the saying goes if it ain't borked don't fix it.

The short and tall of it is that to fix the problem that this 'fix' created, a 'DirIcons' directive needs to be added to each awstats config file:

CODE:

DirIcons="/awstatsicons"

and then an alias line needs to be added to the httpd.conf file:

CODE:

Alias /awstatsicons "/usr/local/www/awstats/icons/"

otherwise the awstats pages look odd because the icon files can't be found.

For posterity there's a GNATS message I drafted to be sent in reply to this FreeBSD Problem Report regarding the recent awstats update in the extended article below.

Continue reading "Awstats Updates and Broken Icons"

Perl User Agent Grep Script

nospam@example.com (munk) — Tue, 07 Sep 2004 12:13:37 +0000

This is here for reminders sake really. Second time in a week I've needed to use a perl sort construct and forgotten how it 'works'. Gist is to search for a certain field (user agent/robot) on each line from STDIN (a httpd access logfile), keep track of how many of each entry (robot) we find and finally print out the count (hits) for each entry (robot).

Makes more sense when you execute the script perhaps. Or not. Whatever, it's just a reminder for the next time I need to use the sort perlfunc on a hash. :P

CODE:

#!/usr/bin/perl
=comment
Displays number of hits per user agent to a httpd access logfile.

Use it by piping logfile lines to it:

./ua_grep somelogfile.log

or

grep text_to_search_for_in_log somelogfile.log | ./ua_grep
=cut

foreach(<>){
    split /"/;
    $UAs{ @_[5] }++;
}

foreach $ua ( sort { $UAs{$a} <=> $UAs{$b} } keys %UAs ) {
    printf "%16s -> %d\n", $ua, $UAs{$ua};
}

Using mod_security to Block Serendipity Weblog / Blog Comment Spam

nospam@example.com (munk) — Fri, 03 Sep 2004 06:53:17 +0000

Have been seeing quite a bit of comment spam recently on this weblog, mainly pimping online pharmacies. Needless to say this is not wanted.

After the fourth lot of spam in not that many days I've added a few lines to the mod_security section in my httpd.conf file to stop apache from allowing access if the POST payload contains certain keywords:

CODE:

SecFilterSelective POST_PAYLOAD "BLOCKED KEYWORDS GO HERE!!!"

You get the gist, adding the keywords would freeze me out so won't bother with that :P

Before I considered mod_security though I was wondering how a spamassassin plugin might work with Serendipity - a plugin that would forward comments to a spamd daemon for spam checking before accepting the comment. Not too sure how spamd handles data that doesn't contain mail headers, I guess it should still parse it ok and return a score. No doubt this won't get pursued any further seeing as this mod_security fix should do the trick.

Attempts To Exploit My_ eGallery Vulnerability Target Random Sites

nospam@example.com (munk) — Thu, 26 Aug 2004 06:21:27 +0000

This is by no means a new attempt to exploit a vulnerability in a PHP web application, but since I started using snort as a full time intrusion detection system I've picked up countless attempts to exploit a vulnerability in an online PHP photo gallery called My_ eGallery. The annoying thing about this is that I've never had this gallery system installed on any of the sites I maintain - particularly the one that keeps getting hit - but they continue to try their exploits. Read on for more...
Continue reading "Attempts To Exploit My_ eGallery Vulnerability Target Random Sites"

Script Kiddies Redirected to NSA Kiddies site

nospam@example.com (munk) — Thu, 26 Aug 2004 06:04:02 +0000

Ok, I'm really bored now. Some morons have been trying to exploit a hole in an online PHP gallery system called E_Gallery for weeks now. The thing is I don't have and never have had E_Gallery on my system - at least not in the location they're trying to 'exploit'.

This really pisses me off, why can't people bother to check whether the thing they're trying to exploit actually exists on the system they're trying to exploit before wasting bandwidth trying to exploit it??? Bloody idiot script kiddies.

Enough's enough, I've just set up this .htaccess file now to redirect them to the NSA kiddies page, let them try and crack something worth cracking:

CODE:

RewriteEngine On
RewriteRule ^modules/my_ egallery/public/displayCategory.php \
http://www.nsa.gov/kids/intro.htm

I'll write something up about the actual exploit attempts in a separate article.

UPDATE:
Ok this was not a good idea, I'm redirecting hits to the URI to somewhere more interesting (like the location of the backdoor servers they're trying to install in the first place :P).

Configuring Apache httpd.conf to Serve Perl Scripts As Plain Text

nospam@example.com (munk) — Tue, 10 Aug 2004 22:21:20 +0000

Here's a small teaser the solution to which is really obvious with the benefit of hindsight.

I needed to make all the files within a certain directory be served as plain text by Apache, regardless of their extension. The solution was to use the following in the httpd.conf file:

CODE:

    <Directory "/usr/local/www/web/www.munk.me.uk/programming/perl">
        SetHandler default-handler
    </Directory>

Specifically I had a directory which contained a lot of perl scripts all ending in the .pl extension and I didn't want them to be treated as executable files - ie I wanted the contents of the files to be visible in a browser to display the source code to users. Adding the above config lines did the trick.

The use of handlers in apache is covered here:

http://httpd.apache.org/docs/handler.html

Apache File Permission Security Model Suitable For Multiple Untrusted Shell Users

nospam@example.com (munk) — Tue, 10 Feb 2004 16:41:43 +0000

This is a post I made to the FreeBSD-Questions list ages ago describing a file permission security model suitable for restricting untrusted shell users without restricting Apache access.

Original post:

here

Post is included in extended article below.

Continue reading "Apache File Permission Security Model Suitable For Multiple Untrusted Shell Users"

PHP HTTPD Web Statistics Frontend For mod_accounting

nospam@example.com (munk) — Mon, 02 Feb 2004 09:30:55 +0000

I'm in the process of hacking at the IPFWstats source code so it displays HTTPD statistical reports for all the vhosts on the server - based on the transfer data logged to a mysql database by the mod_accounting apache module. So far the hacked script displays a rundown of httpd transfer per vhost in descending order of traffic.

You can see the progress so far here:

http://httpdstats.munk.me.uk/

Be warned on average the page will take around 8-10 seconds to load!

I'm fairly happy with it so far, although it does take an age to sort the data - around 8 seconds for 5,000 records. I don't think this can be helped really, perhaps using subselects in MySQL could help.

If anyone is interested in obtaining this hacked up code then let me know.

EDIT:
On second thoughts the source code is not available, although I would be open to offers to configure mod_accounting and install the stats package on a server.

Updated awstats from 5.9 to 6.0

nospam@example.com (munk) — Sat, 24 Jan 2004 14:15:27 +0000

In the weekly portupgrade I just upgraded awstats from 5.9 to 6.0. Looking at the announcement on the awstats site things should go smoothly, although I've added a note for myself in the extended article about customizing the installation of awstats on the server.

Continue reading "Updated awstats from 5.9 to 6.0"

mod_php vs php-cgi

nospam@example.com (munk) — Thu, 22 Jan 2004 23:13:23 +0000

This question came up recently on the apache httpd-users list. The article is here:

http://marc.theaimsgroup.com/?l=apache-httpd-users&m=107153065731172&w=2

The mail can be viewed in the extended entry and discusses when and why you'd want to run php as a cgi rather than as a module in apache.

Continue reading "mod_php vs php-cgi"

AWStats Logging Configuration For Individual Virtual Domains In Apache

nospam@example.com (munk) — Sat, 10 Jan 2004 12:08:00 +0000

Synopsis
This article is based on a 'note to self' I made for configuring the excellent web server statistics package AWStats.

I only decided to start using AWStats after I'd already configured over 50 virtual hosts in apache - therefore I needed a method of easily generating AWStats configuration files and collating AWStats statistics for those virtual hosts that I'd already had running on the server for a while.

This article details:

the use of the script - awsitestats.sh - to 'pregenerate' AWStats for sites that already exist.
a cronjob to update site stats daily
a few PHP scripts to allow admins and visitors to a site easy access to AWStats statistics.
notes on apache httpd configuration to make viewing of AWStats easier.

The complete article is fairly long, so I won't include it all here - please view the extended version for the complete article and source code snippets.

Continue reading "AWStats Logging Configuration For Individual Virtual Domains In Apache"