Update - solved

The problem switched to function as the servers were built with a mount pointing in a file server which was upon the market the 2009 week so not associated with django / mod-python / etc in the end. Upgrading the mount reduced the problem.

Thank you for that comments &lifier help, and apologies for any wild goose chase ... I'll consider improving off mod-python regardless:-)

Summary

I am maintaining a Django webapp which was working fine until a couple of days ago, however all web demands just hang for age range. I am unaware that anything has transformed, so the issue is most likely something quite simple.

I have attempted restarting the webserver, and restarting httpd. 'top' shows the server does acceptable for CPU and memory.

Can anybody suggest other simple stuff that are prone to fail, or any other items to check?

More particulars

I did not produce the web server, so regrettably I am unsure of from the full particulars, or where you can search for all of the logs etc. I understand the net server consists of the next components: implemented using Django operates on a Linux server uses PostgreSQL database lighttpd for static content Apache to deal with incoming HTTP demands and pass them right through to Django via mod_python uses memcached to cache made pages. I actually do have full accessibility Linux server and also the database, so can happily poke around in anything basically know where you can look.

/var/log/httpd/access_log and error_log show lines such as the following whenever I create a new request. I'm not sure when the mod_python line signifies a mistake or otherwise (nothing apparent after i researched this log).

access_log:

127.0.0.1 - - [13/Jan/2011:10:56:11 +0000] "GET /testruns/testrun2176/ HTTP/1.0" 301 - "http://myapp/testruns/" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729; .NET4.0E)"

error_log:

[Thu Jan 13 10:34:19 2011] [notice] mod_python: (Re)importing module 'django.core.handlers.modpython'

Output from running development server

# python manage.py runserver 0.0.0.0:8088
Validating models...
0 errors found

Django version 1.0-final-SVN-unknown, using settings 'myapp.settings'
Development server is running at http://0.0.0.0:8088/
Quit the server with CONTROL-C.
[13/Jan/2011 14:23:31] "GET /testruns/testrun2176/ HTTP/1.1" 301 0

Then nothing happens. Web page just stuck loading.

I'm able to load among the pages that's slightly simpler. Through the dev version the images don't load but it is otherwise fine. The easy page reaches /testruns/ - though for whatever reason I do not observe that page within the runserver output:

# python manage.py runserver 0.0.0.0:8088
Validating models...
0 errors found

Django version 1.0-final-SVN-unknown, using settings 'myapp.settings'
Development server is running at http://0.0.0.0:8088/
Quit the server with CONTROL-C.
[13/Jan/2011 14:27:42] "GET /static/myapp.css HTTP/1.1" 404 1113
[13/Jan/2011 14:27:42] "GET /static/myapp_print.css HTTP/1.1" 404 1113
[13/Jan/2011 14:27:42] "GET /static/myapp_ticks_crosses.css HTTP/1.1" 404 1113

I ought to explain again, it was all working fine a couple of days ago. I've not clearly transformed anything since that time - I simply all of a sudden discovered that the needed pages will not load, and I am attempting to discover why.

The /var/log/httpd/ log files don't appear to exhibit anything particularly interesting, and I am unsure what else I ought to take a look at.

We are managing a second server that creates another group of test results. Running the expansion server on that implies that the 301 redirect is anticipated (.../testrunX/ redirects to .../testrunX/damaged/page1/).

// This trace, on a different server but with a similar setup, shows that the
// 301 redirect is expected, and is not the source of the problem
[server2]# python manage.py runserver 0.0.0.0:8088
Validating models...
0 errors found

Django version 1.0-final-SVN-unknown, using settings 'myapp.settings'
Development server is running at http://0.0.0.0:8088/
Quit the server with CONTROL-C.
[13/Jan/2011 14:47:59] "GET /testruns HTTP/1.1" 301 0
[13/Jan/2011 14:47:59] "GET /testruns/ HTTP/1.1" 200 11568
[13/Jan/2011 14:47:59] "GET /static/myapp.css HTTP/1.1" 404 1131
[13/Jan/2011 14:47:59] "GET /static/myapp_print.css HTTP/1.1" 404 1131
[13/Jan/2011 14:47:59] "GET /static/star.png HTTP/1.1" 404 1131
[13/Jan/2011 14:47:59] "GET /static/myapp_ticks_crosses.css HTTP/1.1" 404 1131
[13/Jan/2011 14:47:59] "GET /static/star.png HTTP/1.1" 404 1131
[13/Jan/2011 14:48:02] "GET /static/star.png HTTP/1.1" 404 1131

[13/Jan/2011 14:48:12] "GET /testruns/testrun1879/ HTTP/1.1" 301 0
[13/Jan/2011 14:48:12] "GET /testruns/testrun1879/broken/page1/ HTTP/1.1" 200 309477
[13/Jan/2011 14:48:12] "GET /static/myapp.css HTTP/1.1" 404 1131
[13/Jan/2011 14:48:12] "GET /static/myapp_print.css HTTP/1.1" 404 1131
[13/Jan/2011 14:48:13] "GET /static/myapp_ticks_crosses.css HTTP/1.1" 404 1131

And So I don't believe there's an infinite loop. Just for whatever reason the request / database query / another thing takes way too lengthy or perhaps is completely stuck.

memcached info

memcached around the bad server appears pretty empty. But that is most likely expected when the web demands are failing, ie nothing's been came back to become saved within the cache (and also the cache expiry time is 12 hrs).

Bad server:

// Top - only using 6K memory (VIRT)
PID   USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
30743 nobody    15   0  6640 4972  476 S  0.0  1.9   0:00.01 memcached

// memcache-top from http://code.google.com/p/memcache-top/
// only using 0.2% available space
memcache-top v0.6       (default port: 11211, color: on, refresh: 3 seconds)
INSTANCE                USAGE   HIT %   CONN    TIME    EVICT/s READ/s  WRITE/s
127.0.0.1:11211         0.2%    0.0%    5       0.8ms   0.0     2       161
AVERAGE:                0.2%    0.0%    5       0.8ms   0.0     2       161
TOTAL:         111.0KB/ 64.0MB          5       0.8ms   0.0     2       161

Good server:

// Top - using ~68K memory (VIRT)
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2503 nobody    15   0 67900 8256  396 S  0.0  3.2   0:01.68 memcached

// memcache-top - using 63% space
memcache-top v0.6       (default port: 11211, color: on, refresh: 3 seconds)
INSTANCE                USAGE   HIT %   CONN    TIME    EVICT/s READ/s  WRITE/s
127.0.0.1:11211         63.2%   0.0%    3       1.2ms   0.0     0       0
AVERAGE:                63.2%   0.0%    3       1.2ms   0.0     0       0
TOTAL:          40.5MB/ 64.0MB          3       1.2ms   0.0     0       0