I am running APC (php cache) on the medium website (13 000 visits each day) on the CentOS server 5 running php 5.3.3 with APC 3.1.4 with 4Gb of RAM. Within the last days I am going through many "unable to allocate memory for pool" errors in logs, and frequently the website goes lower.

I believe the issue is in APC. In the stats I observe that the cache is full after a couple of hours, and also the fragmentation always jumps to 100%. This are my config configurations of apc.ini:

apc.cache_by_default    1
apc.canonicalize    0
apc.coredump_unmap  0
apc.enable_cli  0
apc.enabled 1
apc.file_md5    0
apc.file_update_protection  2
apc.filters 
apc.gc_ttl  3600
apc.include_once_override   0
apc.lazy_classes    00
apc.lazy_functions  0
apc.max_file_size   1M
apc.mmap_file_mask  /dev/zero
apc.num_files_hint  0
apc.preload_path    
apc.report_autofilter   0
apc.rfc1867 0
apc.rfc1867_freq    0
apc.rfc1867_name    APC_UPLOAD_PROGRESS
apc.rfc1867_prefix  upload_
apc.rfc1867_ttl 3600
apc.shm_segments    1
apc.shm_size    512M
apc.slam_defense    1
apc.stat    1
apc.stat_ctime  0
apc.ttl 7200
apc.use_request_time    1
apc.user_entries_hint   0
apc.user_ttl    7200
apc.write_lock  1

From APC stats I can tell that the amount of cached files is extremely high (40 000), which is because of all the different cache files of phpbb and mediawiki. Must i prevent APC caching this files using apc.filters? Exist methods to the issue?

You should set apc.stat= in your production server and it'll prevent APC from really visiting the IO to see if the file continues to be transformed.

also set apc.slam_defense= as it is deprecated

Take a look at documentation for other available choices: http://php.net/manual/en/apc.configuration.php

I believe you need to set apc.mmap_file_mask = /tmp/apc-yourusernamehere.XXXXXX for file-backed mmap make file mask unique with the addition of your specific string XXXXXX (exactly 6 X’s) must still allow APC to include random string OR set to /dev/zero for anonymous mmap if you're able to spare the memory.

I can tell that you're using /dev/zero that takes memory unnecessarily.