Okay and so i am building this site that has functionality much like twitter and facebook and contains highly dynamic content.
The questions etc, and search engines like google I've come across are what create index and therefore are therefore more appropriate having a site with pretty much static content.
I want some recommendations and advice regarding how to begin using these index based search engines like google having a highly dynamic website thinking about new customers is going to be joining every hour, new content produced, and content being edited. It is extremely obvious that repairing a catalog each time simply to keep your search from going stale is quite absurd.
One solution that came near to fixing this problem was using MySQL MYISAM FULLTEXT search posts however i actually need another to that particular because the possible lack of foreign secrets results in data redundancy, plus I want something which can scale because the website develops and become flexible to personalized calculations for ranking etc.
Sphinx enables partial indexes. So, it has a primary index along with a secondary partial index which may be up-to-date whenever. It is also incredibly fast at indexing, to discover that repairing the whole index every, say, a few minutes is fast enough for you personally. When not, make use of the partial index option, and kick them back whenever a bit of submissions are added. Sphinx can be used by craig's list, to ensure that something to the scale. We have had great luck by using it on StartUpHire - it renews our entire index inside a couple seconds, then signals the search daemon to make use of the recently built index.
I'd recommend trying it out before you decide to say it's a bad fit.