I realize that full text indexing and look for a database could be enabled by lots of pre-packed items. However, just from academical curiosity, I question how are individuals full text indexes really implemented. I've attempted to google for results with little answer. Please any feedback could be much appreciated.
Full text searches are based on a number of database engines nowadays like a core feature.
For implementation I believe your best choice is to look at postgres full text searches, as possible
- find lots of material on how it's implemented
- really change and have fun with the parsers (for instance optimize for several domain)
You will find further particulars and concept described on wikipedia:
Most likely by creating dictionaries of "words" and perhaps a little of lexical analysis. (Observe that fulltext searches whole words and never areas of words, so indexing might be restricted to that particular.)