I've got a ~300.000 row table including terminology queried using PHP and MySQL + FULLTEXT indexes. However when I searching an incorrect typed term for instance "hyperpext" naturally giving no results.
I have to "compansate" little writing errors and becoming nearest record from database. The Way I can accomplish such feaure? I understand (really, learned today) about Levenshtein distance, Soundex and Metaphone calculations but presently not getting a good idea to implement this to querying against database.
Sincerely. (Sorry about my poor British, I am attempting to do my best)
Check this out article based on how you may implement Levenshtein distance inside a MySQL saved function.
For offspring, the author's suggestion is to get this done:
CREATE FUNCTION LEVENSHTEIN (s1 VARCHAR(255), s2 VARCHAR(255)) RETURNS INT DETERMINISTIC BEGIN DECLARE s1_len, s2_len, i, j, c, c_temp, cost INT DECLARE s1_char CHAR DECLARE cv0, cv1 VARBINARY(256) SET s1_len = CHAR_LENGTH(s1), s2_len = CHAR_LENGTH(s2), cv1 = 0x00, j = 1, i = 1, c = IF s1 = s2 THEN RETURN ELSEIF s1_len = THEN RETURN s2_len ELSEIF s2_len = THEN RETURN s1_len ELSE WHILE j <= s2_len DO SET cv1 = CONCAT(cv1, UNHEX(HEX(j))), j = j + 1 Finish WHILE As i <= s1_len DO SET s1_char = SUBSTRING(s1, i, 1), c = i, cv0 = UNHEX(HEX(i)), j = 1 WHILE j <= s2_len DO SET c = c + 1 IF s1_char = SUBSTRING(s2, j, 1) THEN SET cost = ELSE SET cost = 1 Finish IF SET c_temp = CONV(HEX(SUBSTRING(cv1, j, 1)), 16, 10) + cost IF c > c_temp THEN SET c = c_temp Finish IF SET c_temp = CONV(HEX(SUBSTRING(cv1, j+1, 1)), 16, 10) + 1 IF c > c_temp THEN SET c = c_temp Finish IF SET cv0 = CONCAT(cv0, UNHEX(HEX(c))), j = j + 1 Finish WHILE SET cv1 = cv0, i = i + 1 Finish WHILE Finish IF RETURN c Finish
Younger crowd offers a LEVENSHTEIN_RATIO assistant method that will evaluate the number of different/total figures, as opposed to a straight edit distance. For example, whether it's 60%, then three-fifths from the figures within the source word aren't the same as the destination word.
CREATE FUNCTION LEVENSHTEIN_RATIO (s1 VARCHAR(255), s2 VARCHAR(255)) RETURNS INT DETERMINISTIC BEGIN DECLARE s1_len, s2_len, max_len INT SET s1_len = LENGTH(s1), s2_len = LENGTH(s2) IF s1_len > s2_len THEN SET max_len = s1_len ELSE SET max_len = s2_len Finish IF RETURN ROUND((1 - LEVENSHTEIN(s1, s2) / max_len) * 100) Finish
In the comments of http://dev.mysql.com/doc/refman/5./en/udf-producing.html
now i download the package in the mysql udf repository http://empyrean.lib.ndsu.nodak.edu/~nem/mysql/
wget http://empyrean.lib.ndsu.nodak.edu/~nem/mysql/udf/dludf.cgi?ckey=28 ll tar -xzvf dludf.cgi?ckey=28 gcc -shared -o libmysqllevenshtein.so mysqllevenshtein.cc -I/usr/include/mysql/ mv libmysqllevenshtein.so /usr/lib mysql -uroot -pPASS mysql> use DATABASE mysql> CREATE FUNCTION levenshtein RETURNS INT SONAME 'libmysqllevenshtein.so' mysql> choose levenshtein(w1.word,w2.word) as dist from word w1, word w2 where ETC........... order by dist asc limit ,10
I would recommend that you simply generate typo versions around the query input.
i.e. hyperpext > etc
One of these simple is certain to function as the correct spelling (specifically for common misspellings)
How you identify probably the most likely match would be to perform a research for every with an index which informs the document frequency from the term. (seem sensible?)