MySQL Forums
Forum List  »  Full-Text Search

Adding stemming to fulltext TF-IDF score queries
Posted by: Jason Wisdom
Date: April 18, 2015 05:51PM

Hi everybody,

(originally posted this in the UDF section for some strange reason; apologies to the management of these fine forums)

I am building a text analytics engine, using scoring through TF-IDF method. Simple as this:


SET @keyword='+deficiencies';

SELECT documentid, documentname, ROUND(MATCH(documenttext) AGAINST (CONCAT('-no ', @keyword) IN BOOLEAN MODE), 7) AS score FROM documenttable;

(fulltext index on longtext column 'documenttext')


What I WANT to do is include words like "deficient" "deficiency" etc. into my search. In other words, stemming.

MATCH() sucks with SOUNDEX(), they don't work together at all.
But how can I include stemming into my fulltext scoring statement?

Options: ReplyQuote


Subject
Views
Written By
Posted
Adding stemming to fulltext TF-IDF score queries
3482
April 18, 2015 05:51PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.