lucene - How does (carrot) clustering work in solr? -


i have running lucene/solr 4 testing different features, "clustering". currently, 1 million documents indexed. every document has following fields:

id (unique key) example1: 10245                example2: 24974 topic (keywords of document) example1: "disaster/japan/nuclear power station"                                  example2: "world/japan/nuclear power" headline (1 line of text): example1: "explosion @ nuclear power plant in japan"                            example2: "news japans nuclear power plant" text (the full text): "in japanese nuclear power plant in fukushima..."

all fields indexed , stored, exapt text, indexed, not stored. use following specific configuration:

  <str name="carrot.title">topic</str>    <str name="carrot.snippet">headline</str>

if looking example see, topic different, japan same. possible configure solr/carrot in way, example1 , example2 in 1 cluster? because of matching "japan"?!

further there 3rd topic "news/nuclear power", no "japan" inside headline , text using words: japans power plant. solr/carrot configuration relevant in order receive 3 news in 1 cluster?

thank you!

carrot2 designed cluster natural / unstructured text , such algorithms produce results human find perfect. unfortunately, such algorithms hard "debug" -- clusters produce depend on many factors, such frequencies words occur in documents. in specific example, word japan may not have been chosen form cluster because it's frequent -- appears in of documents quoted.

here few tips may want try tweak clusters:

  • try separating keywords period followed space rather slash, e.g. "disaster. japan. nuclear power station". if that, carrot2 treat word sequences, such "nuclear power station", phrases rather individual words.

  • try different carrot2 clustering algorithm, e.g. stc.

  • if there chance full story text field stored (or maybe part of it, such first paragraph), use headline carrot.title , full text / excerpt carrot.snippet.

  • play specific settings of carrot2 algorithms. best tool carrot2 clustering workbench. here's how connect solr: http://wiki.apache.org/solr/clusteringcomponent#tuning_carrot2_clustering


Comments

Post a Comment

Popular posts from this blog

c# - SharpSVN - How to get the previous revision? -

i trying find efficient way previous revision of file text comparison using sharpsvn. using (svnclient c = new svnclient()) { c.authentication.defaultcredentials = new networkcredential( configurationmanager.appsettings.get("svnserviceusername") , configurationmanager.appsettings.get("svnservicepassword") , configurationmanager.appsettings.get("svnservicedomain") ); c.authentication.sslservertrusthandlers += new eventhandler<svnsslservertrusteventargs>(authentication_sslservertrusthandlers); collection<svnfileversioneventargs> fileversioncollection = new collection<svnfileversioneventargs>(); svnrevisionrange range = new svnrevisionrange(0, this.hooks.revision); svnfileversionsargs args = new svnfileversionsargs(); args.retrieveproperties = true; args.range = range; foreach (svnchangeitem item in log.changedpaths) { string path = this.repositorypath + it...
Read more

c++ - Is it possible to compile a VST on linux? -

for class project i'm attempting write vst plugin backed cuda. current cuda workflow on linux box, i'd prefer compile , link there. according wikipedia , should possible (i couldn't find steinberg documentation relevant linux) can't find makefile or instructions on how build if aren't using xcode or visual studio. i'm vst 3 sdk doesn't support linux. when try compile plugin under linux, error: ./base/source/fatomic.cpp:39:30: fatal error: libkern/osatomic.h: no such file or directory this issue caused following code in "vst3 sdk/base/source/fatomic.cpp" #if mac #include <libkern/osatomic.h> #if mac_os_x_version_min_required > mac_os_x_version_10_4 #define native_atomic_type (volatile int32_t*) #else #define native_atomic_type (int32_t*) #endif #elif windows #include <windows.h> #endif but hope compiling under linux work vst sdk 2.4. reading. jvstwrapper seems run on lin...
Read more

url - Querystring manipulation of email Address in PHP -

i using third party application (email2http) send me emailid, body , message. unable retrieve email id due name appended on , enclosed < , >. here querystring getting. test.php?from=ajay+reddy &subject=cx&body=somebody when print variable in php returns name , not email id. $from = $_get['from']; echo $from; //output ajay reddy please suggest me how can emailid also. i dont want use post method due testing , debugging. it's working fine. forgot use htmlentities() in debugging.
Read more