Heliosearch – The Next Solr
Project Heliosearch is about creating the next major evolution of Solr. To kick things off, I’ve started a project on github and implemented “off-heap” filters, as a first step toward taking performance to the next level.
For a number of reasons, I felt it best to incubate this project at github, where it could have a community dedicated solely to it’s advancement. The longer term plan is to bring it back to the ASF once it has gained enough traction.
JVMs have never been particularly good at dealing with large heaps. The larger the heap, the more resources the JVM wastes copying memory around that could otherwise be spent doing more useful work. Larger heaps also correlate with larger stop-the-world garbage collection pauses.
Update:The first heliosearch release is out, and the first tests of off-heap filters performance have been very encouraging.
Native Code Optimizations
In addition to moving more large data structures off-heap(like
UnInvertedField), I am planning to implement native code
optimizations for certain hotspots. Native code faceting would be an
obvious first choice since it can often be a CPU bottleneck.