Skip to content

Latest commit

 

History

History
164 lines (135 loc) · 13.2 KB

README.md

File metadata and controls

164 lines (135 loc) · 13.2 KB
Raw

Anserini

Build Status codecov Generic badge Maven Central PyPI LICENSE doi

Anserini is an open-source information retrieval toolkit built on Lucene that aims to bridge the gap between academic information retrieval research and the practice of building real-world search applications. Among other goals, our effort aims to be the opposite of this. Anserini grew out of a reproducibility study of various open-source retrieval engines in 2016 (Lin et al., ECIR 2016). See Yang et al. (SIGIR 2017) and Yang et al. (JDIQ 2018) for overviews.

Getting Started

A low-effort way to try out Anserini is to look at our online notebooks, which will allow you to get started with just a few clicks. For convenience, we've pre-built a few common indexes, available to download here.

If you want to build Anserini itself, then start by verifying the main dependencies:

  • Anserini was upgraded to Java 11 at commit 17b702d (7/11/2019) from Java 8. Maven 3.3+ is also required.
  • Anserini was upgraded to Lucene 8.0 as of commit 75e36f9 (6/12/2019); prior to that, the toolkit uses Lucene 7.6. Based on preliminary experiments, query evaluation latency has been much improved in Lucene 8. As a result of this upgrade, results of all regressions have changed slightly. To replicate old results from Lucene 7.6, use v0.5.1.

After cloning our repo, build using Maven:

mvn clean package appassembler:assemble

The eval/ directory contains evaluation tools and scripts, including trec_eval, gdeval.pl, ndeval. Before using trec_eval, unpack and compile it, as follows:

tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4 && make

Before using ndeval, compile it as follows:

cd ndeval && make

Anserini Regression Experiments

Anserini is designed to support experiments on various standard IR test collections out of the box. The following experiments are backed by rigorous end-to-end regression tests with run_regression.py and the Anserini replicability promise. For the most part, these runs are based on default parameter settings.

Additional Documentation

The experiments described below are not associated with rigorous end-to-end regression testing and thus provide a lower standard of replicability. For the most part, manual copying and pasting of commands into a shell is required to replicate our results:

See this page for additional documentation.

Other Features

How Can I Contribute?

If you've found Anserini to be helpful, we have a simple request for you to contribute back. In the course of replicating baseline results on standard test collections, please let us know if you're successful by sending us a pull request with a simple note, like what appears at the bottom of the Robust04 page. Replicability is important to us, and we'd like to know about successes as well as failures. Since the regression documentation is auto-generated, pull requests should be sent against the raw templates. In turn, you'll be recognized as a contributor.

Beyond that, there are always open issues we would appreciate help on!

Release History

References

Acknowledgments

This research is supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada. Previous support came from the U.S. National Science Foundation under IIS-1423002 and CNS-1405688. Any opinions, findings, and conclusions or recommendations expressed do not necessarily reflect the views of the sponsors.