Test period begins for CLEF 2015 Lab

Today, we are officially entering into the test phase of the LL4IR CLEF 2015 Lab.

There are some changes, detailed on this page, below is an executive summary:

  • Only 2 of the 3 use-cases will be evaluated: product search and web search (local-domain search will not run this year). These two use-cases are fully operational; see this page for updated technical details on the product search use-case.
  • The test period will run from May 1 to 15. Rankings for test queries may be uploaded (and changed) until Apr 30, 23:59 CET.
  • Training queries will continue to run during the test phase and feedback will be made available for them; however, it will not be possible to upload new rankings (either for training or for test queries) during the test period.

If you are planning on submitting a run, please let us know by sending us a short message.

CLEF Lab is underway

There has been a silence on the site over the last few weeks (months), but things have been progressing. The web search use case is operational and running in full swing for some weeks now. We will have more updates on the other two use cases very soon.

You can still sign up for CLEF until April 30. Note that the deadline for submitting official runs is April 15.
We have also prepared a guide specifically for CLEF participants; this should help you to get started right away.

Living Labs runs as a CLEF Lab in 2015

We are happy to announce that Living Labs for IR Evaluation (LL4IR) will run as a Lab at CLEF 2015. The first edition of the lab will focus on three specific use-cases: (1) product search (on an e-commerce site), (2) local domain search (on a university’s website), (3) web search (through a commercial web search engine). See futher details here.

“Give us your ranking, we’ll have it clicked!”