[alife] [GECCO 2017] [Workshop CFP] Deadline: March 27, 2017 - New Standards for Benchmarking in Evolutionary Computation Research

[Randy Olson]

** Apologies for multiple postings **


*************************************************************************
                                  CALL FOR PAPERS
New Standards for Benchmarking in Evolutionary Computation Research
                         July 15-19, 2017, Berlin, Germany
                             http://gecco-2017.sigevo.org/
*************************************************************************

Workshop URL: https://epistasislab.github.io/gecco2017-new-benchmarking-st
andards/

Workshop Description:

Benchmarks are one of the primary tools that machine learning researchers
use to demonstrate the strengths and weaknesses of an algorithm, and to
compare new algorithms to existing ones on a common ground. However,
numerous researchers---including prominent researchers in the evolutionary
computation field---have raised concerns that the current benchmarking
practices in machine learning are insufficient: most commonly-used
benchmarks are too small, lack the complexity of real-world problems, or
are easily solved by basic machine learning algorithms. As such, we need to
establish new standards for benchmarking in evolutionary computation
research so we can objectively compare novel algorithms and fully
demonstrate where they excel and where they can be improved.

This workshop will host speakers from around the world who will propose new
standards for benchmarking evolutionary computation algorithms. These talks
will focus on (i) characterizing current benchmarking methods to better
understand what properties of an algorithm are tested via a benchmark
comparison, and (ii) proposing improvements to benchmarking standards, for
example via new benchmarks that fill gaps in current benchmarking suites or
via better experimental methods. At the end of the workshop, we will host a
panel discussion to review the merits of the proposed benchmarking
standards and how we can integrate them into existing benchmarking
workflows.

Call for Papers:

The focus of this workshop is to highlight promising new standards for
benchmarking practices in evolutionary computation research. As such, we
are soliciting papers on topics that could include but are not limited to:

* Examining the merits or issues regarding benchmarking practices.
* Development or expansion of benchmark data archives or tools.
* The importance of simulated vs. real-world benchmarks.
* Analysis of, or comparison between established benchmarks.
* Targeting benchmarks to different domains.

Important Dates:

March 27, 2017 - Workshop paper submission deadline
April 10, 2017 - Notification of acceptance
April 24, 2017 - Camera-ready deadline:
July 15-19, 2017, GECCO 2017 Conference in Berlin, Germany

Paper Submission:

Submitted papers must not exceed 8 pages and are required to be in
compliance with the GECCO 2017 Call for Papers Preparation Instructions.
However, note that the review process of the workshop is not double-blind,
so authors’ information should be included in the paper.

To submit a paper to the workshop, e-mail your paper to olsonran at upenn.edu.

All accepted papers will be presented at the workshop and appear in the
GECCO Conference Companion Proceedings.

Organizing Committee:

Dr. William La Cava – lacava at upenn.edu
Dr, Randal Olson – olsonran at upenn.edu
Dr. Patryk Orzechowski – patryk at upenn.edu
Dr. Ryan Urbanowicz – ryanurb at upenn.edu






Randal S. Olson, Ph.D.
Senior Data Scientist, Institute for Biomedical Informatics
University of Pennsylvania

E-mail: rso at randalolson.com | Twitter: @randal_olson
<https://twitter.com/randal_olson>
http://www.randalolson.com