Please read all of these instructions carefully when preparing your submissions.

We invite submissions describing novel formats for publishing scientific research, with an emphasis on presentation, accessibility, and reproducibility. Each submission must clearly outline its design goals and discuss the benefits it holds over current publishing formats (e.g., LaTeX + PDF workflows), as well as the challenges involved in widespread adoption of the proposed format changes.

We expect three main types of submissions:

  • Exhibits that present a novel format for scientific research dissemination. More details on exhibits below.
  • Workflows that describe current best practices for scientific presentation. Workflow submissions can be in multiple forms: open-source software, HTML pages, PDFs, or more.
  • Position papers that summarize existing tools for scientific dissemination and discuss viable alternatives. The broad range of topics include, but are not limited to, publication formats and metadata collection practices for reproducible research, code submission/release guidelines, integration of publications with peer-review comments and conference discussion channels, or the impact of social media in modern ML research.

However, the workshop generally is open-ended, and innovative ideas outside of these themes are welcome too. Please reach out to the organizers if in doubt.

Position paper guidelines

Position papers are solicited in PDF format, and must follow the ICLR template. Papers can be up to 8 pages in length, with an unrestricted amount of space allocated for references and appendices. Reviewers will not be required to read the appendices.

Position paper submissions will be double-blind and will need to be appropriately anonymized (please see ICLR guidelines on preparing manuscripts for double-blind reviewing).

Exhibit and workflow submission guidelines

Exhibit and workflow submissions are solicited in any format that allows for review. It could be:

  • Interactive documents, such as Distill.pub, CodaLab articles, code notebooks;
  • Static document(s): PDFs, comics, etc.;
  • Innovative visualization tools;
  • Code repositories;
  • Multimedia formats (videos, audio papers etc., embedded in a web page);
  • Blogs or other formats on content sharing platforms that enable comments/ratings/recommendations;
  • Solutions for collaborative reading/discussion like hypothes.is;
  • Research journalism: newsletters, livetweeting, podcasts, etc.

Due to the difficulty of ensuring peer review anonymity with such new formats, exhibit and workflow submissions will be reviewed in a single-blind process.

The contents of exhibit and workflow submissions can be any existing research project, or simply a description of the workflow or exhibit feature set itself (akin to how LaTeX templates for a venue often end up describing format regulations or features of the template). If the contents of your submission are those of an existing research project, we expect a supplementary section or a separate document introducing the proposed format, discussing its benefits, intended use, and potential limitations and obstacles to wider adoption.

In particular, for exhibits we encourage the authors to reflect on the following:

  • Presentation: How does the proposed new format improve clarity, content delivery, and interactivity? Is the primary use case an asynchronous research presentation, or could it also be used in virtual conferences or education?
  • Review cycle: In what manner is the proposed format best reviewed? What would version control look like, for such a format? What technical skills does this require from a reviewer? Would it be possible to ensure a fully anonymous peer review process?
  • Bibliometrics: Are there more meaningful bibliometrics to measure the impact of articles in the proposed format? If so, how would they integrate into the existing rubric of h-indices, impact factors, and citation counts? How easy would it be to index and cite content in the new format?
  • Interoperability: Can these formats be converted back or from others, such as PDFs, blog posts, etc?
  • Durability: Will these new formats remain accessible in case of server failure or bitrot? How can we ensure that submissions remain accessible for many decades to come?
  • Limitations: Are there foreseeable issues with the proposed format that static documents do not have, that could stand in the way of its wider adoption? How could they be solved?

Adversarial perspectives are also welcome! Submissions arguing in favor of current methods (such as LaTeX + PDF workflows) will be equally well-received.

Accessibility statement

Each submission is required to contain an accessibility statement discussing how the proposed scheme improves accessibility, particularly to individuals with disabilities. This statement, expected to be about 1-2 paragraphs, must

  • Commit to improving accessibility,
  • Establish intended accessibility levels,
  • List the steps taken to improve accessibility concerns,
  • Discuss known exceptions to intended accessibility levels,
  • Comment on how feedback will be gathered and used for improving accessibility.

There are multiple great resources on writing accessibility statements on the Internet, like this one from Nomensa or this one from opentextbc. An example accessibility statement from Nature Journals can be found here.

Submission portal

Our submission portal can be accessed at: https://openreview.net/group?id=ICLR.cc/2021/Workshop/Rethinking_ML_Papers

Important dates

  1. Submissions due for the position paper track (double-blind): 9 March 2021 16 March 2021(2359 hrs, Anywhere-on-Earth)
  2. Submissions due for the exhibit/workflow track (single-blind): 9 March 2021 16 March 2021 (2359 hrs, Anywhere-on-Earth)

Review criteria

In an effort to improve transparency, and to aid in preparing your submission, we have uploaded our review criteria here.