Repeatability and Submission Guidelines
Repeatability in cyber-physical systems research is critical. Therefore, this year we are strongly encouraging authors of accepted regular papers to submit a repeatability package. The program committee will evaluate this package. Accepted repeatable papers will also be highlighted on the conference website. The submission date will be approximately 2 weeks after the notifications for the papers are sent out.
The Repeatability Evaluation Package (REP) consists of several components:
- A copy (in pdf format) of the accepted paper with an appendix that explains the following:
- What elements of the paper are included in the REP (e.g., figures, tables, etc.).
- Instructions for installing the software.
- Instructions for running the software. Ideally, there is a short and easy to run script for each computational component in the paper.
- The system requirements for running the REP (e.g.: OS, compilers, environments, etc.). The document should also include a description of the host platform used to prepare and test the docker image or virtual machine.
- Expected resource requirements. What architecture did you use for your experiments and how long is your code expected to run?
- The software.
-
Host Your Source Code on GitHub
- Create a Public Repository:
- Use GitHub to host your source code in a public repository. If you do not already have a GitHub account, sign up at GitHub.
- Ensure Completeness:
- Include all necessary files for reproducing your results, such as scripts, configurations, and README documentation with clear instructions for building and running your code.
-
Archive Your Repository Using Zenodo
- Connect GitHub to Zenodo:
- Sign up or log in to Zenodo.
- Link your GitHub account to Zenodo by enabling GitHub integration in your Zenodo account settings. This allows Zenodo to archive specific GitHub releases.
- Follow Zenodoโs instructions here to set up the integration.
- Create and archive a GitHub Release:
- After creating a release on GitHub, Zenodo will automatically archive it.
- Zenodo assigns a Digital Object Identifier (DOI) to your archived release, ensuring its long-term availability and citability.
- Provide the DOI:
- Copy the DOI generated by Zenodo for your release and include it in your artifact submission.
-
Host Docker Images on Docker Hub
- Create a Docker Hub Repository:
- If your experiments use Docker, host the Docker images publicly on Docker Hub. Sign up if you do not have an account.
- Build and Push Images:
- Build Docker images for your artifact.
- Push the images to your Docker Hub repository using the following commands:
docker tag <local-image> <dockerhub-username>/<repository-name>:<version-tag>
docker push <dockerhub-username>/<repository-name>:<version-tag>
- Version Tags:
- Tag images with version numbers or descriptive identifiers (e.g., v1.0, baseline, iccps).
- Provide the Docker Hub URL:
- Include the public link to your Docker Hub repository (e.g., https://hub.docker.com/r/<dockerhub-username>/<repository-name>) and specify the tags in your submission.
-
Additional Notes:
-
Licensing: Use an open-source license (e.g., MIT, GPL) for your repository and Docker images to ensure their usability by the research community.
-
Testing: Ensure your Docker images and GitHub repository are functional and reproduce the results claimed in your paper.
-
Support: For questions or assistance, contact the ICCPS artifact evaluation chairs at [ivanor@rpi.edu, etuncali@asu.edu].
- Any data used in your experiments. Please use https://figshare.com/ย to share the data used in your experiments.
The REP should be made available via a link. To be accepted, each REP needs to achieve satisfactory performance along the following three dimensions:
-
Coverage. How many of the computational components in the paper can be reproduced?
-
Documentation quality. Is the REP clearly documented, including system requirements, resource requirements, installation instructions and execution instructions?
-
Ease of reuse. Is the REP generally easy to reuse, i.e., is the code well documented, are the scripts clear and easy to run?
Submission Page
The submission page can be found here.
Important Dates
- Artifact Submission Deadline: February 7, 2025
- Smoke Test: February 14, 2025
- Author Response Deadline: February 21, 2025
- Decisions Announced: March 6, 2025 (tentative)