Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _guidance/preparing-replication-package-details.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ date: 2026-01-07
This document describes how to prepare your code for verification in detail, taking into account some of the most frequent issues that the Data Editor and his team have encountered in submitted replication packages.


> ⚠️❗ **IMPORTANT:** At this point, you should only be seeing this page if you were asked by the Data Editor team to do so, and if your replication package relies on a single software. Admissible containers are listed in the [Step 5 section: authorized containers](#authorized-containers). We are not currently attempting to generalize this to multi-software replication packages, though [it](https://github.com/AEADataEditor/docker-r-gurobi) [is](https://github.com/AEADataEditor/docker-aer-2022-0276) [possible](https://github.com/AEADataEditor/docker-aer-2023-0505) [to do so](https://github.com/AEADataEditor/docker-aer-2023-0700).
> ⚠️❗ **IMPORTANT:** At this point, you should only be seeing this page if you were asked by the Data Editor team to do so, and if your replication package relies on a single software, or a **small number of single-software steps**. Admissible software are listed in the [Step 5 section: authorized containers](preparing-replication-package-step5#authorized-containers). We are not currently attempting to generalize this to arbitrary multi-software replication packages, though [it](https://github.com/AEADataEditor/docker-r-gurobi) [is](https://github.com/AEADataEditor/docker-aer-2022-0276) [possible](https://github.com/AEADataEditor/docker-aer-2023-0505) [to do so](https://github.com/AEADataEditor/docker-aer-2023-0700).



Expand Down
6 changes: 3 additions & 3 deletions _guidance/preparing-replication-package-finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 5](preparing-replication-package-step5)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 5](preparing-replication-package-step5) | [Next: Submitting ▶](preparing-replication-package-submit)

### Finalize README

Expand Down Expand Up @@ -36,7 +36,7 @@ Code ran for about 35 hours.
Code runs about 10 minutes for Stata portion, and about 5 days for MATLAB portion.
```

> NOTE about SIVACOR here: You should not modify your README after your last run on SIVACOR, as the TRO generated by SIVACOR contains a checksum that would highlight the modification to the README. However, the auxiliary files generated by SIVACOR (the `.jsonld` file) contain all the necessary information.

### Submitting

You can now submit your replication package to the Data Editor, along with the completed checklist from above, and the generated `main.log`/`main.Rout` as evidence.
[▶ Next: Submitting](preparing-replication-package-submit)
6 changes: 5 additions & 1 deletion _guidance/preparing-replication-package-step1.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,18 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [Next: Step 2 ▶](preparing-replication-package-step2)

> You may or may not have a main file. The following should be adapted to your circumstances. You do not need to create a file that is called `main.do` if you already have one, but you may need to update your existing main file.

> Reference: <https://larsvilhuber.github.io/self-checking-reproducibility/02-hands_off_running.html>

Creating a single main file is straightforward. However, you will want to make some minor edits depending on where, in the above template setup, the file is located:

> The adjustments described on this page also apply to a **small number (2-4) of single-software main files** that are run in sequence.
>
> If using SIVACOR in [Step 5](preparing-replication-package-step5): SIVACOR allows for multi-step software flows, where a **small** number of single-software containers are run in sequence, with the output of one container being used as the input to the next. See [SIVACOR documentation](https://docs.sivacor.org/docs/step0-prepare/#your-replication-package-only-uses-a-single-software-application-per-step). Each of the main files should be structured as outlined on this page.

## Scenario A: `main` is in the `code` directory

The most frequent scenario we see (which we call **Scenario A**) amongst economists is that the main file is in the `code` directory:
Expand Down
2 changes: 1 addition & 1 deletion _guidance/preparing-replication-package-step2.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 1](preparing-replication-package-step1)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 1](preparing-replication-package-step1) | [Next: Step 3 ▶](preparing-replication-package-step3)

Two issues:

Expand Down
2 changes: 1 addition & 1 deletion _guidance/preparing-replication-package-step3.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 2](preparing-replication-package-step2)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 2](preparing-replication-package-step2) | [Next: Step 4 ▶](preparing-replication-package-step4)

## Stata packages

Expand Down
2 changes: 1 addition & 1 deletion _guidance/preparing-replication-package-step4.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 3](preparing-replication-package-step3)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 3](preparing-replication-package-step3) | [Next: Step 5 ▶](preparing-replication-package-step5)

Displays (figures and tables) should be written out to external files, and the authors' versions, as used in the manuscript, should be provided. In the prototypical replication package structure above, these files would be in the `results` directory.

Expand Down
18 changes: 10 additions & 8 deletions _guidance/preparing-replication-package-step5.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,39 +4,41 @@ toc: true
date: 2026-02-16
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 4](preparing-replication-package-step4)
[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Step 4](preparing-replication-package-step4) | [Next: Finalize README ▶](preparing-replication-package-finalize)

After you have made all the changes, you should test your code. To make this simple, we have set up a public website that hides the complexity of running containers from you. You only need to choose the software, the system will run the properly configured code automatically.

## Using the SIVACOR website

We have developed the [SIVACOR](https://sivacor.org) service, which allows you to run your code using authorized containers without the need to install software on your own computer, producing a Trusted Research Object (TRO).

> In fact, we will run your code using this same system to verify compliance with all of the above steps!
> If successful, your last run on SIVACOR will be the package you [submit](preparing-replication-package-submit#sivacor) to the Data Editor!

[![SIVACOR landing page](/images/sivacor-login.png)](https://sivacor.org)

For more information on how to use SIVACOR, see <https://docs.sivacor.org/>. Once you have successfully run your code on SIVACOR, provide the generated certified ZIP file instead of the original replication package to the Data Editor (via import to your openICPSR draft deposit). A TRO does not need to be re-run by the Data Editor.
For more information on how to use SIVACOR, see <https://docs.sivacor.org/>. Once you have successfully run your code on SIVACOR, you can proceed directly to [Submitting](preparing-replication-package-submit#sivacor). A TRO does not need to be re-run by the Data Editor.


## Authorized containers

SIVACOR uses a curated list of containers, chosen because they are reliably available, and achieve the desired transparency. You can inspect the most current list at <https://docs.sivacor.org/docs/images/>. In general, Stata, R, and MATLAB (with Dynare) are supported.
SIVACOR uses a curated list of containers, chosen because they are reliably available, and achieve the desired transparency. You can inspect the most current list at <https://docs.sivacor.org/docs/images/>. At the moment, Stata, R, and MATLAB (with Dynare) are supported.

> If you know of a different container that we should add to this list, please let us know. The [AEA Data Editor's Github profile](https://github.com/AEADataEditor/) has a few other containers that have worked. However, we do not allow for arbitrary (user-created) containers on SIVACOR, though if you think that they are useful for your replication package, please reach out.

> SIVACOR allows for multi-step software flows, where a **small** number of single-software containers are run in sequence, with the output of one container being used as the input to the next. See [SIVACOR documentation](https://docs.sivacor.org/docs/step0-prepare/#your-replication-package-only-uses-a-single-software-application-per-step).

If you know of a different container that we should add to this list, please let us know. The [AEA Data Editor's Github profile](https://github.com/AEADataEditor/) has a few other containers that have worked. However, we do not allow for arbitrary (user-created) containers on SIVACOR, though if you think that they are useful for your replication package, please reach out.

## A note about multi-software workflows

We are not currently attempting to generalize this to all multi-software replication packages. Building multi-software containers [definitely](https://github.com/AEADataEditor/docker-r-gurobi) [is](https://github.com/AEADataEditor/docker-aer-2022-0276) [possible](https://github.com/AEADataEditor/docker-aer-2023-0505) [with some effort](https://github.com/AEADataEditor/docker-aer-2023-0700).

However, SIVACOR does allow for multi-step software flows, where a **small** number of single-software containers are run in sequence, with the output of one container being used as the input to the next. See [SIVACOR documentation](https://docs.sivacor.org/docs/step0-prepare/#your-replication-package-only-uses-a-single-software-application-per-step).


## Testing using Docker locally (advanced)

If SIVACOR does not work for you, you can either attempt to run it in Docker on your own computer, or skip this step entirely and revert back to the standard (manual) verification process. Installing and running Docker on your computer is straightforward (undergraduate students in the AEA Data Editor team have done this in under half an hour), but may not meet everybody's needs.
If SIVACOR does not work for you, you can either attempt to run it in Docker on your own computer, or skip this step entirely and revert back to the standard (manual) verification process. Installing and running Docker on your computer is straightforward (undergraduate students in the AEA Data Editor team have done this in under half an hour), but may not meet everybody's needs, or your institution's IT policies.

> ⚠️❗ **IMPORTANT:** If you do not have Docker installed on your computer, do not have the rights to install Docker on your computer, or do not have access otherwise to Docker, please do not attempt this, and skip straight [to the alternative approach](#alternative-approach).
> ⚠️❗ **IMPORTANT:** If you do not have Docker installed on your computer, do not have the rights to install Docker on your computer, or do not have access otherwise to Docker, please do not attempt this, and skip straight [to the alternative approach](#fallback-run-on-a-different-computer).

> ⚠️❗ **IMPORTANT:** Do not provide us with a custom container that is not on the above list. Transparency requires that the container be built, using a `Dockerfile` or `apptainer.def` file, from publicly available sources. While we will happily use your container, it must be built from one of the above sources, or well-known "standard" sources, such as "Docker Official Images" in the Dockerhub `library` space (e.g., <https://hub.docker.com/_/python>).

Expand Down
36 changes: 36 additions & 0 deletions _guidance/preparing-replication-package-submit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
title: "Submitting"
toc: true
date: 2026-08-10
---

[◀ Back to Checklist](preparing-replication-package) | [Back to Details](preparing-replication-package-details) | [◀ Previous: Finalize README](preparing-replication-package-finalize)

You can now submit your replication package to the Data Editor.

In all cases, the [**completed checklist**](preparing-replication-package#checklist) should be sent directly to the Data Editor, using `reply-all` to the email that you received from the Data Editor.

## If you ran using SIVACOR {#sivacor}

From the [output from SIVACOR](https://docs.sivacor.org/docs/step4-download/), you should

- use the `Replicated Package` ZIP file, and **replace the entire content** of your draft openICPSR deposit.
- not remove any files from the ZIP file.
- upload the ZIP file using the ["Import"](data-deposit-aea#importing-zip-files) function in your draft openICPSR deposit.


The ZIP file contains your entire upload (minus any explicitly deleted files!), plus any generated output and log files, as well as the **certificate** which documents that you ran this replication package through SIVACOR.

You do not need to email any of the log files or output to the Data Editor separately.

## Submitting other checked code

In addition to the checklist, update the journal deposit with any updated code, data, and README. Ideally, you should also include and document the outputs and log files generated by your reproducible run. In the interest of transparency, these will be published as part of the replication package, without further edits.

However, if you prefer, you can submit the generated logfiles as evidence by email.

## Mark the deposit as complete

In all cases, you should mark the openICPSR deposit as **complete** by [re-submitting it](faq#i-was-asked-to-modify-files-in-my-repository-not-yet-published-but-i-cannot-upload-or-edit-anything).

## You are done.
Loading
Loading