Ajshen: /* Sending new results to the paper */

2026-10-18T23:31:43Z

Sending new results to the paper

← Older revision		Revision as of 23:31, 18 October 2026
Line 181:		Line 181:
	That's it. Refresh the Overleaf editor and recompile, and the new figures are there.		That's it. Refresh the Overleaf editor and recompile, and the new figures are there.

	~~<!-- SCREENSHOT: Overleaf file tree / editor right after a push, showing the updated figure (optionally with the History panel open). -->~~
	~~[[File:Overleaf-after-git-push.png\|thumb\|center\|600px\|Pushed figures appear in Overleaf immediately.]]~~

	<div style="background-color: #e8f4f8; border-left: 5px solid #17a2b8; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">		<div style="background-color: #e8f4f8; border-left: 5px solid #17a2b8; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">

Ajshen: /* Step 3: Clone the paper inside your experiment repository */

2026-10-18T23:30:44Z

Step 3: Clone the paper inside your experiment repository

← Older revision		Revision as of 23:30, 18 October 2026
Line 97:		Line 97:
	</syntaxhighlight>		</syntaxhighlight>

	~~<!-- SCREENSHOT: Terminal showing both `git status` outputs (outer repo clean, inner repo on master). -->~~
	~~[[File:Nested-clone-git-status.png\|thumb\|center\|600px\|The outer repository ignores <code>paper/</code>; the inner repository is independent.]]~~

	<div style="background-color: #f8d7da; border-left: 5px solid #721c24; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">		<div style="background-color: #f8d7da; border-left: 5px solid #721c24; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">

Ajshen: /* Step 2: Find your project's clone URL */

2026-10-18T23:29:58Z

Step 2: Find your project's clone URL

← Older revision		Revision as of 23:29, 18 October 2026
Line 75:		Line 75:
	<!-- SCREENSHOT: Overleaf project Menu, with the "Sync → Git" entry highlighted. -->		<!-- SCREENSHOT: Overleaf project Menu, with the "Sync → Git" entry highlighted. -->
	[[File:Overleaf-menu-sync-git.png\|thumb\|center\|300px\|The Git entry in the Overleaf project menu.]]		[[File:Overleaf-menu-sync-git.png\|thumb\|center\|300px\|The Git entry in the Overleaf project menu.]]

	~~<!-- SCREENSHOT: The Git dialog showing the clone command (blur or crop the project ID if you prefer). -->~~
	~~[[File:Overleaf-git-clone-url.png\|thumb\|center\|500px\|The project's Git clone URL.]]~~

	=== Step 3: Clone the paper inside your experiment repository ===		=== Step 3: Clone the paper inside your experiment repository ===

Ajshen: Ajshen moved page Overleaf GitHub Integration to Overleaf Git Integration

2026-10-18T23:27:23Z

Ajshen moved page Overleaf GitHub Integration to Overleaf Git Integration

← Older revision		Revision as of 23:27, 18 October 2026
(No difference)

Ajshen at 23:24, 18 October 2026

2026-10-18T23:24:07Z

Show changes

Ajshen: /* Step 1: Link Overleaf to GitHub */

2026-10-18T23:23:52Z

Step 1: Link Overleaf to GitHub

← Older revision		Revision as of 23:23, 18 October 2026
Line 62:		Line 62:

	=== Step 1: Link Overleaf to GitHub ===		=== Step 1: Link Overleaf to GitHub ===
	In Overleaf, open '''Account Settings''' and find the ~~GitHub~~ entry under the project synchronization section. Click '''Link''' and ~~authorize Overleaf~~ in the ~~GitHub dialog~~ that pops up.		In Overleaf, open '''Account Settings''' and find the Git entry under the project synchronization section. Click '''Link''' and set up access in the dialogue that pops up.

	<!-- SCREENSHOT: Overleaf Account Settings page with the GitHub "Link" button visible. -->		<!-- SCREENSHOT: Overleaf Account Settings page with the GitHub "Link" button visible. -->

Ajshen: Created page with " XX If your experiment code lives in a GitHub repository and your paper lives on Overleaf, you have probably done '''the upload dance''': rerun an experiment, hunt down the regenerated figure on disk, open Overleaf, delete the old file, upload the new one, and repeat for every figure and table that changed. It is slow, it is easy to le..."

2026-10-18T22:51:47Z

Created page with " XX If your experiment code lives in a GitHub repository and your paper lives on Overleaf, you have probably done '''the upload dance''': rerun an experiment, hunt down the regenerated figure on disk, open Overleaf, delete the old file, upload the new one, and repeat for every figure and table that changed. It is slow, it is easy to le..."

New page

[[Category:Beginner's_Guide_Implementation|XX]]

If your experiment code lives in a GitHub repository and your paper lives on Overleaf, you have probably done '''the upload dance''': rerun an experiment, hunt down the regenerated figure on disk, open Overleaf, delete the old file, upload the new one, and repeat for every figure and table that changed. It is slow, it is easy to leave a stale figure behind, and it leaves no record of which version of your code produced which figure in the paper.

Overleaf Premium's '''GitHub Synchronization''' feature removes most of this busywork. This article describes one particular way to use it: cloning the paper's repository '''inside''' your experiment repository, in a directory that the experiment repository ignores. Your analysis scripts then write figures and tables straight into the paper, and a single push sends them to Overleaf.

<div style="background-color: #e8f4f8; border-left: 5px solid #17a2b8; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">
'''Important: Overleaf Premium is required'''<br>
GitHub Synchronization is a paid Overleaf feature. Check whether you already have access through an institutional license before paying for it yourself. If you don't use Git yet, read up on the basics first, since this workflow assumes you are comfortable with <code>clone</code>, <code>commit</code>, <code>push</code>, and <code>pull</code>.
</div>

== How It Works ==
There are several ways to connect a code repository to an Overleaf project. In this one, the two repositories stay '''completely independent'''. The paper repository is cloned into a subdirectory (here called <code>paper/</code>), and that subdirectory is listed in the experiment repository's <code>.gitignore</code>. The experiment repository never records anything about the paper, and the paper repository never sees your experimental code. Your scripts simply write files into a folder that happens to be a different Git repository.

The final layout looks like this:

<pre>
my-experiments/ ← experiment repo (github.com/you/my-experiments)
├── .gitignore ← contains "paper/"
├── src/
├── scripts/
│ └── make_figures.py ← writes into paper/figures and paper/tables
├── Makefile
└── paper/ ← SEPARATE git repo, ignored by the outer repo
├── .git/ ← points to github.com/you/my-paper
├── main.tex
├── refs.bib
├── figures/ ← written by code
│ └── coverage.pdf
└── tables/ ← written by code
└── results.tex
</pre>

Generated figures and tables flow from your code to Overleaf, and prose written on Overleaf flows back to your local copy:

<pre>
scripts/make_figures.py ──writes──▶ paper/figures, paper/tables
paper/ ──git push──▶ GitHub (my-paper) ──"Pull GitHub changes"──▶ Overleaf
Overleaf ──"Push Overleaf changes"──▶ GitHub (my-paper) ──git pull──▶ paper/
</pre>

{| style="width: 100%; border-collapse: collapse; margin-bottom: 10px;"
|-
! style="width: 50%; background-color: #d4edda; color: #155724; padding: 10px; text-align: left;" | Strengths
! style="width: 50%; background-color: #f8d7da; color: #721c24; padding: 10px; text-align: left;" | Weaknesses
|- style="vertical-align: top; background-color: #f8f9fa;"
| style="padding: 15px; border-right: 2px solid white;" |
* '''No More Uploading:''' Regenerated figures and tables reach the paper with one commit and one push, no manual file shuffling required.
* '''Clean Histories:''' Paper edits never clutter your code history, and code changes never clutter your paper history.
* '''No Coupled Tracking:''' Unlike git submodules, you never need an extra "update the paper pointer" commit in your experiment repository every time the paper changes.
* '''Collaborator Friendly:''' Co-authors can keep writing on Overleaf as usual. Nothing about their workflow changes.
| style="padding: 15px;" |
* '''Sync Is Manual:''' Overleaf does not watch GitHub. Someone has to click a button in Overleaf to pull new changes in (and to push Overleaf edits out).
* '''Provenance Is Up to You:''' Because the repositories are not linked, the only record of which code produced a figure is whatever you write in the paper's commit messages. A small script (below) handles this for you.
* '''One More Thing to Clone:''' Anyone who wants to regenerate the paper's figures needs to clone the paper repository separately.
|-
| colspan="2" style="background-color: #e9ecef; text-align: center; padding: 10px;" | 🔗 '''[https://www.overleaf.com/learn/how-to/GitHub_Synchronization Overleaf Documentation: GitHub Synchronization]'''
|}

== Setting It Up ==

=== Step 1: Link Overleaf to GitHub ===
In Overleaf, open '''Account Settings''' and find the GitHub entry under the project synchronization section. Click '''Link''' and authorize Overleaf in the GitHub dialog that pops up.


[[File:Overleaf-account-settings-github-link.png|thumb|center|600px|Linking a GitHub account from Overleaf's account settings.]]

=== Step 2: Create a GitHub repository from your Overleaf project ===
Open your paper in Overleaf, click '''Menu''' in the top left, and choose '''GitHub''' under the ''Sync'' section. Select '''Create a GitHub repository''', give it a name (e.g. <code>my-paper</code>), and make it private. Overleaf then pushes the current contents of your project to the new repository.


[[File:Overleaf-menu-sync-github.png|thumb|center|300px|The GitHub entry in the Overleaf project menu.]]


[[File:Overleaf-create-github-repo.png|thumb|center|500px|Creating the paper repository from Overleaf.]]

<div style="background-color: #e8f4f8; border-left: 5px solid #17a2b8; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">
'''Tip: Already have the paper on GitHub?'''<br>
You can go the other direction instead. In Overleaf, choose '''New Project → Import from GitHub''' and select your existing repository.
</div>

=== Step 3: Clone the paper inside your experiment repository ===
From the root of your experiment repository, tell Git to ignore the <code>paper/</code> directory '''before''' you clone into it:

<syntaxhighlight lang="bash">
cd my-experiments
echo "paper/" >> .gitignore
git add .gitignore
git commit -m "Ignore nested paper clone"

git clone git@github.com:you/my-paper.git paper
</syntaxhighlight>

Then check that the two repositories don't see each other:

<syntaxhighlight lang="bash">
git status # should NOT list paper/
git -C paper status # shows the paper repo, on its own branch
</syntaxhighlight>


[[File:Nested-clone-git-status.png|thumb|center|600px|The outer repository ignores <code>paper/</code>; the inner repository is independent.]]

<div style="background-color: #f8d7da; border-left: 5px solid #721c24; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">
'''Warning: Don't skip the <code>.gitignore</code> entry'''<br>
Without it, a careless <code>git add .</code> in the experiment repository will register <code>paper/</code> as an "embedded repository," which confuses collaborators and anything else that clones your code. If this happens, <code>git rm --cached paper</code> undoes it.
</div>

=== Step 4: Point your code at the paper directory ===
Rather than hard-coding paths in every script, define the paper location once. Using an environment variable with a sensible default means the code still runs for people who haven't cloned the paper (their figures just land in an ignored folder).

<syntaxhighlight lang="python">
# src/paths.py
import os
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parents[1]
PAPER_DIR = Path(os.environ.get("PAPER_DIR", REPO_ROOT / "paper"))
FIG_DIR = PAPER_DIR / "figures"
TAB_DIR = PAPER_DIR / "tables"

FIG_DIR.mkdir(parents=True, exist_ok=True)
TAB_DIR.mkdir(parents=True, exist_ok=True)
</syntaxhighlight>

For '''figures''', save vector PDFs directly into the paper. Removing the creation timestamp means a figure that didn't actually change produces an identical file, so Git won't report a pointless change:

<syntaxhighlight lang="python">
import matplotlib.pyplot as plt
from src.paths import FIG_DIR

fig, ax = plt.subplots(figsize=(3.25, 2.4)) # match your template's column width
...
fig.savefig(FIG_DIR / "coverage.pdf", bbox_inches="tight",
metadata={"CreationDate": None})
</syntaxhighlight>

For '''tables''', write a LaTeX fragment rather than a full document:

<syntaxhighlight lang="python">
from src.paths import TAB_DIR

tex = df.to_latex(index=False, escape=True, float_format="%.3f")
(TAB_DIR / "results.tex").write_text(tex)
</syntaxhighlight>

and pull both into the paper as usual:

<syntaxhighlight lang="latex">
\includegraphics[width=\linewidth]{figures/coverage}
...
\input{tables/results}
</syntaxhighlight>

For '''numbers quoted in the text''', the same idea works. Have your code write LaTeX macros so the prose never goes stale:

<syntaxhighlight lang="python">
(TAB_DIR / "numbers.tex").write_text(
f"\\newcommand{{\\coverageNinetyFive}}{{{cov95:.1f}\\%}}\n"
)
</syntaxhighlight>

<syntaxhighlight lang="latex">
% preamble
\input{tables/numbers}
% body
Our intervals achieve \coverageNinetyFive{} empirical coverage.
</syntaxhighlight>

== Day-to-Day Workflow ==

=== Sending new results to the paper ===
After rerunning your experiments, commit and push from inside <code>paper/</code>:

<syntaxhighlight lang="bash">
python scripts/make_figures.py

cd paper
git pull # pick up any text edits pushed from Overleaf first
git add figures tables
git commit -m "Regenerate figures from my-experiments@$(git -C .. rev-parse --short HEAD)"
git push
</syntaxhighlight>

Then, in Overleaf, open '''Menu → GitHub''' and click '''Pull GitHub changes into Overleaf'''.


[[File:Overleaf-pull-github-changes.png|thumb|center|500px|Bringing the regenerated figures into Overleaf.]]

<div style="background-color: #e8f4f8; border-left: 5px solid #17a2b8; padding: 15px; margin-bottom: 20px; margin-top: 15px; overflow: hidden;">
'''Important: Overleaf does not sync automatically'''<br>
Overleaf only picks up changes from GitHub when someone clicks the pull button, and only sends its own changes to GitHub when someone clicks the push button. Forgetting the pull is by far the most common reason the paper still shows old figures.
</div>

The commit message above includes the short hash of your experiment repository. Since the two repositories aren't linked, this message is your record of which code produced which figure. Running <code>git log -- figures/coverage.pdf</code> inside <code>paper/</code> will answer "where did this figure come from?" months later.

=== Automating it with a Makefile ===
Once the steps above feel familiar, wrap them in a single command:

<syntaxhighlight lang="make">
PAPER := paper
SHA := $(shell git rev-parse --short HEAD)$(shell git diff --quiet || echo -dirty)

.PHONY: figures paper-sync

figures:
python scripts/make_figures.py

paper-sync: figures
git -C $(PAPER) pull --rebase
git -C $(PAPER) add figures tables
git -C $(PAPER) diff --cached --quiet || \
git -C $(PAPER) commit -m "Regenerate figures from my-experiments@$(SHA)"
git -C $(PAPER) push
@echo "Now click 'Pull GitHub changes into Overleaf' in the Overleaf GitHub menu."
</syntaxhighlight>

Now <code>make paper-sync</code> regenerates everything, skips the commit if nothing actually changed, marks the commit <code>-dirty</code> if your experiment code had uncommitted changes, and reminds you to pull in Overleaf.

=== When co-authors edit on Overleaf ===
Before regenerating figures locally, push any Overleaf edits out to GitHub ('''Menu → GitHub → Push Overleaf changes to GitHub''') so that the <code>git pull</code> in <code>paper/</code> picks them up.


[[File:Overleaf-push-to-github.png|thumb|center|500px|Sending Overleaf edits back to GitHub.]]

=== Avoiding conflicts ===
Conflicts are rare if every file has exactly one "owner":

<div style="background-color: #f8f9fa; border-left: 4px solid #005A9C; padding: 15px; margin-bottom: 10px;">
'''<code>figures/</code> and <code>tables/</code> belong to your code.''' Never edit them by hand on Overleaf; the next regeneration will silently overwrite your changes. Change the formatting in the code that generates them instead.
</div>

<div style="background-color: #f8f9fa; border-left: 4px solid #228B22; padding: 15px; margin-bottom: 10px;">
'''<code>.tex</code> and <code>.bib</code> files belong to people.''' Write them on Overleaf (or locally, then push), and always pull before you push figures.
</div>

== Why Not a Git Submodule? ==
Git submodules are the "official" way to put one repository inside another, so it is natural to ask why this workflow avoids them.

{| style="width: 100%; border-collapse: collapse; margin-bottom: 10px;"
|-
! style="background-color: #e9ecef; padding: 10px; text-align: left;" |
! style="background-color: #e9ecef; padding: 10px; text-align: left;" | Gitignored Clone
! style="background-color: #e9ecef; padding: 10px; text-align: left;" | Git Submodule
! style="background-color: #e9ecef; padding: 10px; text-align: left;" | Git Subtree
|- style="background-color: #f8f9fa;"
| style="padding: 10px;" | '''Experiment repo records the paper version''' || style="padding: 10px;" | No || style="padding: 10px;" | Yes (a pointer) || style="padding: 10px;" | Yes (full history)
|-
| style="padding: 10px;" | '''Extra commit in experiment repo per paper change''' || style="padding: 10px;" | No || style="padding: 10px;" | Yes || style="padding: 10px;" | Yes
|- style="background-color: #f8f9fa;"
| style="padding: 10px;" | '''Paper history mixed into code history''' || style="padding: 10px;" | No || style="padding: 10px;" | No || style="padding: 10px;" | Yes
|-
| style="padding: 10px;" | '''Special clone instructions for collaborators''' || style="padding: 10px;" | No || style="padding: 10px;" | <code>--recurse-submodules</code> || style="padding: 10px;" | No
|- style="background-color: #f8f9fa;"
| style="padding: 10px;" | '''Link between figure and code version''' || style="padding: 10px;" | Commit message || style="padding: 10px;" | Automatic || style="padding: 10px;" | Automatic
|}

A submodule gives you exact, automatic provenance, but at a price: every change to the paper, including every typo fix a co-author pushes from Overleaf, needs a matching "bump the submodule" commit in your experiment repository. For a paper that is edited daily, that noise usually isn't worth it. The gitignored clone trades automatic provenance for a scripted commit message, which is plenty in practice.

== Variations ==

<div style="background-color: #f8f9fa; border-left: 4px solid #005A9C; padding: 15px; margin-bottom: 10px;">
'''Skip GitHub with Overleaf's Git integration:''' Overleaf Premium also lets you clone a project directly (<code>git clone https://git.overleaf.com/<project-id></code>, using a Git token generated in Account Settings). Clone that into the gitignored directory instead, and a <code>git push</code> lands in Overleaf immediately with no pull button. The catch is that you no longer get a GitHub copy of the paper for issues, reviews, or backups.
</div>

<div style="background-color: #f8f9fa; border-left: 4px solid #FFD43B; padding: 15px; margin-bottom: 10px;">
'''Generating figures on a cluster:''' Clone the paper repository on the cluster as well and run <code>make paper-sync</code> there. Alternatively, point <code>PAPER_DIR</code> at a scratch directory on the cluster, copy the outputs back to your laptop, and push from there.
</div>

== Common Pitfalls ==
{| style="width: 100%; border-collapse: collapse; margin-bottom: 10px;"
|-
! style="background-color: #f8d7da; color: #721c24; padding: 10px; text-align: left;" | Things to Watch Out For
|- style="vertical-align: top; background-color: #f8f9fa;"
| style="padding: 15px;" |
* '''Forgetting the Overleaf pull:''' Your figures are on GitHub, but Overleaf still shows the old ones. The reminder in the Makefile helps.
* '''Pushing from both sides at once:''' If Overleaf has unpushed edits and GitHub has new figures, Overleaf has to merge them. Push from Overleaf first, pull locally, then push your figures.
* '''Bloating the paper repository:''' PNGs and timestamped PDFs change on every run. Prefer vector PDFs with the timestamp removed, and only commit figures that actually changed.
* '''Overleaf project limits:''' Overleaf limits project size and file counts. Keep raw outputs, checkpoints, and intermediate data in the experiment repository (or out of Git entirely); only put paper-ready figures and tables in <code>paper/</code>.
* '''Working on branches:''' Overleaf only syncs one branch. Stick to the default branch inside <code>paper/</code>; anything on another branch won't show up in Overleaf.
|}

<div style="background-color: #e8f4f8; padding: 15px; text-align: center; margin-top: 20px; font-style: italic;">
'''Note:''' This is just one of several ways to connect your code and your paper. If the manual pull button bothers you, try the Git integration variation above; if you want airtight provenance, a submodule may be worth the extra commits.
</div>

Overleaf Git Integration - Revision history

Ajshen: /* Sending new results to the paper */

Ajshen: /* Step 3: Clone the paper inside your experiment repository */

Ajshen: /* Step 2: Find your project's clone URL */

Ajshen: Ajshen moved page Overleaf GitHub Integration to Overleaf Git Integration

Ajshen at 23:24, 18 October 2026

Ajshen: /* Step 1: Link Overleaf to GitHub */