Add additional detail about the site deployment pipeline

[willingc]

References and relevant issues

Relaated to #807, #811, #801

Description

This PR adds additional detail about the deployment pipeline from source files to live website.

Additional headings are added to give the reader a better understanding of the logic flow.

[melissawm]

Thank you @willingc - sorry for the late review! I have questions around how we could organize this, although I love the workflow-centered approach (source -> html -> deployment). Do you have ideas for this organization here?

[melissawm]

or napari/napari in the case of docstrings and examples.

[willingc]

Happy to clarify. I want to make sure that I understand the actual approach since it's a bit confusing to have docs build/deploy actions spread across 3 repos.

Source files

• rst/md files in napari/docs
• docstrings in napari/napari source files
• examples in napari/napari examples directory

napari/docs actions

What are they specifically doing related to deployment?

napari/napari actions

What are they doing related to deployment?

---

The reason that I took the workflow approach was I was having difficulty understanding which repo was doing what. There's a bunch of non-typical workflows happening and it's difficult for someone who has plenty of docs experience to follow what is happening with our workflow tooling.

While we can do a local build in napari/docs, it's unclear how the local build differs from the deployed site:

• versioned / unversioned pages
• actions and workflow responsibilities across repos
• differences in Sphinx template locations than a standard Sphinx build

It's totally cool that we're doing something different. My hope is that we can document it for greater clarity for current and future contributors. ❤️

[melissawm]

Just for my understanding, what is the difference? I may be missing something here

[willingc]

docs/.github/workflows/build_and_deploy.yml

Line 4 in 4d5e4ff

# Note this workflow is also triggered by

This was taken from the existing documentation on https://napari.org/stable/developers/contributing/documentation/docs_deployment.html in this line

docs/docs/developers/contributing/documentation/docs_deployment.md

Line 50 in 4d5e4ff

Note that these files are not identical to the `napari/napari` version.

[melissawm]

Yeah, I'm not sure what this means? Might be worth removing, I don't remember why this is here and I don't see a reason why the artifact files would be different...

[melissawm]

I really like the workflow-focused organization, but I'm not sure if this is clear. The build_docs workflow can be triggered by both the napari/napari or napari/docs repos, and in each case it will clone the other corresponding repo, build and deploy to napari.gihub.io, so I think these items don't cover all options.

The following situations are possible:

• A commit to napari/napari is made.
  • The build_docs.yml workflow in napari/napari is triggered
  • The napari/docs repo is cloned to napari/napari actions environment
  • html artifacts are generated and pushed to napari.github.io
• A commit to napari/docs is mae
  • The build_and_deploy.html workflow in napari/docs is triggered
  • The napari/napari repo is cloned to napari/docs actions environment
  • html artifacts are generated and pushed to napari.github.io

[willingc]

This is good context and unclear from the existing docs and the workflows in the repos.

A few questions:

The first situation when changes to napari/napari are made.

1. Are the docs always built?
2. Or are the docs built and deployed only on changes to examples and/or docstrings?
3. Are the raw html artifacts that are pushed to napari.github.io, then acted upon by the actions in napari.github.io?

The second situation when commit is made to napari/docs

1. Are the html files created by the napari/docs actions?
2. Or does a napari/docs action trigger another action to run in napari/napari?
3. Same question as Update contributor docs #3 above

[melissawm]

The first situation when changes to napari/napari are made.

1. Are the docs always built? Yes
2. Or are the docs built and deployed only on changes to examples and/or docstrings? No
3. Are the raw html artifacts that are pushed to napari.github.io, then acted upon by the actions in napari.github.io? Yes

The second situation when commit is made to napari/docs

1. Are the html files created by the napari/docs actions? Yes
2. Or does a napari/docs action trigger another action to run in napari/napari? No, napari/napari gets cloned into the actions environment in napari/docs and the docs are built to generate the html artifacts
3. Same question as 3. above Yes, same process as with napari/napari, html artifacts are pushed to napari.github.io and the workflows there act on the artifacts

We have considered many different workflows in the past, such as having one single action that would run to build the docs when commits are made to either repo, but if that action lives on one repo it would be harder to detect when changes are made in the other repo, for example. We try to keep the two actions that build docs (in napari/napari and napari/docs) synchronized as much as possible though.

[willingc]

To clarify point 3 in both situations:

• napari/napari action can push raw html artifacts to napari.github.io
• napari/docs action can puah raw html artifacts to napari.github.io
• These actions should produce the same raw html artifacts if acting on identical source files. In other words, given the same source files, each action would produce raw html artifacts which are identical.

[melissawm]

Yes exactly - that is my understanding at least!

[willingc]

Thanks @melissawm. I will rework this PR later in the week. Thanks for the additional context. I have a couple of ideas to keep it workflow centric but to handle the case that either repo can trigger the push to napari.github.io

[willingc]

@melissawm Thanks for the review. I've added some clarifying questions.

@psobolewskiPhD brought up a mystery question in the docs meeting about a docs repo action being canceled by a higher priority action in the napari repo.