DOC: Stop rendering plot directive figures to PDF, speeding up documentation builds - #884
Merged
Conversation
Co-Authored-By: Melissa Weber Mendonça <melissawm@gmail.com> Co-Authored-By: Aditi Juneja <91629733+schefflera-arboricola@users.noreply.github.com>
Collaborator
Author
|
While comparing with a single run doesn't prove a lot; the previous PR ( https://app.readthedocs.org/api/v2/build/34868538.txt) vs this PR ( https://app.readthedocs.org/api/v2/build/34872370.txt):
|
Collaborator
Author
|
@Schefflera-Arboricola suggested confirming a bit more reliably since this can go into her blog post, so these are the results from
To reproduce: # setup
git clone https://github.com/PyWavelets/pywt.git
cd pywt
python3.13 -m venv .venv-docs
source .venv-docs/bin/activate
python -m pip install --upgrade pip
python -m pip install -r util/readthedocs/requirements.txt
python -m pip install .# run
cd doc
hyperfine --runs 5 --warmup 0 --export-markdown pdf-vs-png.md \
--prepare 'make clean' \
-n 'PDF on: plot_formats png+pdf' \
'sphinx-build -q -b html -D intersphinx_timeout=3 -D plot_formats=png:96,pdf -d build/doctrees source build/html' \
-n 'PDF off: plot_formats png' \
'sphinx-build -q -b html -D intersphinx_timeout=3 -D plot_formats=png:96 -d build/doctrees source build/html' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of an exercise that @Schefflera-Arboricola, @melissawm, and I are performing to find speedups in docs builds. I ran sphinx-benchmark over a documentation build for PyWavelets, through the output of which I found that a Matplotlib stream that builds PDFs of plots as requested by
.. plot::directives is consuming 66% of the build time in the gap between theobject-description-transformand thedoctree-readevents.Within that, most of it comes from the CWT page. See this comment as well:
pywt/doc/source/ref/cwt.rst
Lines 89 to 90 in cc45b0d
These PDFs are placed into a temporary build directory next to the doctrees, are not used anywhere as part of the PDF build pipeline because PyWavelets does not build PDF documentation (whether via the Sphinx PDF builder or any other tooling), and are later discarded. So I figured that we can perhaps just drop it, until/unless we consider building PDF documentation someday?
Without
sphinx-benchmark, i.e., using a regular Sphinx build (because sphinx-benchmark currently requires a non-parallel build), here are the results from two runs (both with a cold cache) on my macOS machine locally with Sphinx 9 and Python 3.13:plot_formats[('png', 96), 'pdf'][('png', 96)]Therefore, this is a 53% reduction in wall time (34.3s to 16.2s), or, equivalently, the old build took 2.1x as long as the new one! 🚀
I am aware that PyWavelets' docs are not time-consuming by any means in comparison to a much bigger Python library/project, so this doesn't help much in practice, but it's still nice that experimenting here allows us to devise a bit of a playbook for how sphinx-benchmark can be extended to optimise docs builds for other projects, sometimes in simple ways like this. @melissawm, I believe, found another similar improvement that can be made to the Matplotlib plot directive directly IIRC, and is working on upstreaming it in the meantime?
Note that the CPU time drops more than the wall time, because the ~4 seconds of kernel startup and JupyterLite subprocess waiting in the build are unaffected.
xref Quansight/Quansight-website#1011