Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a toc for html and latex exporter #2178

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
fafa016
Add a function to extract the titles and levels of headings from the …
HaudinFlorence Sep 16, 2024
e364a11
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 16, 2024
f7eb4c4
Updates jinja templates.
HaudinFlorence Sep 16, 2024
fab3c55
Update jinja templates and add style to the table of contents.
HaudinFlorence Sep 17, 2024
81e7e6c
Add links to be able to go to the relevant sections when clicking on …
HaudinFlorence Sep 17, 2024
04011ea
Place the css inline in index.html.j2.
HaudinFlorence Sep 17, 2024
0d02f38
Remove index.css from the git index.
HaudinFlorence Sep 17, 2024
05f6f11
Add the include_tableofcontents traitlet and update the jinja templat…
HaudinFlorence Sep 18, 2024
5f26a5c
Add the logics to include a table of contents for the latex exporter.
HaudinFlorence Sep 19, 2024
37485ef
Update nbconvertapp with removing toc from alias and with adding toc …
HaudinFlorence Oct 4, 2024
d5b579f
Try to fix failing CI tests.
HaudinFlorence Oct 4, 2024
6a5f4cb
Try to fix pre-commit test.
HaudinFlorence Oct 7, 2024
b31a287
Try to fix failing linting test.
HaudinFlorence Oct 7, 2024
cb90485
Update markdown.mistune.py and add docstrings to get 100% as a score …
HaudinFlorence Oct 7, 2024
b4d7e3d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
28089fa
Apply suggestions from code review
HaudinFlorence Oct 7, 2024
9720c29
Apply other suggestions from code review.
HaudinFlorence Oct 7, 2024
f88620e
Fix linting issues.
HaudinFlorence Oct 7, 2024
c8483f8
Try to fix failing linting test.
HaudinFlorence Oct 8, 2024
711cee8
Try to remove None check condition.
HaudinFlorence Oct 8, 2024
cdc2743
Restore None check condition in HTML exporter instead of _init_resources
HaudinFlorence Oct 9, 2024
de2ff4e
Change method extract_titles_from_markdown_input to extract_titles_fr…
HaudinFlorence Oct 14, 2024
6fbb968
Update extract_titles_from_notebook_node to fix some parsing issues.
HaudinFlorence Oct 25, 2024
5708233
Try to fix failing tests.
HaudinFlorence Oct 25, 2024
14a3ab8
Remove condition on headers of level 1.
HaudinFlorence Nov 4, 2024
e833379
Update extract_titles_from_notebook_node.
HaudinFlorence Nov 4, 2024
e5d2da6
Change the logics to get the headers using parsing of the markdown ce…
HaudinFlorence Nov 6, 2024
40f971d
Add sysmetically an id to the headers to be sure to have a defined href.
HaudinFlorence Nov 7, 2024
8c66634
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion nbconvert/exporters/html.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,11 @@
from nbformat import NotebookNode

from nbconvert.filters.highlight import Highlight2HTML
from nbconvert.filters.markdown_mistune import IPythonRenderer, MarkdownWithMath
from nbconvert.filters.markdown_mistune import (
IPythonRenderer,
MarkdownWithMath,
extract_titles_from_markdown_input,
)
from nbconvert.filters.widgetsdatatypefilter import WidgetsDataTypeFilter
from nbconvert.utils.iso639_1 import iso639_1

Expand Down Expand Up @@ -256,8 +260,17 @@ def from_notebook_node( # type:ignore[explicit-override, override]
highlight_code = self.filters.get(
"highlight_code", Highlight2HTML(pygments_lexer=lexer, parent=self)
)
markdown_collection = ""
for cell in nb.cells:
if cell.cell_type == "markdown":
markdown_collection = markdown_collection + cell.source + "\n"

resources = self._init_resources(resources)
if resources is None:
resources = {}
resources.update(
{"tableofcontents": extract_titles_from_markdown_input(markdown_collection)}
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like it would better having this in _init_resources as this is the place where we initialize the resources

Copy link
Author

@HaudinFlorence HaudinFlorence Oct 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment. This has been moved to _init_resources. But this is probably not inducing the proper checking on resources not being None since it is breaking again the linting CI test with this type of error

nbconvert/exporters/html.py:269: error: Item "None" of "Optional[Dict[str, Any]]" has no attribute "update"  [union-attr]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using update, you could follow what the rest of the code is doing: resources["tableofcontents"] = extract_titles_from_markdown_input(markdown_collection)

Copy link
Author

@HaudinFlorence HaudinFlorence Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was initially what was used but without any check that resources was not None.
Introducing the suggested change leads to this error message

nbconvert/exporters/html.py:269: error: Unsupported target for indexed assignment ("Optional[Dict[str, Any]]")  [index]
nbconvert/exporters/html.py:358: error: Name "resources" already defined on line 294  [no-redef]

Maybe the type check in the _init_resources function is not valid.

Copy link
Author

@HaudinFlorence HaudinFlorence Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have displaced the None check condition back at its initial place, in the HTMLExporter


filter_data_type = WidgetsDataTypeFilter(
notebook_metadata=self._nb_metadata, parent=self, resources=resources
Expand Down
4 changes: 4 additions & 0 deletions nbconvert/exporters/templateexporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,9 @@ def default_config(self):
enable_async = Bool(False, help="Enable Jinja async template execution").tag(
affects_environment=True
)
include_tableofcontents = Bool(
False, allow_none=True, help="Enable to include a table of contents"
).tag(config=True, affects_template=True)

_last_template_file = ""
_raw_template_key = "<memory>"
Expand Down Expand Up @@ -680,4 +683,5 @@ def get_prefix_root_dirs(self):
def _init_resources(self, resources):
resources = super()._init_resources(resources)
resources["deprecated"] = deprecated
resources["include_tableofcontents"] = self.include_tableofcontents
return resources
38 changes: 38 additions & 0 deletions nbconvert/filters/markdown_mistune.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
from typing import Any, Callable, Dict, Iterable, Match, Optional, Tuple

import bs4
import mistune
from mistune.renderers.markdown import MarkdownRenderer
from pygments import highlight
from pygments.formatters import HtmlFormatter
from pygments.lexer import Lexer
Expand Down Expand Up @@ -487,3 +489,39 @@ def render(self, source: str) -> str:
def markdown2html_mistune(source: str) -> str:
"""Convert a markdown string to HTML using mistune"""
return MarkdownWithMath(renderer=IPythonRenderer(escape=False)).render(source)


class HeadingExtractor(MarkdownRenderer):
"""A renderer to capture headings"""

def __init__(self):
"""Initialize the class."""
super().__init__()
self.headings = []

def heading(self, text, level):
"""Return an empty string for the headings to avoid outputting them."""
self.headings.append((level, text))
return ""


def extract_titles_from_markdown_input(markdown_input):
"""Create a Markdown parser with the HeadingExtractor renderer to collect all the headings of a notebook"""
""" The input argument is markdown_input that is a single string with all the markdown content concatenated """
""" The output is an array containing information about the headings such as their level, their text content, an identifier and a href that can be used in case of html converter.s"""
HaudinFlorence marked this conversation as resolved.
Show resolved Hide resolved
titles_array = []
renderer = HeadingExtractor()
extract_titles = mistune.create_markdown(renderer=renderer)
extract_titles(markdown_input)
headings = renderer.headings

""" Iterate on all headings to get the necessary information on the various titles """
HaudinFlorence marked this conversation as resolved.
Show resolved Hide resolved
for __, title in headings:
children = title["children"]
attrs = title["attrs"]
raw_text = children[0]["raw"]
header_level = attrs["level"]
id = raw_text.replace(" ", "-")
href = "#" + id
titles_array.append([header_level, raw_text, id, href])
return titles_array
5 changes: 5 additions & 0 deletions nbconvert/nbconvertapp.py
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,10 @@ def validate(self, obj, value):
},
"""Whether the HTML in Markdown cells and cell outputs should be sanitized..""",
),
"toc": (
{"TemplateExporter": {"include_tableofcontents": True}},
"Generate a table of contents in the output (only compatible with HTML and Latex exporters)",
),
}
)

Expand Down Expand Up @@ -675,5 +679,6 @@ def _default_export_format(self):
# Main entry point
# -----------------------------------------------------------------------------


main = launch_new_instance = NbConvertApp.launch_instance
dejavu_main = DejavuApp.launch_instance
3 changes: 3 additions & 0 deletions share/templates/lab/base.html.j2
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
{% from 'celltags.j2' import celltags %}
{% from 'cell_id_anchor.j2' import cell_id_anchor %}

{%- block body_header -%}
{%- endblock body_header -%}

martinRenou marked this conversation as resolved.
Show resolved Hide resolved
{% block codecell %}
{%- if not cell.outputs -%}
{%- set no_output_class="jp-mod-noOutputs" -%}
Expand Down
100 changes: 100 additions & 0 deletions share/templates/lab/index.html.j2
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,56 @@ a.anchor-link {
display: block;
}
}
/* Table of Contents for the html exporter */
.jp-RenderedHTMLTOC-Title {
font-family: var(--jp-content-font-family);
font-size: 24px;
margin: 16px 0;
padding-left: 64px;
font-weight: bold;
}

.jp-RenderedHTMLTOC-Item-h1 {
font-family: var(--jp-content-font-family);
font-size: 20px;
margin: 0;
padding-left: 88px;
}

.jp-RenderedHTMLTOC-Item-h2 {
font-family: var(--jp-content-font-family);
font-size: 18px;
margin: 4px;
padding-left: 112px;
}

.jp-RenderedHTMLTOC-Item-h3 {
font-family: var(--jp-content-font-family);
font-size:16px;
margin: 4px;
padding-left: 136px;
}

.jp-RenderedHTMLTOC-Item-h4 {
font-family: var(--jp-content-font-family);
font-size: 14px;
margin: 4px;
padding-left: 160px;
}

.jp-RenderedHTMLTOC-Item-h5 {
font-family: var(--jp-content-font-family);
font-size: 12px;
margin: 4px;
padding-left: 184px;
}

.jp-RenderedHTMLTOC-Item-h6 {
font-family: var(--jp-content-font-family);
font-size: 10px;
margin: 2px;
padding-left: 208px;
}
</style>

{% endblock notebook_css %}
Expand All @@ -126,6 +176,56 @@ a.anchor-link {
<body class="jp-Notebook" data-jp-theme-light="true" data-jp-theme-name="JupyterLab Light">
{% endif %}
<main>
{%- block tableofcontents -%}
{%- if resources.include_tableofcontents -%}
<div class="jp-RenderedHTMLTOC-Title">Table of contents</div>
{%- for item in resources.tableofcontents -%}
{%- set (level, text, id, href) = item -%}
{%- if level==1 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h1">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
{%- if level==2 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h2">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
{%- if level==3 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h3">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
{%- if level==4 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h4">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
{%- if level==5 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h5">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
{%- if level==6 -%}
<div class="jp-RenderedHTMLCommon jp-RenderedHTMLTOC-Item-h6">
<a href={{href}}>
{{text}}
</a>
</div>
{%- endif -%}
HaudinFlorence marked this conversation as resolved.
Show resolved Hide resolved
{%- endfor -%}
{%- endif -%}
{% endblock tableofcontents %}
{%- endblock body_header -%}

{% block body_footer %}
Expand Down
6 changes: 6 additions & 0 deletions share/templates/latex/base.tex.j2
Original file line number Diff line number Diff line change
Expand Up @@ -213,11 +213,17 @@ override this.-=))
((* endblock header *))

((* block body *))

\begin{document}
((* block predoc *))
((* block maketitle *))\maketitle((* endblock maketitle *))
((* block abstract *))((* endblock abstract *))
((* endblock predoc *))
((* block tableofcontents *))
((* if resources.include_tableofcontents *))
\tableofcontents
((* endif *))
((* endblock tableofcontents *))

((( super() )))

Expand Down
Loading
Loading