Maintain cell metadata for RISE

[boazbk]

I wanted to use jupytext to have a paired md+ipynb notebook that I also use for presentation with the RISE plugin https://github.com/damianavila/RISE

Unfortunately, every time I reload the browser all the metadata information about whether a cell is a slide, fragment, or should be skipped, gets lost.

[mwouts]

Hi @boazbk . Thanks for reporting. I am afraid that RISE notebooks have metadata information in cells of every type (code, markdown, raw). At this stage,

• markdown format has no support for cell metadata
• Python, Julia and R scripts, as well as R markdown, only support cell metadata for code cells.

I was thinking of adding support of cell metadata to generic cell types for all cell types in scripts. For markdown documents however,

• we would have to choose a representation of metadata on code cells. Are you aware of any standard way of representing such metadata?
• I am not sure yet how cell metadata could be represented on markdown paragraphs. Any suggestion is welcome!

For now I regret to tell that you can't use Jupytext yet with such documents. Indeed the text version does not store the cell metadata, and that's the cause of the issue you're hitting.

By the way - but I'm not sure you like the idea - have you ever tried RStudio's ioslides? That may sound unexpected to you if you're already using RISE. It works for python as well, but not many people are aware. It is well documented (see previous link), offers a great control of plot size, which code to show, etc. Have a look at this example (an extension that you can open in Jupyter as well), and the resulting HTML5 slides, that include plotly JS graphs generated in python.

[boazbk]

Thanks - I didn't know about ioslides - will look into it. They do have some syntax for markdown metadata inside { ... } in the form

#section { .class1 .class2 }

Perhaps you could use something like

[]{ .slide }

[]{ .fragment }

to have per-cell metadata in markdown, but one should verify this parses correctly (in the sense that it is ignored) in standard markdown parsers such as pandoc.

[mwouts]

I think that cell metadata for plain markdown paragraphs should be commented. I think you suggest something similar, right?

I have found that interesting stackoverflow post on markdown comments. The comment says:
[//]: # "Comment" and [//]: # (Comment) seem to work on a wider variety of [Markdown] implementations.

Now, we have to find a convincing representation of metadata inside the comment. I was thinking of JSON, as it is simple to parse... (all that way from Jupyter notebooks, to come back again to JSON !? 😄)

Indeed we will have to check that the choosen syntax is ignored by

• pandoc
• RStudio
• GitHub

A similar question holds for the code cells in markdown. Language is often found, but we'll also have to represent cell metadata there...

[boazbk]

I was looking at the following example from ioslides:

## Getting up {.smaller}

Seems that .smaller here is a metadata that is applied to the section

But anyway I did not want to cause you much trouble - thanks so much for doing this project!

[mwouts]

You're welcome. No worries, having full support for metadata does matter to me.
In the ioslides example, I think .smaller is a class defined in the CSS file, and it indeed applies to the current section.
Still I am afraid we need a more generic way of representing Jupyter cell metadata. When time permits I will try a prototype that represent cell metadata as JSON in markdown comments.

[kernc]

Why not a YAML (we already have YAML?) header before each markdown cell (that requires it)?

You'd just have to strip the space between # and --- as --- is already a rule in markdown.

[mwouts]

Why not a YAML (we already have YAML?) header before each markdown cell (that requires it)?

Well, @kernc , the reason for which I have a preference for JSON for cell metadata is that it fits on one line... and then, it is it probably easier to comment so that the generated markdown file remains compatible with markdown converters.

@boazbk , the upcoming version v0.7.0 has full cell metadata support for two python formats: light (i.e. the one already available in v0.6) and percent (the format used by Hydrogen, etc). The branch is still going to evolve a bit, but if you want you can already test it.

[kernc]

the generated markdown file remains compatible with markdown converters.

How do the markdown converters fare with inline, say, global (e.g.) Python code? 😛

[mwouts]

Metadata support is available for python formats (light and double percent) in the release candidate. Install it with

pip install jupytext==0.7.0rc0

and read the documentation on the v0.7.0 branch.

@boazbk , could you please test the RC, and close the issue if you are satisfied with the cell metadata support in python scripts?

[goerz]

@mwouts I'm not quite sure what I'm doing wrong, but jupytext --to=py:light <ipynbfile> does not seem to give me cell metadata in the generated python file (v0.8.6)

In any case, for --to=md, I would definitely go with single-line json encoding. For code cells, taking the metadata example

{
  "attributes": {
    "classes": ["myclass"],
    "id": "",
    "n": "2"
  },
  "trusted": false,
  "tags": [
    "mytag"
  ],
  "slideshow": {
    "slide_type": "skip"
  }
}

I would read that into data, and then generate

" ".join(["%s=%s" % (key, json.dumps(val)) for (key, val) in data.items()])

which then goes as metadata on the first line of the fenced code block, as e.g.

```python attributes={"classes": ["myclass"], "id": "", "n": "2"} trusted=false tags=["mytag"] slideshow={"slide_type": "skip"}
x = 1
```

I think attributes might be standard, actually, and could be stripped out of data before generating the json string, except maybe that classes (I'm assuming these are css classes) should extracted, leading to

 ```python .myclass trusted=false tags=["mytag"] slideshow={"slide_type": "skip"}
x = 1
```

This syntax is compatible with a wide variety of Markdown parsers (pandoc, Github), see also e.g. https://python-markdown.github.io/extensions/fenced_code_blocks/#emphasized-lines

For markdown cells, I don't really know what a good syntax is for metadata that gets ignored by the parsers, but the metadata should probably be encoded similarly to the above.