Support `beginning-of-line-text` and `(fill-paragraph 't)` in comments

[jeberger]

I'm not sure if this should be two issues or one, but they feel related.

When programming in C/C++ with cc-mode, when the cursor is inside a comment and I hit M-<home> (which is bound to beginning-of-line-text), then the cursor moves to the first character of the comment text after the comments symbols, e.g.:

   // Comment text
      ^ `beginning-of-line-text` moves the cursor here

In rust-mode, the cursor moves to the first non-whitespace character (same as back-to-indentation).

Possibly related: when I hit C-u M-q (bound to (fill-paragraph 't)) to justify a paragraph in a comment, rust-mode adds extra spaces to the begnining of some lines:

   /// Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
   ///  in reprehenderit  in  voluptate velit  esse  cillum dolore  eu
   /// fugiat nulla pariatur.

whereas cc-mode is able to justify the text correctly:

   /// Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
   /// in  reprehenderit  in voluptate  velit  esse  cillum dolore  eu
   /// fugiat nulla pariatur.

I believe both issues may be related to identifying and setting the fill-prefix when these functions are called inside comments. It would be nice to have those working as in cc-mode.

[psibi]

Is the bug reproducible when you are using with tree-sitter enabled: https://github.com/rust-lang/rust-mode?tab=readme-ov-file#tree-sitter ?

[jeberger]

Is the bug reproducible when you are using with tree-sitter enabled: https://github.com/rust-lang/rust-mode?tab=readme-ov-file#tree-sitter ?

Actually I had tree-sitter enabled, but the behaviour is the same if I disable it.

[psibi]

Hmm, that's strange. This is what happens in my Rust buffer with tree sitter enabled:

• beginning-of-line-text: Works as expected.
• (fill-paragraph 't): Buggy behavior. In fact, it's different in my case - it adds a new / at the beginning of the comment. And I'm confused by it.

Any chance you can run your Emacs only with the rust-mode and see what's the behavior is ? I'm wondering if some other third party package is interfering.

[jeberger]

I've removed my whole .emacs.d, installed just tree-sitter (20220212.1632), tree-sitter-langs (0.12.210), tree-sitter-indent (0.3) and rust-mode (20240520.749) and I have the same behaviour (emacs is v29.4 on ArchLinux).

[psibi]

Hmm, strange. Just to double check - does evaluating this show t for you:

(treesit-ready-p 'rust)

[jeberger]

(treesit-ready-p 'rust)

Gives: Debugger entered--Lisp error: (void-function treesit-ready-p)

But if I type M-x tree-sitter-query-builder, then type the following query:

(function_item (identifier) @func)

Then it correctly highlights the function names.

[psibi]

But if I type M-x tree-sitter-query-builder, then type the following query:

Does the (treesit-ready-p 'rust) work after that ?

[jeberger]

But if I type M-x tree-sitter-query-builder, then type the following query:

Does the (treesit-ready-p 'rust) work after that ?

I don't think so, but I can't check since I'm no longer in front of the work computer where that happened.

Here at home, I have a different behaviour:

• (treesit-ready-p 'rust) returns nil and prints a warning that the grammar for Rust is unavailable
• tree-sitter-query-builder works
• beginning-of-line-text gives the same result i.e. moves to the first non-whitespace character instead of the first character in the comment
• (fill-paragraph 't) random behaviour:
  • Sometimes it works
  • Sometimes it does nothing
  • Sometimes it removes the /// and indentation
  • Sometimes it acts like on my other computer, i.e. it adds extra spaces at the beginning of some lines.

Weird…

[jeberger]

OK I figured out why (treesit-ready-p 'rust) returned nil here: it was picking the system install of treesitter (which didn't include the Rust support) instead of the one from package.el. When I add the Rust support to the system install, I get the same behaviour as at work, except that here (treesit-ready-p 'rust) returns 't

[jeberger]

But if I type M-x tree-sitter-query-builder, then type the following query:

Does the (treesit-ready-p 'rust) work after that ?

I don't think so, but I can't check since I'm no longer in front of the work computer where that happened.

I can confirm that (treesit-ready-p 'rust) still fails with (void-function treesit-ready-p) after I run the query builder.

[psibi]

I can confirm that (treesit-ready-p 'rust) still fails with (void-function treesit-ready-p) after I run the query builder.

Ah, okay. That means that rust-mode is not using the tree sitter implementation for it. What happens if you do rust-ts-mode ? I believe even that won't be working ?

[jeberger]

I hadn't realized that tree-sitter was included in emacs >=29, so I was still installing it from melpa. And I hadn't realized that there were two distinct rust modes with or without tree-sitter.

In Emacs 29.2 on Linux using rust-ts-mode:

• beginning-of-line-text works as expected
• (fill-paragraph) and (fill-paragraph 't) work on single-line comments except that:
  • for doc comments, they replace the space between /// or //! and the text with a / (non-doc comments that start with only two / are not affected).
  • they only consider a single line, so they won't work if you have a text paragraph that has already been filled and you want to re-fill it after doing some changes.
• (fill-paragraph) and (fill-paragraph 't) don't do anything on block comments (i.e. /* … */)

[jeberger]

FWIW, fill-paragraph works as expected for C in Emacs 29.2 on Linux using c-ts-mode.

[jjbarr]

It looks like there was a breaking change between Emacs's c-ts-common mode and the rust parser in treesitter. Apparently doc comment nodes didn't used to be correctly exposed and now that they are things are broken. So if you've pinned the rust parser than it's possible you don't have this problem.