Issue with rendering paragraphs

Initially posted in mdpopups repo: facelessuser/sublime-markdown-popups#163

After author's reply looking in LSP code it turned that the problem is with:

Line 618 in 7981a13

result = re.sub('(\\S) \n', '\\1\n\n', result) # noqa: RUF039

Commenting that regex substitution, shows paragraphs correctly:

Please describe what you think the issue is.

The Markdown content provided in facelessuser/sublime-markdown-popups#163 (comment) has two spaces preceding the linebreaks (\n), so each line should be rendered as a separate paragraph.

See comment at

LSP/plugin/core/views.py

Line 617 in 7981a13

    
           # Workaround CommonMark deficiency: two spaces followed by a newline should result in a new paragraph.

Hm, actually there are competing Markdown specs and I see now that GitHub flavoured Markdown chooses to not turn the trailing spaces into separate paragraphs: https://github.github.com/gfm/#example-196

I wonder was this changed some time in the Markdown specification, or was this PR a mistake: #1441

I think the correct substitution (if needed) according to https://github.github.com/gfm/#example-658 would be

result = re.sub('[ ]{2,}\n', '<br>', result)

But the example also says that it should only apply if it

does not occur at the end of a block

so it might not be that easy.

Yes the problem is the replacement string \1\n\n. If we change it to \1<br> or we use your above substitution paragraphs are rendered correctly.

For this input:

The result is a named 6-tuple with fields corresponding to the  <0x0A>
above. It is either a ParseResult or ParseResultBytes object,  <0x0A>
depending on the type of the url parameter.  <0x0A>
  <0x0A>
The username, password, hostname, and port sub-components of netloc  <0x0A>
can also be accessed as attributes of the returned object.  <0x0A>
  <0x0A>
The scheme argument provides the default value of the scheme  <0x0A>
component when no scheme is found in url.  <0x0A>
  <0x0A>
If allow\_fragments is False, no attempt is made to separate the  <0x0A>
fragment component from the previous component, which can be either  <0x0A>
path or query.  <0x0A>
  <0x0A>
Note that % escapes are not expanded.  <0x0A>
  <0x0A>
urlsplit() should generally be used instead of urlparse().<0x0A>

after re.sub('(\\S) \n', '\\1\n\n', result) it becomes:

The result is a named 6-tuple with fields corresponding to the<0x0A>
<0x0A>
above. It is either a ParseResult or ParseResultBytes object,<0x0A>
<0x0A>
depending on the type of the url parameter.<0x0A>
<0x0A>
  <0x0A>
The username, password, hostname, and port sub-components of netloc<0x0A>
<0x0A>
can also be accessed as attributes of the returned object.<0x0A>
<0x0A>
  <0x0A>
The scheme argument provides the default value of the scheme<0x0A>
<0x0A>
component when no scheme is found in url.<0x0A>
<0x0A>
  <0x0A>
If allow\_fragments is False, no attempt is made to separate the<0x0A>
<0x0A>
fragment component from the previous component, which can be either<0x0A>
<0x0A>
path or query.<0x0A>
<0x0A>
  <0x0A>
Note that % escapes are not expanded.<0x0A>
<0x0A>
  <0x0A>
urlsplit() should generally be used instead of urlparse().<0x0A>

The workaround mentioned was made at the time that mdpopups used a “Daring Fireball Markdown” parser whereas the rest of the world uses CommonMark. But now that mdpopups can also parse CommonMark, I’m not sure if the workaround is needed or even producing wrong results now.

The workaround mentioned was made at the time that mdpopups used a “Daring Fireball Markdown” parser whereas the rest of the world uses CommonMark. But now that mdpopups can also parse CommonMark, I’m not sure if the workaround is needed or even producing wrong results now.

But the change from that PR was doing the opposite. According to the CommonMark / GitHub flavoured Markdown specs, it should not result in a paragraph. So we don't want to replace the spaces with double linebreak (because double linebreak does result in a new paragraph).

Also majority of users (stable release) are still on the old Markdown parser.

I have not looked into the issue myself but just want to point out that that workaround IS only applied when using old parser:

LSP/plugin/core/views.py

Lines 601 to 618 in 7981a13

    
           if MARKO_MD_PARSER_VERSION: 
        
               frontmatter["markdown_parser"] = "marko" 
        
               frontmatter["markdown_extensions"] = ["gfm"] 
        
           else: 
        
               frontmatter["markdown_extensions"] = [ 
        
                   "markdown.extensions.admonition", 
        
                   { 
        
                       "pymdownx.magiclink": { 
        
                           # links are displayed without the initial ftp://, http://, https://, or ftps://. 
        
                           "hide_protocol": True, 
        
                           # GitHub, Bitbucket, and GitLab commit, pull, and issue links are are rendered in a shorthand 
        
                           # syntax. 
        
                           "repo_url_shortener": True 
        
                       } 
        
                   } 
        
               ] 
        
               # Workaround CommonMark deficiency: two spaces followed by a newline should result in a new paragraph. 
        
               result = re.sub('(\\S)  \n', '\\1\n\n', result)  # noqa: RUF039

Huh. Seems like you’re all correct. Let’s just remove it then.

More context in #1441 and especially linked issue.

I remember this was specifically done for clangd, which uses those two spaces at the end of a line for neovim specifically.

I'll also mention that LSP-julia does the same but not sure if on purpose:

https://github.com/sublimelsp/LSP-julia/blob/6575c619fe9604845ddbd0ef0b542e7943b662d5/plugin.py#L226-L227

According to the CommonMark / GitHub flavoured Markdown specs, it should not result in a paragraph.

In this issue on the mdpopups repository, I reason:

There is however this markdown syntax where you can put two whitespace characters at the end of a line, and supposedly this should result in a linebreak

So the intention was that two spaces at the end of a line should result in a hard line break. In HTML syntax that would be </br>. The workaround I've added in 2020 may have resulted in the same visual desired result. Perhaps it introduced a paragraph, but that distinction was not visible to the eye. With the recent changes to use non-monotonic fonts for the text, this may now result in incorrect visual results.

.

In any case, we may just remove the workaround.

I'll also mention that LSP-julia does the same but not sure if on purpose:

I think I just copied that from the LSP code, but it's probably not actually used by the server, so I think it could be removed. Especially if it was unnecessary/wrong all the time anyway.

Removing in #2930 then.

Navigation Menu

Repository navigation

Issue with rendering paragraphs #2928

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if MARKO_MD_PARSER_VERSION:
	frontmatter["markdown_parser"] = "marko"
	frontmatter["markdown_extensions"] = ["gfm"]
	else:
	frontmatter["markdown_extensions"] = [
	"markdown.extensions.admonition",
	{
	"pymdownx.magiclink": {
	# links are displayed without the initial ftp://, http://, https://, or ftps://.
	"hide_protocol": True,
	# GitHub, Bitbucket, and GitLab commit, pull, and issue links are are rendered in a shorthand
	# syntax.
	"repo_url_shortener": True
	}
	}
	]
	# Workaround CommonMark deficiency: two spaces followed by a newline should result in a new paragraph.
	result = re.sub('(\\S) \n', '\\1\n\n', result) # noqa: RUF039

Issue with rendering paragraphs #2928

Description

Activity

jwortmann commented on May 24, 2026

jwortmann commented on May 24, 2026

jwortmann commented on May 24, 2026

klonuo commented on May 24, 2026

rwols commented on May 24, 2026

jwortmann commented on May 24, 2026

rchl commented on May 24, 2026

rwols commented on May 24, 2026

rchl commented on May 24, 2026

rwols commented on May 24, 2026

rchl commented on May 24, 2026

rwols commented on May 24, 2026

jwortmann commented on May 24, 2026

rchl commented on May 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions