Setting the 2024 stable style

See issue for 2023 here: #3407

Here's what this issue is:

I've listed all the features we have enabled under --preview
...in descending order of how controversial I think they are
I've also drawn a dividing line; past "uncontroversial" is where I feel we don't need to think twice about it
I've listed the number of digits in the diff-shade lines changed count and linked original PR
I tried to list any open issues next to the relevant preview feature, but it's possible I missed something
- Issues tagged with preview I haven't yet associated with a feature: Preview style: Preserve empty lines between async def and async for #3614 and string_processing: makes it impossible to use a comment to ignore a static type violation #3802 (sorted these out–Jelle)

Here's the current call to action:

Any comments / opinions / vibes on these features is very welcome
Please test your codebase with --preview and let us know your thoughts!
If you have a specific problem with a preview feature, especially something that feels like an implementation issue, it's best to open a separate issue (that we can then link to from here)

For many of the changes, diff-shades logs have expired. We should find a way to keep these around for longer!

It's currently November, so there's still time to make more uncontroversial changes and hopefuly iron out some issues in the not uncontroversial ones.

Not uncontroversial

string_processing

This one is controversial, vast majority of preview style issues are related to string processing
Open issues I believe are caused by string_processing: #2334, #2314, #3466, #3497, #3409, #3424, #3428, #3210, #3410, #3747, #3665, #3663, #2553, #3855, #3802

hug_parens_with_braces_and_square_brackets (#3964, #3992)

Causes O(10000) changes, although counting lines changed is unfavourable to this change.
A very substantial change in style for Black. We can't revert once it's in stable.
Outstanding crash: #4036

parenthesize_conditional_expressions (#2278)

Causes O(1000) changes. Need to look at diff
~~Has unfortunate interactions with comments: #3555~~

wrap_long_dict_values_in_parens (#3440)

Causes O(100) changes. Need to look at diff
~~Outstanding crash: #3758~~
Report of worse splitting: #3452
~~Unnecessary nesting: #4129~~
Removes parentheses: #4158

multiline_string_handling (#1879)

Causes O(1000) changes. Need to look at diff
Maybe has some synergy with hug_parens_with_braces_and_square_brackets.
Unfortunate white space preservation: #4159

dummy_implementations (#3796)

I can't find up-to-date diff-shades, but maybe O(1000) changes
Requires a change in flake8 config: #3887
Can result in an extra line, but no change expected to be made: #3872

prefer_splitting_right_hand_side_of_assignments (#3368)

Causes O(100) changes.

hex_codes_in_unicode_sequences (#2916)

Causes O(100) changes
Open issue whether lowercase or uppercase is better #3568

allow_empty_first_line_before_new_block_or_comment (#3967)

Causes O(1) changes.
This change itself is fine, but it's related to a controversial and churn causing change in 2023 style. I think we should probably revert that 2023 change; I posted a longer issue about this over here #4043

parenthesize_long_type_hints (#3899)

Causes O(100) changes

module_docstring_newlines (#3932)

Causes O(100) changes

Uncontroversial

respect_magic_trailing_comma_in_return_type (#3916)

O(10) changes.
Shouldn't affect existing formatting too much, unless there's a buggy trailing comma
All changes in diff-shades are incidental, but are good and what you'd expect Black to do

no_blank_line_before_class_docstring (#3692)

O(100) changes. Makes consistent with function behaviour
Blank lines have been a little controversial, but I think this should be totally fine

wrap_multiple_context_managers_in_parens (#3489)

O(100) changes. Naively, looks great to me

fix_power_op_line_length (#3942)

O(10) changes. A bug fix

add_trailing_comma_consistently (#3393)

O(1) changes. A bug fix

skip_magic_trailing_comma_in_subscript (#3209)

O(1) changes. A bug fix, affects a non-default option

blank_line_after_nested_stub_class (#3564)

Only affects stubs, adds some blank lines in an uncommon case

blank_line_between_nested_and_def_stub_file (#3862)

Only affects stubs, adds some blank lines in even less common case

accept_raw_docstrings (#3947)

O(1) changes. Just a bug fix

long_case_block_line_splitting (#4024)

0 changes. Seems strictly good

walrus_subscript (#3823)

0 changes. Straightforward style change

improved_async_statements_handling (#3609)

0 changes. More consistent

single_line_format_skip_with_multiple_comments (#3959)

0 changes. Makes life better

My vibes:

I feel very comfortable with shipping dummy_implementations and lower
I think parenthesize_conditional_expressions / wrap_long_dict_values_in_parens / multiline_string_handling are also all good, but I want to look over diff and address the existing issues
I feel we shouldn't ship string_processing in its current state
I'm fairly wary of hug_parens_with_braces_and_square_brackets. It's a big change to Black's style! I'm very interested in hearing from folks on this one; if we don't feel like we get much feedback on it we should strongly consider keeping in --preview
I'll evaluate some more on large work codebase and post any findings

@hauntsaninja

My ¢2 as a Black user:

I generally agree with @hauntsaninja's vibes/analysis overall!
I ran Black with --preview on a few projects, and in those projects, I'm sad to say that I dislike most of the string-formatting-related changes the preview style make.
I really, really like hug_parens_with_braces_and_square_brackets and hope it gets in! One of my biggest gripes with black is how annoying its formatting of multiline frozensets etc. has been in the past, and this addresses that. The one thing I'd say is that it surprised me that the preview style does this:
```
-    return AnnotationStats(
-        *[random.randint(0, 1000) for _ in AnnotationStats.__annotations__]
-    )
+    return AnnotationStats(*[
+        random.randint(0, 1000) for _ in AnnotationStats.__annotations__
+    ])
```
I prefer the non-preview style there -- I believe this is basically the same thing @JelleZijlstra's is saying in Fix #4011: Handle more huggable immediately nested parens/brackets. #4012 (comment)

Thanks, this is great!

We should try to make a 24.1a1 soon that implements a draft stable style, so we can get more feedback.

I haven't thought too hard about each change, but a few thoughts:

It might be time to pull the plug on string_processing, since it's been years and the feature is still buggy. Possibly we can keep a more limited version, e.g. only merging adjacent string literals that are on the same line.
I like the hug_parens change personally but I appreciate it's going to be controversial. Especially since it went into the preview style pretty late in the year, it might be better to hold off on it for now and gather more feedback.

On string_processing:

We have been using it internally (Google) since the beginning of this year and it has been working well for us. So I'm motivated to keep this and help graduate it to the stable style (too late for the 2024 stable style though).

Possibly we can keep a more limited version, e.g. only merging adjacent string literals that are on the same line.

This is a good strategy, I think we can split string_processing into a few separate style changes. Moving all of them at once is hard to reason about and is too disruptive.

I tested preview on a code base of mine, and it mostly looked good. The only change I was really surprised by was from string processing; the same issue as #3663 and #3210 . A little testing showed that it's from the string_paren_strip transformer. I like the idea of that part of string_processing, but that's a pretty ugly downside. It seems like it's stripping the parentheses and then re-inserting them at a worse place, which is consistent behavior with non-preview style if I manually remove the parentheses first. Not strictly a problem with string_paren_strip, I guess, but not great.

For hug_parens, I like it in theory but it does feel less consistent than the current behavior, at least in some cases. I'm not totally sure about it. I can understand why you'd compare it and multiline_string_handling, but I don't see any issues with the latter - that one looks great to me.

I think that parenthesize_conditional_expressions is a big improvement for conditionals in the original context of function arguments, but I don't think I like what it's doing in other contexts.

I started putting together a PR to prepare an alpha for the 2024 stable style, and here's a few thoughts:

hex_codes_in_unicode_sequences: I think there are good reasons to prefer lowercase. I would propose to keep this out of the 2024 stable style and make the 2024 preview style use lowercase.
allow_form_feeds was just added (Permit standalone form feed characters at the module level #4021) as another preview feature. It should be low-impact, so I would favor adding it to the new stable style.
I wouldn't be comfortable shipping parenthesize_conditional_expressions and wrap_long_dict_values_in_parens with the open issues about them. Nevertheless, I'll include them initially so we can gather more feedback. It's better if we put something in the alpha and then walk it back, instead of adding a new style change after the alpha.
dummy_implementation is surprisingly disruptive in our test suite, but that's atypical code.
The test suite found a new crash with parenthesize_long_type_hints: Crash with preview style + line length 1 #4062
Reported suboptimal formatting with dummy_implementation: Preview style: Don't collapse dummy implementation with comment #4063

Is there a chance of module_docstring_newlines making it in? As far as I can see there has only been support for the feature since it was first requested in 2020 (#1872) and I haven't seen issues about it being opened since it was added to preview.

If it means anything it was also quickly picked up by ruff in astral-sh/ruff#7995 without any discussion about it being potentially controversial.

It seems to me to be controversial as it would create a lot of diffs across projects, but doesn't change styling in any way that users would say "hey, what's black doing there". I would really like to see this change make it in to standardize these newlines across the projects I work on (and which don't want to run with preview style).

@DanielNoord

@DanielNoord yes, that one is expected to go in (unless either lots of people speak up and say they don't like it, or we discover bugs caused by it).

I'm still writing it up but we track Ruff's preview style compatibility in astral-sh/ruff#8678
Ruff's stable style already includes some of the mentioned uncontroversial preview styles.

Regarding multiline_string_handling. Ruff stable already formats

MULTILINE = """
foobar
""".replace("\n", "")

and we've generally received positive feedback. I haven't yet come around to implement the special casing inside call expressions. So I can't say how people feel about that.

I personally like hug_parens_with_braces_and_square_brackets that that's probably because I'm biased from Prettier ;)

Ruff formats long dict values slightly different from black (wrap_long_dict_values_in_parens). Parenthesizing the values would align the behavior. Although it sets a new precedence of removing parentheses from sub-expression, something Black so far very explicitly avoids (in comparison to Prettier that normalises all parentheses). There's a similar issue around slice indexes that may benefit from parenthesizing as well astral-sh/ruff#7316

I'll share more feedback when I had time to take a closer look at the changes and thanks for drafting this up, it helps to understand how Black's styleguide will change.

Release notes states the following regarding #3445 :

Please try out the preview style with black --preview and tell us your feedback. All changes in the preview style are expected to become part of Black’s stable style in January 2024.

Isn't this a topic for this discussion?

@takanakahiko

@takanakahiko I'm not sure exactly what you're asking. This issue is definitely the right place to discuss what should go into the 2024 stable style. The default is that everything in the preview style will move to stable, unless there's some reason not to. As discussed above, string processing (which is what #3445 affects) probably will not go into the 2024 stable style because there are many open bugs around it.

@JelleZijlstra

@JelleZijlstra I may have had a problem with my English, sorry.

I thought all Preview styles were listed because it said the following.

I've listed all the features we have enabled under --preview

However, I was concerned that my interest #3445 was not listed, so I asked my question.

Thanks for the answer.
I understand that it will not be introduced in the stable style because it is related to the string processing issue.

I started putting together a PR to prepare an alpha for the 2024 stable style, and here's a few thoughts:
* The test suite found a new crash with `parenthesize_long_type_hints`: [Crash with preview style + line length 1 #4062](https://github.com/psf/black/issues/4062)

Fixed the crash, feel free to tag me if any other issues with either of the type-hint features I added cause any other problems :)

Here is my feedback. TLDR: I like the changes with --preview flag more than without it.

I was testing black 23.11.0 with --preview on https://github.com/Fatal1ty/mashumaro

I like this idea with Ellipsis on the same line (there is no longer a need to exclude it from coverage)

-    def decode(self, data: Any) -> T:  # pragma: no cover
-        ...
+    def decode(self, data: Any) -> T: ...

It found an unnecessary string concatenation:

         raise ValueError(
-            f"Time zone {s} must be either UTC " f"or in format UTC[+-]hh:mm"
+            f"Time zone {s} must be either UTC or in format UTC[+-]hh:mm"
         )

This became more readable:

         schema = JSONObjectSchema(
-            additionalProperties=_get_schema_or_none(
-                instance.derive(type=args[1]), ctx
-            )
-            if args
-            else None,
-            propertyNames=_get_schema_or_none(
-                instance.derive(type=args[0]), ctx
-            )
-            if args
-            else None,
+            additionalProperties=(
+                _get_schema_or_none(instance.derive(type=args[1]), ctx)
+                if args
+                else None
+            ),
+            propertyNames=(
+                _get_schema_or_none(instance.derive(type=args[0]), ctx)
+                if args
+                else None
+            ),
         )

Special thanks for this change, I like it a lot:

     serialize_by_alias: Union[
-        bool, Literal[Sentinel.MISSING]
-    ] = Sentinel.MISSING
-    namedtuple_as_dict: Union[
-        bool, Literal[Sentinel.MISSING]
+        bool, Literal[Sentinel.MISSING], float, List[str]
     ] = Sentinel.MISSING
+    namedtuple_as_dict: Union[bool, Literal[Sentinel.MISSING]] = (
+        Sentinel.MISSING
+    )

What interesting is that ruff turns all this changes back, so all the credits go to black!

I really like hug_parens_with_braces_and_square_brackets, hope it gets merged in 2024 so that we don't have to wait a year.
It really made code look better in all cases I saw.

As for multiline string processing, I am not a big fan, I find implicit concatenation of string literals somewhat confusing and would prefer single (but long) literal.

One case where I find it actively worse then it were:

Original:

assert x == "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo o oo  o oo o oo  o o o o o o oooo oo o oo ng text"

Old black: (clunky but ok)

assert (
    x
    == "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo o oo  o oo o oo  o o o o o o oooo oo o oo ng text"
)

New black (with preview): == is so unnoticable here

assert (
    x
    == "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo"
    " o oo  o oo o oo  o o o o o o oooo oo o oo ng text"
)

Ideal result (with new hugging style) considering we need to have splitting

assert x == (
    "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo"
    " o oo  o oo o oo  o o o o o o oooo oo o oo ng text"
)

Probably good enough is this:

assert (
    x == (
        "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo"
        " o oo  o oo o oo  o o o o o o oooo oo o oo ng text"
    )
)

Also, I think multiline string should almost always be in parens (except in cases where it's already in cases when they are already in parens from the function call. There are cases when it's not this way (in asserts as above), in list comprehensions:

original:

lst = [
    "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo o oo  o oo o oo  o o o o o o oooo oo o oo ng text"
    for line in range(5)
]

black with preview

 lst = [
    "looooooooooooooooooooooooooooooo o o o oo o o oo oo o o oo o oo o oo ooo o o oo o"
    " oo  o oo o oo  o o o o o o oooo oo o oo ng text"
    for line in range(5)
]

I hope this is the correct place to comment on this. I "accidentally" ran the alpha version on my project and I think the new dummy_implementations rule made the following change(s) (slightly simplified to highlight a certain type of change):

# Before
class Parser(typing.Protocol):
    def set_defaults(self, **kwargs: typing.Any) -> None:
        ...  # pragma: no cover

# After
class Parser(typing.Protocol):
    def set_defaults(
        self, **kwargs: typing.Any
    ) -> None: ...  # pragma: no cover

This doesn't seem to me to be much of an improvement in readability. I think I understand the intent of the rule to move the elipsis onto the same line as the def but doing so at the expense of then having to split the definition across multiple lines seems much less pleasant than just leaving it as-is IMO. Thanks for listening!

@phlogistonjohn

@phlogistonjohn that's issue #3872. As I wrote on the issue, I think it's still a net positive on balance, but there's definitely some risk, especially in cases where there is a comment after the ....

@AlexeyDmitriev

@AlexeyDmitriev string splitting won't be in the 2024 stable style.

(And for everyone commenting, thanks for your feedback!)

@JelleZijlstra

@JelleZijlstra thanks for your feedback.
What is the expected time when black 2024.* is published?

@JelleZijlstra

@JelleZijlstra I was looking at #4106, Do I understand correctly that parens hugging will not make it either?

We'll publish it by the end of the month and the hugging changes will not be included.

Sorry if these pieces have already been covered, but I ran the preview style (from 24.1a1) over a small portion of Meta's codebase and have some feedback. Most of the style changes seem like a positive change, but I think the changes to collapsing/compacting elements and especially multiline strings is a net-negative.

First some changes I'm happy to see:

# before
for (a, b) in things:
    ...

# after
for a, b in things:
    ...

# before
value: Optional[
    int
] = None  # some really long comment

# after
value: Optional[int] = (
    None  # some really long comment
)

Love it!

# before
func(
    arg1,
    "long string"
    if condition
    else "other string",
)

# after
func(
    arg1,
    (
        "long string"
        if condition
        else "other string"
    ),
)

Much clearer, exactly how I would have rewritten it myself! 💜

Now, some examples where I think the changes result in less-readable code, particularly where black takes aligned triple quotes and then unaligns them:

# before 
Thing(
    """
    value
    """
)

# after
Thing("""
    value
    """)

or worse:

# before
VALUE = [
    Thing(
        """
        value
        """
    )
]

# after
VALUE = [Thing("""
    value
    """)]

# before
func(
    (
        value
        + """
        something
        """
    ),
    argument,
)

# after
func(
    (value + """
        something
        """),
    argument,
)

Not pretty, and not really any more readable than before? ¯\(ツ)/¯

Is that related to the "paren hugging" you mentioned, or is this still scheduled for the 2024 style?

Is that related to the "paren hugging" you mentioned

Maybe not, because I just found this change as well:

# before
def foo():
    result = (
        """
        something
        """
        + value
    )

# after
def foo():
    result = """
        something
        """ + value

Distinctly less readable IMO.

@amyreese

Update:

I am going to take out wrap_long_dict_values_in_parens, because Suboptimal line split choices with wrap_long_dict_values_in_parens #3452 is still open (and I just added wrap_long_dict_values_in_parens can remove parentheses #4158)
@amyreese's bad examples (thanks for reporting these!) are from multiline_string_handling, Improve multiline string handling #1879. I opened multiline_string_handling sometimes leads to worse formatting #4159 about your feedback. I'm inclined to take out this feature from the new stable style, but I'm open to persuasion otherwise; I think most changes it makes on my codebase are good, but it may be highly dependent on how individual codebases use multiline strings.
dummy_implementations might be the most controversial remaining change, but I think it's a net positive.
hug_parens_with_braces_and_square_brackets: Doesn't hug if all entries fit on a line #4099, blank_line_after_nested_stub_class: blank line got added after a nested class with ... #4119, allow_empty_first_line_before_new_block_or_comment can lead to inconsistent formatting #4121 have some valuable feedback, but I don't think they should lead to any changes.

py-black: update to 23.12.0.

Navigation Menu

Repository navigation

Setting the 2024 stable style #4042

Not uncontroversial

string_processing

hug_parens_with_braces_and_square_brackets (#3964, #3992)

parenthesize_conditional_expressions (#2278)

wrap_long_dict_values_in_parens (#3440)

multiline_string_handling (#1879)

dummy_implementations (#3796)

prefer_splitting_right_hand_side_of_assignments (#3368)

hex_codes_in_unicode_sequences (#2916)

allow_empty_first_line_before_new_block_or_comment (#3967)

parenthesize_long_type_hints (#3899)

module_docstring_newlines (#3932)

Uncontroversial

respect_magic_trailing_comma_in_return_type (#3916)

no_blank_line_before_class_docstring (#3692)

wrap_multiple_context_managers_in_parens (#3489)

fix_power_op_line_length (#3942)

add_trailing_comma_consistently (#3393)

skip_magic_trailing_comma_in_subscript (#3209)

blank_line_after_nested_stub_class (#3564)

blank_line_between_nested_and_def_stub_file (#3862)

accept_raw_docstrings (#3947)

long_case_block_line_splitting (#4024)

walrus_subscript (#3823)

improved_async_statements_handling (#3609)

single_line_format_skip_with_multiple_comments (#3959)

10 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Setting the 2024 stable style #4042

Description

Not uncontroversial

string_processing

hug_parens_with_braces_and_square_brackets (#3964, #3992)

parenthesize_conditional_expressions (#2278)

wrap_long_dict_values_in_parens (#3440)

multiline_string_handling (#1879)

dummy_implementations (#3796)

prefer_splitting_right_hand_side_of_assignments (#3368)

hex_codes_in_unicode_sequences (#2916)

allow_empty_first_line_before_new_block_or_comment (#3967)

parenthesize_long_type_hints (#3899)

module_docstring_newlines (#3932)

Uncontroversial

respect_magic_trailing_comma_in_return_type (#3916)

no_blank_line_before_class_docstring (#3692)

wrap_multiple_context_managers_in_parens (#3489)

fix_power_op_line_length (#3942)

add_trailing_comma_consistently (#3393)

skip_magic_trailing_comma_in_subscript (#3209)

blank_line_after_nested_stub_class (#3564)

blank_line_between_nested_and_def_stub_file (#3862)

accept_raw_docstrings (#3947)

long_case_block_line_splitting (#4024)

walrus_subscript (#3823)

improved_async_statements_handling (#3609)

single_line_format_skip_with_multiple_comments (#3959)

Activity

hauntsaninja commented on Nov 13, 2023

AlexWaygood commented on Nov 13, 2023

JelleZijlstra commented on Nov 14, 2023

yilei commented on Nov 14, 2023

tungol commented on Nov 15, 2023

JelleZijlstra commented on Nov 21, 2023

DanielNoord commented on Nov 23, 2023

JelleZijlstra commented on Nov 23, 2023

MichaReiser commented on Nov 28, 2023

takanakahiko commented on Nov 28, 2023

JelleZijlstra commented on Nov 28, 2023

takanakahiko commented on Nov 28, 2023

jakkdl commented on Dec 6, 2023

Fatal1ty commented on Dec 7, 2023

10 remaining items

AlexeyDmitriev commented on Dec 28, 2023

phlogistonjohn commented on Jan 3, 2024

JelleZijlstra commented on Jan 11, 2024

JelleZijlstra commented on Jan 11, 2024

AlexeyDmitriev commented on Jan 12, 2024

AlexeyDmitriev commented on Jan 12, 2024

JelleZijlstra commented on Jan 12, 2024

amyreese commented on Jan 13, 2024

amyreese commented on Jan 13, 2024

JelleZijlstra commented on Jan 20, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions