Please document this in What's New athttps://docs.python.org/3.12/whatsnew/3.12.html#deprecated

And is this operator documented in the main reference? Let's also mention the deprecation there.

Is the plan to turn this into an error in 3.14, or keep it open ended?

We should probably add a comment about this tohttps://github.com/python/cpython/blob/main/Doc/whatsnew/3.12.rst.

We should probably add a comment about this tohttps://github.com/python/cpython/blob/main/Doc/whatsnew/3.12.rst.

Yes!

I've added the deprecation notice to whatsnew.

Is the plan to turn this into an error in 3.14, or keep it open ended?

I suggest to turn this into an error (and have indicated so in the whatsnew). There is a real danger that some users writeif ~condition (which is always True). Warnings are often overlooked, so it's better to eventually error on this. In contrast, I can hardly imagine any intended use for~some_bool, so this error should not annoy any rightful users.

The new behavior should also be mentioned inhttps://docs.python.org/3.12/library/stdtypes.html#boolean-values

It currently says "In numeric contexts (for example when used as the argument to an arithmetic operator), they behave like the integers 0 and 1, respectively.", which will no longer be true in all contexts.

I suggest to not document this as it would become more confusing than helpful:

In my view, there should be no reason to use bitwise operators onbool; one should always use the boolean operators (or convert toint explicitly). I therefore consider all bitwise operators ofbool an implementation detail. While this is public API, it's not something we should advertize.

In particular, if we document~, we'd also need to leave some words on&,| and^. And then it gets quite messy. If they operate onbool, they returnbool. In contrast,~ was returningint, will now warn and will error out in the future. Changing to returningbool for~ meaning a logic negation was rejected in the original issue discussion as too much of an API break (still working but with changed behavior). If I had to document this correctly this would be:

The binary bitwise operators&,| and^ on twobools return abool and behave like their logical equivalents. When applying these binary bitwise operators to mixedbool andint arguments, they return an int and interpret thebool as the underlyingint (i.e. 0 or 1). The bitwise negation~ of a bool currently returns the bitwise complement of the underlyingint. This is deprecated and will raise an error in 3.14. Generally, the use of bitwise operators on bools is discouraged. Use the logical operatorsand,or,not andxor instead, or convert toint explicitly.

But maybe you have better ideas what to document.

The new behavior should also be mentioned inhttps://docs.python.org/3.12/library/stdtypes.html#boolean-values
It currently says "In numeric contexts (for example when used as the argument to an arithmetic operator), they behave like the integers 0 and 1, respectively.", which will no longer be true in all contexts.

I suggest to not document this as it would become more confusing than helpful:

In my view, there should be no reason to use bitwise operators onbool; one should always use the boolean operators (or convert toint explicitly). I therefore consider all bitwise operators ofbool an implementation detail. While this is public API, it's not something we should advertize.

Whether we like it or not, these operators have worked this way for a long time in Python, and backwards compatibility alone means we can't just make them go away. So it's better to document clearly how they work and what is changing.

In particular, if we document~, we'd also need to leave some words on&,| and^. And then it gets quite messy. If they operate onbool, they returnbool. In contrast,~ was returningint, will now warn and will error out in the future. Changing to returningbool for~ meaning a logic negation was rejected in the original issue discussion as too much of an API break (still working but with changed behavior).

I think the whole reason that we're deprecating only the~ operator is that it's unlike the others: the results of&,|, and^ make sense whether you think of bools or ints, it's only~ that turns something from a bool into a non-bool int.

If I had to document this correctly this would be:

The binary bitwise operators&,| and^ on twobools return abool and behave like their logical equivalents. When applying these binary bitwise operators to mixedbool andint arguments, they return an int and interpret thebool as the underlyingint (i.e. 0 or 1). The bitwise negation~ of a bool currently returns the bitwise complement of the underlyingint. This is deprecated and will raise an error in 3.14. Generally, the use of bitwise operators on bools is discouraged. Use the logical operatorsand,or,not andxor instead, or convert toint explicitly.

But maybe you have better ideas what to document.

Your wording sounds good to me!

@JelleZijlstra thanks for the feedback. I've rewritten and restructured the bool docs a bit so that it's hopefully more clear and still precise. Please check if this is ok. - It's in a seprate commit, so could be easily modified/reverted.

In particular, I've created a top-level section "Boolean Type - bool", which does the importance of the type more justice.
Before, the bool description was only in documented in the section "Other Built-in Types / Boolean Values". I've moved most of the description over from there, but left the section as a short stub; both because (1) I'm unclear whether we still need "Boolean Values" explicitly in addtion to "Boolean Type", and (2) because that keeps the link so that internal and external references are not broken..

I think I'd prefer to remove theBoolean Values section. We can preserve the.. _bltin-boolean-values: andindex roles, which should hopefully preserve links.

You mean moving them to the new "Boolean Type" section?

Yup!

I think I'd prefer to remove theBoolean Values section. We can preserve the.. _bltin-boolean-values: andindex roles, which should hopefully preserve links.

HTML links to the section cannot be preseved the URL washttps://docs.python.org/3.12/library/stdtypes.html#boolean-values and the anchor#boolean-values is constructed from the section title, not from the label.

I've rewritten internal references (because the label_booltype is more in line with the labeling conventation in the file than_bltin-boolean-values). And the movedindex roll works.

Is this waiting for anyone?

Not anymore. Thanks all!

IMHO deprecating the~ operator while keeping&,| and^ seems (very?) inconsistent to me.
Asbool is a subclass ofint one could expect that~bool(0) returnsTrue, sadly it returns-1 ...
As a Hardware guy I see thebool as a constrainedint with a bit width of 1, with 0 being equivalent to False and 1 to True, so the negation~ would simply flip the bit. Especially as e.g.int(bool(1e6)) returns1
With this you are breaking a lot ofMyHDL code and probably a lot of others too.
I guess if you fix~bool() less code will get broken, possibly none at all. I can hardly imagine anyone relying on~bool(0) returning-1 and~bool(1) returning-2

Asbool is a subclass ofint one could expect that~bool(0) returnsTrue, sadly it returns-1 ... As a Hardware guy I see thebool as a constrainedint with a bit width of 1, with 0 being equivalent to False and 1 to True, so the negation~ would simply flip the bit. Especially as e.g.int(bool(1e6)) returns1

Unfortunately, this is not howbool was implemented. It's a full integer underneith. This means that be bit inversion of True (0000 ... 0001) is 1111 ... 1110. This in particular means thatbit inversion does not behave as a logical negation:bool(~True) is True. This is a major pitfall and the motivation for the deprecation. To be clear:If you have used~ as a logical negation, your code is broken.

IMHO deprecating the~ operator while keeping&,| and^ seems (very?) inconsistent to me.

The other bitwise operations happen to behave like the logical variants, e.g.bool(a & b) == a and b. So, while not recommended (The logical and operator isand, not&), there's not the danger of getting unexpected results. To keep changes minimal, we've kept these operators.

With this you are breaking a lot ofMyHDL code and probably a lot of others too.

As above, if you have used~ as logical negation, the existing code does not do what you think. The deprecation should make you aware of this. Please switch to the logical negationnot.

It is not quite true that "If you have used ~ as a logical negation, your code is broken."

Using bitwise operations has some advantages over logical operations, e.g., in the context of MyHDL package, which is for hardware design. Not matter how many bits are in signals, we can write the same expression. We just need to mask out unnecessary bits in the end.

output_1bit = (a & ~b) & 1output_4bits = (x & ~y) & 0xF

Edit: I did more testing. It seems the above code actually works. I only saw warnings on (single) bit indexing.

Using bitwise operations have some advantages over logical operations

Agreed. But to have and advantage of bitwise, your data should have multiple bits, i.e. the underlying data type should beint notbool. Otherwise you're implicitly casting from bool to int somewhere in your expressions. In general I would claim: If you have bool data, use logical operators, if you have int data, you can use the bitwise operators. And do any typecasts explicitly.

In that sense, inputs to your examples should be ints.

Remark: While I agree that it would be nice if~b would return the negated True/False on a bool, we decided against that for safety. Unfortunately, the current behavior is different and switching to another working behavior was considered too dangerous, see the original discussion in (#82012).

I have been unavailable this Sunday, so I will respond chronologically

@timhoffm
#103487 (comment):

Unfortunately, this is not how bool was implemented. It's a full integer underneith.

So it is basically flawed? (rhetorical question ...) Abool that nonetheless pretends to be anint?

The other bitwise operations happen to behave like the logical variants, e.g. bool(a & b) == a and b

Oh, we can get away with this ...
Actually CPython takes care to return abool - see the code later

In MyHDL (which is 100% Python) this:

a=Signal(intbv(0)[2:])y.next= (~a[1]&a[0])| (a[1]&~a[0])

a holds a constrained unsigned integer with bit width 2; indexinga returns abool
y holds abool

Above code simulates (== running the actual code as it is Python) returning correct results - so it is not broken, but we will have to fix it ...

For the record: above codes comes from@zhijieshi; I always usenot,and,or; as in VHDL , but in Verilog it is~,&,| ; having bothstyles in Python can be / is confusing; but IMNSHO they should act the same.

@zhijieshi
#103487 (comment)
Your final & 1 is not always necessary, but it is better to be safe than sorry.
It will have a time penalty in the simulation; the conversion to either Verilog or VHDL will also have this superfluous finaland but the synthesizer will remove that.
Yes, the only issue is thesingular case as:

y.next=~r

But MyHDL will throw a ValueError as -1 nor -2 will fit into the destination; be it anintbv(0)[1:] or abool()

@timhoffm
#103487 (comment)

But to have and advantage of bitwise, your data should have multiple bits, i.e. the underlying data type should be int not bool

You are defending the choice of deprecating the~ operator onbool instead of fixing it.
@gvanrossum words:
#82012 (comment)

Probably when we introduced bool we should have thought harder about it, but I don't think we should change anything at this point, so I'm not sure why whether it's worth trying to uncover the original deep motivations (probably they weren't so deep).

Withbool being asubclass ofint we can expect thatbool to follow theint rules. You say that there is an unsafe case because of the actual implementation ofbool and there is. I am quite sure that the followers of~ will have encountered that; in fact@zhijieshi has, so he added the and 1 to protect himself.

I took a look into the GitHub repo:

/* Arithmetic operations redefined to return bool if both args are bool. */staticPyObject*bool_invert(PyObject*v){if (PyErr_WarnEx(PyExc_DeprecationWarning,"Bitwise inversion '~' on bool is deprecated. This ""returns the bitwise inversion of the underlying int ""object and is usually not what you expect from negating ""a bool. Use the 'not' operator for boolean negation or ""~int(x) if you really want the bitwise inversion of the ""underlying int.",1)<0) {returnNULL;    }returnPyLong_Type.tp_as_number->nb_invert(v);}staticPyObject*bool_and(PyObject*a,PyObject*b){if (!PyBool_Check(a)|| !PyBool_Check(b))returnPyLong_Type.tp_as_number->nb_and(a,b);returnPyBool_FromLong((a==Py_True)& (b==Py_True));}staticPyObject*bool_or(PyObject*a,PyObject*b){if (!PyBool_Check(a)|| !PyBool_Check(b))returnPyLong_Type.tp_as_number->nb_or(a,b);returnPyBool_FromLong((a==Py_True) | (b==Py_True));}staticPyObject*bool_xor(PyObject*a,PyObject*b){if (!PyBool_Check(a)|| !PyBool_Check(b))returnPyLong_Type.tp_as_number->nb_xor(a,b);returnPyBool_FromLong((a==Py_True) ^ (b==Py_True));}

You can fix it by:

staticPyObject*bool_invert(PyObject*v){returnbool_xor(v,Py_True);}

(I am aware it may not be as simple as that, but I guess I am pretty close ...)
There may be more work to do as e.g.

>>>~False&True1

I would have expected to seeTrue as the answer not1, but as the~ results in anint we get an 'int' regardless that we supplied twobool

This fix will break far less code than the deprecation, if any at all; certainly not@zhijieshi 's
Like I said before: there should be no one in the whole world relying on~False returning-1 and~True returning-2

@josyb thanks for your feedback. Allow me to answer on a higher level. I think we're drifting too much into details. Yes, the current implementation is flawed. (*)

I am quite sure that the followers of ~ will have encountered that; in fact@zhijieshi has, so he added the and 1 to protect himself.

Good job@zhijieshi, but I'm not that optimistic. It's easy to writeif ~state:, which unexpectedly is always fulfilled. You'll find lots of such cases digging throughhttps://github.com/search?q=language%3Apython+%2Fif+%7E%5Cw%2F&type=code. APIs must be easy to use right and hard to use wrong.

Starting from the current bool implementation, we have three options:

1. leave everything as is
2. change the behavior so that~True is False
3. deprecate and remove~ onbool

All three options have their downsides which have been discussed in#82012: (1) is prone to misuse and can lead to unnoticted incorrect results; (2) has conceptual arguments related to expected behavior as aint subclass against it. Also, changing behavior is risky: If somebody relied on the exact behavior - rare but not to be excluded - they will get a different (=wrong) result, possibly without noticing (errors should never pass silently). OTOH if you remove the behavior (3), any previous users will be informed. We assumed that the majority of these are incorrect uses and intended correct usage is rare.

So there are trade offs to be made. I'm unclear whether your case makes a difference in the conclusion. That's beyond my level and I leave that to the core devs.

(*)Theoretical background: The issues we see are caused by a variant of thecircle-ellipse problem. The base classint implements~ as bit inversion of the number in two's complement representation, or mathematically equivalent:$x \rightarrow -(x+1)$. This is reasonable (even though the actual bit representation is different due to infinite precision, but that does not matter here). The problem occurs now for the derived classbool: We logically considerbool as equivalent to anunsigned 1-bit integer. But the inherited two's complement semantics of~ assumes signed integers, in particular$~1 = -2$. We cannot simultaneously fulfill the Liskov principle for bool as an int subclass, and maintain the unsigned 1-bit integer semantics for bool. To put it simple:bool is not suited as a subclass forint.

@zhijieshi

Using bitwise operations has some advantages over logical operations, e.g., in the context of MyHDL package, which is for hardware design. Not matter how many bits are in signals, we can write the same expression. We just need to mask out unnecessary bits in the end.

output_1bit= (a&~b)&1output_4bits= (x&~y)&0xF

There should be no need to mask out the unused bits;intbv handles that by itself but unfortunately thebool doesn't.

@timhoffm
Sorry, I am a but low level guy; making your hardware ...

You'll find lots of such cases digging throughhttps://github.com/search?q=language%3Apython+%2Fif+%7E%5Cw%2F&type=code.

I checked several examples of that list. It looks to me that they all test for the actual variable being 0. Theyget away by using the~ although I agree 100% that should have usednot. They will all be (rudely) awakened by the deprecation message.
(being a positive guy?) I actually most of the times avoid usingnot and write code like:

ifs.some_condition:# a comment why there is nothing to do here (which incidentally also documents what is done in the other case)passelse:do_this()

in stead of:

ifnots.some_condition:do_this()

OTOH if you remove the behavior (3), any previous users will be informed.

Same can be said for 2. -> you could do something fundamental: remove theint base class frombool andinform any user of~something_bool that the behavior will change in the future. The user can inspect the code and decide how or what to mitigate. I understand that this too much to ask :)

Let's try to wind down.
The final outcome is thatclass bool is flawed. But we can get away by restricting the use of it.
Of course this decision ripples down to MyHDL (in our case ).
If we desperately want to keep the ~, &, | and ^ operators for single bit values we would have to implement a new classbit to replacebool but that would breakall MyHDL code. Reminds me of the infamous Python 2 to 3 step ...
And it will have an impact on the simulation speed, as I expect that the CPythonbool will be faster than a Python classbit
I am a strong believer innot breaking code - something where the open source has no issue with. So we will have to stick withbool and the deprecation message in 3.12. I am slightly worried
I have always felt that the original MyHDL author and later maintainers made a wrong decision to use as much as possible native Python objects to represent hardware constructs - the flaw inbool underlines this. Funny that it took more than 20 years to show up, both for MyHDL as for Python itself...

You'll find lots of such cases digging throughhttps://github.com/search?q=language%3Apython+%2Fif+%7E%5Cw%2F&type=code.

I checked several examples of that list. It looks to me that they all test for the actual variable being 0. Theyget away by using the~ although I agree 100% that should have usednot. They will all be (rudely) awakened by the deprecation message.

The majority of the above cases is likely valid and not affected: Note that e.g. numpy provides its ownnp.bool_ type, which interprets~ as negation, so that e.g.~np.isfinite(5) is not affected. However, there are obvious cases likethis andthis, which hopefully will be awakened because that code is broken.

OTOH if you remove the behavior (3), any previous users will be informed.

Same can be said for 2. -> you could do something fundamental: remove theint base class frombool andinform any user of~something_bool that the behavior will change in the future. The user can inspect the code and decide how or what to mitigate.

It's not quite the same. Very few people read release notes and quite some people ignore warnings. So an announced change may well slip through unnoticed. While ignoring warnings is a flaw, we try to also care for the less able/experienced users. Eventually breaking code is the ultimate way to force them to take notice. - In that sense, removing functionality is safer than changing behavior.

If we desperately want to keep the ~, &, | and ^ operators for single bit values we would have to implement a new classbit to replacebool but that would breakall MyHDL code.

Not necessarily. There might be other ways around. As the author of the deprecation, I'd be willing to take a look at that, if you point me to two or three representative examples that now give warnings. But that's getting off-topic here. Let's do that inmyhdl/myhdl#429.

It's not quite the same. Very few people read release notes and quite some people ignore warnings. So an announced change may well slip through unnoticed.

I meant issue aFunctionality Change Warning like theDeprecation Warning not just adding it to the release notes.

Not necessarily. There might be other ways around. As the author of the deprecation, I'd be willing to take a look at that, if you point me to two or three representative examples that now give warnings.

I read the mail before going to bed andbetween sleeping and waking found a possible way out: as in MyHDL single hardware connections are declared as eitherSignal(False) orSignal(bool(0)) we couldsilently replace thebool by a newbit class; the only pitfall may be thead hoc use of abool as a variable but there probably aren't that many users practicing this. And maybe we adopt aDeprecation Warning. It will be a bit of work, though. And we might just accept the reduced functionality ofbool.
So far the only example is the one produced by@zhijieshi, but perhaps one or two will pop up when we test the MyHDL package against Python3.12 which will happen soon.

As you indicate we can close here.

Best regards,
Josy

P.S. I started checking MyHDL against 3.12, and get quite a few deprecation warnings ...

Since there is no real use-case for the current behavior

I can hardly imagine anyone relying on~bool(0) returning-1 and~bool(1) returning-2

In contrast, I can hardly imagine any intended use for~some_bool, so

Like I said before: there should be no one in the whole world relying on~False returning-1 and~True returning-2

You've broken indexing. I've just been bitten by this change.

I sometimes want to get the first or second value of a list based on a Boolean condition. Naturally, I usemylist[cond].

And I sometimes want to index from the end. Python supports negative indexes, so naturally I usemylist[~0] for the last element and in generalmylist[~i] for indexi from the back. This is very useful.

Now I needed both, so naturally I usedmylist[~cond]. And got a DeprecationWarning. Now I need to writemylist[~int(cond)] for no good reason? I'm not happy.

Thanks for the feedback. I‘m sorry this affects your code.mylist[cond] andmylist[~i] are unambiguous and not affected by the deprecation.

I regard the inferredmylist[~cond] as problematic. While the current implementation „bitwise inversion of the underlying int“ is what you wanted, there are numerous examples that users have interpreted~ as boolean negation. They would writemylist[~cond] to try and get the 2nd or first element of the list (and not necessarily realize that‘s not what they get). Also they would interpret your technically correct code wrongly. In the face of these issues, I still believe that the depreciation is justified and it‘s better to be explicit and writemylist[~int(cond)] when you need it. I‘m sorry for the inconvenience that you have to adapt your code, but it’s in the interest of preventing potential misuse of the pattern and making Python a more intuitive and safe language for all.

I have two things I would like to say.

Firstly, from reading the comments here, I feel like very little consideration was taken into people that actually do make use of the~ operator in their code. I'm here because my code broke because of this change. A long time feature of Python has been that you have always can interchangably use Booleans and integers, for exampleisinstance(True, int) isTrue. This change creates an awkward tear in this close relationship between integers and Booleans. I've identified 3 cases where my code breaks. Code involving bitmasks, code making use of~ for "reverse indexing" of a list, and the codegolf trick of using-~x to incrementx by 1.

Secondly, to anyone whose code broke because of this change. The easiest workaround that I've found is to switch out all~ with~+. This can safely be done with a simple search replace. The reason why this fixes the issue is that the unary plus operator converts Booleans into int, while not affecting ints (also it doesn't affect more exotic data types likenp.uint8). Another benefit of using this workaround over~int(x) is that~int(x) can cause "silent errors" in the case wherex happens to be for example a float.

Thanks for the feedback. I'm sorry that this broke your code. Let me assure you that the effect of this change has been carefully considered. There were good reasons to change and also good reasons not to change. In the end, it was a trade-off decision. Breaking some rare justified usage and violating the Liskov substitution principle vs. having an API that is prone to misuse and whose misuse is not easily detected. See#82012 (comment)

The broken cases I've seen so far were either real bugs or cases that were technically correct but could be written more clearly without bitwise inversion of bools. I'm happy to discuss your cases if you want to.

See also the discussionhttps://discuss.python.org/t/bool-deprecation/62232