#9120 (comment)

I haven't had a chance to really look at and grok how the units framework works. I suppose it's possible that units are inferred from type and that everything gets translated into a number under the hood and plotted on the same plot,

Thats indeed what happens; datetimes are translated to an ordinal day (days since 1 Jan 0000 plus one). So the proposed behaviour for time deltas in bar plots (for instance) is to translate them to days so they can be relative to an actual datetime value that anchors the bars.

I don't think the approach in#9120 will help much - users are going to supply lists that mix datetime and timedelta objects, so this PR just decides on an element by element basis what to do w the object.

This is ready for review

I am not sure how much this addresses my main concerns in#9120. My biggest issue is thattimedelta anddatetime are fundamentally different creatures, despite the fact that they both have to do with time.

Consider:

1. It doesn't make sense to plotdatetime andtimedelta objects on the same axis, since one is an absolute position and the other is a delta.
2. The default axis fortimedelta and fordatetime should be completely different. Fortimedelta, you expect an elapsed-time axis denoted in minutes, seconds, hours, days, etc. Fordatetime you expect absolute dates.
3. Basically all the special time-handling stuff isdatetime specific. It has to handle weird calendrical arithmetic, time zones and a number of other funky things.timedelta can be losslessly converted to a float withtotal_seconds() and converted back totimedelta withtimedelta(seconds=x).

I think thereare complications that you'll run into, particularly having to do with units (and compatibility withnp.timedelta64), but they're completely different from the kinds of complications you expect withdatetime, which is further evidence for not conflating the two abstractions into one.

@pganssle Unfortunately, we have lots of API that require x and delta x be specified in the same tuple or array, and the units machinery is just called on the whole tuple and array. That works fine for most data (i.e 30 kg +/- 2 kg are both the same units...) but time is time, and in order to support some functions we need to be able to process both t and dt in the same call todate2num((t, dt)). Refactoring those functions to make different calls to the units machinery seems like a far heavier lift than detecting the types here and doing the right thing.

@jklymak What do you mean by "the right thing"?

If I doplt.plot([1], [datetime.timedelta(3)]), what would be plotted? What axis would be used?

What happens if I do this:

plt.plot([1,2], [datetime.timedelta(hours=3),datetime.datetime(2018,1,1)])`

Will it throw an exception?

I realize that this is not a simple problem to solve, but if you merge something that just forcesdatetime andtimedelta into the same units automatically, you are creating a situation that is hard to reverse without breaking backwards compatibility. It is worth taking the time to get this right.

(i.e 30 kg +/- 2 kg are both the same units...)

My suggestion for handling spans and other situations like this is the same as it was in#9120. I believe these problems would go away if you did unit conversionsafter any arithmetical operations. By converting everything to float before calculating spans and such, you lose all custom__add__ behavior and the like, which means you can't have a span that is arelativedelta or any other interesting custom behavior.

I strongly recommend against treatingtimedelta and its ilk like adatetime for the short-term expedience of being able toignore the unit analysis.

The first will have a point at t = 1 Jan 0000, 03:00, and the second will have a point there and at 2018. I don't consider thatso bad, though I agree the user shouldn't mix the vectors in this case, but, garbage in, garbage out.

My suggestion for handling spans and other situations like this is the same as it was in#9120. I believe these problems would go away if you did unit conversionsafter any arithmetical operations.

That would be great, but thatsnot how unit support works right now, and would require alot of engineering to make it work that way.@dopplershift has been thinking about these sorts of issues, and its on our radar to fix units in a "better" way. Though to be honest, the idea of assuming variable 'x' inplot has an__add__ method might be a heavy ask.

I strongly recommend against treatingtimedelta and its ilk like adatetime for the short-term expedience of being able toignore the unit analysis.

Any major changes to the units machinery is likely tonot be backwards compatible anyways. Given that, and the fact such an overhaul is not being actively worked on right now, I think making the current system work better is the less of two evils. I certainly agree with your main point, but practically I still think this PR is specifically helpful.

BTW, I think folks would be willing to have someone dive deep on units if you are interested, but it'd require a MEP first to make sure we all think it is heading in the right direction.

The first will have a point at t = 1 Jan 0000, 03:00, and the second will have a point there and at 2018. I don't consider that so bad, though I agree the user shouldn't mix the vectors in this case, but, garbage in, garbage out.

Both of these are decidedly the wrong thing. The first one isn't "garbage in", it's a very reasonable thing to do, actually. There are many experiments where the "time" axis is "time elapsed" not "absolute time" (includingnearly everything I've ever published). Real support for that in the form of a "TimeDeltaLocator" or "ElapsedTimeLocator" would be great.

The second one is nonsense and should either throw an exception or plot some garbage, depending on how it's usually handled when you inappropriately mix units in a single axis.

Any major changes to the units machinery is likely to not be backwards compatible anyways. Given that, and the fact such an overhaul is not being actively worked on right now, I think making the current system work better is the less of two evils. I certainly agree with your main point, but practically I still think this PR is specifically helpful.

I think it's fine to make incremental improvements, but this is the wrong way to do it. I don't have time to dig into this right now, but I think you're in a hole at the moment and the correct thing to do is to stop digging.

If you're looking to fix things like#11290 and#12862, you shouldn't do it in such a way thatalso makes fixing#8869 correctly a backwards-incompatible change.

I haven't understood all the nuances here, but the problem seems to stem from the fact thattimedelta has valid uses standalone (e.g. should just be another registered type for the unit conversion machinery) and has uses as one of two classes used arithmetically with dates and times.

Is there a particular reason why we're favoring the latter use case over the former? As far as I can tell, we're not successfully solving both use cases here.

OK, I hadn't checked, but, because we aren't registering timedeltas of either flavours the date converter does not get called if atimedelta is the first element of the array. So, as is, the PR doesn't block whatever needs to be done to make#8869 work.

If I addtimedelta to the registry, then there arestill errors, but since this is not meant to make#8869 work, I'm not planning to work on this.

Overall, its not clear to me that this PR precludes properly handling fulltimedelta arrays, and adapting a new Locator (though is that necessary?) and Formatter (probably more necessary) for that kind of data.

importmatplotlib.pyplotaspltfromdatetimeimportdatetime,timedelta,timezoneplt.scatter([timedelta(hours=3),timedelta(hours=4),timedelta(hours=5)], [1,2,3]);plt.show()

gives

TypeError: float() argument must be a string or a number, not 'datetime.timedelta'

plt.plot([1,2], [datetime(2018,1,1),timedelta(hours=3)])

gives

ValueError: view limit minimum -36834.61875 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units

(Actually, I'm not 100% sure why this happens)

Switching order yields:

plt.plot([1,2], [timedelta(hours=3),datetime(2018,1,1)])

TypeError: float() argument must be a string or a number, not 'datetime.datetime'

Which is all to say that there is still scope for backwards compatible changes...

@dopplershift sorry, cross post. But you are correct, we are not solving the vector-of-timedelta case. But we are not precluding it from working here either.

But we are not precluding it from working here either.

I feel nervous given how much code is floating around special casing all of the various date/time things in the ecosystem that we can make this statement for certain, given that we don't know how to solve#8869.

I'm not saying this is right or wrong, just that we should exercise caution when adding more duct tape to our mess here.

I feel nervous given how much code is floating around special casing all of the various date/time things in the ecosystem that we can make this statement for certain, given that we don't know how to solve#8869.

I think theright way to solve#8869and the various "datetime axes can't have a span" issues floating around is clear. It's just not the approach anyone has taken, and if the various people working on it are to be believed (a qualification I only put in place because Ipersonally have not been able to dig into this, I see no reason to doubt them), it's hard to do.

I would be alot more comfortable with a refactoring based on dispatch. Also, throwing this out there, but how about something like this as a temporary measure (using@dataclass because it makes example code shorter):

@dataclassclassDeltaWrapper(float):_value :floatdef__add__(self,other):returnself._value+other

Doesn't have to befloat and it probably needs to be somewhat more complicated than this, but the idea would be that in theplotting code you can detect the presence of a_DeltaWrapper and raise an exception if you try to plot it. In span code it will work as expected. If necessary, this could become a stepping stone to a version that looks like this:

@dataclassclassDeltaWrapper(float):_delta:object# The pre-conversion object_value :float# The post-conversion objectdef__add__(self,other):try:returnself._delta+otherexceptException:returnself._value+other

Then you can incrementally convert over to the unit framework performing arithmetic pre-conversion without it throwing errors. Once everything is changed over, you can dropDeltaWrapper as vestigial.

Perhaps an easy idea is to add aconvert_units_delta function that could go in parallel toconvert_units but did the right thing for deltas. By default, it could just callconvert_units. Then the task would be finding the places where a delta is expected and callingconvert_units_delta instead ofconvert_units.

This new commit is a different approach as described above... The converter gets a newdef convert_delta(value, units, axis) method thataxis.convert_units_delta(self, dx) tries to call if the method exists.broken_barh then simply goes through its x values and converts the first element in the tuple differently than the second (delta) value.

I've not changed broken_barv, or the other axes methods where we expect the second element in a tuple etc to be a delta, but this gets the concept across as a proof of concept.

Ah, OK, I see what the fundamental problem w/broken_barh was, but its not actually a problem for other functions with deltax or deltay (that I've seen).

The issue is thatbarh was doing the conversion on dx, whereas other functions indeed do the conversion onx+dx (i.e. errorbar or fill_between). So as@pganssle said above, we should do the math first and then convert. I mistakenly thought this issue was everywhere, but its not.

New PR on the way for the original issue. Closing this, but I'll open a new one based on this branch, since it does 75% of the work needed for#8869.

Thanks a lot@pganssle for your comments, which were largely correct.

I may have stumbled across an issue with the timedelta64 handling. The following produces a numpy SystemError:

import matplotlib.pyplot as pltimport matplotlib.dates as plt_dtimport pandas as pdfrom datetime import timedeltatimedelta_series = [timedelta(minutes=n) for n in range(0, 25, 5)]td_df = pd.DataFrame([1, 2, 3, 4, 5], timedelta_series, columns=["Val"])plt.plot(td_df.index, td_df.Val)ax = plt.gca()ax.xaxis.set_major_formatter(plt_dt.DateFormatter('%H:%M'))plt.show()

Looking into the error, it seems thatdates._from_ordinalf is receivingx as atimedelta64[ns] integer, which then produces a meaningless result when:

    dt = (np.datetime64(get_epoch()) +          np.timedelta64(int(np.round(x * MUSECONDS_PER_DAY)), 'us'))

This may only be a symptom rather than the root cause, and perhaps DateFormatter is not passing the expected value?

If it is the expected value being passed in this case, then the following would not produce the error...

    dt = (np.datetime64(get_epoch()) +          np.timedelta64(int(np.round(x), 'ns'))

... noting thatx is already in ns (rather than 'us').