Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork18.6k
BUG: Raise clear error for duplicate id_vars in melt (GH61475)#61484
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation
pandas/core/reshape/melt.py Outdated
| # GH61475 - prevent AttributeError when duplicate column | ||
| if not hasattr(id_data, "dtype"): | ||
| raise Exception(f"{col} is a duplicate column header") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
- This should check
if not frame.columns.is_uniqueat the beginning of the function. - A
ValueErroris more appropriate here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for the feedback! I've moved the check fornot frame.columns.is_unique to the beginning of the function and updated the exception type toValueError as suggested.
A quick clarification question: currentlymelt allows duplicate column names in 'value_vars', as seen in the testtest_melt_with_duplicate_columns.With this change, are we treating any duplicate columns in the inputDataFrame as aValueError? Not just when the duplicates appear inid_vars?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ah good point. I guess this specifically whenid_vars is not empty we'll want to raiseif not frame.columns.is_unique
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks! Just to make sure I understand:
You're saying that we should only raise an error in duplicate column names if the duplicate is inid_vars, correct? Essentially if the duplicates are invalue_vars then, we can let the melt function work as is, as long as no errors are occuring?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yup, correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I have updated logic to raise aValueErrorwhen there are duplicates columns in id_vars and not invalue_vars, as discussed and moved it to the beginning of the melt function.
I considered using theframes.columns.is_unique, but that would raise for any duplicate name - even invalue_vars which is supported. To keep the behavior consistent, I checked only for duplicate column names referenced inid_vars. Let me know if further adjustment is needed!
pandas/core/reshape/melt.py Outdated
| @@ -173,6 +173,9 @@ def melt( | |||
| 1 b B E 3 | |||
| 2 c B E 5 | |||
| """ | |||
| # GH61475 - prevent AttributeError when duplicate column in id_vars | |||
| if id_vars and any(frame.columns.tolist().count(col) > 1 for col in id_vars): | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
- Can you move this after the
id_vars = ensure_list_vars(id_vars, "id_vars", frame.columns)line - Specify this as
if id_vars and not frame.columns.is_unique and any(id_var in frame.column for id_var in id_vars):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If I addednot frame.columns.is_unique at the beginning of the function, wouldn't it causetest_melt_with_duplicate_columns to fail, since that test currently allows duplicate column names invalue_vars?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'm not sure. You'll need to run the test to find out
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I added not frame.columns.is_unique at and it causedtest_melt_with_duplicate_columnsto fail since that test currently allows duplicate column names invalue_vars, so we might need to handle duplicates differently since we allow certain duplicates.
mroeschkeMay 29, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ah ok you'll need to check that theid_var values is actually a duplicate in the columns.
You can uselen(frame.columns.get_indexer_for(id_vars)) > len(id_vars)
Uh oh!
There was an error while loading.Please reload this page.
8981a0f intopandas-dev:mainUh oh!
There was an error while loading.Please reload this page.
Thanks@ZanirP |
Uh oh!
There was an error while loading.Please reload this page.
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.