Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork187
Fix manifest parsing#250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| if name == 'all': | ||
| raise ValueError('a group named "all" is not allowed ("all" is reserved for selecting the full set of declared benchmarks)') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This should still be the case. So the Pyston benchmarks' manifest needs to be fixed.
| if entries is None: | ||
| if name in self._raw_groups: | ||
| return | ||
| self._raw_groups[name] = None | ||
| elif name in self._raw_groups and self._raw_groups[name] is not None: | ||
| raise ValueError(f'a group named {name} was already defined') | ||
| self._raw_groups[name].extend(list(entries) if entries else []) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This rule should apply for a single manifest, with the exception only for includes (which should be merged like you are doing). I expect the logic for distinguishing between the two cases is lacking, either in `_parse_manifest() below or in the code that consumes its output.
cbbdc0e tob4871bdCompare
I'm a little unclear about how the manifest parsing is supposed to work, so I'm not sure this is the correct solution.
To reproduce the bug, use a "meta" manifest file that loads both the pyperformance suite and the pyston python-macrobenchmarks suite (as we do in our infra for the faster-cpython project):
There seems to be two bugs:
Both pyperformance and pyston have an empty
[group default]entry. I'm not 100% sure what that is supposed to do in practice, but it causes the "a group named default was already defined" error to raise. Here, I just make the groups concatenate instead.The pyston manifest has:
It seems like thisshould be valid, but the code here disallows any mention of
all.Again, I have no idea if this fix is correct, but it does at least let these two suites of benchmarks combine correctly.