Hi, Could you share the expected timeline or next steps for merging this PR? Thanks!

I was morbidly curious and read the README so added minor copyedits.

@tylerjereddy, Thanks, I have addressed them. Feel free to give it a second round.

Hi, Could you share the expected timeline or next steps for merging this PR? Thanks!

@ixgbe, I'm was aiming to get feedback on the approach first that outlined in your PR#28490 but so far no strong opinions against it, then we can move forward with merging. Once this PR gets merged we can follow on your PR as fast as possible. Please, let me know if you have any specific suggestions or any concerns against this approach.

First read, will process more over time

@Mousius, Thank you! I've updated the README to cover your questions. Please give it a second read.

I was morbidly curious and read the README so added minor copyedits.

@tylerjereddy, Thanks, I have addressed them. Feel free to give it a second round.

Hi, Could you share the expected timeline or next steps for merging this PR? Thanks!

@ixgbe, I'm was aiming to get feedback on the approach first that outlined in your PR#28490 but so far no strong opinions against it, then we can move forward with merging. Once this PR gets merged we can follow on your PR as fast as possible. Please, let me know if you have any specific suggestions or any concerns against this approach.

First read, will process more over time

@Mousius, Thank you! I've updated the README to cover your questions. Please give it a second read.
LGTM (Looks Good To Me)! Thanks for coordinating this.

when I compile numpy source code(#28608) on RVV1.0 platform, We’ve identified two pain points that require further investigation: 1、constexpr int UNROLL = (NPY_SIMD == 128 ? 4 : 2); Since we have migrated from NPY_SIMD to NPY_SIMDX, the NPY_SIMD macro is now undefined for RVV (RISC-V Vector), which will trigger compile-time errors."
2、NPY_SIMD、NPY_SIMD_F32、NPY_SIMD_F64 for RISC-V Vector are undefined. The compiler error messages are shown in the figure below. How should we resolve this? Any insights would be greatly appreciated!

@ixgbe,
I have added kMaxLanes which can be used in the first case:

constexprint UNROLL = (NPY_SIMD ==128 ?4 :2);

to:

constexprintkUnrollSize =kMaxLanes<uint8_t> ==16 ?4 :2;

Forloadable_* andstoreable_*, you should avoid using them since Highway's scatter/gather intrinsics use scalar operations. However, you will need to implement another utility function to check if the stride is divisible by the size of the type.

That makes sense to me. Centralized variable definitions for the tags would avoid the (minor) problem of people choosing different names for them, and shorten the code a bit.

There was also a concern that tags complicate python bindings because many more wrapper functions would have to be generated there - one for half, quarter vectors etc. I suppose we can for purposes of that python binding only export the full-vector version that the currently proposed wrapper functions here would provide.

@r-devulap,

I appreciate your thoughtful analysis. You make valid points about the trade-offs involved with this wrapper approach.
The primary motivation for this wrapper is indeed to make SIMD code more approachable by eliminating the tag-based syntax, because the current way of Highway seems to encourages users to produce more complicated meta-programming code in their final kernels.

Also, modern C++17 features allow us to introduce simpler constants to deal with SFINAE with cleaner code: e.g.,kMaxLanes<float> rather thanHWY_MAX_LANES_D<ScalableTag<float>> orkSupportLane<double> rather than#ifdef HWY_HAVE_FLOAT64.

This adds an extra layer of abstraction for the intrinsics that needs to be maintained and potentially add an extra layer to debug.

This wrapper is exceptionally thin. All functions are simple one-liners that just forward to Highway with the appropriate tag. There's very little to maintain or debug in the wrapper itself. If an issue occurs, the debugging process is straightforward - the problem will either be in the calling code or in Highway itself, with the wrapper simply passing values through. The mapping between wrapper functions and Highway functions is direct and transparent. See:

If we want all our highway kernels to be tag free, then this wrapper needs to be expanded with time for all highway ops used which is extra work.

Not all Highway operations need to be functionally wrapped. Most of Highway operations are already tag-free; extra work is nothing compared to the benefit of having cleaner kernels. Developers can still access the full Highway API through thehn namespace alias that's already included in the wrapper before deciding to add a wrapper for a specific operation. Do you think this expansion process is a burden?

While the approach proposed here avoids explicit tags in Highway operations, the added abstraction layer introduces maintenance overhead without significant benefits.

The benefits are clear: code becomes more concise, more readable, less prone to errors and encourages users to produce simpler code. As we decided to replace C template sources.c.src with C++ code for simplicity, would it be wise to choose a cleaner approach even if that requires a thin wrapper to Highway that potentially will be moved to Highway itself in the end?

Here is my proposed alternative:
Centralize type definitions in simd.hpp with predefined tags/vectors
Reuse these definitions across all Highway kernels
Avoid op-specific expansions through generic implementations
This way we avoid introducing an extra abstraction layer and eliminate the need to expand wrappers for every Highway operation.

Not sure what the point of predefining tags is? Why not justhn::ScalableTag<float>{}? ortemplate<typename T> constexpr auto kScalable = hn::ScalableTag<T>{}; withtemplate <typename T> using ScalableVec = hn::Vec<hn::ScalableTag<T>>;?

Using your previous example, your suggestion would be like this:

template<typename T>voidadd(T* src, T* dst,size_t N) {// ScalableVec<T> v = hn::LoadU(kScalable<T>, src);// bad practice to use auto everywhere but can we tolerate it?auto v =hn::LoadU(kScalable<T>, src);    v =hn::Add(v, v);hn::StoreU(v,kScalable<T>, dst);}

How aboutkMaxLanes,kSupportLane, and constantsNPY_SIMDX_*? Based on your alternative, are we going to define them in simd.hpp too?

Do you think this would be a simpler and more maintainable solution?

I'm more concerned about maintaining the kernels than the wrapper itself.
With your suggestion, we still create some form of abstraction - just to avoid simple one-liner wrappers where needed.
All code involves abstraction; the question is at what level. At NumPy's scale, the thin wrapper approach is absolutely worth it.

There was also a concern that tags complicate python bindings because many more wrapper functions would have to be generated there - one for half, quarter vectors etc.
I suppose we can for purposes of that python binding only export the full-vector version that the currently proposed wrapper functions here would provide.

@jan-wassenberg, Thank you for your input. If we moved the wrapper to Highway to replacegoogle-tests withpytest, it would still be possible to usehwy::scalable::<HWY_TARGETS>,hwy::fixed128::<TARGETS>, etc. in the bindings.
pytest is dynamic enough to handle this.

One quick comment:

// bad practice to use auto everywhere but can we tolerate it?

I'd advise against using auto everywhere. Kernels that involve range reduction may involve multiple types (at least int32 and f32, possibly more), and I think it's helpful to see the type mentioned in the code.

This wrapper is exceptionally thin. All functions are simple one-liners that just forward to Highway with the appropriate tag. There's very little to maintain or debug in the wrapper itself.

This is potentially my most pressing question: Will it continue to be thin? It seems we'll have a lot of thin wrappers.

@jan-wassenberg do you see issues with these being contributed directly to Highway? I'd imagine a lot of the C++17 stuff is a welcome addition to the macro variants?

I'm not seeing anything that is NumPy-centric or needs to be NumPy-centric just yet, and it appears we already know roughly what we'd like to contribute.

Not all Highway operations need to be functionally wrapped. Most of the Highway operations are already tag-free; extra work is nothing compared to the benefit of having cleaner kernels.

I'd also suggest that the extra work may not be necessary if it's this minimal for users. I prefer@r-devulap's ideas of centralised tags for NumPy types and explicit typing in the SIMD kernels.

The benefits are clear: code becomes more concise, more readable, less prone to errors and encourages users to produce simpler code.

This could be debated. As@jan-wassenberg has mentioned above, it is valuable to see the types and conversions explicitly in SIMD kernels to avoid errors. The code itself may contain fewer characters, but that doesn't necessarily correlate with being easier to read. Personally, I've found the verbosity of the SIMD kernels justified due to their details.

@jan-wassenberg,

I'd advise against using auto everywhere. Kernels that involve range reduction may involve multiple types (at least int32 and f32, possibly more), and I think it's helpful to see the type mentioned in the code.

Totally agree with you as mentioned "bad practice". Based on@r-devulap's idea of using centralized tags, the most simplified example would be like:

template<typename T>voidadd(T* src, T* dst,size_t N) {    ScalableVec<T> v =hn::LoadU(kScalable<T>, src);    v =hn::Add(v, v);hn::StoreU(v,kScalable<T>, dst);}

And that's why I believe simplifying Highway will encourages users to produce more clearer and simple code.

@Mousius,

Will it continue to be thin? It seems we'll have a lot of thin wrappers.

Yes definitely. We're just erasing tags and using modern C++ features. We're in 2025 - if we have a chance to make our meta-programming much simpler by leveraging modern C++ features, we should take it. Highway had legacy reasons to support C++11, but we don't.

I'm not seeing anything that is NumPy-centric or needs to be NumPy-centric just yet, and it appears we already know roughly what we'd like to contribute.

I agree that most of this wrapper isn't NumPy-specific and could benefit the broader Highway ecosystem. That's why it would be great to move it to Highway as we agreed before.

This could be debated. As@jan-wassenberg has mentioned above, it is valuable to see the types and conversions explicitly in SIMD kernels to avoid errors. The code itself may contain fewer characters, but that doesn't necessarily correlate with being easier to read. Personally, I've found the verbosity of the SIMD kernels justified due to their details.

I think you misunderstood my comment#28622 (comment). Let me explain with more details:

@r-devulap suggested using centralized tags, and my response was "there's no need for them, what's the point of having them?" Instead, we can use template variables (introduced in C++14) rather than explicitly defining each tag (if we decide to tolerate tags).

So instead of explicitly defining each tag:

const hn::ScalableTag<float> f32;const hn::ScalableTag<int32_t> s32;

we can define a template variable like this:

template<typename T>constexprautokScalable = hn::ScalableTag<T>{};

And by relying on the lane type, you can define any tag simply like this:

kScalable<float>;// equivalent to f32kScalable<int32_t>;// equivalent to s32

Instead, we can use template variables (introduced in C++14) rather than explicitly defining each tag (if we decide to tolerate tags).

template<typename T> constexpr auto kScalable = hn::ScalableTag<T>{};template<typename T>void add(T* src, T* dst, size_t N) {    ScalableVec<T> v = hn::LoadU(kScalable<T>, src);    v = hn::Add(v, v);    hn::StoreU(v, kScalable<T>, dst);}

@seiko2plus this sounds good to me. My preference would be to tolerate tags in highway ops and avoid having a wrapper around them. Thin or not, it is still additional code we have to maintain and expand over time and I personally think doesn't add much value. As for the tags and vector themselves, a central definition that kernels can draw from sounds perfect.

How about kMaxLanes, kSupportLane, and constants NPY_SIMDX_*? Based on your alternative, are we going to define them in simd.hpp too?

I think so. These are useful to be defined in a central place that all the highway kernels can use.

I agree that most of this wrapper isn't NumPy-specific and could benefit the broader Highway ecosystem. That's why it would be great to move it to Highway as we agreed before.

I'm sorry, my question was probably misformed. What prevents us from contributing this directly to Highway now?

Instead, we can use template variables (introduced in C++14) rather than explicitly defining each tag (if we decide to tolerate tags).

Couldn't we also useScalableTag<float> everywhere, which is two characters more thankScalable<float>?

I personally think doesn't add much value

It's different preferences, I guess. To me, counting on ADL and being able to force change SIMD width with just a single line when needed is a great advantage.

Thin or not, it is still additional code

But we are going to transferring the additional code from the kernels to the wrapper.

What prevents us from contributing this directly to Highway now?

Well, once it becomes mature enough we can pass it. New ideas may emerge anytime - why should we worry about backward compatibility at this stage when other projects aren't yet depending on it?

Couldn't we also use ScalableTag everywhere, which is two characters more than kScalable?

ScalableTag is the type, whilekScalable would be the tag instance. Yes, we could use the type directly with extra characters() or{}.

It's different preferences, I guess. To me, counting on ADL and being able to force change SIMD width with just a single line when needed is a great advantage.

@seiko2plus if you find it advantageous and feel strongly about it, I don't mind putting this in. As you said in an earlier comment: developers can still access the full Highway API through thehn namespace alias.

@seiko2plus Can we make this change before merging?https://github.com/numpy/numpy/pull/28622/files#r2057043403

Discussing in the Optimisation team meeting:

• #define NPY_SIMDX (HWY_TARGET != HWY_SCALAR) will always beTrue as Highway is going to defaultEMU128 which will allow for compiler auto-vectorisation by default. ThereforeNPY_SIMDX is likely unnecessary?
• Highway is happy to host the wrapper to remove lane tags under contrib given it's fairly clear why it exists and why it's useful, which would save us hosting it in NumPy. We should contribute it directly undercontrib for Highway which equally allows for it to evolve further if necessary?

@seiko2plus Can we make this change before merging?https://github.com/numpy/numpy/pull/28622/files#r2057043403

Done.

#define NPY_SIMDX (HWY_TARGET != HWY_SCALAR) will always be True as Highway is going to default EMU128 which will allow for compiler auto-vectorisation by default. Therefore NPY_SIMDX is likely unnecessary?

NPY_HWY (NPY_SIMDX) is needed for two important reasons: to completely disable optimization via disable-optimization and to clear the baseline features viacpu-baseline=none. This is important because Highway will still enableSSE3 by default even ifSSE flags aren't provided, sinceSSE3 is enabled by default ongcc/clang. By the way,Highway seems doesn't correctly emulate severalSSE41 elementary intrinsics, but this isn't a concern since PR#28896 will raise the baseline to x86-64-v2 (level 2).

Highway is happy to host the wrapper to remove lane tags under contrib given it's fairly clear why it exists and why it's useful, which would save us hosting it in NumPy. We should contribute it directly under contrib for Highway which equally allows for it to evolve further if necessary?

As I mentioned before, I'd prefer to use it within NumPy first rather than moving it to Highway until it's mature enough. Hosting in NumPy will make it easier to make changes as we refine the implementation.

NPY_HWY (NPY_SIMDX) is needed for two important reasons: to completely disable optimization via disable-optimization and to clear the baseline features viacpu-baseline=none. This is important because Highway will still enableSSE3 by default even ifSSE flags aren't provided, sinceSSE3 is enabled by default ongcc/clang. By the way,Highway seems doesn't correctly emulate severalSSE41 elementary intrinsics, but this isn't a concern since PR#28896 will raise the baseline to x86-64-v2 (level 2).

I believe this is already possible by only enablingEMU128? I don't see why we needNPY_SIMDX for this.

As I mentioned before, I'd prefer to use it within NumPy first rather than moving it to Highway until it's mature enough. Hosting in NumPy will make it easier to make changes as we refine the implementation.

I think this is unnecessary extra work, as it's simple to contribute this to Highway directly without creating our own mini-framework straight away. In my mind, this is setting a precedent that we don't try to work with Highway in the first instance.@jan-wassenberg has already said he has no issues and it's a small change. However, if@r-devulap is happy to merge this, then I will get out of the way.

I believe this is already possible by only enabling EMU128? I don't see why we need NPY_SIMDX for this.

When the optiondisable-optimization is set to true, we should disable all CPU optimized code including dispatch, SIMD, and loop unrolling.
After a quick look at theEMU128 implementation, I realized that we should disable it too to avoid any kind of fast math operations. Also, compiling withfp-strict may not lead to auto-vectorization and may end up with register spilling. Lets stay safe for now, till we get done from moving to Highway.

When the optiondisable-optimization is set to true, we should disable all CPU optimized code including dispatch, SIMD, and loop unrolling.
After a quick look at theEMU128 implementation, I realized that we should disable it too to avoid any kind of fast math operations. Also, compiling withfp-strict may not lead to auto-vectorization and may end up with register spilling. Lets stay safe for now, till we get done from moving to Highway.

We already haveNPY_DISABLE_OPTIMIZATION to completely disable the optimised kernels?

I'm not sure whether a lack of optimisation inEMU128 is hugely worrying given it's the fallback.