I don't know what the icon means, but current status of OpenCL back-end is: it works with supported models, but is buggy and perhaps, slower than it could be.

Yup, this was my impression from reading a few issues lately. If you think it's not the case, feel free to update it. I just haven't set up OpenCL in my environment and cannot do tests atm

Parallel decoding is also called "batched decoding" hencebatched. Theparallel example demonstrates a basic server that serves clients in parallel - it just happens to have the continuous batching feature as an option.

Naming things is hard :) Sorry if these are confusing

Yes, it should be added. The list is far from complete

Fwiw, for me beam search is broken even without CUDA in a sense that when I run the example, nothing happens (it just hangs for minutesat this line until I CTRL+C it).

If it's an unknown problem, I'll open an issue (tbh, it's strange that nobody mentioned it before so maybe I'm doing something wrong).

Update: when it hangs on the above mentioned line, I have 0 hard page fauls/sec.

With CUDA it works for a while, but then it starts generating gibberish. I think that the calls tollama_decode are failing and it is not catching it. It's probably missing some KV cache management after the batched decoding change.

The beam search functionality should be moved out from the library and implemented as a standalone example.

The criteria is that if it runs correctly on your machine, then it is ✅ until someone reports a problem that is reproducible - then it becomes ☁️ or ❌ depending on how broken the thing is

Alright, turning the green light then!

maybe you can ditch the icons for something Like Scoring Like (A+, A, A-, B, ...) this will make it obvious if something working fine but needs improvements has a score with A- and so on, maybe something like this :
[ A+ ] or [ A ] : working like charm
[ A- ] : Working correctly but needs improvement
[ B ] : Partially working
[ B- ] : Partially working with big issues to be resolved
[ C ] : Status unknown (needs testing)
[ D+ ] : Under investigation
[ D ] : Currently in development
[ F ] : Failing

maybe you should add a column for tier support for example, if a feature is tier 1 or 2, ... what do you think?