return_full_text is a legacy option linked totransformers.pipelines initial implementation (something like 4+ years ago).
We had API dependencies on that behavior and therefore implemented here, it can mostly be disregarded nowadays.

What you did here is effectively deactivate its job. Which is really simply to decode all the decoder tokens, instead of only the new ones.

I was under the impression that T5 was a encoder/decoder model so I'm surprised to see a decoder-only model here.

I'm pretty sure the "bug" will simply end up being a tokenizer issue, where possibly you're hitting a very old T5 non flash version that doesn't support some flags used in those tokenizers.

Thanks for the fix for your use case, I'll leave it up for others to see, but if we're going to fix it, we need to find the root cause and fix that instead.

@Narsil thanks for looking into this and explaining it to me to give me a sense of what the root of the problem could be. it was nice to see madlad being served with good results and may just use this for the time being. looking forward to tracking this solution and learning more about this great project💪

Any movement on this one? Would be excellent to fully support MadLad as a translation T5 model.