cc@greenrazer@FL33TW00D@Vaibhavs10 for opinions and feedback.

I like this solution. however, I cannot get thecanImport approach inTokenizersWrapper to work for the tests due to how Swift Package Manager handles module dependencies and conditional imports within the same package. All the tests work great with almost no modification (besides changingTokenizers toTokenizersCore) except for theChatTemplatesTests.swift.

I've tried a few different options:

• The best option: Just addingTokenizersTemplates as a dependency to the test, similar to how you would for an external package. This doesn’t work because it doesn’t trigger recompilation for the test.
• AddingswiftSettings: [.define("USE_TEMPLATES")] to the test and then adding another build condition toTokenizersWrapper. Same issue as above—it doesn’t trigger recompilation for the test.
• Creating two targets pointing to the sameTokenizersWrapper source file, with one includingUSE_TEMPLATES. This doesn’t work because you can't have two targets referencing the same source file.
• Adding.target(name: "TokenizersTemplates", condition: .when(platforms: nil)) as a dependency to hopefully run only during testing. This doesn’t work in newer Swift versions.
• AddingswiftSettings: [.define("ENABLE_TEMPLATES", .when(configuration: .debug))] to theTokenizers target. This almost works, butTokenizersTemplates must then be a dependency ofTokenizers, which defeats the whole purpose.

Since users must explicitly importTokenizersTemplates anyway, I think the best approach is to create two wrappers and two targets. That way, users only need to import eitherTokenizersTemplates orTokenizers, depending on their needs.