Access Token component

Use case (TL;DR)

When you develop a Symfony project thatneeds to consume data from a remote service/API, inB2B (Business To Business) /MTM (Machine To Machine) mode,without any user interaction, in most cases you will need toauthenticate your HTTP requests for this remote service. It's rather common that this authentication is done by aBearer access token.

This component aims toprovide ways to configuring credentials then retrieve those tokens in a generic manner. It also aimsimprove performance improvement using caching for storing those access tokens for their lifetime, and toensure consistency by using locking around remote token fetch, in order to avoid parallel accesses to the remote endpoint when a token expires.

In this scenario, theSymfony application is a client, not a server, requesting tokens and consuming data, and not the other way around.

Rationale

This main purpose component is toprovide a common interface for fetching and refreshing remote access tokens, for webservices bearer tokens, SMTP XOAUTH, etc... as well as a some commonly used bridges.

The obvious use case is for managing OAuth2 token credentials, using theclient_credentials andrefresh_token grants, which are provided by a remote service and have short life time, in order to use them as aclient only. Theauthorization_code flow is voluntarily excluded because it's meant to trigger a user interaction, which is out of scope right now (but may be later).

It's important to keep this in mind, OAuth2 is a standard, and security wise very hazardous to implement by one self without using a production-ready, industry grade, well maintained specialized component.This component is not going to provide any crypto related or server side code: it's meant for B2B / MTM token exchange only. This component is not likely to introduce any security issue.

The main goal being to provide a token fetcher and manager generic API interface, which is not OAuth2 specific, thus may be used for various other kind of tokens. In real life, most token-based authentication seem to be inspired or variations of one of the OAuth2 grant flows.

Why would Symfony need this? For a few different reasons:

• To start withthere's already a few bits of code doing this, such as a fewsymfony/notifier transports, especially theOrangeSmsTransport that handles the token fetch by itself.
• To go further, thesymfony/mailer component holds theXOAuth2Authenticator whichonly accepts an hardcoded access token from configuration, without any refresh logic, which is carried by theDsn instance upper in the stack. Tokens are almost always bound to a life time, sometimes a few minutes, sometimes a year, and those tokens cannot simply be set in configuration for this exact reason. Even if given via environment variables: changing it almost always in the current state requires a manual operation, or sometimes some services restart.
• Using tokens forBearer authentication or similar to authenticate into third party web servicesis a common task that all developer probably writes at least once per project.
• The submitted[Mailer] Improve oauth setup #52585 PR (not merged yet) confirms that it is a common need, and that this should be well thought to be future proof, and easy to use.

And why would it be better to have a component than doing it manually, like it is done in theOrangeSmsTransport ? Well, for many reasons, one being that a few mistakes are common to see:

• If you look at theOrangeSmsTransport,it will fetch a token for each notification being sent, but the token it gets is valid for a certain amount of time:it's a waste of resources and a performance hog not to reuse it.
• Handling caching for those tokens is often forgotten, yet quite trivial, but may be error prone in some edge cases, having the available code being generic, well tested, and community-maintained seems to be a good idea.
• Sometimes, in thesymfony/mailerDsn class use case, the configuration and the token will be cached in memory, which means that whenused through a service that lives during asymfony/messenger bus process long-lived runtime, the token could get invalidated during runtime. Fixing it requires a restart. We can fix this by forcing the token to be systematically fetched for cache, validated, and refreshed programmatically, for each use.
• When a project is subject tohigh concurrency / high throughput scenario,parallel requests and buses consumptions might invalidate tokens while others are using it. Sometimes they also might fetch all remote tokens in parallel. It is,at best, a performance red flag, at worst, concurrency might cause valid token used by some threads to be invalidated by other threads, if error handling is wrongly handled, cause real domain bugs. For this,we need to handle lock carefully and gracefully around cache accesses and remote accesses.

When you start to put all pieces altogether, it shows to be more complex than simply doing an HTTP request to fetch a token.In most real life scenarios we always want caching, validating and refreshing features to be implemented. This is even more true now that thesymfony/messenger component is widely adopted by the community.

My personal experience with this is that, in the company I work for, each project owns its own implementation of this lock, cache, validate and refresh algorithm. Sometimes more than once in a single project because we need to interconnect with various third party remote web services API, each having its own dialect for fetching tokens. That is the main reason why we want a generic, reusable and well-maintained component for this.We hope and do expect it will also be in the interest of many other people.

Proposed design

Main concepts

There the following concepts/interfaces which are really important here:

• A bridge is a set of classes that contains a specific token implementation.
• ProviderInterface is what the bridge must implement: it has one method for fetching a new token, another one for refreshing an existing token. The abstract base implementation merges the two methods into a single fetch new token interface. This interface is hidden for the end user.
• CredentialsInterface is a DTO that is typed and carries user credentials information, bridges must implement their credentials as well, this interface is exposed to the end user. Its only generic feature is to be able to produce a reproducible identifier for caching and locking.
• FactoryInterface is what the bridge can implement in order to enable user-driven credentials configuration. From an URL schemes, it produces aCredentialsInterface instance suitable for this component usage.
• AccessTokenManagerInterface is the user facade, it provides the same two methods, but they are different in the sense that they only work with aCredentialsInterface parameter.
• AccessTokenFetcher is a service created from user YAML configuration, it is a replica ofAccessTokenManagerInterface without the credentials parameter, since it already knows the user configuration.

The access token manager works in this way:

• When a method is called, find the appropriate provider depending upon the credentials class.
• Call the appropriate method on the provider to get the token as a result.
• Validate the token with information that it carries (lifetime mostly).
• If invalid, call a forced refresh on the provider once again and get the new valid token.
• Return the valid token.
• We're done here, let the user do whatever it needs to do with the token.

Caching and locking

Caching and locking are done by decorating theAccessTokenManagerInterface.

Caching should do:

• Cache access uses the SymfonyCacheInterface contract.
• Get token method will attempt a cache fetch first, if it gets a token then return it.
• If no token was found, calls the decorated get token implementation, get the new token, save it and returns it.
• In case of force refresh, delete the existing cache item (if it exists), then calls the refresh token method, save it to cache and return the result.

Locking:

• Uses the core Symfony Lock component.
• Get token method will do a blocking lock, and await for somebody to release it in case it is already taken by someone else. Once lock is released by others, keeps the lock, calls the decorated get token method and returns the result (lock is released prior to returning).
• Refresh token method will do a non blocking lock, in case it fails it will call the get method instead. Get method will block and wait for a new token to be fetched by someone else.

Note: for locking, there is probably ways to improve it, such as getting a read-only lock on first cache get, we are eager to hear your suggestions.

For this to work, cache and lock, we need for each credential a unique and reproducible identifier, that's why the only public visible method onCredentialsInterface isgetId(): string. It's up to the implementation to compute a unique identifier.

Note:getId(): string method could boil down to a__toString(): string method which returns the URI (which is supposed to be unique). I have no strong opinion about this so all suggestions are welcome.

In default configuration, lock decorates cache which decorates the default implementation in order to prevent concurrent cache accesses: a thread could fetch an invalid token from cache while another thread is fetching a new one, here this scenario is impossible. In case you prefer performance over consistency, it is possible, per configuration, to disable the lock completely.

Configuration and factories

In order to allow user configuration in YAML and environment, a piece of code is needed to glue user strings toCredentialsInterface instances, it's theFactoryInterface:

• User gives an arbitrary name and a URI for each credentials in YAML configuration.
• Each user given URI must have a scheme.
• Each factory is registered with a scheme name.
• A single scheme, hence a single factory, can be polymorphic and return various credentials implementations, the implementation is opaque (for example, a single schemeoauth for all OAuth implementations, that requires agrant_type= parameter in the URL to discriminate).

During container compilation, each user given credentials in YAML configuration is parsed, in order to check for validality, then an associated service factory is registered that creates anAccessTokenFetch at runtime when requested.

Usage and configuration

Overview

Component in its current state can be used with three different workflows:

• First one, by using directly theAccessTokenManagerInterface service: user handles its credentials configuration manually, creates aCredentialsInterface implementation instance and passes it to the manager service for fetching the token. For example, sinceOrangeSmsTransport already carries its credentials in the user configured DSN from the mailer component, it would be appropriate to keep it that way for backward compatibility and simplicity.
• Second one, by configuring arbitrary tokens in YAML configuration, using a DSN specific for theaccess-token component (which can be retrieved from environment variables like the database URL or SMTP DNS are) which is injected as aAccessTokenFetcher instance into the container, one per configured credentials. This method would be preferable for users that need to extensively use this component for various services at once: it allows them to register all in configuration, pass credentials via environment variables, and keep their services implementations simple.
• Third one, by configuring arbitrary tokens in YAML configuration, using a DSN specific for theaccess-token component (which can be retrieved from environment variables like the database URL or SMTP DNS are) which is injected as aCredentialsInterface instance into the container, one per configured credentials. The user can use theAccessTokenManagerInterface by passing over the credentials that were injected.

UsingAccessTokenManagerInterface directly

Simple API user code example using OAuthclient_credentials grant flow:

namespaceApp\Services;useSymfony\Component\AccessToken\AccessTokenManagerInterface;useSymfony\Component\AccessToken\Bridge\OAuth\ClientCredentials;class SomeService{publicfunction__construct(        #[\SensitiveParameter]privatereadonlystring$clientId,        #[\SensitiveParameter]privatereadonlystring$clientSecret,privatereadonlyAccessTokenManagerInterface$accessTokenManager,    ) {}publicfunctionsomeMethod()    {// Create implementation specific credentials.$credentials =newClientCredentials(            clientId:$this->cliendId,            clientSecret:$this->clientSecret,            endpoint:"https://thirdpartyservice.com/oauth/2.0/token"            scope:'some_scope',// can also be an array such as ['do_something', 'some_other_thing']        );// Variant is possible using factories:$credentials =$this->accessTokenManager->createCredentials(sprintf('oauth://%s:%s@thirdpartyservice.com/oauth/2.0/token?grant_type=client_credentials&scope=some_scope'$this->clientId,$this->clientSecret,            ),        );$token =$this->accessTokenManager->getToken($credentials);try {$this->callSomeRpc($token);        }catch (SomeRemoteInvalidTokenDependingUponTheSomeRpcMethodError) {$token =$this->accessTokenManager->refreshToken($credentials);$this->callSomeRpc($token);// Let exception pass in case of a second failure. Remote service is probably// down, misconfigured, or user identity is disabled or is missing access rights.        }    }}

And that's pretty much it. Considering the user doesn't need to implement its own bridge and one of the core provided one fits the need.

Using YAML configurationAccessTokenFetcher

Let's start by configuration:

access_token:credentials:my_thirdparty_service:url:'%env(resolve:MY_THIRDPARTY_SERVICE_TOKEN)%'

The environment variable would look like:

MY_THIRDPARTY_SERVICE_TOKEN=oauth://CLIENT_ID:CLIENT_SECRET@thirdpartyservice.com/oauth/2.0/token?grant_type=client_credentials&scope=some_scope

Then, the same user code as upper would boil down to:

namespaceApp\Services;useSymfony\Component\AccessToken\AccessTokenFetcher;class SomeService{publicfunction__construct(// Uses the same mecanism as cache pool for naming services arguments:privatereadonlyAccessTokenFetcher$myThirdPartyService,    ) {}publicfunctionsomeMethod()    {$token =$this->myThirdPartyService->getToken();try {$this->callSomeRpc($token);        }catch (SomeRemoteInvalidTokenDependingUponTheSomeRpcMethodError) {$token =$this->myThirdPartyService->refreshToken();$this->callSomeRpc($token);// Let exception pass in case of a second failure. Remote service is probably// down, misconfigured, or user identity is disabled or is missing access rights.        }    }}

Implementing a bridge

Implementing a bridge consists in three different things:

• First, one or moreCredentialsInterface concrete implementations. This is mandatory.
• Second, one or moreProviderInterface concrete implementations that deals with the associated credentials. This is mandatory.
• Third, one or moreFactoryInterface concrete implementations, that allows the credentials to be tied to a scheme and registered to allow the user to drive their configuration in YAML. This is optional, but if not implemented, cannot be YAML configured.

About the current code state

It may be incomplete, nevertheless it's a fully functional, working, MVP. It holds the most important structural pieces and our opinion is a sufficiently good design to be used :

• Interfaces everywhere it makes sense, in some cases in order to allow decoration and other because remote services inner working are implementation details.It's flexible and extensible to the bone.
• It allows either user to drive its own configuration via direct use of credentials classes or factories, or provide a common, documented and tested, static configuration tooling in YAML and environment variables.
• It must remain simple to use.
• Cache and lock features are implemented via the decoration pattern, lock feature can be disabled if the user prefers speed over consistency. Default configuration is that lock decorates the cache accesses, in order to maximize consistency over speed.
• A single bridge provided for OAuth2, which covers most common use cases. It is known that many providers sometime will deviate from the standard in details (Google has long been known for violating standards they push forward for example) but it's easy to extend the default OAuth2 bridge and adapt.

Per default, the choice has been made to prefer consistency over speed, in all cases, locking then cache access will be significantly faster than doing an HTTP request for fetching a token. In high throughput scenarios, this is probably a sensible default, for all others, it won't make a visible difference.

Naming can be wrong at some places, we are opened to any suggestion. Especially for shortening some interfaces names. Actual names are the most verbose and expressive names we could find, but they are great chances that you might have better ideas than us!

Known limitations

It's not possible, in the current state, to detect remote access token invalidity in a generic manner (e.g. you send the token to an SMTP server and it gets rejected) because the remote service implementation is opaque, errors can differ depending upon the remote service ("535 Authentication Unsucessful" for SMTP, "401 Authorization Required" for HTTP, etc... and which\Exception class ? It depends upon the user implementation or library in use). So, the final try/catch-error/refresh-token/retry algorithm must be implemented by the user, as shown in the previous example.

DSN parsing is simplistic for now, and improvements will be done, in order, for example, to be able to combine HTTP Basic Auth with OAuth2 client credentials, this is quite common to stumble upon remote services requiring it.

What it is not

I'm repeating it, because it's important:

• It is NOT an OAuth2 or OIDC server implementation, there is no security issues here.
• It is NOT manipulating, validating, encrypting anything such as JWT tokens, for example, thus considerably lowers the risks of messing up with crypto. Really, it's not doing anything in that domain.

So now, what?

We are seeking for reviews, and will glady accept any opinion, any criticism. We do really hope this would get adopted, because this would improve:

• For existing implementations, performance (through token caching) and consistency (through locking).
• IT may fix many annoying bugs or behaviors a lot of access-token-unexperienced people may experience on a daily basis (for example, a command bus that requires restart because you start seeing errors in logs due to expired tokens).
• Easier instrumentation by using extensive logging in manager and providers (not yet implemented).
• This is a feature that many people implement everyday without benefiting from the experience that many have on that topic. This component is exaclty this: experience of some, which only awaits for the community experience to improve it even more.
• As a final note, I have to admit, this is fun to develop, but I have children and not so much time for myself, I am bored re-implementing all of this in every project I work for.

Future plans

• Implement more bridges.
• Add an early self-expiration of tokens (ie. consider a token invalid before it is actually, a few minutes, a few days, ... sometimes tokens might be received with a year long lifetime, the user or site ops might want to force a shorter refresh time).
• Allow usage of both basic auth and client credentials in the current OAuth2 implementation.
• Add instrumentation (logging) everyhwere it seems pertinent to.
• Later in the future, add profiler and debuguer integration.
• Allow max life time configuration for each configured credentials.
• Add fine grained configuration options per credentials (including forced expiry time).
• Whatever the users need !

Note about dependency

In order to avoid creating hard dependencies to other components (notifier, mailer), there is two different ways I can see right now:

• Either provide a contract interface for access token fetcher/manager ? Then implement a bridge for it in theaccess-token component, and inject it properly with some container compilation magic.
• Or a simpler one, maybe with a single getToken() method ?
• Or simply let each component have its own contract, and provide its own bridge for this contract.