Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7k
Override model serializer globally, big integer field#9722
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
First of all thanks for this project, i can't use Django without drf 😀 Im looking for a way to override serializers.ModelSerializer globally Little bit of context: My problem:
So the best solution in my opinion is to serialize bigint as string. classSnowflakeField(serializers.Field):defto_representation(self,value):returnstr(value)defto_internal_value(self,data):try:returnint(data)exceptValueError:raiseserializers.ValidationError("Invalid integer value")classSnowflakeModelSerializer(serializers.ModelSerializer):""" Snowflake is a bigint so convert to string to avoid frontend memory handling problems """defbuild_standard_field(self,field_name,model_field):field_class,field_kwargs=super().build_standard_field(field_name,model_field)# Use BigIntAsStringField per BigIntegerFieldifisinstance(model_field,models.BigIntegerField):field_class=SnowflakeFieldreturnfield_class,field_kwargs Anyway, since django uses bigint as id, serializing it as sting could be a better option in any case since the id can grow big Possible ways
classMyModelSerializer(SnowflakeModelSerializer):classMeta:model=MyModelfields='__all__'
and do something like this with a lot of assumptions, spoiler bad programming classSnowflakeJSONRenderer(JSONRenderer):def_convert_big_integers(self,obj):ifisinstance(obj,dict):return {key:self._convert_big_integers(value)forkey,valueinobj.items()}elifisinstance(obj,list):return [self._convert_big_integers(item)foriteminobj]elifisinstance(obj,int)andobj>2**53:returnstr(obj)returnobj What im looking forAs far as i know there is no option to do this REST_FRAMEWORK = { Am i missing something? In your opinion what could be the best solution? I'm open to any suggestion |
BetaWas this translation helpful?Give feedback.
All reactions
I'd recommend going with the first option. It doesn't look pleasant at first, but it's easily maintainable if you add aDjango system check to validate that the correct class is used.
Do you have a specific reason to use snowflake btw? If you are open to alternatives, I'd recommend UUID7 as a solid one.
Replies: 2 comments 6 replies
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
I'd recommend going with the first option. It doesn't look pleasant at first, but it's easily maintainable if you add aDjango system check to validate that the correct class is used. Do you have a specific reason to use snowflake btw? If you are open to alternatives, I'd recommend UUID7 as a solid one. |
BetaWas this translation helpful?Give feedback.
All reactions
👍 2
-
Thx for the replay@ulgens Actually we are also considering uuid, for sure it's a simpler implementation and probably a better compromise in our situation I'm try to evaluate snowflake option for this reasons:
Probably it's overkill but i'm not committed to the switch yet I've made some tests with this code and it's working, it's still a fast implementation.
fromdjango.dbimportmodelsfromdjango.utils.encodingimportsmart_strfromrest_frameworkimportserializersclassSnowflakeField(serializers.Field):"""Convert snowflake data to string"""defto_representation(self,value):returnsmart_str(value)defto_internal_value(self,data):try:returnint(data)exceptValueError:raiseserializers.ValidationError("Invalid integer value")classSnowflakePrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField,SnowflakeField):"""Handle related primary keys as snowflake"""defto_representation(self,value):ifself.pk_fieldisnotNone:returnself.pk_field.to_representation(value.pk)returnstr(value.pk)defto_internal_value(self,data):data=super(SnowflakeField).to_representation(data)returnsuper(serializers.PrimaryKeyRelatedField).to_internal_value(data)classSnowflakeSlugRelatedField(serializers.SlugRelatedField):"""Implement if used"""classSnowflakeModelSerializer(serializers.ModelSerializer):""" Snowflake is a bigint so convert to string to avoid frontend memory handling problems """serializer_field_mapping= {**serializers.ModelSerializer.serializer_field_mapping,models.BigAutoField:SnowflakeField,models.BigIntegerField:SnowflakeField }serializer_related_field=SnowflakePrimaryKeyRelatedFieldserializer_related_to_field=SnowflakeSlugRelatedField My pk model fromdjango.dbimportmodelsclassSnowflakeBigAutoField(models.BigAutoField):defpre_save(self,model_instance,add):ifaddandnotgetattr(model_instance,self.attname,None):value=self.get_next_value()setattr(model_instance,self.attname,value)returnsuper().pre_save(model_instance,add)@staticmethoddefget_next_value():returnnext(sf)# settings.pyDEFAULT_AUTO_FIELD='core.snowflake.SnowflakeBigAutoField' If we deoloy with snowfake i like to share the production code here, because i didn't find any useful resource about snowflake id with django DEFAULT_MODEL_SERIALIZER_CLASSIn any case, is it possible in your opinion try to introduce this setting? I can submit a pull request. |
BetaWas this translation helpful?Give feedback.
All reactions
-
We are doing this right now almost everywhere except on some dashboard elements, but it's kinda a pain 🥲 At one point we also introduced the ability for the user to customize the slug to reach certain resources, it was more confusing than useful, so we switched back to autogeneration. I like something like discord only one id in and out, no names no personalization no problems, it's ok even if uuid it's a bit longer. I don't see problems in this aproach, am i wrong?
Next week i try to implemente this solution and let you know
There is actually a kind of bug: if you take a BigInt in DB, for example It is possible to find a workaround on client side, but it is not guaranteed to work on all browsers. |
BetaWas this translation helpful?Give feedback.
All reactions
-
Sorry if I missed it but what's the result of this behaviour? Does it fail? Does it produce an incorrect response? If a Django native datatype can't be converted to a proper json by DRF, we may consider to add a warning and update the docs about it. |
BetaWas this translation helpful?Give feedback.
All reactions
-
Sorry for the late reply, in the meantime i've made some research and extensive tests. JSON does not have a native BigInt type — it only supports double-precision floating-point numbers (IEEE 754), that are safely precise for integers only up to 2⁵³ − 1 (i.e. 9,007,199,254,740,991). Beyond this threshold, numbers lose precision and may be silently truncated or rounded when parsed in JavaScript or other languages. It depends on the json library implementation, this particular issue doesn’t happen in Python but it’s not consistent across all languages JSON libraries The twitter example:Twitter introduced via api id both numeric and id_str as string to avoid this exact problem. Why DRF test frontend works?Because it uses FormData, basically it is all string data, and since the server side render is handled by python no bigint errors occour Resources: Since the majority of frontends that will call api are javascript, this is an issue that have to be addressed in my opinion, i know that the bigint problem with ids will happen only at very large scale and probably a lot of people will never encounter this, but if we find a solution it would be wonderfull. COERCE_DECIMAL_TO_STRING |
BetaWas this translation helpful?Give feedback.
All reactions
❤️ 1
-
BetaWas this translation helpful?Give feedback.
All reactions
👍 1❤️ 1
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
👏🏻 I think this approach has better focus on the actual issue, and precisely defines what needs to be handled. Thanks for all the details. I'll be following people's reactions under#9733, and I can help with the PR when it's ready. |
BetaWas this translation helpful?Give feedback.
All reactions
This discussion was converted from issue #9721 on June 20, 2025 07:19.