I have time series data with 2 columns ( ds(timestamp) and y(numerical column)) for around 10 months. There is 10 minutes of difference between consecutive rows of dataframe. y is between 0 and 1.

My task is to do trend analysis or to fit a trend graph with required change points to see pattern in data. Currently i am using neural prophet model. I am using basic approach of dividing dataset into train, val and test and I am using randomsearch to fin d best model and calculate performance metric(RMSE) on test set.

There are also NaN values in dataset. I have data for may different orgs and enviornmens, for all orgs there are different NaN values, like, 80 %, 60%, 70%, 30%, 24%, 41% ..etc. I am confused as there is lot of missing values. Basically I have applied 10 minutes of grouping in data, so if in that 10 minutes of interval value of y doesn't exit, then it will be NaN for that interval, so because of this mission values are more. If consecutive missing gap length is less , then there is default hyperparameter in neural prophet by which it was able to fill missing values. But in larger gap, it was creating probelm.

What other approach and model I can use here? How to deal with these so many NaN values? Should I fill it by any method, but by this it will not be real data. As this data is realtime. If i remove missing values, then how can I apply time series model then?

• $\begingroup$Welcome, can you add the details in your comment to the question itself? You simply edit the question.$\endgroup$

  Rick Hass

  – Rick Hass

  2025-08-24 11:09:31 +00:00

  CommentedAug 24 at 11:09

1 Answer1

Start asking to get answers

Find the answer to your question by asking.

Explore related questions

See similar questions with these tags.