That's one way but I think the following way is nicer since a single thread processes contiguous array elements (from which it is easier to trace any errors which might have occurred during the process):

defsplit(xs,n):"""Yields n roughly even-sized chunks from xs."""size=len(xs)q=size//nr=size%noffset=0foriinrange(r):yieldxs[offset :offset+ (q+1)]offset+=q+1foriinrange(n-r):yieldxs[offset :offset+q]offset+=qassertoffset==size

One should also sort the list of files beforehand via:

video_list=os.listdir(VIDEO_ROOT)video_list.sort(key=lambdax:int(x.split(".")[0]))

EDIT: On the other hand, your method makes it so that the dataset is mostly processed "in order", assuming the threads are roughly synced.