The internal format that DeepJet writes to while converting from ROOT Ntuples to Numpy Arrays needs to be done away with. The new workflow should write directly into TFRecords.
TFRecords can then be read into a tf.data.Dataset and manipulated (batched, shuffled, custom-function applied to all elements, reshaped, etc.).
For reference on converting data from ROOT Ntuples to TFRecords, refer to the [Batch] Data Preprocessing Notebook in TrackingNtuples.
The internal format that DeepJet writes to while converting from ROOT Ntuples to Numpy Arrays needs to be done away with. The new workflow should write directly into TFRecords.
TFRecords can then be read into a
tf.data.Datasetand manipulated (batched, shuffled, custom-function applied to all elements, reshaped, etc.).For reference on converting data from ROOT Ntuples to TFRecords, refer to the [Batch] Data Preprocessing Notebook in TrackingNtuples.