ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
NVIDIA Time Series Prediction Platform
Resource
NVIDIA
NVIDIA
NVIDIA Time Series Prediction Platform

NVIDIA Time Series Prediction Platform is a tool designed to compare easily and experiment with arbitrary combinations of forecasting models, time-series datasets, and other configurations.

3_ByoDataset.ipynb
In [1]:
# Copyright 2023 NVIDIA Corporation. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
In [2]:
%load_ext autoreload
%autoreload 2

NOTE: These notebooks are designed to highlight different features of NVIDIA-TSPP. For this reason, all the examples are created to run quickly with only few iterations. The parameters should be tuned to get optimal results.

1. Introduction

NVIDIA-TSPP has been designed to work with most CSV inputs. This notebook will cover all the steps required to add a custom dataset to the NVIDIA-TSPP tool. As an example, we will use a time-series dataset from the kaggle competition on Tokyo Stock Exchange Prediction and cover all the steps including dataset download, feature engineering, NVIDIA-TSPP dataset configuration creation, NVIDIA-TSPP dataset preprocessing, and training on the custom dataset.

Imports

In [3]:
import os
import hydra
import warnings
import torch
from omegaconf import OmegaConf
from hydra import compose, initialize
from hydra.core.global_hydra import GlobalHydra
from hydra.core.hydra_config import HydraConfig
from hydra.utils import get_original_cwd
import conf.conf_utils

from hydra_utils import get_config
from data.data_utils import Preprocessor
from training.utils import set_seed
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from decimal import ROUND_HALF_UP, Decimal
import datetime
import pickle

warnings.filterwarnings("ignore")

curr_workdir = os.getcwd()
/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Using backend: pytorch
In [4]:
#Since this notebook uses the NVIDIA-TSPP container. TSPP code is available at /workspace directory
tspp_ws = "/workspace"

2. NVIDIA-TSPP Datasets

NVIDIA-TSPP provides configuration files for several common time-series datasets

In [5]:
!ls {tspp_ws}/conf/dataset
electricity.yaml  traffic.yaml
In [6]:
dataset = "electricity"
#dataset = "traffic"

For some of the datasets, NVIDIA-TSPP supports end-to-end flow starting from automatic download all the way to creating train, valid and test binaries (for example: electricity dataset). For others, it expects user to download the dataset and from there it will do the remaining steps.
For downloading and cleaning up the datasets, the NVIDIA-TSPP provides the script: {tspp_ws}/data/script_download_data.py --dataset {dataset_name} --output_dir {output_dir_path} This script generates the output at the directory: {output_dir}/{dataset}

In the cell below, we will show the steps to download dataset directly by running the download script. If you have already downloaded the dataset, the next cell can be skipped.

In [7]:
!python {tspp_ws}/data/script_download_data.py --dataset {dataset} --output_dir {tspp_ws}/datasets
Using backend: pytorch
#### Running download script ###
Getting electricity data...
/workspace/datasets/electricity
Pulling data from https://archive.ics.uci.edu/ml/machine-learning-databases/00321/LD2011_2014.txt.zip to /workspace/datasets/electricity/LD2011_2014.txt.zip
100% [..................................................] 261335609 / 261335609done
Unzipping file: /workspace/datasets/electricity/LD2011_2014.txt.zip
Done.
Aggregating to hourly data
Done.
Download completed.

After download and initial dataset cleanup, we call NVIDIA-TSPP preprocessing functions for dataset splits, feature mapping, normalization, etc. Cell below will generate the train.csv, valid.csv and test.csv (also the .bin files respectively). More details about the method can be found in the notebook: 1_TsppOverview. If you have already preprocessed the dataset, next cell can be skipped.

In [8]:
!python {tspp_ws}/launch_preproc.py dataset={dataset}
{'_target_': 'data.data_utils.Preprocessor', 'config': {'graph': False, 'source_path': '/workspace/datasets/electricity/electricity.csv', 'dest_path': '/workspace/datasets/electricity/', 'time_ids': 'days_from_start', 'train_range': [0, 1315], 'valid_range': [1308, 1339], 'test_range': [1332, 10000], 'dataset_stride': 1, 'scale_per_id': True, 'encoder_length': 168, 'example_length': 192, 'MultiID': False, 'features': [{'name': 'categorical_id', 'feature_type': 'ID', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 371}, {'name': 'hours_from_start', 'feature_type': 'TIME', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'power_usage_weight', 'feature_type': 'WEIGHT', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'power_usage', 'feature_type': 'TARGET', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'hour', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 25}, {'name': 'day_of_week', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 8}, {'name': 'hours_from_start', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'categorical_id', 'feature_type': 'STATIC', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 371}], 'train_samples': 450000, 'valid_samples': 50000, 'binarized': True, 'time_series_count': 369}}
Reading in data from CSV File: /workspace/datasets/electricity/electricity.csv
Sorting on time feature
Mapping nodes
Mapping categoricals to bounded range
Splitting datasets
Calculating scalers
Applying scalers
Applying scalers
Applying scalers
Fixing any nans in continuous features
Fixing any nans in continuous features
Fixing any nans in continuous features
Saving preprocessor state at /workspace/datasets/electricity/tspp_preprocess.bin
Saving processed data at /workspace/datasets/electricity/

Train a quick TFT model on a single epoch for one of the available datasets in NVIDIA-TSPP. Take a look at the 1_TsppOverview notebook for data download and preprocessing instructions in addition to an in depth description on training.

In [9]:
checkpoint_dir = "outputs/3_ByoDataset_tft_" + dataset
!python {tspp_ws}/launch_training.py model=tft dataset={dataset} trainer/criterion=quantile \
    trainer.config.num_epochs=1 +trainer.config.force_rerun=true hydra.run.dir={checkpoint_dir}
Using backend: pytorch
 Training with 1 epochs 
 Epoch 0 
Epoch 0 | step 0 |avg loss 1.424 |walltime 0.861 |
Epoch 0 | step 25 |avg loss 0.604 |walltime 3.693 |
Epoch 0 | step 50 |avg loss 0.414 |walltime 6.526 |
Epoch 0 | step 75 |avg loss 0.324 |walltime 9.364 |
Epoch 0 | step 100 |avg loss 0.276 |walltime 12.220 |
Epoch 0 | step 125 |avg loss 0.251 |walltime 15.058 |
Epoch 0 | step 150 |avg loss 0.239 |walltime 17.904 |
Epoch 0 | step 175 |avg loss 0.231 |walltime 20.732 |
Epoch 0 | step 200 |avg loss 0.228 |walltime 23.556 |
Epoch 0 | step 225 |avg loss 0.219 |walltime 26.381 |
Epoch 0 | step 250 |avg loss 0.216 |walltime 29.208 |
Epoch 0 | step 275 |avg loss 0.215 |walltime 32.041 |
Epoch 0 | step 300 |avg loss 0.208 |walltime 34.869 |
Epoch 0 | step 325 |avg loss 0.208 |walltime 37.700 |
Epoch 0 | step 350 |avg loss 0.205 |walltime 40.526 |
Epoch 0 | step 375 |avg loss 0.206 |walltime 43.362 |
Epoch 0 | step 400 |avg loss 0.206 |walltime 46.194 |
Epoch 0 | step 425 |avg loss 0.203 |walltime 49.030 |
 Calculating Validation Metrics 
 Epoch 0 Validation Metrics: {'val_loss': 0.2297} 
Epoch 0 | step event |avg loss 0.202 |walltime 52.611 |
 Saving checkpoint to best_checkpoint.zip 
 Saving checkpoint to last_checkpoint.zip 
 Training Stopped 
 MAE : 47.242537606396986  RMSE : 409.92711211571003  SMAPE : 9.563726724549715  ND : 0.06218640786648368

3 | Kaggle Dataset

In this section, we will download the dataset from the kaggle competition: https://www.kaggle.com/competitions/jpx-tokyo-stock-exchange-prediction/data

Kaggle dataset can be easily downloaded with kaggle-api: https://github.com/Kaggle/kaggle-api. You need to have an active account on kaggle for this

In [10]:
!pip install --user kaggle --upgrade
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: kaggle in ./.local/lib/python3.8/site-packages (1.5.12)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.8/dist-packages (from kaggle) (1.26.13)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.8/dist-packages (from kaggle) (1.16.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.8/dist-packages (from kaggle) (2022.12.7)
Requirement already satisfied: python-slugify in ./.local/lib/python3.8/site-packages (from kaggle) (7.0.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.8/dist-packages (from kaggle) (2.8.2)
Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from kaggle) (2.28.1)
Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from kaggle) (4.64.1)
Requirement already satisfied: text-unidecode>=1.3 in ./.local/lib/python3.8/site-packages (from python-slugify->kaggle) (1.3)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->kaggle) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.8/dist-packages (from requests->kaggle) (2.1.1)
WARNING: You are using pip version 21.2.4; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.

Now download the kaggle.json in the directory: kaggle_credentials following the instructions: https://github.com/Kaggle/kaggle-api#api-credentials

In [11]:
os.makedirs("kaggle_credentials", exist_ok=True)
os.environ['KAGGLE_CONFIG_DIR'] = os.path.abspath("kaggle_credentials")
In [12]:
kaggle_json_path = os.path.join(os.path.abspath("kaggle_credentials"), "kaggle.json")
!chmod 600 {kaggle_json_path}
In [13]:
!~/.local/bin/kaggle competitions download -c jpx-tokyo-stock-exchange-prediction
Downloading jpx-tokyo-stock-exchange-prediction.zip to /home/jupyter
 93%|██████████████████████████████████████▏  | 224M/241M [00:01<00:00, 224MB/s]
100%|█████████████████████████████████████████| 241M/241M [00:01<00:00, 206MB/s]
In [14]:
!unzip jpx-tokyo-stock-exchange-prediction.zip -d jpx-tokyo-stock-exchange-prediction
Archive:  jpx-tokyo-stock-exchange-prediction.zip
  inflating: jpx-tokyo-stock-exchange-prediction/data_specifications/options_spec.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/data_specifications/stock_fin_spec.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/data_specifications/stock_list_spec.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/data_specifications/stock_price_spec.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/data_specifications/trades_spec.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/financials.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/options.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/sample_submission.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/secondary_stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/example_test_files/trades.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/jpx_tokyo_market_prediction/__init__.py  
  inflating: jpx-tokyo-stock-exchange-prediction/jpx_tokyo_market_prediction/competition.cpython-37m-x86_64-linux-gnu.so  
  inflating: jpx-tokyo-stock-exchange-prediction/stock_list.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/supplemental_files/financials.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/supplemental_files/options.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/supplemental_files/secondary_stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/supplemental_files/stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/supplemental_files/trades.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/train_files/financials.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/train_files/options.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/train_files/secondary_stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/train_files/stock_prices.csv  
  inflating: jpx-tokyo-stock-exchange-prediction/train_files/trades.csv

4 | Feature Engineering

In this section, we will review the dataset and create features for training. Our features are various aggregates on lag variables. We provide these as examples, and these could certainly be modified and improved.
Load stock_prices training dataset

In [15]:
dataset_path = os.path.join(curr_workdir, "jpx-tokyo-stock-exchange-prediction")
train_sp_file = "train_files/stock_prices.csv"
train_sp_file = os.path.join(dataset_path, train_sp_file)
In [16]:
df = pd.read_csv(train_sp_file)
print(df.head())
           RowId        Date  SecuritiesCode    Open    High     Low   Close  \
0  20170104_1301  2017-01-04            1301  2734.0  2755.0  2730.0  2742.0   
1  20170104_1332  2017-01-04            1332   568.0   576.0   563.0   571.0   
2  20170104_1333  2017-01-04            1333  3150.0  3210.0  3140.0  3210.0   
3  20170104_1376  2017-01-04            1376  1510.0  1550.0  1510.0  1550.0   
4  20170104_1377  2017-01-04            1377  3270.0  3350.0  3270.0  3330.0   

    Volume  AdjustmentFactor  ExpectedDividend  SupervisionFlag    Target  
0    31400               1.0               NaN            False  0.000730  
1  2798500               1.0               NaN            False  0.012324  
2   270800               1.0               NaN            False  0.006154  
3    11300               1.0               NaN            False  0.011053  
4   150800               1.0               NaN            False  0.003026

Cell below checks if all the 2000 stocks are available on all dates

In [17]:
%%script false --no-raise-error
idcount = df.groupby("Date")["SecuritiesCode"].count().reset_index()
plt.plot(idcount["Date"],idcount["SecuritiesCode"])
idcount.loc[idcount["SecuritiesCode"]==2000,:]

We will update the prices using AdjustmentFactor value. This should reduce historical price gap caused by split/reverse-split.

In [18]:
def adjust_price(price):
    """
    Ref: https://www.kaggle.com/code/smeitoma/train-demo#Generating-AdjustedClose-price
    Args:
        price (pd.DataFrame)  : pd.DataFrame include stock_price
    Returns:
        price DataFrame (pd.DataFrame): stock_price with generated Adjusted Prices
    """

    def generate_adjusted_price(df):
        """
        Args:
            df (pd.DataFrame)  : stock_price for a single SecuritiesCode
        Returns:
            df (pd.DataFrame): stock_price with Adjusted Price for a single SecuritiesCode
        """
        # sort data to generate CumulativeAdjustmentFactor
        df = df.sort_values("Date", ascending=False)
        # generate CumulativeAdjustmentFactor
        df.loc[:, "CumulativeAdjustmentFactor"] = df["AdjustmentFactor"].cumprod()
        # generate AdjustedClose
        df.loc[:, "AdjustedClose"] = (
            df["CumulativeAdjustmentFactor"] * df["Close"]
        ).map(lambda x: float(
            Decimal(str(x)).quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)
        ))
        df.loc[:, "AdjustedOpen"] = (
            df["CumulativeAdjustmentFactor"] * df["Open"]
        ).map(lambda x: float(
            Decimal(str(x)).quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)
        ))
        df.loc[:, "AdjustedLow"] = (
            df["CumulativeAdjustmentFactor"] * df["Low"]
        ).map(lambda x: float(
            Decimal(str(x)).quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)
        ))
        df.loc[:, "AdjustedHigh"] = (
            df["CumulativeAdjustmentFactor"] * df["High"]
        ).map(lambda x: float(
            Decimal(str(x)).quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)
        ))
        # reverse order
        df = df.sort_values("Date")
        # to fill AdjustedClose, replace 0 into np.nan
        df.loc[df["AdjustedClose"] == 0, "AdjustedClose"] = np.nan
        # forward fill AdjustedClose
        df.loc[:, "AdjustedClose"] = df.loc[:, "AdjustedClose"].ffill()

        df.loc[df["AdjustedOpen"] == 0, "AdjustedOpen"] = np.nan
        df.loc[:, "AdjustedOpen"] = df.loc[:, "AdjustedOpen"].ffill()

        df.loc[df["AdjustedLow"] == 0, "AdjustedLow"] = np.nan
        df.loc[:, "AdjustedLow"] = df.loc[:, "AdjustedLow"].ffill()
        
        df.loc[df["AdjustedHigh"] == 0, "AdjustedHigh"] = np.nan
        df.loc[:, "AdjustedHigh"] = df.loc[:, "AdjustedHigh"].ffill()
        
        return df

    # generate AdjustedClose
    price = price.sort_values(["SecuritiesCode", "Date"])
    price = price.groupby("SecuritiesCode").apply(generate_adjusted_price).reset_index(drop=True)

    return price

We will create a new set of features with Adjusted Close Price: mean, std, min, and max each over a period of 5 and 20 days respectively.

In [19]:
def create_features(price):
    """
    Args:
        price (pd.DataFrame)  : pd.DataFrame include stock_price
    Returns:
        price DataFrame (pd.DataFrame): stock_price with new generated features
    """

    def generate_features_single_stock(df):
        """
        Args:
            df (pd.DataFrame)  : stock_price for a single SecuritiesCode
        Returns:
            df (pd.DataFrame): stock_price with new features for a single SecuritiesCode
        """
        
        df['Close_1week_mean'] = df['AdjustedClose'].rolling(window = 5).mean().fillna(0)
        df['Close_4weeks_mean'] = df['AdjustedClose'].rolling(window = 20).mean().fillna(0)
        df['Close_1week_std'] = df['AdjustedClose'].rolling(window = 5).std().fillna(0)
        df['Close_4weeks_std'] = df['AdjustedClose'].rolling(window = 20).std().fillna(0)
        df['Close_1week_min'] = df['AdjustedClose'].rolling(window = 5).min().fillna(0)
        df['Close_4weeks_min'] = df['AdjustedClose'].rolling(window = 20).min().fillna(0)
        df['Close_1week_max'] = df['AdjustedClose'].rolling(window = 5).max().fillna(0)
        df['Close_4weeks_max'] = df['AdjustedClose'].rolling(window = 20).max().fillna(0)
        return df
    
    price = price.sort_values(["SecuritiesCode", "Date"])
    price = price.groupby("SecuritiesCode").apply(generate_features_single_stock).reset_index(drop=True)

    return price

Here, we define a function for creating dataset features for NVIDIA-TSPP's preprocessing

In [20]:
def process_raw_data(df, min_timestamp=None):
    df["Date"] = pd.to_datetime(df["Date"])
    df['DayOfWeek'] = df['Date'].apply(lambda x: x.dayofweek)
    df['Month'] = df['Date'].apply(lambda x: x.month)
    df['Day'] = df['Date'].apply(lambda x: x.day)
    df['Hour'] = df['Date'].apply(lambda x: x.hour)
    df['Minute'] = df['Date'].apply(lambda x: x.minute)
    df['Timestamp'] = df['Date']
    df['id'] = df["SecuritiesCode"]
    df['Weight'] = 1
    df = adjust_price(df)
    df = create_features(df)
    df['DayOfYear'] = df['Date'].apply(lambda x: x.dayofyear)
    df['Year'] = df['Date'].apply(lambda x: x.year)
    df['IsMonday'] = (df['DayOfWeek'] == 0).astype(int)
    # Add continuous timeline
    if min_timestamp:
        min_timestamp = pd.to_datetime(min_timestamp)
    else:
        min_timestamp = min(df['Timestamp'])
    df['Timestep'] = df['Timestamp'].apply(lambda x: (x-min_timestamp)/datetime.timedelta(days=1))
    df['Timestep_id'] = df['Timestep']
    df = df.set_index('Date')
    return df

Create output directory for dataset

In [21]:
out_dataset_dir = F"{tspp_ws}/datasets/jpx"
os.makedirs(out_dataset_dir, exist_ok=True)

Note that we can model missing stocks too, because NVIDIA-TSPP maintains a separate scaler per stock id. We went for simplicity here.

Select stock prices for dates from 2021-01-01 onwards and save the updated features dataframe

In [22]:
train_df = df.loc[df["Date"]>= "2021-01-01"].reset_index(drop=True)
train_df = process_raw_data(train_df)
train_df.to_csv(os.path.join(out_dataset_dir, 'filtered_stock_prices.csv'))

5 | Data Preprocessing

In this section, we will preprocess the dataset for NVIDIA-TSPP

NVIDIA-TSPP dataset parameters

TSPP Features

As NVIDIA-TSPP works with yaml-based configuration files, we need to create a configuration file for the kaggle dataset.
Cell below will print the configuration file for the electricity dataset

In [23]:
with open(os.path.join(tspp_ws, "conf/dataset/electricity.yaml"), "rb") as f:
    elec_config = OmegaConf.load(f)
    print("\nDataset Config\n", OmegaConf.to_yaml(elec_config))

Dataset Config
 _target_: data.datasets.create_datasets
config:
  graph: false
  source_path: /workspace/datasets/electricity/electricity.csv
  dest_path: /workspace/datasets/electricity/
  time_ids: days_from_start
  train_range:
  - 0
  - 1315
  valid_range:
  - 1308
  - 1339
  test_range:
  - 1332
  - 10000
  dataset_stride: 1
  scale_per_id: true
  encoder_length: 168
  example_length: 192
  MultiID: false
  features:
  - name: categorical_id
    feature_type: ID
    feature_embed_type: CATEGORICAL
    cardinality: 371
  - name: hours_from_start
    feature_type: TIME
    feature_embed_type: CONTINUOUS
  - name: power_usage_weight
    feature_type: WEIGHT
    feature_embed_type: CONTINUOUS
  - name: power_usage
    feature_type: TARGET
    feature_embed_type: CONTINUOUS
    scaler:
      _target_: sklearn.preprocessing.StandardScaler
  - name: hour
    feature_type: KNOWN
    feature_embed_type: CATEGORICAL
    cardinality: 25
  - name: day_of_week
    feature_type: KNOWN
    feature_embed_type: CATEGORICAL
    cardinality: 8
  - name: hours_from_start
    feature_type: KNOWN
    feature_embed_type: CONTINUOUS
    scaler:
      _target_: sklearn.preprocessing.StandardScaler
  - name: categorical_id
    feature_type: STATIC
    feature_embed_type: CATEGORICAL
    cardinality: 371
  train_samples: 450000
  valid_samples: 50000
  binarized: true
  time_series_count: 369

Some of the relevant fields are described below:

  1. config.source_path: path to the source CSV file that contains the dataset
  2. config.dest_path: output directory path for NVIDIA-TSPP's preprocessing
  3. config.time_ids: The name of the column within the source CSV that is used to split training, validation, and test datasets. In the configuration, we can specify the range as shown above using: config.train_range for the train dataset, the example above has the range [0, 1315). Similarly we can use config.valid_range & config.test_range for the validation and test sets respectively. Remember that there can be overlap between subsets since predicting the first ‘unseen element’ requires the input of the seen elements before it.
  4. config.dataset_stride: This is used to define how far apart in time different examples are in the dataset. When set to 1, all nearest examples will differ only by one time step
  5. config.scale_per_id: If true, preprocessing uses different scaling factors and biases for each of the time-series in the dataset
  6. config.encoder_length: The length of data known up until the ‘present’ for a single sample
  7. config.example_length: The length of all data, including data known into the future for a single sample. The target you are predicting lies on the difference between the example_length and encoder_length.
  8. config.input_length: same as encoder_length
  9. config.features: We will go over features and their properties in a bit more detail in the next cell.
  10. config.train_samples, config.valid_samples: Randomly subsample train and valid splits to reduce amount of correlated data fed to the model during a training epoch
  11. config.binarized: Store train, validation and test splits in binarized versions to improve the data-loading speed (by default csv files are created)
  12. config.time_series_count: The number of unique time-series contained in the dataset
  13. config.graph: option is relevant for models that can use graph related information. For example, time-series forecasting using gnns
  14. config.MultiID: option is relevant for models that can use multiple time-series as inputs or predict multiple time-series in a single step

Feature Specification in Dataset configuration files:

The NVIDIA-TSPP requires a feature list for each dataset that the models take as input. Each feature should be represented by an object containing descriptive attributes based on the supported types. These are used for mapping input dataset columns to types. Different types of features are treated differently during preprocessing and model runtime.

Each feature should have atleast:
config.features.feature_type: OPTIONS(ID, KNOWN, STATIC, OBSERVED, TARGET, WEIGHT, TIME)

ID: Unique Identifier representing a time-series node/channel/sensor. Example: Securities Code
KNOWN: Features known in advance for both history and future. Example: Hour, day of week, etc
STATIC: Constant feature throughout a single time series. Ex: physical location of a sensor
OBSERVED: Features known only for historical data
TARGET: Target value to predict. Use history target value as model's input. Use Future targets for calculating loss. Example: Power Usage, Stock Closing Price
WEIGHT: Some of the target values could be missing from the dataset for some timesteps. This can be used to mask those entries during loss calculation.
TIME: Unique timestamp to create the time-series. Example: Hours from Start, Time Steps

and
config.features.feature_embed_type: OPTIONS(CATEGORICAL, CONTINUOUS)

CATEGORICAL: Features that usually have a fixed number possible values. For some models, they can be passed through categorical embedding tables that are trainable. Categorical columns should have a cardinality attribute that represents the number of unique values that the feature takes using config.features.cardinality. Cardinality is 1+number of categories where extra category is for supporting NaNs.
CONTINUOUS: Represents real-valued features. Continuous features may have a scaler attribute that represents the type of scaler used in preprocessing using config.features.scaler

Required features are one TIME feature, at least one ID feature, one TARGET feature, and at least one KNOWN, OBSERVED, or STATIC feature.

A single input sample to the model is a dictionary of features grouped into the categories specified above. The Dataset class provides a utility to iterate over a monolithic csv (or it's binarized version) creating examples on the fly. The NVIDIA-TSPP assumes that a model doesn't need any more information than the type of a group and the underlying feature tensor.

Create a new dataset configuration file

In the cell below, we will start with the electricity dataset configuration that is already available within NVIDIA-TSPP's code and make changes to it for this competition's dataset

In [24]:
# CREATE DATASET CONFIG HERE
config = None
forecast_len = 2
out_conf_dir = f"{tspp_ws}/conf/dataset"
with open(os.path.join(tspp_ws, "conf/dataset/electricity.yaml"), "rb") as f:
    config = OmegaConf.load(f)
#print(config)
config.config.source_path = os.path.join(out_dataset_dir, 'filtered_stock_prices.csv')
config.config.dest_path = out_dataset_dir
config.config.time_ids = 'Timestep_id'

config.config.encoder_length = 10
config.config.example_length = 10 + forecast_len
config.config.input_length = 10

# This feature is using by xgboost model only: Its similar to setting encoder_length of 10 xgboost
config.config.lag_features=[{'name': 'AdjustedClose', 'min_value': 1, 'max_value': 10}]

# Train-Valid-Test Split
num_samples = len(train_df["Timestep_id"].unique())
num_test = round(num_samples * 0.1)
num_train = round(num_samples * 0.8)
num_val = num_samples - num_test - num_train
timesteps = train_df["Timestep_id"].unique()
train_start = int(timesteps[0])
train_end = int(timesteps[num_train])
valid_start = int(timesteps[num_train - config.config.encoder_length])
valid_end = int(timesteps[num_train + num_val])
test_start = int(timesteps[num_train + num_val - config.config.encoder_length])
test_end = int(timesteps[-1]+1)
print(F"Train Range: [{train_start}, {train_end})")
print(F"Valid Range: [{valid_start}, {valid_end})")
print(F"Test Range: [{test_start}, {test_end})")
config.config.train_range = [train_start, train_end]
config.config.valid_range = [valid_start, valid_end]
config.config.test_range = [test_start, test_end]

config.config.scale_per_id = True
#Generate Feature Spec

time_series_count = len(train_df["SecuritiesCode"].unique())

features = [
    {'name': 'id', 'feature_type': 'ID', 'feature_embed_type': 'CATEGORICAL', 'cardinality': time_series_count+1},
    {'name': 'Timestep', 'feature_type': 'TIME', 'feature_embed_type': 'CONTINUOUS'},
    {'name': 'Weight', 'feature_type': 'WEIGHT', 'feature_embed_type': 'CONTINUOUS'},
    {'name': 'AdjustedClose', 'feature_type': 'TARGET', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'DayOfWeek', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 6},
    {'name': 'Timestep', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Month', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 13},
    {'name': 'Day', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 32},
    {'name': 'AdjustedOpen', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'AdjustedHigh', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'AdjustedLow', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Volume', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'id', 'feature_type': 'STATIC', 'feature_embed_type': 'CATEGORICAL', 'cardinality': time_series_count+1},
    {'name': 'IsMonday', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 3},
    {'name': 'Close_1week_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_1week_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_1week_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_1week_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_4weeks_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_4weeks_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_4weeks_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Close_4weeks_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'DayOfYear', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
    {'name': 'Year', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}},
]
    
config.config.features = features
config.config.time_series_count = time_series_count
config.config.train_samples = -1
config.config.valid_samples = -1

with open(os.path.join(out_conf_dir, "jpx.yaml"), "wb") as f:
    OmegaConf.save(config=config, f=f.name)

print(config)
Train Range: [0, 269)
Valid Range: [253, 301)
Test Range: [287, 334)
{'_target_': 'data.datasets.create_datasets', 'config': {'graph': False, 'source_path': '/workspace/datasets/jpx/filtered_stock_prices.csv', 'dest_path': '/workspace/datasets/jpx', 'time_ids': 'Timestep_id', 'train_range': [0, 269], 'valid_range': [253, 301], 'test_range': [287, 334], 'dataset_stride': 1, 'scale_per_id': True, 'encoder_length': 10, 'example_length': 12, 'MultiID': False, 'features': [{'name': 'id', 'feature_type': 'ID', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 2001}, {'name': 'Timestep', 'feature_type': 'TIME', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'Weight', 'feature_type': 'WEIGHT', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'AdjustedClose', 'feature_type': 'TARGET', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'DayOfWeek', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 6}, {'name': 'Timestep', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Month', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 13}, {'name': 'Day', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 32}, {'name': 'AdjustedOpen', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'AdjustedHigh', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'AdjustedLow', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Volume', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'id', 'feature_type': 'STATIC', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 2001}, {'name': 'IsMonday', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 3}, {'name': 'Close_1week_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'DayOfYear', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Year', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}], 'train_samples': -1, 'valid_samples': -1, 'binarized': True, 'time_series_count': 2000, 'input_length': 10, 'lag_features': [{'name': 'AdjustedClose', 'min_value': 1, 'max_value': 10}]}}
In [25]:
!ls {tspp_ws}/conf/dataset
electricity.yaml  jpx.yaml  traffic.yaml

NVIDIA-TSPP dataset preprocessing

Cell below will generate train.csv, valid.csv and test.csv (also .bin files if config.binarized is set to True) at the output directory set with option: config.dest_path above

In [26]:
dataset = "jpx"
In [27]:
skip_preprocessing = False

# Dataset Proprocessing
if not skip_preprocessing:
    !python {tspp_ws}/launch_preproc.py dataset={dataset}
{'_target_': 'data.data_utils.Preprocessor', 'config': {'graph': False, 'source_path': '/workspace/datasets/jpx/filtered_stock_prices.csv', 'dest_path': '/workspace/datasets/jpx', 'time_ids': 'Timestep_id', 'train_range': [0, 269], 'valid_range': [253, 301], 'test_range': [287, 334], 'dataset_stride': 1, 'scale_per_id': True, 'encoder_length': 10, 'example_length': 12, 'MultiID': False, 'features': [{'name': 'id', 'feature_type': 'ID', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 2001}, {'name': 'Timestep', 'feature_type': 'TIME', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'Weight', 'feature_type': 'WEIGHT', 'feature_embed_type': 'CONTINUOUS'}, {'name': 'AdjustedClose', 'feature_type': 'TARGET', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'DayOfWeek', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 6}, {'name': 'Timestep', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Month', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 13}, {'name': 'Day', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 32}, {'name': 'AdjustedOpen', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'AdjustedHigh', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'AdjustedLow', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Volume', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'id', 'feature_type': 'STATIC', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 2001}, {'name': 'IsMonday', 'feature_type': 'KNOWN', 'feature_embed_type': 'CATEGORICAL', 'cardinality': 3}, {'name': 'Close_1week_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_1week_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_mean', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_std', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_min', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Close_4weeks_max', 'feature_type': 'OBSERVED', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'DayOfYear', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}, {'name': 'Year', 'feature_type': 'KNOWN', 'feature_embed_type': 'CONTINUOUS', 'scaler': {'_target_': 'sklearn.preprocessing.StandardScaler'}}], 'train_samples': -1, 'valid_samples': -1, 'binarized': True, 'time_series_count': 2000, 'input_length': 10, 'lag_features': [{'name': 'AdjustedClose', 'min_value': 1, 'max_value': 10}]}}
Reading in data from CSV File: /workspace/datasets/jpx/filtered_stock_prices.csv
Sorting on time feature
Mapping nodes
Mapping categoricals to bounded range
Splitting datasets
Calculating scalers
Applying scalers
Applying scalers
Applying scalers
Fixing any nans in continuous features
Fixing any nans in continuous features
Fixing any nans in continuous features
Saving preprocessor state at /workspace/datasets/jpx/tspp_preprocess.bin
Saving processed data at /workspace/datasets/jpx

6 | Training

Launch training through the command-line
All the model related parameter are represented as model.config.parameter_name. This will depend on the model implementation.
hydra.run.dir: explicitly sets the output directory for training to prevent the autogenerated output directory by hydra.
tft model related parameters can be found as shown below

In [28]:
with open(os.path.join(tspp_ws, "conf/model/tft.yaml"), "rb") as f:
    elec_config = OmegaConf.load(f)
    print("\nModel Config\n", OmegaConf.to_yaml(elec_config))

Model Config
 _target_: models.tft_pyt.modeling.TemporalFusionTransformer
config:
  quantiles:
  - 0.5
  n_head: 4
  hidden_size: 160
  dropout: 0.1
  attn_dropout: 0
  output_selector: 0
defaults:
- _self_
- /trainer@_global_/trainer: ctltrainer
In [29]:
model_name = "tft"
output_workdir = os.path.join(curr_workdir, F'outputs/3_ByoDataset_{model_name}_{dataset}')
os.makedirs(output_workdir, exist_ok = True)
! python {tspp_ws}/launch_training.py model={model_name} dataset={dataset} trainer.config.num_epochs=4 hydra.run.dir={output_workdir} \
    +trainer.config.force_rerun=True ++trainer.config.log_interval=100 trainer.config.batch_size=1024 \
    model.config.hidden_size=168 model.config.n_head=4 model.config.dropout=0.1 trainer.optimizer.lr=0.0009
Using backend: pytorch
 Training with 4 epochs 
 Epoch 0 
Epoch 0 | step 0 |avg loss 1.193 |walltime 0.668 |
Epoch 0 | step 100 |avg loss 0.187 |walltime 7.322 |
Epoch 0 | step 200 |avg loss 0.092 |walltime 13.966 |
Epoch 0 | step 300 |avg loss 0.087 |walltime 20.537 |
 Calculating Validation Metrics 
 Epoch 0 Validation Metrics: {'val_loss': 0.2271} 
Epoch 0 | step event |avg loss 0.087 |walltime 24.218 |
 Saving checkpoint to best_checkpoint.zip 
 Saving checkpoint to last_checkpoint.zip 
 Epoch 1 
Epoch 1 | step 400 |avg loss 0.086 |walltime 29.254 |
Epoch 1 | step 500 |avg loss 0.084 |walltime 35.879 |
Epoch 1 | step 600 |avg loss 0.083 |walltime 42.457 |
 Calculating Validation Metrics 
 Epoch 1 Validation Metrics: {'val_loss': 0.3053} 
Epoch 1 | step event |avg loss 0.081 |walltime 48.297 |
 Saving checkpoint to last_checkpoint.zip 
 Epoch 2 
Epoch 2 | step 700 |avg loss 0.088 |walltime 51.414 |
Epoch 2 | step 800 |avg loss 0.079 |walltime 58.039 |
Epoch 2 | step 900 |avg loss 0.080 |walltime 64.762 |
 Calculating Validation Metrics 
 Epoch 2 Validation Metrics: {'val_loss': 0.2264} 
Epoch 2 | step event |avg loss 0.077 |walltime 72.821 |
 Saving checkpoint to best_checkpoint.zip 
 Saving checkpoint to last_checkpoint.zip 
 Epoch 3 
Epoch 3 | step 1000 |avg loss 0.091 |walltime 74.187 |
Epoch 3 | step 1100 |avg loss 0.076 |walltime 80.808 |
Epoch 3 | step 1200 |avg loss 0.078 |walltime 87.439 |
Epoch 3 | step 1300 |avg loss 0.076 |walltime 93.996 |
 Calculating Validation Metrics 
 Epoch 3 Validation Metrics: {'val_loss': 0.2137} 
Epoch 3 | step event |avg loss 0.076 |walltime 97.590 |
 Saving checkpoint to best_checkpoint.zip 
 Saving checkpoint to last_checkpoint.zip 
 Training Stopped 
 MAE : 98.33266601211375  RMSE : 279.1561004009957  SMAPE : 3.3435420170672643  ND : 0.03407858407773056

7 | Evaluation

After training is finished, the trained model is used to evaluate performance on the test set.

In [30]:
! python {tspp_ws}/launch_inference.py checkpoint={output_workdir}
{'inference': {'_target_': 'inference.inference.run_inference', 'config': {'checkpoint': '???', 'batch_size': 64, 'precision': 'fp32', 'device': 'cuda'}}, 'checkpoint': '/home/jupyter/outputs/3_ByoDataset_tft_jpx'}
Using backend: pytorch
 Evaluation Metrics: {'MAE': 98.33266601211375, 'RMSE': 279.1561004009957, 'SMAPE': 3.3435420170672643, 'ND': 0.03407858407773056}

This notebook shows how to create an inference pipeline with NVIDIA-TSPP for daily stock predictions

Back To Top

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.