Getting Started

To use the tool, perform the following steps. For the specifics concerning generation and training, refer to the Advanced section.

1. Clone the repository.

git clone https://github.com/NVIDIA/DeepLearningExamples

2. Go to the SyntheticGraphGeneration tool directory within the DeepLearningExamples repository:

cd DeepLearningExamples/Tools/DGLPyTorch/SyntheticGraphGeneration

3. Build the SyntheticGraphGeneration container.

bash docker_scripts/build_docker.sh

4. Download the datasets. (It is advisable to run this command inside docker interactive container to ensure environment setup, see 6.1)

bash scripts/get_datasets.sh

Note: This script requires a manual download of 4 datasets (tabformer, ieee, paysim, credit) and putting them into ./data directory with the correct naming. The instruction for the manual download will be printed during the preprocessing. If the raw data is not present or the dataset is already preprocessed, the preprocessing will be skipped. 5. Run the SyntheticGraphGeneration Jupyter notebook.

5.1. Run the Docker notebook container.

bash docker_scripts/run_docker_notebook.sh

5.2 Open Jupyter notebook.

http://localhost:9916/tree/demos

6. Run the SyntheticGraphGeneration CLI.

6.1. Run the Docker interactive container.

bash docker_scripts/run_docker_interactive.sh

6.2. Run Command Line Interface (CLI) command.

The tool contains two run commands: synthesize and pretrain

For example, to synthesize a graph similar to the IEEE dataset, run the following command:

syngen synthesize \
--synthesizer static_bipartite_graph \
--preprocessing ieee \
--aligner xg_boost \
--graph-generator rmat_bipartite \
--gg-seed 42 \
--edge-generator ctgan \
--eg-batch-size 2000 \
--eg-epochs 10 \
--num-nodes-src-set 17091 \
--num-nodes-dst-set 198 \
--num-edges-src-dst 52008 \
--num-edges-dst-src 52008 \
--data-path '/workspace/data/ieee-fraud/data.csv' \
--save-path '/workspace/ieee/' \
--features-to-correlate-edge "{\"TransactionAmt\": \"continuous\"}"

In this example run, a graph of similar size as the IEEE dataset is generated using the fitted synthesizer.

Note: In the above command, the static_bipartite_graph synthesizer is used, along with ctgan as the edge-generator and xg_boost aligner for assigning features to nodes. Accordingly, --data-path points to the location where the IEEE dataset is extracted, and --save-path points to the location where the generated data is saved.

Following the above command, the pretrain command can be used to pre-train or fine-tune the given generated sample.

syngen pretrain \
--model gat_ec \
--hidden-dim 64 \
--out-dim 32 \
--n-layers 1 \
--n-heads 2 \
--weight-decay 0.0 \
--learning-rate 0.0005 \
--batch-size 256 \
--pretrain-epochs 5 \
--finetune-epochs 5 \
--data-path '/workspace/data/ieee-fraud/data.csv' \
--pretraining-path '/workspace/ieee/' \
--preprocessing ieee \
--task ec \
--target-col isFraud \
--num-classes 2

Note: The current set of tasks and models are solely provided as use case examples on how to use the generated synthetic data to pretrain/fine-tune on a downstream task, and generally would need extension/modifications to accomodate very large graphs or arbitrary models.

For the complete CLI usage of the synthesize command run:

syngen synthesize --help

Similarly for the pretrain command run:

syngen pretrain --help