Repository structure

.
|-- demos            # Directory with all the Jupyter examples
|-- docker_scripts   # Directory with Docker scripts
|-- scripts          # Directory with datasets scripts
|-- syngen              # Directory with Synthetic Graph Generation source code
│  |-- analyzer             # Directory with tools for getting graph visualisation and statistics
│  │   |-- graph                    # Directory with graph structure analyzer
│  │   |-- tabular                  # Directory with tabular features analyzer
│  |-- benchmark            # Directory with pretraining tools
│  │   |-- data_loader              # Directory with pre-defined node and edge classification datasets
│  │   |-- models                   # Directory with GNN model definitions
│  │   |-- tasks                    # Directory with set of tasks that are supported for training
│  |-- generator            # Directory with all the generators
│  │   |-- graph                    # Directory with graph generators and graph
│  │   |-- tabular                  # Directory with tabular generators
│  │   |   |-- data_transformer         # Directory with tabular data transformations used by generators
│  │   |   |-- transforms               # Directory with tabular column transforms
│  |-- graph_aligner      # Directory with all the aligners
│  |-- preprocessing        # Directory with the preprocessings for the supported datasets
│  │   |-- datasets                 # Directory with example dataset preprocessing scripts used to generate data
│  |-- synthesizer          # Directory with all the synthesizers
│  |-- utils                # Directory with the utilities
│      |-- types                    # Directory with common data types used in the tool

Important scripts and files

• scripts/get_datasets.sh - Bash script downloading and preprocessing supported datastes
• docker_scripts/build_docker.sh - Bash script that builds the Docker image
• docker_scripts/run_docker_notebook.sh - Bash script that runs Jupyter notebook in the Docker container
• docker_scripts/run_docker_interactive.sh - Bash script that runs the Docker container in interactive mode
• syngen/__main__.py - Python script that defines Synthetic Graph Generation CLI
• syngen/synthesizer/static_graph_synthesizer.py - Python file with non-partite graph synthesizer
• syngen/synthesizer/static_bipartite_graph_synthesizer.py - Python file with bipartite graph synthesizer

Parameters

For the synthesizer, refer to the parameters in the following table.

For the pretraining refer to the Command-line options, as the parameters depend on the model choice.

Command-line options

To display the full list of available options and their descriptions, use the -h or --help command-line option:

syngen --help

The tool currently support the synthesize and pretrain commands. To display the full list of available options for the respective command run:

syngen <command> --help

Define the synthesizer pipeline

In this example, we show how to define the synthesizer pipeline for IEEE dataset. A full example can be found in ieee_notebook.

Prepare synthesizer

• Feature generator is used to generate tabular features. For the final graph, we use only edge features as there are no node features in the IEEE dataset. In this example, we use the CTGAN generator.

edge_feature_generator = CTGANGenerator(epochs=10, batch_size=2000, verbose=False)

• Structure generator is used to generate graph structure. In this example, we use RMAT implementation

static_graph_generator = RMATBipartiteGenerator()

• Preprocessing is required for all datasets to work. For custom datasets, users need to create their own preprocessing following the base API. In the repository, we provide implementations for all the supported datasets.

preprocessing = IEEEPreprocessing(cached=False)

• Aligner is necessary to properly align generated structure with tabular features, as those two processes are independent of each other. In this example, we use an XGBoost aligner and specify which features to correlate to which structure.

graph_aligner = XGBoostAligner(features_to_correlate_edge={'TransactionAmt': ColumnType.CONTINUOUS})

• Synthesizer is a class that combines all the generators and allows the user to run end-to-end fitting and generation. We use StaticBipartiteGraphSynthesizer because ieee is a bipartite dataset. To select whether to run the computation on GPU or not, use the gpu flag. By setting is_directed=True, we say that the graph is undirected.

synthesizer = StaticBipartiteGraphSynthesizer(
                                    graph_generator=static_graph_generator,
                                    graph_info=preprocessing.graph_info,
                                    edge_feature_generator=edge_feature_generator,
                                    graph_aligner=graph_aligner,
                                    is_directed=False,
                                    gpu=True)

Generate graph

• To generate a graph, first we need to extract graph data (structure and features) from the preprocessing. This can be done by calling provided .transform method on the dataset path.

data = preprocessing.transform(dataset_path)

• To run fitting for all the generators, we use the fit method provided by the synthesizer. We pass only edge_data as ieee is a bipartite dataset with edge features only.

synthesizer.fit(edge_data=data[MetaData.EDGE_DATA])

• To run generation, we call the generate method provided by the synthesizer. We can provide the number of nodes in both partites and the number of edges for each direction. In our case, this is the same number because we specified that we have an undirected graph during synthesizer instantiation.

data_proper = synthesizer.generate(num_nodes_src_set,
                                   num_nodes_dst_set,
                                   num_edges_src_dst,
                                   num_edges_dst_src)

Getting the data

To download the datasets used as an example , use get_datasets.sh script

bash scripts/get_datasets.sh

Note: Certain datasets require a Kaggle API key, hence may require manual download. Refer to the links below. Note: Each user is responsible for checking the content of datasets and the applicable licenses and determining if they are suitable for the intended use

List of datasets

Supported datasets:

• Twitch
• LastFM
• Orkut
• Tabformer
• IEEE
• Paysim
• Credit
• CORA
• Rating