Description

AutoGEMM is a software package that offers a family of Python code generators for high-performance GEMM (General Matrix-Matrix Multiplication) implementations based on Apache TVM.

This guide complements the submitted paper Automatic Generators for a Family of Matrix Multiplication Routines with Apache TVM and includes basic setup and execution steps for the package.

The provided software package includes all the generators described in the paper (listed in Sections 5, 6, and 7 of the manuscript) and used to extract the performance plots in Section 8, together with a simplified high-level driver program to easily use them.

Package structure

The software bundle consists on two main directories:

• doc/

• src/: Main code directory containing the driver and configuration files prepared to conduct experiments.

Within the src/ directory, the Python scripts for the generators described in the paper can be found in the generators subdirectory:

• src/generators/: Directory containing Python generators leveraging Apache TVM to generate architecture-specific GEMM codes. Specifically, the included files mimic the code generators listed in the manuscript, namely:

Installation steps

Prerequisites and supported platforms

AutoGEMM relies on two software prerequisites: any installation of Python 3, and Apache TVM. The prerequisites for the latter are not considered in this documentation, but will be automatically managed and installed by the following installation steps. To ease the installation process, the following instructions are based on the creation of a virtual environment via Python.

Creation of a virtual environment

# Installation of Python3 virtualenv (considering a Debian-based Linux distribution)
user@machine:~$ apt-get install python3-virtualenv
# Creation of a virtual Python environment with name .tvm
user@machine:~$ python3 -m venv .tvm
# Activation of the virtual environment
user@machine:~$ source .tvm/bin/activate
(.tvm) user@machine:~$

Additional software prerequisites

# Installation of additional software requisites via pip
(.tvm) user@machine:~$ pip install typing_extensions pytest

Installation of Apache TVM

# Installation of Apache TVM and dependences
(.tvm) user@machine:~$ pip install apache-tvm

Driver configuration and usage

Configuration files

Execution configuration file (exec.cfg)

This configuration file includes the specific conditions for the execution of one experiment. The file is structured in sections, each one including a number of mandatory parameters (we consider next the operation with matrices input matrices A and B, and output matrix C, C := A * B:

• Section [GENERAL]:
  • repeats: number of repetitions of the test (boolean).
  • assembly: generation of the assembly code for further analysis (boolean).
• Section [EXPERIMENT]:
  • variant: algorithmic variant for GEMM (string). Supported values are B3A2C0, A3C2B0 and C3A2B0.
  • opt_level: optimization level of the TVM generator (string). Accepted values for variant B3A2C0 are basic, blocked, packed, packed_ukernel, opt_ukernel and opt_ukernel_par. Variants A3C2B0 and C3A2B0 only accept the opt_ukernel optimization level.
  • dtype: datatype to use (string).
• Section [PROBLEM]:
  • M: number of rows of matrix C.
  • N: number of columns of matrix C.
  • K: number of columns of matrix A/rows of matrix B.
• Section [BLOCKSIZES]:
  • mr: register blocksize m_r (integer).
  • nr: register blocksize n_r (integer).
  • kr: register blocksize k_r (integer).
  • MC: cache blocksize m_c (integer).
  • NC: cache blocksize n_c (integer).
  • KC: cache blocksize k_c (integer).
  • ls: lanesize, numer of lanes in vector registers of the target architecture (integer).

Example of configuration file exec.cfg:

#########################
[GENERAL]
repeats = 1
assembly = 1

[EXPERMIENT]
variant = B3A2C0
opt_level = opt_ukernel
dtype = float32

[PROBLEM]
M = 4096
N = 4096
K = 4096

[BLOCKSIZES]
mr = 4
nr = 16
kr = 4

MC = 256
NC = 1280
KC = 128

ls = 16
#########################

Machine configuration file (machine.cfg)

This configuration file includes the specific characteristics of the target architecture. In this version, this information includes a label describing the architecture, and the llvm target used to generate code, as follows:

• Section [MACHINE]:
  • name: label to identify the experiment, under user’s choice (string).
  • target: llvm target identifier (string).

Example of configuration file machine.cfg (in this example, targeting an Intel Icelake architecture):

#########################
[MACHINE]
name = icelake

target = llvm -mcpu=icelake-server
#########################

Some examples for target used in the manuscript include:

Driver execution

The file driver.py accepts two mandatory arguments that indicate the path to the configuration files. A detailed usage guide can be extrated upon execution with the -h argument:

(.tvm) user@machine:~$ python3 driver.py -h
## Test driver for BLAS generators
usage: Test driver. [-h] -c EXECCONFIG -m MACHINECONFIG

Description

optional arguments:
  -h, --help            show this help message and exit
  -c EXECCONFIG, --execconfig EXECCONFIG
                        Execution configuration file
  -m MACHINECONFIG, --machineconfig MACHINECONFIG
                        Machine configuration file

An example execution of the driver is illustrated next:

(.tvm) user@machine:~$ python3 driver.py -c exec.cfg -m machine.cfg
## Test driver for BLAS generators
icelake,B3A2C0_opt_ukernel_par,4096,4096,4096,256,1280,128,4,16,4,465.45

Note that the CSV output includes the execution information, namely:

• Machine name.
• Execution configuration in the format variant_optimizationlevel.
• Matrix dimensions (M, N, K).
• Cache blocksizes (MC, NC, KC).
• Register blocksizes (mr, nr, kr).
• Performance (in terms of GFLOPS).

If the assembly file has been requested in the execution configuration file, it will be placed in the assembly folder with name variant_MxNxK_mrxnr.s (e.g. B3A2C0_256x1280x128_4x16.s).

Parallel executions

For codes generated with support for multithreading, the environment variable TVM_NUM_THREADS controls the number of threads deployed for execution. As an example, the following execution:

(.tvm) user@machine:~$ TVM_NUM_THREADS=1 python3 driver.py -c exec.cfg -m machine.cfg
## Test driver for BLAS generators
icelake,B3A2C0_opt_ukernel_par,4096,4096,4096,256,1280,128,4,16,4,32.46

would execute a sequential version of the corresponding GEMM code, whereas

(.tvm) user@machine:~$ TVM_NUM_THREADS=64 python3 driver.py -c exec.cfg -m machine.cfg
## Test driver for BLAS generators
icelake,B3A2C0_opt_ukernel_par,4096,4096,4096,256,1280,128,4,16,4,464.63

would execute a parallel version deploying 64 threads (observe the difference in performance reported as the last field of the comma-separated output). By default, TVM deploys as many threads as cores are available in the system.

Prototypes of the generators

• Prototypes:

      def basic_GEMM_B3A2C0(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def blocked_GEMM_B3A2C0(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def packed_GEMM_B3A2C0(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def opt_GEMM_B3A2C0_ukernel(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def opt_GEMM_B3A2C0_ukernel_parallel(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def opt_GEMM_C3A2C0_ukernel(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

      def opt_GEMM_A3C2B0_ukernel(m, n, k, mc, nc, kc, mr, nr, kr, lanesize, dtype, target)

• Parameters:

• Return value:

  func: Code module for the target device