User Guide

PyFR can be obtained here.

PyFR 1.5.0 has a hard dependency on Python 3.3+ and the following Python packages:

Note that due to a bug in numpy PyFR is not compatible with 32-bit Python distributions.

The CUDA backend targets NVIDIA GPUs with a compute capability of 2.0 or greater. The backend requires:

The MIC backend targets Intel Xeon Phi co-processors. The backend requires:

- ICC >= 14.0
- Intel MKL >= 11.1
- Intel MPSS >= 3.3
- pymic >= 0.7 (post commit 4d8a2da)

The OpenCL backend targets a range of accelerators including GPUs from AMD and NVIDIA. The backend requires:

The OpenMP backend targets multi-core CPUs. The backend requires:

- GCC >= 4.9
- A BLAS library compiled as a shared library (e.g. OpenBLAS)

To partition meshes for running in parallel it is also necessary to have one of the following partitioners installed:

To import CGNS meshes it is necessary to have the following installed:

- CGNS >= 3.3 (develop branch post commit e0faea6)

Before running PyFR 1.5.0 it is first necessary to either install
the software using the provided `setup.py` installer or add the root
PyFR directory to `PYTHONPATH` using:

`user@computer ~/PyFR$ export PYTHONPATH=.:$PYTHONPATH`

To manage installation of Python dependencies we strongly recommend using pip and virtualenv.

PyFR 1.5.0 uses three distinct file formats:

`.ini`— configuration file`.pyfrm`— mesh file`.pyfrs`— solution file

The following commands are available from the `pyfr` program:

`pyfr import`— convert a Gmsh .msh file or CGNS .cgns file into a PyFR .pyfrm file.Example:

pyfr import mesh.msh mesh.pyfrm

`pyfr partition`— partition an existing mesh and associated solution files.Example:

pyfr partition 2 mesh.pyfrm solution.pyfrs .

`pyfr run`— start a new PyFR simulation. Example:pyfr run mesh.pyfrm configuration.ini

`pyfr restart`— restart a PyFR simulation from an existing solution file. Example:pyfr restart mesh.pyfrm solution.pyfrs

`pyfr export`— convert a PyFR .pyfrs file into an unstructured VTK .vtu or .pvtu file. Example:pyfr export mesh.pyfrm solution.pyfrs solution.vtu

`pyfr` can be run in parallel. To do so prefix `pyfr` with
`mpirun -n <cores/devices>`. Note that the mesh must be
pre-partitioned, and the number of cores or devices must be equal to
the number of partitions.

The .ini configuration file parameterises the simulation. It is written in the INI format. Parameters are grouped into sections. The roles of each section and their associated parameters are described below.

Parameterises the backend with

`precision`— number precision:`single`|`double``rank-allocator`— MPI rank allocator:`linear`|`random`

Example:

```
[backend]
precision = double
rank-allocator = linear
```

Parameterises the CUDA backend with

`device-id`— method for selecting which device(s) to run on:*int*|`round-robin`|`local-rank``gimmik-max-nnz`— cutoff for GiMMiK in terms of the number of non-zero entires in a constant matrix:*int*`mpi-type`— type of MPI library that is being used:`standard`|`cuda-aware``block-1d`— block size for one dimensional pointwise kernels:*int*`block-2d`— block size for two dimensional pointwise kernels:*int*,*int*

Example:

```
[backend-cuda]
device-id = round-robin
gimmik-max-nnz = 512
mpi-type = standard
block-1d = 64
block-2d = 128, 2
```

Parameterises the MIC backend with

`device-id`— for selecting which device(s) to run on:*int*|`local-rank``mkl-root`— path to MKL root directory:*string*

Parameterises the OpenCL backend with

`platform-id`— for selecting platform id:*int*|*string*`device-type`— for selecting what type of device(s) to run on:`all`|`cpu`|`gpu`|`accelerator``device-id`— for selecting which device(s) to run on:*int*|*string*|`local-rank``gimmik-max-nnz`— cutoff for GiMMiK in terms of the number of non-zero entires in a constant matrix:*int*`local-size-1d`— local work size for one dimensional pointwise kernels:*int*`local-size-2d`— local work size for two dimensional pointwise kernels:*int*,*int*

Example:

```
[backend-opencl]
platform-id = 0
device-type = gpu
device-id = local-rank
gimmik-max-nnz = 512
local-size-1d = 16
local-size-2d = 128, 1
```

Parameterises the OpenMP backend with

`cc`— C compiler:*string*`cflags`— additional C compiler flags:*string*`cblas`— path to shared C BLAS library:*string*`cblas-type`— type of BLAS library:`serial`|`parallel`

Example:

```
[backend-openmp]
cc = gcc
cblas= example/path/libBLAS.dylib
cblas-type = parallel
```

Sets constants used in the simulation with

`gamma`— ratio of specific heats:*float*`mu`— dynamic viscosity:*float*`Pr`— Prandtl number:*float*`cpTref`— product of specific heat at constant pressure and reference temperature for Sutherland’s Law:*float*`cpTs`— product of specific heat at constant pressure and Sutherland temperature for Sutherland’s Law:*float*

Example:

```
[constants]
gamma = 1.4
mu = 0.001
Pr = 0.72
```

Parameterises the solver with

`system`— governing system:`euler`|`navier-stokes``order`— order of polynomial solution basis:*int*`anti-alias`— type of anti-aliasing:`flux`|`surf-flux`|`div-flux`|`flux, surf-flux`|`flux, div-flux`|`surf-flux, div-flux`|`flux, surf-flux, div-flux``viscosity-correction`— viscosity correction:`none`|`sutherland``shock-capturing`— shock capturing scheme:`none`|`artificial-viscosity`

Example:

```
[solver]
system = navier-stokes
order = 3
anti-alias = flux
viscosity-correction = none
shock-capturing = artificial-viscosity
```

Parameterises the time-integration scheme used by the solver with

`formulation`— formulation:`std`|`dual`where

`std`requires`scheme`— time-integration scheme`euler`|`rk34`|`rk4`|`rk45`|`tvd-rk3``tstart`— initial time*float*`tend`— final time*float*`dt`— time-step*float*`controller`— time-step controller`none`|`pi`where

`pi`only works with`rk34`and`rk45`and requires`atol`— absolute error tolerance*float*`rtol`— relative error tolerance*float*`errest-norm`— norm to use for estimating the error`uniform`|`l2``safety-fact`— safety factor for step size adjustment (suitable range 0.80-0.95)*float*`min-fact`— minimum factor that the time-step can change between iterations (suitable range 0.1-0.5)*float*`max-fact`— maximum factor that the time-step can change between iterations (suitable range 2.0-6.0)*float*

`dual`requires`scheme`— time-integration scheme`backward-euler`|`bdf2`|`bdf3``pseudo-scheme`— pseudo-time-integration scheme`euler`|`tvd-rk3`|`rk4``tstart`— initial time*float*`tend`— final time*float*`dt`— time-step*float*`pseudo-dt`— pseudo-time-step*float*`controller`— pseudo-time-step controller`none`where

`none`requires`pseudo-niters-max`— minimum number of iterations*int*`pseudo-niters-min`— maximum number of iterations*int*`pseudo-aresid`— absolute residual tolerance*float*`pseudo-rresid`— relative residual tolerance*float*

Example:

```
[solver-time-integrator]
formulation = std
scheme = rk45
controller = pi
tstart = 0.0
tend = 10.0
dt = 0.001
atol = 0.00001
rtol = 0.00001
errest-norm = l2
safety-fact = 0.9
min-fact = 0.3
max-fact = 2.5
```

Parameterises the interfaces with

`riemann-solver`— type of Riemann solver:`rusanov`|`hll`|`hllc`|`roe`|`roem``ldg-beta`— beta parameter used for LDG:*float*`ldg-tau`— tau parameter used for LDG:*float*

Example:

```
[solver-interfaces]
riemann-solver = rusanov
ldg-beta = 0.5
ldg-tau = 0.1
```

Parameterises the line interfaces with

`flux-pts`— location of the flux points on a line interface:`gauss-legendre`|`gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing on a line interface:*int*`quad-pts`— name of quadrature rule for anti-aliasing on a line interface:`gauss-legendre`|`gauss-legendre-lobatto`

Example:

```
[solver-interfaces-line]
flux-pts = gauss-legendre
quad-deg = 10
quad-pts = gauss-legendre
```

Parameterises the triangular interfaces with

`flux-pts`— location of the flux points on a triangular interface:`williams-shunn``quad-deg`— degree of quadrature rule for anti-aliasing on a triangular interface:*int*`quad-pts`— name of quadrature rule for anti-aliasing on a triangular interface:`williams-shunn`|`witherden-vincent`

Example:

```
[solver-interfaces-tri]
flux-pts = williams-shunn
quad-deg = 10
quad-pts = williams-shunn
```

Parameterises the quadrilateral interfaces with

`flux-pts`— location of the flux points on a quadrilateral interface:`gauss-legendre`|`gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing on a quadrilateral interface:*int*`quad-pts`— name of quadrature rule for anti-aliasing on a quadrilateral interface:`gauss-legendre`|`gauss-legendre-lobatto`|`witherden-vincent`

Example:

```
[solver-interfaces-quad]
flux-pts = gauss-legendre
quad-deg = 10
quad-pts = gauss-legendre
```

Parameterises the triangular elements with

`soln-pts`— location of the solution points in a triangular element:`williams-shunn``quad-deg`— degree of quadrature rule for anti-aliasing in a triangular element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a triangular element:`williams-shunn`|`witherden-vincent`

Example:

```
[solver-elements-tri]
soln-pts = williams-shunn
quad-deg = 10
quad-pts = williams-shunn
```

Parameterises the quadrilateral elements with

`soln-pts`— location of the solution points in a quadrilateral element:`gauss-legendre`|`gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing in a quadrilateral element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a quadrilateral element:`gauss-legendre`|`gauss-legendre-lobatto`|`witherden-vincent`

Example:

```
[solver-elements-quad]
soln-pts = gauss-legendre
quad-deg = 10
quad-pts = gauss-legendre
```

Parameterises the hexahedral elements with

`soln-pts`— location of the solution points in a hexahedral element:`gauss-legendre`|`gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing in a hexahedral element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a hexahedral element:`gauss-legendre`|`gauss-legendre-lobatto`|`witherden-vincent`

Example:

```
[solver-elements-hex]
soln-pts = gauss-legendre
quad-deg = 10
quad-pts = gauss-legendre
```

Parameterises the tetrahedral elements with

`soln-pts`— location of the solution points in a tetrahedral element:`shunn-ham``quad-deg`— degree of quadrature rule for anti-aliasing in a tetrahedral element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a tetrahedral element:`shunn-ham`|`witherden-vincent`

Example:

```
[solver-elements-tet]
soln-pts = shunn-ham
quad-deg = 10
quad-pts = shunn-ham
```

Parameterises the prismatic elements with

`soln-pts`— location of the solution points in a prismatic element:`williams-shunn~gauss-legendre`|`williams-shunn~gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing in a prismatic element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a prismatic element:`williams-shunn~gauss-legendre`|`williams-shunn~gauss-legendre-lobatto`|`witherden-vincent`

Example:

```
[solver-elements-pri]
soln-pts = williams-shunn~gauss-legendre
quad-deg = 10
quad-pts = williams-shunn~gauss-legendre
```

Parameterises the pyramidal elements with

`soln-pts`— location of the solution points in a pyramidal element:`gauss-legendre`|`gauss-legendre-lobatto``quad-deg`— degree of quadrature rule for anti-aliasing in a pyramidal element:*int*`quad-pts`— name of quadrature rule for anti-aliasing in a pyramidal element:`witherden-vincent`

Example:

```
[solver-elements-pyr]
soln-pts = gauss-legendre
quad-deg = 10
quad-pts = witherden-vincent
```

Parameterises solution, space (x, y, [z]), and time (t) dependent source terms with

`rho`— density source term:*string*`rhou`— x-momentum source term:*string*`rhov`— y-momentum source term:*string*`rhow`— z-momentum source term:*string*`E`— energy source term:*string*

Example:

```
[solver-source-terms]
rho = t
rhou = x*y*sin(y)
rhov = z*rho
rhow = 1.0
E = 1.0/(1.0+x)
```

Parameterises artificial viscosity for shock capturing with

`max-artvisc`— maximum artificial viscosity:*float*`s0`— sensor cut-off:*float*`kappa`— sensor range:*float*

Example:

```
[solver-artificial-viscosity]
max-artvisc = 0.01
s0 = 0.01
kappa = 5.0
```

Parameterises an exponential solution filter with

`nsteps`— apply filter every`nsteps`:*int*`alpha`— strength of filter:*float*`order`— order of filter:*int*`cutoff`— cutoff frequency below which no filtering is applied:*int*

Example:

```
[soln-filter]
nsteps = 10
alpha = 36.0
order = 16
cutoff = 1
```

Periodically write the solution to disk in the pyfrs format. Parameterised with

`dt-out`— write to disk every`dt-out`time units:*float*`basedir`— relative path to directory where outputs will be written:*string*`basename`— pattern of output names:*string*`post-action`— command to execute after writing the file:*string*`post-action-mode`— how the post-action command should be executed:`blocking`|`non-blocking`

Example:

```
[soln-plugin-writer]
dt-out = 0.01
basedir = .
basename = files-{t:.2f}
post-action = echo "Wrote file {soln} at time {t} for mesh {mesh}."
post-action-mode = blocking
```

Periodically integrates the pressure and viscous stress on the boundary
labelled `name` and writes out the resulting force vectors to a CSV
file. Parameterised with

`nsteps`— integrate every`nsteps`:*int*`file`— output file path; should the file already exist it will be appended to:*string*`header`— if to output a header row or not:*boolean*

Example:

```
[soln-plugin-fluidforce-wing]
nsteps = 10
file = wing-forces.csv
header = true
```

Periodically checks the solution for NaN values. Parameterised with

`nsteps`— check every`nsteps`:*int*

Example:

```
[soln-plugin-nancheck]
nsteps = 10
```

Periodically calculates the residual and writes it out to a CSV file. Parameterised with

`nsteps`— calculate every`nsteps`:*int*`file`— output file path; should the file already exist it will be appended to:*string*`header`— if to output a header row or not:*boolean*

Example:

```
[soln-plugin-residual]
nsteps = 10
file = residual.csv
header = true
```

Write time-step statistics out to a CSV file. Parameterised with

`flushsteps`— flush to disk every`flushsteps`:*int*`file`— output file path; should the file already exist it will be appended to:*string*`header`— if to output a header row or not:*boolean*

Example:

```
[soln-plugin-dtstats]
flushsteps = 100
file = dtstats.csv
header = true
```

Periodically samples specific points in the volume and writes them out to a CSV file. The plugin actually samples the solution point closest to each sample point, hence a slight discrepancy in the output sampling locations is to be expected. A nearest-neighbour search is used to locate the closest solution point to the sample point. The location process automatically takes advantage of scipy.spatial.cKDTree where available. Parameterised with

`nsteps`— sample every`nsteps`:*int*`samp-pts`— list of points to sample:`[(x, y), (x, y), ...]`|`[(x, y, z), (x, y, z), ...]``format`— output variable format:`primitive`|`conservative``file`— output file path; should the file already exist it will be appended to:*string*`header`— if to output a header row or not:*boolean*

Example:

```
[soln-plugin-sampler]
nsteps = 10
samp-pts = [(1.0, 0.7, 0.0), (1.0, 0.8, 0.0)]
format = primative
file = point-data.csv
header = true
```

Time average quantities. Parameterised with

`nsteps`— accumulate the average every`nsteps`time steps:*int*`dt-out`— write to disk every`dt-out`time units:*float*`basedir`— relative path to directory where outputs will be written:*string*`basename`— pattern of output names:*string*`avg-name`— expression as a function of the primitive variables, time (t), and space (x, y, [z]) to time average; multiple expressions, each with their own*name*, may be specified:*string*

Example:

```
[soln-plugin-tavg]
nsteps = 10
dt-out = 2.0
basedir = .
basename = files-{t:06.2f}
avg-p = p
avg-p2 = p*p
avg-vel = sqrt(u*u + v*v)
```

Parameterises constant, or if available space (x, y, [z]) and time (t)
dependent, boundary condition labelled `name` in the .pyfrm file
with

`type`— type of boundary condition:`char-riem-inv`|`no-slp-adia-wall`|`no-slp-isot-wall`|`slp-adia-wall`|`sub-in-frv`|`sub-in-ftpttang`|`sub-out-fp`|`sup-in-fa`|`sup-out-fn`where

`char-riem-inv`requires`rho`— density*float*|*string*`u`— x-velocity*float*|*string*`v`— y-velocity*float*|*string*`w`— z-velocity*float*|*string*`p`— static pressure*float*|*string*

`no-slp-isot-wall`requires`u`— x-velocity of wall*float*`v`— y-velocity of wall*float*`w`— z-velocity of wall*float*`cpTw`— product of specific heat capacity at constant pressure and temperature of wall*float*

`sub-in-frv`requires`rho`— density*float*|*string*`u`— x-velocity*float*|*string*`v`— y-velocity*float*|*string*`w`— z-velocity*float*|*string*

`sub-in-ftpttang`requires`pt`— total pressure*float*`cpTt`— product of specific heat capacity at constant pressure and total temperature*float*`theta`— azimuth angle (in degrees) of inflow measured in the x-y plane relative to the positive x-axis*float*`phi`— inclination angle (in degrees) of inflow measured relative to the positive z-axis*float*

`sub-out-fp`requires`p`— static pressure*float*|*string*

`sup-in-fa`requires`rho`— density*float*|*string*`u`— x-velocity*float*|*string*`v`— y-velocity*float*|*string*`w`— z-velocity*float*|*string*`p`— static pressure*float*|*string*

Example:

```
[soln-bcs-bcwallupper]
type = no-slp-isot-wall
cpTw = 10.0
u = 1.0
```

Parameterises space (x, y, [z]) dependent initial conditions with

`rho`— initial density distribution:*string*`u`— initial x-velocity distribution:*string*`v`— initial y-velocity distribution:*string*`w`— initial z-velocity distribution:*string*`p`— initial static pressure distribution:*string*

Example:

```
[soln-ics]
rho = 1.0
u = x*y*sin(y)
v = z
w = 1.0
p = 1.0/(1.0+x)
```

Proceed with the following steps to run a serial 2D Couette flow simulation on a mixed unstructured mesh:

Create a working directory called

`couette_flow_2d/`Copy the configuration file

`PyFR/examples/couette_flow_2d/couette_flow_2d.ini`into`couette_flow_2d/`Copy the Gmsh mesh file

`PyFR/examples/couette_flow_2d/couette_flow_2d.msh`into`couette_flow_2d/`Run pyfr to covert the Gmsh mesh file into a PyFR mesh file called

`couette_flow_2d.pyfrm`:pyfr import couette_flow_2d.msh couette_flow_2d.pyfrm

Run pyfr to solve the Navier-Stokes equations on the mesh, generating a series of PyFR solution files called

`couette_flow_2d-*.pyfrs`:pyfr run -b cuda -p couette_flow_2d.pyfrm couette_flow_2d.ini

Run pyfr on the solution file

`couette_flow_2d-040.pyfrs`converting it into an unstructured VTK file called`couette_flow_2d-040.vtu`. Note that in order to visualise the high-order data, each high-order element is sub-divided into smaller linear elements. The level of sub-division is controlled by the integer at the end of the command:pyfr export couette_flow_2d.pyfrm couette_flow_2d-040.pyfrs couette_flow_2d-040.vtu -d 4

Visualise the unstructured VTK file in Paraview

Proceed with the following steps to run a parallel 2D Euler vortex simulation on a structured mesh:

Create a working directory called

`euler_vortex_2d/`Copy the configuration file

`PyFR/examples/euler_vortex_2d/euler_vortex_2d.ini`into`euler_vortex_2d/`Copy the Gmsh file

`PyFR/examples/euler_vortex_2d/euler_vortex_2d.msh`into`euler_vortex_2d/`Run pyfr to convert the Gmsh mesh file into a PyFR mesh file called

`euler_vortex_2d.pyfrm`:pyfr import euler_vortex_2d.msh euler_vortex_2d.pyfrm

Run pyfr to partition the PyFR mesh file into two pieces:

pyfr partition 2 euler_vortex_2d.pyfrm .

Run pyfr to solve the Euler equations on the mesh, generating a series of PyFR solution files called

`euler_vortex_2d*.pyfrs`:mpirun -n 2 pyfr run -b cuda -p euler_vortex_2d.pyfrm euler_vortex_2d.ini

Run pyfr on the solution file

`euler_vortex_2d-100.0.pyfrs`converting it into an unstructured VTK file called`euler_vortex_2d-100.0.vtu`. Note that in order to visualise the high-order data, each high-order element is sub-divided into smaller linear elements. The level of sub-division is controlled by the integer at the end of the command:pyfr export euler_vortex_2d.pyfrm euler_vortex_2d-100.0.pyfrs euler_vortex_2d-100.0.vtu -d 4

Visualise the unstructured VTK file in Paraview