Modern Tools for TUFLOW Model Version Control

Version control is the practice of systematically tracking changes over time.  It is an important part of TUFLOW modelling workflow assisting with quality control, allowing users to track changes and revert to an earlier version of the model input and configuration should problems arise from more recent changes. The flexibility of TUFLOW allows users to develop their own version control system although the commonly used approach is to apply a version numbering system to the TUFLOW control files as a suffix.  There are examples of this approach within the TUFLOW Tutorials.

The nature of the TUFLOW model build is such that changes to inputs are probably most common at the GIS input level and therefore the version control suffix convention was applied to the GIS input layers as well as TUFLOW control files as shown below. 

This tried and tested approach provides a robust version control system utilised by many TUFLOW modellers.  The QGIS TUFLOW Plugin provides an Increment Selected Layer tool to automate the labelling GIS layers using this approach.  However, a downside of the approach is that it generates a lot of files, including data duplication, which can look cluttered, consumes more disk space than neccesary, and increases the chance of introducing mistakes.

There are other, more modern, software-based version control systems, primarily used within the software development space. Perhaps the most commonly known is Git, which is a distributed version control system used to track changes, typically during the development of source code, which is very suitable for tracking any text-based file types, for instance TUFLOW control files.  Git allows version control of existing files in-place, enabling users to commit the current state of their files at any time and to recall that state later, or switch between alternatives and make comparisons.

As well as tracking changes during model development, Git also provides approaches to enable branching and merging multiple versions, collaboration between multiple users and a distributed system which enables offline work, synchronisation between users, and robust backups. It is not intended to give a full description of a Git process here, but it is possible to apply Git to an existing folder of TUFLOW project files and selectively start committing changes as required. This provides a local version control which can be extended by utilising remote repositories on Gitlab/Github or similar to a multi-user TUFLOW model build workflow. The TUFLOW CATCH manual provides a possible workflow for multi-user development of a TUFLOW CATCH model, which could just as easily be followed for a TUFLOW Classic/HPC or TUFLOW FV only workflow.

However, although Git will provide version control and can show which configuration and other text data has changed, it will not provide any details on where GIS file changes occur spatially. For this, you would need to add on something similar to the Kart Project which provides version control specialised for geospatial and tabular data within a Git environment.

Kart is built on Git and uses standard Git repositories and Git-like commands to provide a workflow familiar to Git users. It has the benefit of directly supporting common vector and raster GIS formats, so can provide information of what spatial data has changed between different commits.

Kart comes in particularly useful when utilising the Geopackage GIS format, such that only one Geopackage, with a single file extension, is required for a TUFLOW model containing all GIS layers required to define a TUFLOW model geometry.  With Kart, one Geopackage can be used containing current and previous versions, allowing efficient GIS version control without duplication.

Kart also comes with a QGIS Plugin which provides a simple interface for the version control including easy-to-use tools for reviewing changes, as shown in the below video, as well as interaction with remote repositories.

Using a version control system such as Kart, in conjunction with Geopackages, provides a much neater set of model files, with the full commit history including commit comments.  The Kart QGIS plugin provides an easy-to-use interface for reviewing changes including the geospatial details of changes within GIS layers, with the ability to use remote repositories.

The TUFLOW wiki page here provides more information on using the Kart approach with the Kart QGIS plugin during the development of a TUFLOW model. The approach can be used to version control GIS data only, either using a single geopackage for the whole TUFLOW model, or separate files. If you have an interest, the repository here hosted on the TUFLOW Gitlab User Group, based on the TUFLOW tutorial dataset, can be cloned using the Kart QGIS Plugin to provide a demo dataset which shows the potential usage of the tools.

As a further extension, Kart can be used in conjunction with Git to provide a version control approach for the TUFLOW model including both geospatial data and TUFLOW control file data. The wiki page here provides some information on how this may be done.