Authors: Payman Alemi and Aakash Prashar
Last Updated On: 2026-09-10
This repository contains files related to several ongoing development projects managed by the Waterboards' Supply and Demand Assessment (SDA) section. The three main folders are "W1_Watershed_Demand", "W2_Russian_River", and "W3_LSPC_Watershed".
For every watershed in California, water rightholders self-report monthly diversion data to the Waterboards on an annual basis. These scripts can help users gather relevant water rights for a watershed and flag potential issues in their reporting data. In combination with several manual QA/QC processes, these scripts ultimately generate a demand dataset for the watershed that can be used to simulate withdrawals in various sub-basins of a watershed on a monthly timescale. This dataset has applications in both SDA's model development and water availability analysis procedures.
Originally a process inherited from the Waterboards' Cannabis Instream Flows unit, this workflow integrates hydrologic modeling and water availability analysis—with exclusive focus on the Russian River watershed. It involves gathering weather data, applying QA/QC procedures to the gauge data, forecasting conditions for the remainder of the water year, and modeling flows via PRMS and SRP. This produces a supply dataset that the scripts use in conjunction with the demand dataset to perform a water availability analysis via Paradigm DWRAT (a modified version of the Drought Water Rights Allocation Tool).
For watersheds where SDA and its contractors have developed a Loading Simulation Program in C++ (LSPC) model, this is the counterpart to the Russian River Workflow. The scripts in this folder perform similar processes, obtaining weather data, performing QA/QC analyses, forecasting conditions in the current water year, and modeling hydrologic flows (via LSPC in this case). Furthermore, later scripts in the workflow integrate the resultant supply dataset with the watershed's demand dataset to run Paradigm DWRAT.
This folder contains scripts that treat Calwatrs flat files as the intake source for demand data, instead of ewrims. It is currently under development and not officially part of the workflow but SDA's goal is to replace the W1_Watershed_Demand workflow with this one by the end of 2026.
This repository contains several additional folders that support the three main workflows.
Several processes are shared between different workflows. This folder contains scripts whose functions can be imported by any procedure to perform these overlapping actions.
An SDA package is currently under development. All shared scripts will gradually be compiled into a package with proper documentation for easier access.
Tools such as LSPC and Paradigm DWRAT are stored here.
The Demand and Russian River Workflow "Scripts" folders both contain a sub-folder called Extra. The scripts in these locations are not formally integrated into their respective workflows, but they may be useful complements.
The scripts in this folder are not specific to any workflow, but they may contain useful supplemental or one-off procedures. The Archive sub-folder also contains files from obsolete workflows and scripts. Another particularly important directory is the XLSX_Tracking folder (discussed in the next section).
To improve the structure and usability of the workflows' control files, they are primarily shared as spreadsheets. However, the downside is that they are binary files, which are not tracked well by git. The imperfect workaround to this is an R script that extracts the underlying XML of these spreadsheets and stores them in this folder. These XML files are text-based and can be tracked by git. In addition, by compressing these files together and changing the "zip" extension to "xlsx", the spreadsheet can be recovered.
The scripts and procedures are primarily developed for use on computers with Windows operating systems. In addition, the following programs are required:
-
A program that can read and edit XLSX files (such as Microsoft Excel, Google Sheets, or LibreOffice Calc)
-
Git (some scripts submit git commands using Command Prompt)
-
Google Chrome (only used in backup procedures that rely on RSelenium)
In addition, GIS software such as ArcGIS Pro or QGIS is not mandatory, but it may assist users with the geospatial components of these workflows.
The scripts in this repository are mainly coded in R and Python. However, the primary orchestrators are generally R scripts. All of the repository's R scripts rely on the "DWRAT_DataScraping" RStudio Project, which can be activated using the file DWRAT_DataScraping.Rproj. Please ensure that this project is active when running any of the repository's R scripts.
The required packages for this repository's R and Python scripts are tracked via renv and Anaconda YAML files, respectively.
Assuming the "DWRAT_DataScraping" project is active, the supplemental script Project_Setup.R can install renv and all required R packages. In the console of RStudio, the command source("Additional_Scripts/Project_Setup.R") can execute this process. For Python, the two required Anaconda environments are detailed in Paradigm_DWRAT_Environment.yml and LSPC_Climate_Environment.yml. To setup these environments, open Anaconda Prompt (or similar) and use the command conda create --file [PATH_TO_YML].1
NOTE: For the "W2_Russian_River" workflow, dependency setup is performed automatically by the scripts. Eventually, the other workflows will be updated to do the same as well.
Updating Master_Control_File.xlsx is optional for most users. Each workflow has its own specific control file(s) that must be filled in, but this spreadsheet specifically is only relevant for supplemental project-level features. Most notably, this file enables the scripts to check locally synced cloud repositories like SharePoint.
If this file is left blank, there is one warning that may appear for users when running some of the workflow scripts. Once per session per day, users will receive a warning message if the "INITIAL_SHAREPOINT_FILE_PORTION" field has no associated value. If cloud-based repositories will not be used, this message can be silenced permanently by inputting any value into Cell C10 of the spreadsheet (e.g., typing in "NA").
The remaining setup requirements depend on the chosen workflow. For more information, please consult the workflows' primary scripts and associated documentation.
Please review the Documentation folder and Demand_Master_Script.R.
Formal documentation is still pending. Please review ReadMe.txt and MSTR_v1_RRW.R.
Formal documentation is still pending.
Footnotes
-
Please remember to encase the path name in quotation marks if it contains spaces. ↩