Dear Pathfinder development team,
First of all, thank you making this model open-source and sharing it with the climate modeling community!
I was doing some experiments on the model and noticed that xarray operations were a bit slow compared to numpy-based operations only (np is 20x faster average).
I thus implemented a POC version of a run_xarray()-like numpy-based method called run_numpy here, if needed:
https://github.com/pjbarjhoux/Pathfinder/blob/0a4768b4fdb0b8440fa553a1644376901148a3d2/core_fct/cls_model.py#L217
with a script running a simple evaluation of the model and compares execution times:
https://github.com/pjbarjhoux/Pathfinder/blob/ft/numpy_operations/run_scripts/run_eval.py
It is not generic enough to cover all Forcings input xarray DataSets structure, but it should work if they are modified to match the implementation (eg. the case of Forcings with shapes (years, scenario) is not handled but could work if transformed as (years, scenario, config)). One of xarray's strengths is indeed its ability to perform extended broadcasting operations over named dimensions, a feature not available in NumPy. While the proposed implementation already covers many cases, there is still room for improvement if a more generic approach is preferred.
Regards
Pierre-Jean
Dear Pathfinder development team,
First of all, thank you making this model open-source and sharing it with the climate modeling community!
I was doing some experiments on the model and noticed that xarray operations were a bit slow compared to numpy-based operations only (np is 20x faster average).
I thus implemented a POC version of a run_xarray()-like numpy-based method called run_numpy here, if needed:
https://github.com/pjbarjhoux/Pathfinder/blob/0a4768b4fdb0b8440fa553a1644376901148a3d2/core_fct/cls_model.py#L217
with a script running a simple evaluation of the model and compares execution times:
https://github.com/pjbarjhoux/Pathfinder/blob/ft/numpy_operations/run_scripts/run_eval.py
It is not generic enough to cover all Forcings input xarray DataSets structure, but it should work if they are modified to match the implementation (eg. the case of Forcings with shapes (years, scenario) is not handled but could work if transformed as (years, scenario, config)). One of xarray's strengths is indeed its ability to perform extended broadcasting operations over named dimensions, a feature not available in NumPy. While the proposed implementation already covers many cases, there is still room for improvement if a more generic approach is preferred.
Regards
Pierre-Jean