What’s New

v3.13.4 (2024/02/29)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix initialization dependencies for pysd.py_backend.statefuls.Smooth in the builder pysd.builders.python.python_expressions_builder.SmoothBuilder (issue #440). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.13.3 (2024/02/02)

New Features

Breaking changes

Deprecations

Bug fixes

Documentation

• Improve documentation for netCDF4 dependency. (@enekomartinmartinez)

Performance

Internal Changes

• Support for Python 3.12. (@enekomartinmartinez)

• Support for numpy >= 1.24. (@enekomartinmartinez)

• Correct some warnings management in the tests. (@enekomartinmartinez)

• Set numpy requirements to >= 1.23 to follow NEP29. (@enekomartinmartinez)

• Custom error messages when netCDF4 is missing (issue #435). (@enekomartinmartinez)

v3.13.2 (2024/01/09)

New Features

Breaking changes

Deprecations

Bug fixes

• Add support for XMILE’s DELAY function through DelayFixed (issue #433). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.13.1 (2024/01/02)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bug with pysd.py_backend.output.ModelOutput initializing all the handlers, and thus, importing netCDF4 when unnecessary (issue #431). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.13.0 (2023/12/25)

New Features

• Include new method pysd.py_backend.model.Model.copy() which allows copying a model (issue #131). (@enekomartinmartinez)

• pysd.py_backend.model.Model.select_submodel() now takes an optional argument inplace when set to False it will return a modified copy of the model instead of modifying the original model (issue #131). (@enekomartinmartinez)

• pysd.py_backend.model.Model.export() will now save also time component information if changed (e.g. final time, time step…). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Set the pointer of pysd.py_backend.statefuls.DelayFixed to 0 during initialization (issue #427). (@enekomartinmartinez)

• pysd.py_backend.model.Model.export() now works with Macros. (@enekomartinmartinez)

Documentation

• Improve documentation of methods in pysd.py_backend.model.Model and pysd.py_backend.model.Macro includying cross-references and rewrite the one from pysd.py_backend.model.Macro.set_components(). (@enekomartinmartinez)

• Include documentation about the new method pysd.py_backend.model.Model.copy() and update documentation from pysd.py_backend.model.Model.select_submodel(). (@enekomartinmartinez)

Performance

• Improved performace of pysd.py_backend.output.DataFrameHandler by creating the dataframe at the end of the run (issue #374 and issue #330). (@easyas314159 and @enekomartinmartinez)

Internal Changes

• Move old pysd.py_backend.model.Macro.set_components() to pysd.py_backend.model.Macro._set_components(), and create new method with the same name without the new argument. (@enekomartinmartinez)

• Move old pysd.py_backend.model.Macro.set_stateful() to pysd.py_backend.model.Macro._set_stateful(). (@enekomartinmartinez)

• Make integration tests filter only specific warnings. (@enekomartinmartinez)

• Include warnings in pysd.py_backend.model.Macro.set_components() when changing the behaviour of the component (issue #58). (@enekomartinmartinez)

v3.12.0 (2023/10/02)

New Features

Breaking changes

• Remove support for Python 3.7 and 3.8. (@enekomartinmartinez)

• Update the cellrange retrievel to be compatible with openpyxl >= 3.1. It breaks the code for lower versions. (@enekomartinmartinez)

Deprecations

Bug fixes

• Update benchmarking tools to avoid FutureWarning for positional keys from pandas. (@enekomartinmartinez)

Documentation

• Update requirements. (@enekomartinmartinez)

Performance

Internal Changes

• Update libraries used for building documentation and .readthedocs.yml. (@enekomartinmartinez)

• Force to use xarray >= 2023.09. (@enekomartinmartinez)

• Use pandas[excel] instead of xlrd dependency in the requirements. (@gdrosos)

v3.11.0 (2023/08/01)

New Features

• Add the possibility to run a model one or several steps at a time, updating model variables in the process. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

Documentation

• Add the Running models one (or more) step(s) at a time section in Advanced Usage. (@rogersamso)

Performance

Internal Changes

• Modify signature of the pysd.py_backend.output.ModelOutput class. It now only accepts the path of the results file (@rogersamso)

• Add the pysd.py_backend.output.ModelOutput.collect() method to the pysd.py_backend.output.ModelOutput class. (@rogersamso)

• Add the pysd.py_backend.model.Model.set_stepper() and pysd.py_backend.model.Model.step() methods to the pysd.py_backend.model.Model class. (@rogersamso)

• Add several internal methods to the pysd.py_backend.model.Model class, to avoid code repetition. (@rogersamso)

• Add FutureWarning for Python 3.7 support deprecation. (@enekomartinmartinez)

v3.10.0 (2023/04/28)

New Features

• Parse TABBED ARRAYS Vensim function. (@rogersamso)

• Add support for Vensim’s POWER function. (@rogersamso)

• Add possibility to pass data_files in netCDF format. (@rogersamso)

• Add support for XMILE’s non-negative flows and stocks. (@enekomartinmartinez)

• Add support for XMILE’s MIN and MAX functions with one argument. (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Set the final_subscripts to an empty dictionary for ELMCOUNT function in pysd.builders.python_expressions_builder.CallBuilder.build_function_call(). (@rogersamso)

• Define comp_subtype of Unchangeable tabbed arrays as Unchangeable. This is done in pysd.builders.python.python_expressions_builder.ArrayBuilder.build(). (@rogersamso)

Documentation

• Add information about slack channel https://slofile.com/slack/sdtoolsandmet-slj3251. (@enekomartinmartinez)

• Update XMILE stocks section. (@enekomartinmartinez)

Performance

Internal Changes

• Add a weekly scheduled run to all CI workflows, which run each Monday at 06:00 UTC. (@EwoutH)

• Fix CI pipeline for Python 3.11 and remove Python 3.10 pipeline in favour of 3.11. (@kinow)

• Add non_negative argument in pysd.translators.structures.abstract_expressions.IntegStructure. (@enekomartinmartinez)

v3.9.1 (2023/03/11)

New Features

• Add numpy.py as translation for the call to the function PI(). (@lionel42)

Breaking changes

Deprecations

Bug fixes

• Set numpy <1.24 to avoid errors with least squares equation in pysd.py_backend.allocation.allocate_available(). (@enekomartinmartinez)

• Keep the attributes of a component when using pysd.py_backend.model.Macro.set_components() to avoid losing coords or arguments information. (@enekomartinmartinez)

• Set openpyxl <3.1 to avoid errors due to non-backwards compatible changes. (@enekomartinmartinez)

• Include time dependency in random functions to avoid them using constant cache. (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Run test for Python 3.11 with ubuntu-latest (hdf5-headers need to be installed using apt manager). (@enekomartinmartinez)

v3.9.0 (2022/12/15)

New Features

• Parses and ignores reality check functions during translation of Vensim models. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

• Fix issue with the classification of variables in modules and submodules (issue #388). When a model had a view with 3 sublevels (e.g. energy-transformation.losses) but another view was defined with only two of them (e.g. energy-transformation), the variables in the second view were placed in the main model file. Now, if this happens, the variables in the second view will be placed in a main.py file (i.e. energy/transformation/main.py). (@rogersamso)

• Fix bug on the CLI when passing a hyphen as first value to the –subview-sep argument (issue #388). (@rogersamso)

• Fix bug on the CLI when parsing initial conditions (issue #395). (@rogersamso)

Documentation

• The Splitting Vensim views in different files section in Command Line Usage has been updated to include an example of the usage of the –subview-sep CLI argument. (@rogersamso)

Performance

Internal Changes

• The _merge_nested_dicts() method from the pysd.translators.vensim.vensim_file.VensimFile class has been made a static method, as it does not need to access any attribute of the instance, and it does facilitate unit testing. (@rogersamso)

• The pysd/translators/vensim/parsing_grammars/element_object.peg grammar has been modified to be able to parse reality check elements. (@rogersamso)

• pysd.translators.vensim.vensim_element.Constraint and pysd.translators.vensim.vensim_element.TestInputs classes have been added, which inherit from the also newly created pysd.translators.vensim.vensim_element.GenericComponent, which include the parse() and get_abstract_component() methods. (@rogersamso and @enekomartinmartinez)

• The pysd.translators.structures.abstract_model.AbstractSection class now has two extra attributes (constraints and input_tests), which hold the pysd.translators.structures.abstract_model.AbstractConstraint and pysd.translators.structures.abstract_model.AbstractTestInputs objects. (@rogersamso)

v3.8.0 (2022/11/03)

New Features

• Adds ncfile.py module with helper functions to export a subset or all of the data_vars in netCDF files generated with PySD to pandas.DataFrame, csv or tab files. (@rogersamso)

• Adds possibility to initialize and export a subset or all external objects to netCDF, and then initialize the external objects from the file. (@rogersamso)

Breaking changes

Deprecations

• Deprecate pysd.py_backend.model.Model._get_dependencies() replacing it with pysd.py_backend.model.Model.get_dependencies(). (@enekomartinmartinez)

Bug fixes

• Include new class pysd.py_backend.utils.Dependencies to return by pysd.py_backend.model.Model.get_dependencies() (issue #379). (@lionel42)

Documentation

• Updates the Getting Started page with instructions on how to use the new helper functions for netCDF files. (@rogersamso)

• Updates the Advanced Usage page with instructions on how to export externals to netCDF and initialize a model from it. (@rogersamso)

• Update citation information to include the new paper published in JOSS. (@enekomartinmartinez)

Performance

• Initializing external objects from netCDF file is much faster than reading from spreadsheet files.(@rogersamso)

Internal Changes

• Adds the pysd.py_backend.model.Macro.serialize_externals() and pysd.py_backend.model.Macro.initialize_external_data() methods, and a few other private methods.(@rogersamso)

• Adds the pysd.py_backend.utils.UniqueDims class for renaming model dimensions with unique names.(@rogersamso)

• Force pysd.py_backend.external.External objects to always have the full element dimensions, missing dimensions are filled with numpy.nan. (@enekomartinmartinez)

• Add dependabot configuration for GitHub Actions updates. (@EwoutH)

• Include new error messages for initialization of pysd.py_backend.lookups.HardcodedLookups (issue #376). (@enekomartinmartinez)

• Include new warning message when a translated variable has several types or subtypes. (@enekomartinmartinez)

• Set CI test to run in parallel in 2 cores. (@enekomartinmartinez)

v3.7.1 (2022/09/19)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bugs with pandas.DataFrame 1.5.0 (issue #366). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.7.0 (2022/09/19)

New Features

• Simulation results can now be stored as netCDF4 files. (@rogersamso)

• The CLI also accepts netCDF4 file paths after the -o argument. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

• Fix bug when a WITH LOOKUPS argument has subscripts. (@enekomartinmartinez)

• Fix bug of exporting csv files with multiple subscripts variables. (@rogersamso)

• Fix bug of missing dimensions in variables defined with not all the subscripts of a range (issue #364). (@enekomartinmartinez)

• Fix bug when running a model with variable final time or time step and progressbar (issue #361). (@enekomartinmartinez)

Documentation

• Add Storing simulation results on a file section in the Getting Started page. (@rogersamso)

• Include cookbook information in the Getting Started page. (@enekomartinmartinez)

• Include an introduction of main historical changes in the About the Project page. (@enekomartinmartinez)

Performance

• Exporting outputs as netCDF4 is much faster than exporting a pandas DataFrame, especially for large models. (@rogersamso)

Internal Changes

• Make PySD work with parsimonius 0.10.0. (@enekomartinmartinez)

• Add netCDF4 dependency for tests. (@rogersamso)

• Improve warning message when replacing a stock with a parameter. (@enekomartinmartinez)

• Include more pytest parametrizations in some test and make them translate the models in temporary directories. (@enekomartinmartinez)

• Include lychee-action in the GHA workflow to check the links. (@enekomartinmartinez)

• Update License. (@enekomartinmartinez)

• Include Maintained? Yes and Contributions welcome badges. (@enekomartinmartinez)

• Update links to the new repository location. (@enekomartinmartinez)

• Reduce relative precision from 1e-10 to 1e-5 to compute the saving times and final time. (@enekomartinmartinez)

• Add convergence tests for euler integration method. (@enekomartinmartinez)

• Include build docs check in the GHA workflow to avoid warnings with sphinx. (@enekomartinmartinez)

v3.6.1 (2022/09/05)

New Features

Breaking changes

Deprecations

Bug fixes

Documentation

Performance

Internal Changes

• Set parsimonius requirement to 0.9.0 to avoid a breaking-change in the newest version. Pending to update PySD to run it with parsimonious 0.10.0. (@enekomartinmartinez)

v3.6.0 (2022/08/31)

New Features

• Include warning messages when a variable is defined in more than one view, when a control variable appears in a view or when a variable doesn’t appear in any view as a workbench variable (issue #357). (@enekomartinmartinez)

• Force variables in a module to be saved alphabetically for being able to compare differences between versions (only for the models that are split by views). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Classify control variables in the main file always (issue #357). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Include pysd.translators.structures.abstract_model.AbstractControlElement child of pysd.translators.structures.abstract_model.AbstractElement to differentiate the control variables. (@enekomartinmartinez)

v3.5.2 (2022/08/15)

New Features

Breaking changes

Deprecations

Bug fixes

• Make sketch’s font_size optional. (@enekomartinmartinez)

Documentation

• Correct typos.

Performance

Internal Changes

v3.5.1 (2022/08/11)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bug generated when :EXCEPT: keyword is used with subscript subranges (issue #352). (@enekomartinmartinez)

• Fix bug of precision error for pysd.py_backend.allocation.allocate_by_priority() (issue #353). (@enekomartinmartinez)

• Fix bug of constant cache assignment. (@enekomartinmartinez)

Documentation

Performance

• Improve the performance of reading pysd.py_backend.external.External data with cellrange names by loading the data in memory with pandas. As recommended by openpyxl developers, this is a possible way of improving performance to avoid parsing all rows up each time for getting the data (issue 1867 in openpyxl). (@enekomartinmartinez)

Internal Changes

v3.5.0 (2022/07/25)

New Features

• Add support for subscripted arguments in pysd.py_backend.functions.ramp() and pysd.py_backend.functions.step() (issue #344). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Fix bug related to the order of elements in 1D GET expressions (issue #343). (@enekomartinmartinez)

• Fix bug in request 0 values in allocate by priority (issue #345). (@enekomartinmartinez)

• Fix a numerical error in starting time of step and ramp. (@enekomartinmartinez)

Documentation

• Include new PySD logo. (@enekomartinmartinez)

Performance

Internal Changes

• Ignore ‘distutils Version classes are deprecated. Use packaging.version instead’ error in tests as it is an internal error of xarray. (@enekomartinmartinez)

• Add a warning message when a subscript range is duplicated in a variable reference. (@enekomartinmartinez)

v3.4.0 (2022/06/29)

New Features

• Add support for Vensim’s ALLOCATE AVAILABLE (pysd.py_backend.allocation.allocate_available()) function (issue #339). Integer allocation cases have not been implemented neither the fixed quantity and constant elasticity curve priority functions. (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

• Improve the documentation of the pysd.py_backend.allocation module. (@enekomartinmartinez)

Performance

Internal Changes

• Add a class to manage priority profiles so it can be also used by the many-to-many allocation. (@enekomartinmartinez)

v3.3.0 (2022/06/22)

New Features

• Add support for Vensim’s ALLOCATE BY PRIORITY (pysd.py_backend.allocation.allocate_by_priority()) function (issue #263). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Fix bug of using subranges to define a bigger range (issue #335). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Improve error messages for pysd.py_backend.External objects. (@enekomartinmartinez)

v3.2.0 (2022/06/10)

New Features

• Add support for Vensim’s GET TIME VALUE (pysd.py_backend.functions.get_time_value()) function (issue #332). Not all cases have been implemented. (@enekomartinmartinez)

• Add support for Vensim’s VECTOR SELECT (pysd.py_backend.functions.vector_select()) function (issue #266). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

Performance

Internal Changes

v3.1.0 (2022/06/02)

New Features

• Add support for Vensim’s VECTOR SORT ORDER (pysd.py_backend.functions.vector_sort_order()) function (issue #326). (@enekomartinmartinez)

• Add support for Vensim’s VECTOR RANK (pysd.py_backend.functions.vector_rank()) function (issue #326). (@enekomartinmartinez)

• Add support for Vensim’s VECTOR REORDER (pysd.py_backend.functions.vector_reorder()) function (issue #326). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

• Add the section Adding new functions with examples for developers. (@enekomartinmartinez)

Performance

Internal Changes

• Include a template for PR.

v3.0.1 (2022/05/26)

New Features

Breaking changes

Deprecations

Bug fixes

• Simplify subscripts dictionaries for pysd.py_backend.data.TabData objects. (@enekomartinmartinez)

Documentation

• Improve tests/README.md.

• Minor improvements in the documentation.

Performance

Internal Changes

• Add Python 3.10 to CI pipeline and include it in the supported versions list. (@enekomartinmartinez)

• Correct LICENSE file extension in the setup.py. (@enekomartinmartinez)

• Move from importlib’s load_module() to exec_module(). (@enekomartinmartinez)

• Remove warnings related to set usage. (@enekomartinmartinez)

• Move all the missing test to pytest. (@enekomartinmartinez)

• Remove warning messages from test and make test fail if there is any warning. (@enekomartinmartinez)

v3.0.0 (2022/05/23)

New Features

• The new Abstract Model Representation translation and building workflow will allow to add new output languages in the future. (@enekomartinmartinez)

• Added new properties to the pysd.py_backend.model.Macro to make more accessible some information: namespace, subscripts, dependencies, modules, doc. (@enekomartinmartinez)

• Cleaner Python models: (@enekomartinmartinez)

  • _namespace and _dependencies dictionaries have been removed from the file.

  • Variables original names, dependencies metadata now are given through pysd.py_backend.components.Component.add() decorator, instead of having them in the docstring.

  • Merging of variable equations is now done using the coordinates to a pre-allocated array, instead of using the magic function pysd.py_backend.utils.xrmerge().

  • Arranging and subseting arrays are now done inplace instead of using the magic function pysd.py_backend.utils.rearrange().

Breaking changes

• Set the argument flatten_output from run() to True by default. Previously it was set to False by default. (@enekomartinmartinez)

• Move the docstring of the model to a property, doc. Thus, it is not callable anymore. (@enekomartinmartinez)

• Allow the function pysd.py_backend.functions.pulse() to also perform the operations performed by pysd.py_backend.functions.pulse_train() and pysd.py_backend.functions.pulse_magnitude(). (@enekomartinmartinez)

• Change first argument of pysd.py_backend.functions.active_initial(), now it is the stage of the model and not the time. (@enekomartinmartinez)

• Simplify the function pysd.py_backend.utils.rearrange() orienting it to perform simple rearrange cases for user interaction. (@enekomartinmartinez)

• Move pysd.py_backend.statefuls.Model and pysd.py_backend.statefuls.Macro to pysd.py_backend.model.Model and pysd.py_backend.model.Macro, respectively. (@enekomartinmartinez)

• Manage all kinds of lookups with the pysd.py_backend.lookups.Lookups class. (@enekomartinmartinez)

• Include a second optional argument to lookups functions to set the final coordinates when a subscripted variable is passed as an argument. (@enekomartinmartinez)

Deprecations

• Remove pysd.py_backend.utils.xrmerge(), pysd.py_backend.functions.pulse_train(), pysd.py_backend.functions.pulse_magnitude(), pysd.py_backend.functions.lookup(), pysd.py_backend.functions.lookup_discrete(), pysd.py_backend.functions.lookup_extrapolation(), pysd.py_backend.functions.logical_and(), pysd.py_backend.functions.logical_or(), pysd.py_backend.functions.bounded_normal(), pysd.py_backend.functions.log(). (@enekomartinmartinez)

• Remove old translation and building files (pysd.translation). (@enekomartinmartinez)

Bug fixes

• Generate the documentation of the model when loading it to avoid losing information when replacing a variable value (issue #310, PR #312). (@enekomartinmartinez)

• Make random functions return arrays of the same shape as the variable, to avoid repeating values over a dimension (issue #309, PR #312). (@enekomartinmartinez)

• Fix bug when Vensim’s :MACRO: definition is not at the top of the model file (issue #306, PR #312). (@enekomartinmartinez)

• Make builder identify the subscripts using a main range and subrange to allow using subscripts as numeric values as Vensim does (issue #296, issue #301, PR #312). (@enekomartinmartinez)

• Fix bug of missmatching of functions and lookups names (issue #116, PR #312). (@enekomartinmartinez)

• Parse Xmile models case insensitively and ignoring the new lines characters (issue #203, issue #253, PR #312). (@enekomartinmartinez)

• Add support for Vensim’s :EXCEPT: keyword (issue #168, issue #253, PR #312). (@enekomartinmartinez)

• Add support for Xmile’s FORCST and SAFEDIV functions (issue #154, PR #312). (@enekomartinmartinez)

• Add subscripts support for Xmile (issue #289, PR #312). (@enekomartinmartinez)

• Fix numeric error bug when using return_timestamps and time step with non-integer values. (@enekomartinmartinez)

Documentation

• Review the whole documentation, refract it, and describe the new features. (@enekomartinmartinez)

Performance

• The variables defined in several equations are now assigned to a pre-allocated array instead of using pysd.py_backend.utils.xrmerge(). (@enekomartinmartinez)

• The arranging and subseting of arrays is now done inplace instead of using the magic function pysd.py_backend.utils.rearrange(). (@enekomartinmartinez)

• The grammars for Parsimonious are only compiled once per translation. (@enekomartinmartinez)

Internal Changes

• The translation and the building of models has been totally modified to use the Abstract Model Representation. (@enekomartinmartinez)