Vensim Translation

PySD allows parsing a Vensim .mdl file and translates the result to an AbstractModel object that can later (building process) be used to build the model in another programming language.

Translation workflow

The following translation workflow allows splitting the Vensim file while parsing its contents in order to build an AbstractModel object. The workflow may be summarized as follows:

  1. Vensim file: splits the model equations from the sketch and allows splitting the model in sections (main section and macro sections).

  2. Vensim section: is a full set of varibles and definitions that is integrable. The Vensim section can then be split into model expressions.

  3. Vensim element: a definition in the mdl file which could be a subscript (sub)range definition or a variable definition. It includes units and comments. Definitions for the same variable are grouped after in the same AbstractElement object. Allows parsing its left hand side (LHS) to get the name of the subscript (sub)range or variable and it is returned as a specific type of component depending on the used assing operator (=, ==, :=, (), :)

  4. Vensim component: the classified object for a variable definition, it depends on the opperator used to define the variable. Its right hand side (RHS) can be parsed to get the Abstract Syntax Tree (AST) of the expression.

Once the model is parsed and broken following the previous steps, the AbstractModel is returned.

Vensim file parts

Vensim file

The VensimFile class allows reading the original Vensim model file, parsing it into Section elements using the FileSectionsVisitor, parsing its sketch using SketchVisitor in order to classify the varibales per view. The final result can be exported to an AbstractModel class in order to build the model in another programming language.

class pysd.translators.vensim.vensim_file.VensimFile(mdl_path: Union[str, pathlib.Path], encoding: Optional[str] = None)[source]

The VensimFile class allows parsing an mdl file. When the object is created, the model file is automatically opened; unnecessary tabs, whitespaces, and linebreaks are removed; and the sketch is split from the model equations.

Parameters
  • mdl_path (str or pathlib.Path) – Path to the Vensim model.

  • encoding (str or None (optional)) – Encoding of the source model file. If None, the encoding will be read from the model, if the encoding is not defined in the model file it will be set to ‘UTF-8’. Default is None.

property verbose

Print model information to standard output.

parse(parse_all: bool = True) None[source]

Parse model file with parsimonious using the grammar given in ‘parsing_grammars/file_sections.peg’ and the class FileSectionsVisitor to visit the parsed expressions.

This breaks the model file in VensimSections, which correspond to the main model section and the macros.

Parameters

parse_all (bool (optional)) – If True, the VensimSection objects created will be automatically parsed. Otherwise, these objects will only be added to self.sections but not parsed. Default is True.

parse_sketch(subview_sep: List[str]) None[source]

Parse the sketch of the model with parsimonious using the grammar given in ‘parsing_grammars/sketch.peg’ and the class SketchVisitor to visit the parsed expressions.

It will modify the views_dict of the first section, including the dictionary of the classification of variables by views. This, method should be called after calling the self.parse method.

Parameters

subview_sep (list) – List of the separators to use to classify the model views in folders and subfolders. The sepparator must be ordered in the same order they appear in the view name. For example, if a view is named “economy:demand.exports” if subview_sep=[“:”, “.”] this view’s variables will be included in the file ‘exports.py’ and inside the folders economy/demand.

get_abstract_model() pysd.translators.structures.abstract_model.AbstractModel[source]

Instantiate the AbstractModel object used during building. This, method should be called after parsing the model (self.parse) and, in case you want to split the variables per views, also after parsing the sketch (self.parse_sketch). This automatically calls the get_abstract_section method from the model sections.

Returns

AbstractModel – Abstract Model object that can be used for building the model in another language.

Return type

AbstractModel

Vensim section

The Section class allows parsing a model section into Elements using the SectionElementsVisitor. The final result can be exported to an AbstractSection class in order to build a model in another programming language. A section is either the main model (without the macros), or a macro definition.

class pysd.translators.vensim.vensim_section.Section(name: str, path: pathlib.Path, section_type: str, params: List[str], returns: List[str], content: str, split: bool, views_dict: Optional[dict])[source]

The Section class allows parsing the elements of a model section.

Parameters
  • name (str) – Section name. That is, ‘__main__’ for the main section, and the macro name for macros.

  • path (pathlib.Path) – Section path. The model name for the main section and the clean macro name for a macro.

  • section_type (str ('main' or 'macro')) – The section type.

  • params (list) – List of parameters that the section takes. If it is the main section, it will be an empty list.

  • returns (list) – List of variables that returns the section. If it is the main section, it will be an empty list.

  • content (str) – Section content as string.

  • split (bool) – If True, the created section will split the variables depending on the views_dict.

  • views_dict (dict) – The dictionary of the views. Giving the variables classified at any level in order to split them by files.

property verbose

Print section information to standard output.

parse(parse_all: bool = True) None[source]

Parse section object with parsimonious using the grammar given in ‘parsing_grammars/section_elements.peg’ and the class SectionElementsVisitor to visit the parsed expressions.

This will break the section (__main__ or macro) in VensimElements, which are each model expression LHS and RHS with already parsed units and description.

Parameters

parse_all (bool (optional)) – If True, the created VensimElement objects will be automatically parsed. Otherwise, these objects will only be added to self.elements but not parsed. Default is True.

get_abstract_section() pysd.translators.structures.abstract_model.AbstractSection[source]

Instantiate an object of the AbstractSection class that will be used during the building process. This method should be called after parsing the section (self.parse). This method is automatically called by Model’s get_abstract_model method, and automatically generates the AbstractSubscript ranges and merges the components in elements. It also calls the get_abstract_components method from each model component.

Returns

AbstractSection – AbstractSection object that can be used for building the model in another programming language.

Return type

AbstractSection

Vensim element

The Element class allows parsing the LHS of a model equation. Depending on the LHS value, either a SubscriptRange object or a Component object will be returned. There are 4 components types:

  • Component: Regular component, defined with ‘=’.

  • UnchangeableConstant: Unchangeable constant, defined with ‘==’.

  • Data: Data component, defined with ‘:=’

  • Lookup: Lookup component, defined with ‘()’

Lookup components have their own parser for the RHS of the expression, while the other 3 components share the same parser. The final result from a parsed component can be exported to an AbstractComponent object in order to build a model in other programming languages.

class pysd.translators.vensim.vensim_element.Element(equation: str, units: str, documentation: str)[source]

Element object allows parsing the LHS of the Vensim expressions.

Parameters
  • equation (str) – Original equation in the Vensim file.

  • units (str) – The units of the element with the limits, i.e., the content after the first ‘~’ symbol.

  • documentation (str) – The comment of the element, i.e., the content after the second ‘~’ symbol.

property verbose

Print element information to standard output.

parse() object[source]

Parse an Element object with parsimonious using the grammar given in ‘parsing_grammars/element_object.peg’ and the class ElementsComponentVisitor to visit the parsed expressions.

Splits the LHS from the RHS of the equation. If the returned object is a SubscriptRange, no more parsing is needed. Otherwise, the RHS of the returned object (Component) should be parsed to get the AbstractSyntax Tree.

Returns

self.component – The subscript range definition object or component object.

Return type

SubscriptRange or Component

class pysd.translators.vensim.vensim_element.SubscriptRange(name: str, definition: Union[List[str], str, dict], mapping: List[str] = [])[source]

Subscript range definition, defined by “:” or “<->” in Vensim.

property verbose

Print subscript range information to standard output.

get_abstract_subscript_range() pysd.translators.structures.abstract_model.AbstractSubscriptRange[source]

Instantiates an AbstractSubscriptRange object used for building. This method is automatically called by the Sections’s get_abstract_section method.

Returns

AbstractSubscriptRange – AbstractSubscriptRange object that can be used for building the model in another programming language.

Return type

AbstractSubscriptRange

class pysd.translators.vensim.vensim_element.Component(name: str, subscripts: Tuple[list, list], expression: str)[source]

Model component defined by “name = expr” in Vensim.

Parameters
  • name (str) – The original name of the component.

  • subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appears after the :EXCEPT: keyword (if used) as the second argument.

  • expression (str) – The RHS of the element, expression to parse.

property verbose

Print component information to standard output.

parse() None[source]

Parse Component object with parsimonious using the grammar given in ‘parsing_grammars/components.peg’ and the class EquationVisitor to visit the RHS of the expressions.

get_abstract_component() Union[pysd.translators.structures.abstract_model.AbstractComponent, pysd.translators.structures.abstract_model.AbstractLookup][source]

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns

AbstractComponent – Abstract Component object that can be used for building the model in another language. If the component equations include external lookups (GET XLS/DIRECT LOOKUPS), an AbstractLookup class will be used.

Return type

AbstractComponent or AbstractLookup

class pysd.translators.vensim.vensim_element.UnchangeableConstant(name: str, subscripts: Tuple[list, list], expression: str)[source]

Unchangeable constant defined by “name == expr” in Vensim. This class inherits from the Component class.

Parameters
  • name (str) – The original name of the component.

  • subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appears after the :EXCEPT: keyword (if used) as second argument.

  • expression (str) – The RHS of the element, expression to parse.

get_abstract_component() pysd.translators.structures.abstract_model.AbstractUnchangeableConstant[source]

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns

AbstractComponent – Abstract Component object that can be used for building the model in another language.

Return type

AbstractUnchangeableConstant

class pysd.translators.vensim.vensim_element.Lookup(name: str, subscripts: Tuple[list, list], expression: str)[source]

Lookup component, defined by “name(expr)” in Vensim. This class inherits from the Component class.

Parameters
  • name (str) – The original name of the component.

  • subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appear after the :EXCEPT: keyword (if used) as second argument.

  • expression (str) – The RHS of the element, expression to parse.

parse() None[source]

Parse component object with parsimonious using the grammar given in ‘parsing_grammars/lookups.peg’ and the class LookupsVisitor to visit the RHS of the expressions.

get_abstract_component() pysd.translators.structures.abstract_model.AbstractLookup[source]

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns

AbstractComponent – Abstract Component object that may be used for building the model in another language.

Return type

AbstractLookup

class pysd.translators.vensim.vensim_element.Data(name: str, subscripts: Tuple[list, list], keyword: str, expression: str)[source]

Data component, defined by “name := expr” in Vensim. This class inherits from the Component class.

Parameters
  • name (str) – The original name of the component.

  • subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appear after the :EXCEPT: keyword (if used) as second argument.

  • keyword (str) – The keyword used before the “:=” symbol. The following values are possible: ‘interpolate’, ‘raw’, ‘hold_backward’ and ‘look_forward’.

  • expression (str) – The RHS of the element, expression to parse.

parse() None[source]

Parse component object with parsimonious using the grammar given in ‘parsing_grammars/components.peg’ and the class EquationVisitor to visit the RHS of the expressions.

If the expression is None, the data will be read from a VDF file in Vensim.

get_abstract_component() pysd.translators.structures.abstract_model.AbstractData[source]

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns

AbstractComponent – Abstract Component object that can be used for building the model in another language.

Return type

AbstractData

Supported Functions and Features

Ongoing development of the translator will support the full subset of Vensim functionality. The current release supports the following operators, functions and features.

Operators

All the basic operators are supported, this includes the ones shown in the tables below.

Supported unary operators

Vensim

Vensim example

Abstract Syntax

-

-A

LogicStructure([‘negative’], (A,))

+

+A

A

:NOT:

:NOT: A

LogicStructure([‘:NOT:’], (A,))

Supported binary operators

Vensim

Vensim example

Abstract Syntax

^

A ^ B

ArithmeticStructure([‘^’], (A, B))

*

A * B

ArithmeticStructure([‘*’], (A, B))

/

A / B

ArithmeticStructure([‘/’], (A, B))

+

A + B

ArithmeticStructure([‘+’], (A, B))

-

A - B

ArithmeticStructure([‘-‘], (A, B))

=

A = B

LogicStructure([‘=’], (A, B))

<>

A <> B

LogicStructure([‘<>’], (A, B))

<

A < B

LogicStructure([‘<’], (A, B))

>

A > B

LogicStructure([‘>’], (A, B))

>=

A >= B

LogicStructure([‘>=’], (A, B))

<=

A <= B

LogicStructure([‘<=’], (A, B))

:AND:

A :AND: B

LogicStructure([‘:AND:’], (A, B))

:OR:

A :OR: B

LogicStructure([‘:OR:’], (A, B))

Moreover, the Vensim :EXCEPT: operator is also supported to manage exceptions in the subscripts. See the Subscripts section.

Functions

The list of currentlty supported Vensim functions are detailed below:

Supported basic functions

Vensim

Vensim example

Abstract Syntax

Vensim comments

ABS

ABS(A)

CallStructure(‘abs’, (A,))

MIN

MIN(A, B)

CallStructure(‘min’, (A, B))

MAX

MAX(A, B)

CallStructure(‘max’, (A, B))

SQRT

SQRT(A)

CallStructure(‘sqrt’, (A,))

EXP

EXP(A)

CallStructure(‘exp’, (A,))

LN

LN(A)

CallStructure(‘ln’, (A,))

SIN

SIN(A)

CallStructure(‘sin’, (A,))

COS

COS(A)

CallStructure(‘cos’, (A,))

TAN

TAN(A)

CallStructure(‘tan’, (A,))

ARCSIN

ARCSIN(A)

CallStructure(‘arcsin’, (A,))

ARCCOS

ARCCOS(A)

CallStructure(‘arccos’, (A,))

ARCTAN

ARCTAN(A)

CallStructure(‘arctan’, (A,))

INVERT MATRIX

INVERT MATRIX(A)

CallStructure(‘invert_matrix’, (A,))

ELMCOUNT

ELMCOUNT(A)

CallStructure(‘elmcount’, (A,))

INTEGER

INTEGER(A)

CallStructure(‘int’, (A,))

QUANTUM

QUANTUM(A, B)

CallStructure(‘quantum’, (A, B))

MODULO

MODULO(A, B)

CallStructure(‘modulo’, (A, B))

IF THEN ELSE

IF THEN ELSE(A, B, C)

CallStructure(‘if_then_else’, (A, B))

XIDZ

XIDZ(A, B, X)

CallStructure(‘xidz’, (A, B, X))

ZIDZ

ZIDZ(A, B)

CallStructure(‘zidz’, (A, B))

VMIN

VMIN(A[dim!])

CallStructure(‘vmin’, (A,))

VMAX

VMAX(A[dim!])

CallStructure(‘vmax’, (A,))

SUM

SUM(A[dim!])

CallStructure(‘sum’, (A,))

PROD

PROD(A[dim!])

CallStructure(‘prod’, (A,))

PULSE

PULSE(start, width)

CallStructure(‘pulse’, (start, width))

PULSE TRAIN

PULSE TRAIN(start, width, tbetween, end)

CallStructure(‘pulse_train’, (start, tbetween, width, end))

RAMP

RAMP(slope, start_time, end_time)

CallStructure(‘ramp’, (slope, start_time, end_time))

STEP

STEP(height, step_time)

CallStructure(‘step’, (height, step_time))

GET TIME VALUE

GET TIME VALUE(relativeto, offset, measure)

CallStructure(‘get_time_value’, (relativeto, offset, measure))

Not all the cases implemented!

VECTOR SELECT

VECTOR SELECT(sel_array[dim!], exp_array[dim!], miss_val, n_action, e_action)

CallStructure(‘vector_select’, (sel_array, exp_array, miss_val, n_action, e_action))

VECTOR RANK

VECTOR RANK(vec, direction)

CallStructure(‘vector_rank’, (vec, direction))

VECTOR REORDER

VECTOR REORDER(vec, svec)

CallStructure(‘vector_reorder’, (vec, svec))

VECTOR SORT ORDER

VECTOR SORT ORDER(vec, direction)

CallStructure(‘vector_sort_order’, (vec, direction))

GAME

GAME(A)

GameStructure(A)

ALLOCATE AVAILABLE

ALLOCATE AVAILABLE(request, pp, avail)

AllocateAvailableStructure(request, pp, avail)

ALLOCATE BY PRIORITY

ALLOCATE BY PRIORITY(request, priority, size, width, supply)

AllocateByPriorityStructure(request, priority, size, width, supply)

INITIAL

INITIAL(value)

InitialStructure(value)

SAMPLE IF TRUE

SAMPLE IF TRUE(condition, input, initial_value)

SampleIfTrueStructure(condition, input, initial_value)

Supported delay functions

Vensim

Vensim example

Abstract Syntax

DELAY1I

DELAY1I(input, delay_time, initial_value)

DelayStructure(input, delay_time, initial_value, 1)

DELAY1

DELAY1(input, delay_time)

DelayStructure(input, delay_time, input, 1)

DELAY3I

DELAY3I(input, delay_time, initial_value)

DelayStructure(input, delay_time, initial_value, 3)

DELAY3

DELAY3(input, delay_time)

DelayStructure(input, delay_time, input, 3)

DELAY N

DELAY N(input, delay_time, initial_value, n)

DelayNStructure(input, delay_time, initial_value, n)

DELAY FIXED

DELAY FIXED(input, delay_time, initial_value)

DelayFixed(input, delay_time, initial_value)

SMOOTHI

SMOOTH1I(input, delay_time, initial_value)

SmoothStructure(input, smth_time, initial_value, 1)

SMOOTH

SMOOTH1(input, delay_time)

SmoothStructure(input, smth_time, input, 1)

SMOOTH3I

SMOOTH3I(input, delay_time, initial_value)

SmoothStructure(input, smth_time, initial_value, 3)

SMOOTH3

SMOOTH3(input, delay_time)

SmoothStructure(input, smth_time, input, 3)

SMOOTH N

SMOOTH N(input, delay_time, initial_value, n)

SmoothNStructure(input, smth_time, initial_value, n)

FORECAST

FORECAST(input, average_time, horizon)

ForecastStructure(input, average_time, horizon, 0)

TREND

TREND(input, average_time, initial_trend)

TrendStructure(input, average_time, initial_trend)

Supported get functions

Vensim

Vensim example

Abstract Syntax

GET XLS DATA

GET XLS DATA(‘file’, ‘sheet’, ‘time_row_or_col’, ‘cell’)

GetDataStructure(‘file’, ‘sheet’, ‘time_row_or_col’, ‘cell’)

GET DIRECT DATA

GET DIRECT DATA(‘file’, ‘sheet’, ‘time_row_or_col’, ‘cell’)

GetDataStructure(‘file’, ‘sheet’, ‘time_row_or_col’, ‘cell’)

GET XLS LOOKUPS

GET XLS LOOKUPS(‘file’, ‘sheet’, ‘x_row_or_col’, ‘cell’)

GetLookupsStructure(‘file’, ‘sheet’, ‘x_row_or_col’, ‘cell’)

GET DIRECT LOOKUPS

GET DIRECT LOOKUPS(‘file’, ‘sheet’, ‘x_row_or_col’, ‘cell’)

GetLookupsStructure(‘file’, ‘sheet’, ‘x_row_or_col’, ‘cell’)

GET XLS CONSTANTS

GET XLS CONSTANTS(‘file’, ‘sheet’, ‘cell’)

GetConstantsStructure(‘file’, ‘sheet’, ‘cell’)

GET DIRECT CONSTANTS

GET DIRECT CONSTANTS(‘file’, ‘sheet’, ‘cell’)

GetConstantsStructure(‘file’, ‘sheet’, ‘cell’)

GET XLS SUBSCRIPT

GET XLS SUBSCRIPT(‘file’, ‘sheet’, ‘first_cell’, ‘last_cell’, ‘prefix’)

GET DIRECT SUBSCRIPT

GET DIRECT SUBSCRIPT(‘file’, ‘sheet’, ‘first_cell’, ‘last_cell’, ‘prefix’)

Stocks

Stocks defined in Vensim as INTEG(flow, initial_value) are supported and are translated to the AST as IntegStructure(flow, initial_value).

Subscripts

Several subscript related features are also supported. Thiese include:

  • Basic subscript operations with different ranges.

  • Subscript ranges and subranges definitions.

  • Basic subscript mapping, where the subscript range is mapping to a full range (e.g. new_dim: A, B, C -> dim, dim_other). Mapping to a partial range is not yet supported (e.g. new_dim: A, B, C -> dim: E, F, G).

  • Subscript copy (e.g. new_dim <-> dim).

  • :EXCEPT: operator with any number of arguments.

  • Subscript usage as a variable (e.g. my_var[dim] = another var * dim).

  • Subscript vectorial opperations (e.g. SUM(my var[dim, dim!])).

Lookups

Vensim Lookups expressions are supported. They can be defined using hardcoded values, using GET LOOKUPS function or using WITH LOOKUPS function.

Data

Data definitions with GET functions and empty data definitions (no expressions, Vensim uses a VDF file) are supported. These definitions may or may not include any of the possible interpolation keywords: :INTERPOLATE:, :LOOK FORWARD:, :HOLD BACKWARD:, :RAW:. These keywords will be stored in the ‘keyword’ argument of AbstractData as ‘interpolate’, ‘look_forward’, ‘hold_backward’ and ‘raw’, respectively. The Abstract Structure for GET XLS/DATA is given in the supported GET functions table. The Abstract Structure for the empty Data declarations is a DataStructure.

For the moment, any specific functions applying over data are supported (e.g. SHIFT IF TRUE, TIME SHIFT…), but new ones may be includded in the future.

Macro

Vensim macros are supported. The macro content between the keywords :MACRO: and :END OF MACRO: is classified as a section of the model and is subsequently sused to build an independent section from the rest of the model.

Planed New Functions and Features

Discarded Functions and Features

SHIFT IF TRUE

Due to the complexity of SHIFT IF TRUE function and the need to mutate already computed values in another variable, the implementation of this function has been discarded for now.

However, it is possible to reproduce the same behaviour using IF THEN ELSE in the stock flow. A model with a workaround to avoid SHIFT IF TRUE and make the model work with PySD is given in issue #265.