Vensim Translation

PySD allows parsing a Vensim .mdl file and translates the result to an AbstractModel object that can later (building process) be used to build the model in another programming language.

Translation workflow

The following translation workflow allows splitting the Vensim file while parsing its contents in order to build an AbstractModel object. The workflow may be summarized as follows:

1. Vensim file: splits the model equations from the sketch and allows splitting the model in sections (main section and macro sections).

2. Vensim section: is a full set of variables and definitions that is integrable. The Vensim section can then be split into model expressions.

3. Vensim element: a definition in the mdl file which could be a subscript (sub)range definition or a variable definition. It includes units and comments. Definitions for the same variable are grouped after in the same AbstractElement object. Allows parsing its left hand side (LHS) to get the name of the subscript (sub)range or variable and it is returned as a specific type of component depending on the used assign operator (=, ==, :=, (), :)

4. Vensim component: the classified object for a variable definition, it depends on the operator used to define the variable. Its right hand side (RHS) can be parsed to get the Abstract Syntax Tree (AST) of the expression.

Once the model is parsed and broken following the previous steps, the AbstractModel is returned.

Vensim file

The VensimFile class allows reading the original Vensim model file, parsing it into Section elements using the FileSectionsVisitor, parsing its sketch using SketchVisitor in order to classify the varibales per view. The final result can be exported to an AbstractModel class in order to build the model in another programming language.

class pysd.translators.vensim.vensim_file.VensimFile(mdl_path: str | Path, encoding: None | str = None)

The VensimFile class allows parsing an mdl file. When the object is created, the model file is automatically opened; unnecessary tabs, whitespaces, and linebreaks are removed; and the sketch is split from the model equations.

Parameters:

• mdl_path (str or pathlib.Path) – Path to the Vensim model.

• encoding (str or None (optional)) – Encoding of the source model file. If None, the encoding will be read from the model, if the encoding is not defined in the model file it will be set to ‘UTF-8’. Default is None.

property verbose

Print model information to standard output.

parse(parse_all: bool = True) → None

Parse model file with parsimonious using the grammar given in ‘parsing_grammars/file_sections.peg’ and the class FileSectionsVisitor to visit the parsed expressions.

This breaks the model file in VensimSections, which correspond to the main model section and the macros.

Parameters:

parse_all (bool (optional)) – If True, the VensimSection objects created will be automatically parsed. Otherwise, these objects will only be added to self.sections but not parsed. Default is True.

parse_sketch(subview_sep: List[str]) → None

Parse the sketch of the model with parsimonious using the grammar given in ‘parsing_grammars/sketch.peg’ and the class SketchVisitor to visit the parsed expressions.

It will modify the views_dict of the first section, including the dictionary of the classification of variables by views. This, method should be called after calling the self.parse method.

Parameters:

subview_sep (list) – List of the separators to use to classify the model views in folders and subfolders. The sepparator must be ordered in the same order they appear in the view name. For example, if a view is named “economy:demand.exports” if subview_sep=[“:”, “.”] this view’s variables will be included in the file ‘exports.py’ and inside the folders economy/demand.

get_abstract_model() → AbstractModel

Instantiate the AbstractModel object used during building. This, method should be called after parsing the model (self.parse) and, in case you want to split the variables per views, also after parsing the sketch (self.parse_sketch). This automatically calls the get_abstract_section method from the model sections.

Returns:

AbstractModel – Abstract Model object that can be used for building the model in another language.

Return type:

AbstractModel

Vensim section

The Section class allows parsing a model section into Elements using the SectionElementsVisitor. The final result can be exported to an AbstractSection class in order to build a model in another programming language. A section is either the main model (without the macros), or a macro definition.

class pysd.translators.vensim.vensim_section.Section(name: str, path: Path, section_type: str, params: List[str], returns: List[str], content: str, split: bool, views_dict: dict | None)

The Section class allows parsing the elements of a model section.

Parameters:

• name (str) – Section name. That is, ‘__main__’ for the main section, and the macro name for macros.

• path (pathlib.Path) – Section path. The model name for the main section and the clean macro name for a macro.

• section_type (str ('main' or 'macro')) – The section type.

• params (list) – List of parameters that the section takes. If it is the main section, it will be an empty list.

• returns (list) – List of variables that returns the section. If it is the main section, it will be an empty list.

• content (str) – Section content as string.

• split (bool) – If True, the created section will split the variables depending on the views_dict.

• views_dict (dict) – The dictionary of the views. Giving the variables classified at any level in order to split them by files.

property verbose

Print section information to standard output.

parse(parse_all: bool = True) → None

Parse section object with parsimonious using the grammar given in ‘parsing_grammars/section_elements.peg’ and the class SectionElementsVisitor to visit the parsed expressions.

This will break the section (__main__ or macro) in VensimElements, which are each model expression LHS and RHS with already parsed units and description.

Parameters:

parse_all (bool (optional)) – If True, the created VensimElement objects will be automatically parsed. Otherwise, these objects will only be added to self.elements but not parsed. Default is True.

get_abstract_section() → AbstractSection

Instantiate an object of the AbstractSection class that will be used during the building process. This method should be called after parsing the section (self.parse). This method is automatically called by Model’s get_abstract_model method, and automatically generates the AbstractSubscript ranges and merges the components in elements. It also calls the get_abstract_components method from each model component.

Returns:

AbstractSection – AbstractSection object that can be used for building the model in another programming language.

Return type:

AbstractSection

Vensim element

The Element class allows parsing the LHS of a model equation. Depending on the LHS value, either a SubscriptRange object or a Component object will be returned. There are four components types:

• Component: Regular component, defined with ‘=’.

• UnchangeableConstant: Unchangeable constant, defined with ‘==’.

• Data: Data component, defined with ‘:=’

• Lookup: Lookup component, defined with ‘()’

Lookup components have their own parser for the RHS of the expression, while the other 3 components share the same parser. The final result from a parsed component can be exported to an AbstractComponent object in order to build a model in other programming languages. Two more element-like objects could be defined, which are only used for testing:

• Constraint: constraint for Reality check, defined with ‘:THE CONDITION:’

• TestInput: inputs for testing, defined with ‘:TEST INPUT:’

class pysd.translators.vensim.vensim_element.Element(equation: str, units: str, documentation: str)

Element object allows parsing the LHS of the Vensim expressions.

Parameters:

• equation (str) – Original equation in the Vensim file.

• units (str) – The units of the element with the limits, i.e., the content after the first ‘~’ symbol.

• documentation (str) – The comment of the element, i.e., the content after the second ‘~’ symbol.

property verbose

Print element information to standard output.

parse() → object

Parse an Element object with parsimonious using the grammar given in ‘parsing_grammars/element_object.peg’ and the class ElementsComponentVisitor to visit the parsed expressions.

Splits the LHS from the RHS of the equation. If the returned object is a SubscriptRange, no more parsing is needed. Otherwise, the RHS of the returned object (Component) should be parsed to get the AbstractSyntax Tree.

Returns:

self.component – The subscript range definition object or component object.

Return type:

SubscriptRange or Component

class pysd.translators.vensim.vensim_element.SubscriptRange(name: str, definition: List[str] | str | dict, mapping: List[str] = [])

Subscript range definition, defined by “:” or “<->” in Vensim.

property verbose

Print subscript range information to standard output.

get_abstract_subscript_range() → AbstractSubscriptRange

Instantiates an AbstractSubscriptRange object used for building. This method is automatically called by the Sections’s get_abstract_section method.

Returns:

AbstractSubscriptRange – AbstractSubscriptRange object that can be used for building the model in another programming language.

Return type:

AbstractSubscriptRange

class pysd.translators.vensim.vensim_element.Component(name: str, subscripts: Tuple[list, list], expression: str)

Model component defined by “name = expr” in Vensim.

Parameters:

• name (str) – The original name of the component.

• subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appears after the :EXCEPT: keyword (if used) as the second argument.

• expression (str) – The RHS of the element, expression to parse.

parse() → None

Parse Component object with parsimonious using the grammar given in ‘parsing_grammars/components.peg’ and the class EquationVisitor to visit the RHS of the expressions.

get_abstract_component() → AbstractComponent | AbstractLookup

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns:

AbstractComponent – Abstract Component object that can be used for building the model in another language. If the component equations include external lookups (GET XLS/DIRECT LOOKUPS), an AbstractLookup class will be used.

Return type:

AbstractComponent or AbstractLookup

class pysd.translators.vensim.vensim_element.UnchangeableConstant(name: str, subscripts: Tuple[list, list], expression: str)

Unchangeable constant defined by “name == expr” in Vensim. This class inherits from the Component class.

Parameters:

• name (str) – The original name of the component.

• subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appears after the :EXCEPT: keyword (if used) as second argument.

• expression (str) – The RHS of the element, expression to parse.

get_abstract_component() → AbstractUnchangeableConstant

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns:

AbstractComponent – Abstract Component object that can be used for building the model in another language.

Return type:

AbstractUnchangeableConstant

class pysd.translators.vensim.vensim_element.Lookup(name: str, subscripts: Tuple[list, list], expression: str)

Lookup component, defined by “name(expr)” in Vensim. This class inherits from the Component class.

Parameters:

• name (str) – The original name of the component.

• subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appear after the :EXCEPT: keyword (if used) as second argument.

• expression (str) – The RHS of the element, expression to parse.

parse() → None

Parse component object with parsimonious using the grammar given in ‘parsing_grammars/lookups.peg’ and the class LookupsVisitor to visit the RHS of the expressions.

get_abstract_component() → AbstractLookup

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns:

AbstractComponent – Abstract Component object that may be used for building the model in another language.

Return type:

AbstractLookup

class pysd.translators.vensim.vensim_element.Data(name: str, subscripts: Tuple[list, list], keyword: str, expression: str)

Data component, defined by “name := expr” in Vensim. This class inherits from the Component class.

Parameters:

• name (str) – The original name of the component.

• subscripts (tuple) – Tuple of length two with the list of subscripts in the variable definition as first argument and the list of subscripts that appear after the :EXCEPT: keyword (if used) as second argument.

• keyword (str) – The keyword used before the “:=” symbol. The following values are possible: ‘interpolate’, ‘raw’, ‘hold_backward’ and ‘look_forward’.

• expression (str) – The RHS of the element, expression to parse.

parse() → None

Parse component object with parsimonious using the grammar given in ‘parsing_grammars/components.peg’ and the class EquationVisitor to visit the RHS of the expressions.

If the expression is None, the data will be read from a VDF file in Vensim.

get_abstract_component() → AbstractData

Get Abstract Component used for building. This method is automatically called by Sections’s get_abstract_section method.

Returns:

AbstractComponent – Abstract Component object that can be used for building the model in another language.

Return type:

AbstractData

Supported Functions and Features

Ongoing development of the translator will support the full subset of Vensim functionality. The current release supports the following operators, functions and features.

Operators

All the basic operators are supported, this includes the ones shown in the tables below.

Supported unary operators

Vensim	Vensim example	Abstract Syntax
-	-A	LogicStructure([‘negative’], (A,))
+	+A	A
:NOT:	:NOT: A	LogicStructure([‘:NOT:’], (A,))

Supported binary operators

Vensim	Vensim example	Abstract Syntax
^	A ^ B	ArithmeticStructure([‘^’], (A, B))
*	A * B	ArithmeticStructure([‘*’], (A, B))
/	A / B	ArithmeticStructure([‘/’], (A, B))
+	A + B	ArithmeticStructure([‘+’], (A, B))
-	A - B	ArithmeticStructure([‘-‘], (A, B))
=	A = B	LogicStructure([‘=’], (A, B))
<>	A <> B	LogicStructure([‘<>’], (A, B))
<	A < B	LogicStructure([‘<’], (A, B))
>	A > B	LogicStructure([‘>’], (A, B))
>=	A >= B	LogicStructure([‘>=’], (A, B))
<=	A <= B	LogicStructure([‘<=’], (A, B))
:AND:	A :AND: B	LogicStructure([‘:AND:’], (A, B))
:OR:	A :OR: B	LogicStructure([‘:OR:’], (A, B))

Moreover, the Vensim :EXCEPT: operator is also supported to manage exceptions in the subscripts. See the Subscripts section.

Functions

The list of currently supported Vensim functions are detailed below:

Supported basic functions

Vensim	Vensim example	Abstract Syntax	Vensim comments
ABS	ABS(A)	CallStructure(‘abs’, (A,))
MIN	MIN(A, B)	CallStructure(‘min’, (A, B))
MAX	MAX(A, B)	CallStructure(‘max’, (A, B))
SQRT	SQRT(A)	CallStructure(‘sqrt’, (A,))
EXP	EXP(A)	CallStructure(‘exp’, (A,))
LN	LN(A)	CallStructure(‘ln’, (A,))
SIN	SIN(A)	CallStructure(‘sin’, (A,))
COS	COS(A)	CallStructure(‘cos’, (A,))
TAN	TAN(A)	CallStructure(‘tan’, (A,))
ARCSIN	ARCSIN(A)	CallStructure(‘arcsin’, (A,))
ARCCOS	ARCCOS(A)	CallStructure(‘arccos’, (A,))
ARCTAN	ARCTAN(A)	CallStructure(‘arctan’, (A,))
INVERT MATRIX	INVERT MATRIX(A)	CallStructure(‘invert_matrix’, (A,))
ELMCOUNT	ELMCOUNT(A)	CallStructure(‘elmcount’, (A,))
INTEGER	INTEGER(A)	CallStructure(‘int’, (A,))
QUANTUM	QUANTUM(A, B)	CallStructure(‘quantum’, (A, B))
MODULO	MODULO(A, B)	CallStructure(‘modulo’, (A, B))
IF THEN ELSE	IF THEN ELSE(A, B, C)	CallStructure(‘if_then_else’, (A, B))
XIDZ	XIDZ(A, B, X)	CallStructure(‘xidz’, (A, B, X))
ZIDZ	ZIDZ(A, B)	CallStructure(‘zidz’, (A, B))
VMIN	VMIN(A[dim!])	CallStructure(‘vmin’, (A,))
VMAX	VMAX(A[dim!])	CallStructure(‘vmax’, (A,))
SUM	SUM(A[dim!])	CallStructure(‘sum’, (A,))
PROD	PROD(A[dim!])	CallStructure(‘prod’, (A,))
PULSE	PULSE(start, width)	CallStructure(‘pulse’, (start, width))
PULSE TRAIN	PULSE TRAIN(start, width, tbetween, end)	CallStructure(‘pulse_train’, (start, tbetween, width, end))
RAMP	RAMP(slope, start_time, end_time)	CallStructure(‘ramp’, (slope, start_time, end_time))
STEP	STEP(height, step_time)	CallStructure(‘step’, (height, step_time))
GET TIME VALUE	GET TIME VALUE(relativeto, offset, measure)	CallStructure(‘get_time_value’, (relativeto, offset, measure))	Not all the cases implemented!
VECTOR SELECT	VECTOR SELECT(sel_array[dim!], exp_array[dim!], miss_val, n_action, e_action)	CallStructure(‘vector_select’, (sel_array, exp_array, miss_val, n_action, e_action))
VECTOR RANK	VECTOR RANK(vec, direction)	CallStructure(‘vector_rank’, (vec, direction))
VECTOR REORDER	VECTOR REORDER(vec, svec)	CallStructure(‘vector_reorder’, (vec, svec))
VECTOR SORT ORDER	VECTOR SORT ORDER(vec, direction)	CallStructure(‘vector_sort_order’, (vec, direction))
GAME	GAME(A)	GameStructure(A)
ALLOCATE AVAILABLE	ALLOCATE AVAILABLE(request, pp, avail)	AllocateAvailableStructure(request, pp, avail)
ALLOCATE BY PRIORITY	ALLOCATE BY PRIORITY(request, priority, size, width, supply)	AllocateByPriorityStructure(request, priority, size, width, supply)
INITIAL	INITIAL(value)	InitialStructure(value)
SAMPLE IF TRUE	SAMPLE IF TRUE(condition, input, initial_value)	SampleIfTrueStructure(condition, input, initial_value)
RANDOM 0 1	RANDOM 0 1()	CallStructure(‘random_0_1’, ())
RANDOM UNIFORM	RANDOM UNIFORM(m, x, s)	CallStructure(‘random_uniform’, (m, x, s))
RANDOM NORMAL	RANDOM NORMAL(m, x, h, r, s)	CallStructure(‘random_normal’, (m, x, h, r, s))
RANDOM EXPONENTIAL	RANDOM EXPONENTIAL(m, x, h, r, s)	CallStructure(‘random_exponential’, (m, x, h, r, s))

Supported delay functions

Vensim	Vensim example	Abstract Syntax
DELAY1I	DELAY1I(input, delay_time, initial_value)	DelayStructure(input, delay_time, initial_value, 1)
DELAY1	DELAY1(input, delay_time)	DelayStructure(input, delay_time, input, 1)
DELAY3I	DELAY3I(input, delay_time, initial_value)	DelayStructure(input, delay_time, initial_value, 3)
DELAY3	DELAY3(input, delay_time)	DelayStructure(input, delay_time, input, 3)
DELAY N	DELAY N(input, delay_time, initial_value, n)	DelayNStructure(input, delay_time, initial_value, n)
DELAY FIXED	DELAY FIXED(input, delay_time, initial_value)	DelayFixed(input, delay_time, initial_value)
SMOOTHI	SMOOTH1I(input, delay_time, initial_value)	SmoothStructure(input, smth_time, initial_value, 1)
SMOOTH	SMOOTH1(input, delay_time)	SmoothStructure(input, smth_time, input, 1)
SMOOTH3I	SMOOTH3I(input, delay_time, initial_value)	SmoothStructure(input, smth_time, initial_value, 3)
SMOOTH3	SMOOTH3(input, delay_time)	SmoothStructure(input, smth_time, input, 3)
SMOOTH N	SMOOTH N(input, delay_time, initial_value, n)	SmoothNStructure(input, smth_time, initial_value, n)
FORECAST	FORECAST(input, average_time, horizon)	ForecastStructure(input, average_time, horizon, 0)
TREND	TREND(input, average_time, initial_trend)	TrendStructure(input, average_time, initial_trend)

Supported get functions

Vensim	Vensim example	Abstract Syntax
GET XLS DATA	GET XLS DATA(‘file’, ‘tab’, ‘time_row_or_col’, ‘cell’)	GetDataStructure(‘file’, ‘tab’, ‘time_row_or_col’, ‘cell’)
GET DIRECT DATA	GET DIRECT DATA(‘file’, ‘tab’, ‘time_row_or_col’, ‘cell’)	GetDataStructure(‘file’, ‘tab’, ‘time_row_or_col’, ‘cell’)
GET XLS LOOKUPS	GET XLS LOOKUPS(‘file’, ‘tab’, ‘x_row_or_col’, ‘cell’)	GetLookupsStructure(‘file’, ‘tab’, ‘x_row_or_col’, ‘cell’)
GET DIRECT LOOKUPS	GET DIRECT LOOKUPS(‘file’, ‘tab’, ‘x_row_or_col’, ‘cell’)	GetLookupsStructure(‘file’, ‘tab’, ‘x_row_or_col’, ‘cell’)
GET XLS CONSTANTS	GET XLS CONSTANTS(‘file’, ‘tab’, ‘cell’)	GetConstantsStructure(‘file’, ‘tab’, ‘cell’)
GET DIRECT CONSTANTS	GET DIRECT CONSTANTS(‘file’, ‘tab’, ‘cell’)	GetConstantsStructure(‘file’, ‘tab’, ‘cell’)
GET XLS SUBSCRIPT	GET XLS SUBSCRIPT(‘file’, ‘tab’, ‘first_cell’, ‘last_cell’, ‘prefix’)
GET DIRECT SUBSCRIPT	GET DIRECT SUBSCRIPT(‘file’, ‘tab’, ‘first_cell’, ‘last_cell’, ‘prefix’)

Stocks

Stocks defined in Vensim as INTEG(flow, initial_value) are supported and are translated to the AST as IntegStructure(flow, initial_value).

Subscripts

Several subscript related features are also supported. These include:

• Basic subscript operations with different ranges.

• Subscript ranges and subranges definitions.

• Basic subscript mapping, where the subscript range is mapping to a full range (e.g. new_dim: A, B, C -> dim, dim_other). Mapping to a partial range is not yet supported (e.g. new_dim: A, B, C -> dim: E, F, G).

• Subscript copy (e.g. new_dim <-> dim).

• :EXCEPT: operator with any number of arguments.

• Subscript usage as a variable (e.g. my_var[dim] = another var * dim).

• Subscript vectorial operations (e.g. SUM(my var[dim, dim!])).

Lookups

Vensim Lookups expressions are supported. They can be defined using hardcoded values, using GET LOOKUPS function or using WITH LOOKUPS function.

Data

Data definitions with GET functions and empty data definitions (no expressions, Vensim uses a VDF file) are supported. These definitions may or may not include any of the possible interpolation keywords: :INTERPOLATE:, :LOOK FORWARD:, :HOLD BACKWARD:, :RAW:. These keywords will be stored in the ‘keyword’ argument of AbstractData as ‘interpolate’, ‘look_forward’, ‘hold_backward’ and ‘raw’, respectively. The Abstract Structure for GET XLS/DATA is given in the supported GET functions table. The Abstract Structure for the empty Data declarations is a DataStructure.

For the moment, any specific functions applying over data are supported (e.g. SHIFT IF TRUE, TIME SHIFT…), but new ones may be included in the future.

Macro

Vensim macros are supported. The macro content between the keywords :MACRO: and :END OF MACRO: is classified as a section of the model and is subsequently used to build an independent section from the rest of the model.

Planed New Functions and Features

Discarded Functions and Features

SHIFT IF TRUE

Due to the complexity of SHIFT IF TRUE function and the need to mutate already computed values in another variable, the implementation of this function has been discarded for now.

However, it is possible to reproduce the same behaviour using IF THEN ELSE in the stock flow. A model with a workaround to avoid SHIFT IF TRUE and make the model work with PySD is given in issue #265.