Skip to content

SympyWriter #2923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hbrunie opened this issue Mar 17, 2025 · 14 comments · May be fixed by #2869
Open

SympyWriter #2923

hbrunie opened this issue Mar 17, 2025 · 14 comments · May be fixed by #2869
Assignees
Labels

Comments

@hbrunie
Copy link
Collaborator

hbrunie commented Mar 17, 2025

Hi @hiker , am I missing something here on the way to use SympyWriter?

I wanted to use it that way:

    def evaluate_with_sympy(self, condition: nodes.Node) -> bool:

        sympy_writer = SymPyWriter()
        expr_sympy = sympy_writer([condition])[0]
        new_expr = sympy.simplify(expr_sympy)
        sympy_reader = SymPyReader(sympy_writer)

        psyir_expr: nodes.Node = sympy_reader.psyir_from_expression(new_expr)
        return self._evaluate_to_bool(psyir_expr)

but I managed to show this could not work because of that behavior:

@pytest.mark.parametrize("expressions", [(".true. .and. .false.", "True .AND. False"),
                                         ])
def  test_sym_writer_boolean_expr(fortran_reader, expressions):
    '''Test that booleans are written in the way that SymPy accepts.
    '''
    # A dummy program to easily create the PSyIR for the
    # expressions we need. We just take the RHS of the assignments
    source = f'''program test_prog
                logical :: bool_expr
                bool_expr = {expressions[0]}
                end program test_prog '''

    psyir = fortran_reader.psyir_from_source(source)
    lit = psyir.children[0].children[0].rhs
    assert SymPyWriter()._to_str(lit) == expressions[1]
    sympy_writer = SymPyWriter()
    assert sympy_writer([lit]) == expressions[1]

This test throws this error:

        for expr in expression_str_list:
            try:
                result.append(parse_expr(expr, self.type_map))
            except SyntaxError as err:
>               raise VisitorError(f"Invalid SymPy expression: '{expr}'.") \
                    from err
E               psyclone.psyir.backend.visitor.VisitorError: Visitor Error: Invalid SymPy expression: 'True .AND. False'.

psyclone/psyir/backend/sympy_writer.py:499: VisitorError

You can find all the code in that branch / MR #2869

@hbrunie hbrunie added the bug label Mar 17, 2025
@hbrunie hbrunie added question and removed bug labels Mar 17, 2025
@hbrunie
Copy link
Collaborator Author

hbrunie commented Mar 17, 2025

Besides I think the .AND. should be replaced by and to be understandable by sympy for sympification.

@hiker
Copy link
Collaborator

hiker commented Mar 17, 2025

Yes, I just realised that I have only very rudimentary support for boolean implemented (ok, that might actually not even be rudimentary ... only True and False). I've tried, and it seems not too much work:

yclone/psyir/backend$ git diff sympy_writer.py
diff --git a/src/psyclone/psyir/backend/sympy_writer.py b/src/psyclone/psyir/backend/sympy_writer.py
index cabf7c980..495844508 100644
--- a/src/psyclone/psyir/backend/sympy_writer.py
+++ b/src/psyclone/psyir/backend/sympy_writer.py
@@ -47,7 +47,7 @@ from psyclone.psyir.backend.fortran import FortranWriter
 from psyclone.psyir.backend.visitor import VisitorError
 from psyclone.psyir.frontend.sympy_reader import SymPyReader
 from psyclone.psyir.nodes import (
-    DataNode, Range, Reference, IntrinsicCall, Call)
+    BinaryOperation, DataNode, Range, Reference, IntrinsicCall, Call)
 from psyclone.psyir.symbols import (ArrayType, ScalarType, SymbolTable)
 
 
@@ -697,6 +697,15 @@ class SymPyWriter(FortranWriter):
         return (f"{name}{self.array_parenthesis[0]}"
                 f"{','.join(result)}{self.array_parenthesis[1]}")
 
+    # ------------------------------------------------------------------------
+    def binaryoperation_node(self, node):
+        if node.operator == BinaryOperation.Operator.AND:
+            lhs = self._visit(node.children[0])
+            rhs = self._visit(node.children[1])
+
+            return f"{lhs} & {rhs}"   # sympy 'and'
+        return super().binaryoperation_node(node)
+
     # ------------------------------------------------------------------------
     def gen_indices(self, indices, var_name=None):
         '''Given a list of PSyIR nodes representing the dimensions of an

Then I can run the following test:

@pytest.mark.parametrize("expressions", [(".true. .and. .false.", "False"),
                                         (".true. .and. .true.", "True"),
                                         ])
def  test_sym_writer_boolean_expr(fortran_reader, expressions):
    '''Test that booleans are written in the way that SymPy accepts.
    '''
    # A dummy program to easily create the PSyIR for the
    # expressions we need. We just take the RHS of the assignments
    source = f'''program test_prog
                logical :: bool_expr
                bool_expr = {expressions[0]}
                bool_expr = {expressions[1]}
                end program test_prog '''

    psyir = fortran_reader.psyir_from_source(source)
    lit0 = psyir.children[0].children[0].rhs
    lit1 = psyir.children[0].children[1].rhs
    #assert SymPyWriter()._to_str(lit) == expressions[1]
    sympy_writer = SymPyWriter()

    sympy_expr = sympy_writer(lit0)
    print("XX", lit0, lit1)
    assert sympy_expr == sympy_writer(lit1)

So basically, the .and. expression is converted to sympy (which in many cases will already simplify it - as in this case). Then I converted the second part of the parameterisation (i.e. the expected result) to sympy, and check that the (simplified) parameterised expression 1 is indeed the same as the result (the debug print xx prints the psyir, so you can see the first is an expression, the second a literal.

Of course, it needs a proper fix (there is already code that rewrites intrinsics, so I can add the same for binary operations). I'll try to fix this up in the next couple of days.

@hiker
Copy link
Collaborator

hiker commented Mar 18, 2025

@hbrunie , when do you need this? Also, do you need to convert the logical expressions back to Fortran, or are you ok with just testing the results using sympy (which has its own true/false)?

@hbrunie
Copy link
Collaborator Author

hbrunie commented Mar 24, 2025

@hiker this is not a hurry. I have implemented a draft (non robust) version in our own psyclone fork for the moment.
The usage is essentially to evaluate an psyir expressions in which all Reference have been replaced by Literals and thus could be completely evaluated. Then I get a boolean as returned value.
Thus no back to fortran should be needed.

@hbrunie
Copy link
Collaborator Author

hbrunie commented May 6, 2025

@hiker I would like to wrap the #2869 branch up. It has a dependency with the EvaluateCondition class in tools I have developed. Have you got the time to fix this "bug" on boolean expressions or would you like me to work on that?
Best,
Hugo

@hbrunie hbrunie linked a pull request May 6, 2025 that will close this issue
hiker added a commit that referenced this issue May 7, 2025
hiker added a commit that referenced this issue May 7, 2025
@hiker
Copy link
Collaborator

hiker commented May 7, 2025

@hbrunie , I have created a new branch: https://github.com/stfc/PSyclone/tree/2923_support_boolean_in_sympy
This should already allow you to convert Fortran expression to sympy and then evaluate them. Converting the result back to Fortran is still outstanding.

@hiker
Copy link
Collaborator

hiker commented May 8, 2025

I just realised that you want to evaluate the PSyIR expression, and as such you need to get back from SymPy to PSyIR. Looking at it, that is not that trivial, since logical sympy expression do not directly translate back to Fortran (the Sympy Reader basically converts the SymPy expression to a string, and then parses it with the Fortran parser).

While I am certain that this can be done, I do wonder if we would actually need that support. Can't you evaluate the result in SymPy, e.g.:

>>> (x and y).subs(x, True)
y

I.e. you provide the values in the subs method, and then use SymPy for that?

@hiker
Copy link
Collaborator

hiker commented May 8, 2025

I believe I found a way to convert logical expressions back to PSyIR. While I still think that evaluation in SymPy might be better, this should give you both options.

@hiker
Copy link
Collaborator

hiker commented May 8, 2025

Your example works now as follows:

@pytest.mark.parametrize("expressions", [(".true. .and. .false.", false),
                                         (".true. .and. .true.", true),
                                         ])
def  test_sym_writer_boolean_expr_add_test(fortran_reader, fortran_writer, expressions):
    '''Test that booleans are written in the way that SymPy accepts.
    '''
    # A dummy program to easily create the PSyIR for the
    # expressions we need. We just take the RHS of the assignments
    source = f'''program test_prog
                logical :: bool_expr
                bool_expr = {expressions[0]}
                end program test_prog '''

    psyir = fortran_reader.psyir_from_source(source)
    lit = psyir.children[0].children[0].rhs
    sympy_writer = SymPyWriter()
    sympy_expr = sympy_writer(lit)
    assert sympy_expr == expressions[1]

    # Now convert the sympy expression back to Fortran
    # and do a string comparison
    symbol_table = psyir.children[0].symbol_table
    from psyclone.psyir.frontend.sympy_reader import SymPyReader
    sympy_reader = SymPyReader(sympy_writer)
    new_psyir = sympy_reader.psyir_from_expression(sympy_expr, symbol_table)
    assert fortran_writer(new_psyir) == str(expressions[1])

Notice that I converted the 2nd element of expressions to SymPy booleans (not string). The first assert is the sympy test, the second after converting back to PSyIR.

It needs more testing :)

@hbrunie
Copy link
Collaborator Author

hbrunie commented May 12, 2025

Hi @hiker ,
that works indeed with simple boolean expressions, thank you.
What if the expression is now something with other Literal (not just boolean) like : 3+2 - 5 .and. .true. .or. 3.2 > 0

I have tried with these expressions:

test_sym_writer_boolean_expr_add_test

@pytest.mark.parametrize("expressions", [(".true. .and. .false.", False),
(".true. .and. .true.", True),
(".false. .or. .true.", True),
("3 .eq. 3", True),
(" ((3 -2 + 4 - 5) .eq. 0 .and. .false.) .or. .true.",True),
(" (3 -2 + 4 - 5) .eq. 0 .and. .false. .and. .true.",False),
(" ((3 -2 + 4 - 5) .eq. 0 .and. .false.) .and. .true.",False),
(" .false. .and. ((3 -2 + 4 - 5) .eq. 0 .and. .false.)",False),
])
def test_sym_writer_boolean_expr_add_test(fortran_reader, fortran_writer, expressions):
'''Test that booleans are written in the way that SymPy accepts.
'''
# A dummy program to easily create the PSyIR for the
# expressions we need. We just take the RHS of the assignments
source = f'''program test_prog
logical :: bool_expr
bool_expr = {expressions[0]}
end program test_prog '''

psyir = fortran_reader.psyir_from_source(source)
lit = psyir.children[0].children[0].rhs
sympy_writer = SymPyWriter()
sympy_expr = sympy_writer(lit)
assert sympy_expr == expressions[1]

And the results is wrong for the last 4 ones.

It seems I need to call sympy.sympify maybe?

@hiker
Copy link
Collaborator

hiker commented May 13, 2025

Can you try again with the latest version? I forgot to push the latest changes that support converting back to PSyIR - and as a side effect I had to replaced & with AND(..., ...) etc. The latter seems to fix your issue.

@hbrunie
Copy link
Collaborator Author

hbrunie commented May 14, 2025

Can you try again with the latest version? I forgot to push the latest changes that support converting back to PSyIR - and as a side effect I had to replaced & with AND(..., ...) etc. The latter seems to fix your issue.

Thank you! these 2 commits make all the tests pass. I will clean the MR #2869 , are you ok if it will contain your commits on this issue? (I merged 2923_support_boolean_in_sympy into mine)

hbrunie pushed a commit to hbrunie/PSyclone that referenced this issue May 14, 2025
hbrunie added a commit to hbrunie/PSyclone that referenced this issue May 14, 2025
@hiker
Copy link
Collaborator

hiker commented May 15, 2025

I think it would be easier for the reviewer if this were a standalone PRs (smaller scope). I'll try to submit that tomorrow, I just need to check if I need additional tests for covering the individual files.

hiker added a commit that referenced this issue May 16, 2025
hiker added a commit that referenced this issue May 16, 2025
@hiker
Copy link
Collaborator

hiker commented May 16, 2025

I have #2997 ready (I've added your example as test, and also fixed coverage). I still need to run CI, but will wait till Monday to submit this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants