Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature[next]: Temporary extraction heuristics #1340

Closed
wants to merge 233 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
233 commits
Select commit Hold shift + click to select a range
f8755c2
Extend common subexpression elimination to support expressions contai…
tehrengruber Feb 21, 2023
defb2dd
Cleanup
tehrengruber Feb 21, 2023
88742b1
Address reviewer comments
tehrengruber Mar 2, 2023
f78c958
Poor man's cache for gtfn backend
tehrengruber Mar 6, 2023
5fd56f7
Tiny fix
tehrengruber Mar 6, 2023
5867201
Propagate foast typing information to ITIR
tehrengruber Mar 6, 2023
117838e
Add CachedWorkflow
tehrengruber Mar 8, 2023
abd283e
Fix format
tehrengruber Mar 8, 2023
6f97022
Small improvements to ITIR type inference
tehrengruber Mar 8, 2023
597d771
Merge branch 'cse_enhancements' into itir_temporaries
tehrengruber Mar 8, 2023
2dc6ff8
Merge branch 'poor_mans_caching' into itir_temporaries
tehrengruber Mar 8, 2023
00b5892
Merge branch 'itir_type_inference_improvements' into itir_temporaries
tehrengruber Mar 8, 2023
9a41559
Fix inlining lambda if parameter is not referenced
tehrengruber Mar 9, 2023
e729f79
Merge branch 'fix_lambda_inliner' into itir_temporaries
tehrengruber Mar 9, 2023
f9cbfaa
Address reviewer comments
tehrengruber Mar 11, 2023
952bb61
Address reviewer comments
tehrengruber Mar 11, 2023
1aa59df
Fix doctests and address comment from @egparedes
tehrengruber Mar 17, 2023
ce7b57b
Merge remote-tracking branch 'origin/main' into poor_mans_caching
tehrengruber Mar 17, 2023
5ef8e44
Polish & add test
tehrengruber Mar 17, 2023
f4fc239
Merge remote-tracking branch 'origin/main' into propagate_foast_typin…
tehrengruber Mar 17, 2023
6b058f7
Fix format
tehrengruber Mar 17, 2023
cd9b338
Merge `poor_mans_caching`
tehrengruber Mar 17, 2023
6dce122
Merge `origin/main`
tehrengruber Mar 17, 2023
5456656
Fix gtfn caching
tehrengruber Mar 17, 2023
525dd31
Merge branch 'fix_gtfn_caching' into itir_temporaries
tehrengruber Mar 17, 2023
18ce6d1
Merge branch 'propagate_foast_typing_to_itir' into itir_temporaries
tehrengruber Mar 17, 2023
32cc82f
Simplify TraceShifts pass
tehrengruber Mar 21, 2023
92c1ce1
Tiny fix
tehrengruber Mar 22, 2023
a2bf0ce
Fix wrong comment
tehrengruber Mar 22, 2023
13b1a0d
Address reviewer comments
tehrengruber Mar 23, 2023
c36a08a
Address reviewer comments
tehrengruber Mar 23, 2023
b7cab5d
Merge origin/main
tehrengruber Mar 23, 2023
10df9ab
Address reviewer comment
tehrengruber Mar 23, 2023
8050ece
Fix format
tehrengruber Mar 23, 2023
435d076
Merge remote-tracking branch 'origin/main' into propagate_foast_typin…
tehrengruber Mar 23, 2023
1073647
Fix broken tests
tehrengruber Mar 24, 2023
df5856b
Merge branch 'simplify_trace_shifts' into itir_temporaries
tehrengruber Mar 24, 2023
984e638
Add map_ builtin to TraceShift
tehrengruber Mar 25, 2023
1aece74
Merge branch 'simplify_trace_shifts' into itir_temporaries
tehrengruber Mar 25, 2023
27c82f8
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber Mar 27, 2023
b6e5b19
Add support for local fields
tehrengruber Apr 3, 2023
d294ca4
Merge remote-tracking branch 'origin/main' into propagate_foast_typin…
tehrengruber Apr 3, 2023
8048d7a
Merge branch 'propagate_foast_typing_to_itir' into itir_temporaries
tehrengruber Apr 4, 2023
4b8ace2
Small fixes
tehrengruber Apr 4, 2023
fc9a22c
Merge branch 'propagate_foast_typing_to_itir' into itir_temporaries
tehrengruber Apr 4, 2023
6928af6
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber Apr 11, 2023
b8766c2
Merge origin/main
tehrengruber May 1, 2023
a534e2a
Performance enhancements
tehrengruber May 13, 2023
965238a
Enhance cse pass
tehrengruber May 16, 2023
5dee2a2
Improve common subexpression elimination (CSE) pass
tehrengruber May 16, 2023
fc6f27f
Fix annotation
tehrengruber May 17, 2023
d7d4e55
Merge improve origin_tehrengruber/improve_cse_pass
tehrengruber May 22, 2023
2b1dbb9
Typo
tehrengruber May 22, 2023
e83e183
Remove int type (without size) from FOAST, PAST, ITIR
tehrengruber May 22, 2023
3b12e66
Merge origin/main
tehrengruber May 22, 2023
7b40a51
Cleanup
tehrengruber May 22, 2023
7caa673
Fix type inference
tehrengruber May 22, 2023
6844afb
Small fixes
tehrengruber May 22, 2023
ce3956f
Fix type inference tests
tehrengruber May 22, 2023
9971c4c
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber May 22, 2023
686c328
Merge branch 'remove_int_type' into itir_temporaries
tehrengruber May 22, 2023
13bb9f6
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber May 22, 2023
90132c5
Fix doctests
tehrengruber May 22, 2023
3f72f87
Fix tests
tehrengruber May 22, 2023
6e71f9f
Fix format
tehrengruber May 22, 2023
d9c409d
Fix tests
tehrengruber May 22, 2023
0689e8f
Merge branch 'remove_int_type' into itir_temporaries
tehrengruber May 22, 2023
d66c64f
Fix failing tests
tehrengruber May 24, 2023
5a4e5af
Merge remote-tracking branch 'origin/main' into remove_int_type
tehrengruber May 24, 2023
8967788
Address reviewer comments
tehrengruber May 25, 2023
b4e6887
Merge remote-tracking branch 'origin/main' into improve_cse_pass
tehrengruber May 28, 2023
bcf123a
Fix broken test
tehrengruber May 28, 2023
2c182ec
Merge branch 'remove_int_type' into itir_temporaries
tehrengruber May 28, 2023
4f70fb6
Change default datatype for tests to float
tehrengruber May 30, 2023
7aebcc2
Fix format
tehrengruber May 30, 2023
81bdde7
Address reviewer comments
tehrengruber May 30, 2023
b4c3f81
Address reviewer comments
tehrengruber May 30, 2023
5e59445
Save type to annex in ITIR type inference
tehrengruber May 30, 2023
3dcc350
Use type information from annex for CollapseTuple pass
tehrengruber May 30, 2023
c8dff55
Merge branch 'improve_cse_pass' into itir_temporaries
tehrengruber May 30, 2023
bc3714b
Merge branch 'improve_collapse_tuple_get' into itir_temporaries
tehrengruber May 30, 2023
eb3cbf6
Refactoring of CommonSubexpression pass
tehrengruber May 30, 2023
d20775b
Fix format
tehrengruber May 30, 2023
f949abf
Merge origin/main
tehrengruber May 30, 2023
e57cb6d
Merge branch 'change_default_case_to_float' into itir_temporaries
tehrengruber May 30, 2023
4d928b2
Fix format
tehrengruber May 30, 2023
404bf35
Merge origin_tehrengruber/refactor_cse_pass
tehrengruber May 30, 2023
62d04b4
Change default dtype in testing to float for unstructured
tehrengruber May 31, 2023
9f03c43
Remove IJKIntField type alias
tehrengruber May 31, 2023
1e9e191
Merge branch 'change_default_case_to_float' into remove_int_type
tehrengruber May 31, 2023
a3fce3a
Fix broken tests
tehrengruber May 31, 2023
7ac5876
Fix broken assert
tehrengruber May 31, 2023
cd66946
Merge branch 'save_types_to_annex' into improve_collapse_tuple_get
tehrengruber May 31, 2023
a09f18f
Use regular type inference instead of relying on annex information
tehrengruber May 31, 2023
cfc867f
Fix format
tehrengruber May 31, 2023
bdc70e4
Small fix
tehrengruber May 31, 2023
115feca
Merge branch 'save_types_to_annex' into improve_collapse_tuple_get
tehrengruber May 31, 2023
ed17421
Add comment
tehrengruber May 31, 2023
103cd35
Merge branch 'save_types_to_annex' into improve_collapse_tuple_get
tehrengruber May 31, 2023
f8aa0f3
Fix format
tehrengruber May 31, 2023
cdb33ba
Fix doctest
tehrengruber May 31, 2023
f6dd124
Merge origin_tehrengruber/improve_collapse_tuple_get
tehrengruber May 31, 2023
9730ac2
Merge origin_tehrengruber/remove_int_type
tehrengruber May 31, 2023
8ad7d90
Merge remote-tracking branch 'origin/main' into remove_int_type
tehrengruber May 31, 2023
b5f05d1
Fix failing tests
tehrengruber May 31, 2023
f3532e3
Fix failing tests
tehrengruber May 31, 2023
49f5118
Fix failing tests
tehrengruber May 31, 2023
9cb36df
Fix format
tehrengruber May 31, 2023
25c55ad
Fix failing tests
tehrengruber May 31, 2023
cedd50d
Address reviewer comments
tehrengruber May 31, 2023
1ad3eae
Address reviewer comments
tehrengruber May 31, 2023
1094086
Merge branch 'save_types_to_annex' into improve_collapse_tuple_get
tehrengruber May 31, 2023
d513e8e
Merge branch 'improve_collapse_tuple_get' into itir_temporaries
tehrengruber Jun 1, 2023
8a5359e
Merge branch 'remove_int_type' into itir_temporaries
tehrengruber Jun 1, 2023
d6a7d64
Merge origin/main
tehrengruber Jun 6, 2023
bb9ed6e
Merge origin/main
tehrengruber Jun 12, 2023
24d6c7a
Address review comments
tehrengruber Jun 12, 2023
d024d91
Merge remote-tracking branch 'origin/main' into change_default_case_t…
tehrengruber Jun 12, 2023
c4aab5b
Merge origin/main
tehrengruber Jun 12, 2023
3b9c47a
Use int32 as default data type
tehrengruber Jun 12, 2023
c239bf9
Merge origin_tehrengruber/change_default_case_to_float
tehrengruber Jun 12, 2023
8f85deb
Fix tests
tehrengruber Jun 12, 2023
a7155a8
Fix tests
tehrengruber Jun 12, 2023
46d588e
Fix tests
tehrengruber Jun 12, 2023
8581783
Fix typo
tehrengruber Jun 12, 2023
0c6f399
Fix tests
tehrengruber Jun 13, 2023
47d82b6
Fix typo
tehrengruber Jun 13, 2023
fdd792b
Merge origin/main
tehrengruber Jun 13, 2023
69cd797
Merge origin/main
tehrengruber Jun 13, 2023
efd8baa
Merge origin/main
tehrengruber Jun 14, 2023
97f3bc0
Small fix
tehrengruber Jun 14, 2023
5c7a76a
Small fix
tehrengruber Jun 14, 2023
8246e8f
Small fix
tehrengruber Jun 14, 2023
2b18b1a
Cleanup
tehrengruber Jun 14, 2023
d0e3cd2
Add support for temporaries
tehrengruber Jun 14, 2023
bf766fd
Merge origin_tehrengruber/remove_int_type
tehrengruber Jun 14, 2023
ddef8b0
Merge origin/main
tehrengruber Jun 14, 2023
f69a014
Merge remote-tracking branch 'origin/main' into improve_collapse_tupl…
tehrengruber Jun 14, 2023
60fbaf2
Fix type inference
tehrengruber Jun 14, 2023
d2b73fe
Merge origin_tehrengruber/improve_collapse_tuple_get
tehrengruber Jun 14, 2023
218b0be
Fix format
tehrengruber Jun 14, 2023
fe7ca93
Fix broken merge
tehrengruber Jun 15, 2023
c1aaabd
Floordiv fix
tehrengruber Jun 15, 2023
06dde8b
Fix format
tehrengruber Jun 15, 2023
538d9bb
Fix broken merge
tehrengruber Jun 15, 2023
d76ef97
Fix mypy
tehrengruber Jun 19, 2023
6f436e5
Cleanup
tehrengruber Jun 19, 2023
aa17d22
Cleanup
tehrengruber Jun 19, 2023
eb55967
Fix broken merge
tehrengruber Jun 19, 2023
5e1f53f
Fix nabla
tehrengruber Jun 19, 2023
5f5186e
Fix nabla
tehrengruber Jun 19, 2023
27f14c0
Fix format
tehrengruber Jun 19, 2023
a174500
Fix format
tehrengruber Jun 19, 2023
367e34d
Fix tuple tests
tehrengruber Jun 19, 2023
cb3cdd2
Fix tuple tests
tehrengruber Jun 19, 2023
bcc6a11
Fix tuple tests
tehrengruber Jun 19, 2023
ecf7c3a
Fix extent analysis
tehrengruber Jun 19, 2023
40b1c15
Cleanup
tehrengruber Jun 19, 2023
877899d
Fix temporary sizes in gtfn backend + enable laplacian frontend test …
tehrengruber Jun 19, 2023
6a37daa
Merge origin/main
tehrengruber Jun 19, 2023
852f0e7
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber Jun 19, 2023
25ee62e
Address reviewer comment
tehrengruber Jun 23, 2023
40235e3
Extent cse test case
tehrengruber Jun 30, 2023
9a692ae
Fix CollectSubExpression ordering and add doctests
tehrengruber Jul 26, 2023
3be5761
Enhance doctests
tehrengruber Jul 26, 2023
74b8c62
Merge refactor_cse_pass
tehrengruber Jul 26, 2023
887cdec
Merge origin/main
tehrengruber Jul 27, 2023
52e0736
Fix typo
tehrengruber Jul 27, 2023
80430e4
Merge origin/main
tehrengruber Jul 27, 2023
ce0e031
Improve ITIR type inference
tehrengruber Jul 27, 2023
7324dd8
Merge origin_tehrengruber/itir_type_inference_power_fix
tehrengruber Jul 27, 2023
1e3478a
Merge origin/main
tehrengruber Jul 27, 2023
4f81a92
Merge remote-tracking branch 'origin/main' into refactor_cse_pass
tehrengruber Jul 27, 2023
d79e98b
Merge remote-tracking branch 'origin/main' into itir_type_inference_p…
tehrengruber Jul 27, 2023
5122bc6
Fix gtfn_ir
tehrengruber Jul 27, 2023
b77b3b7
Improve ITIR CountSymbolRefs pass
tehrengruber Jul 27, 2023
38fc31a
Fix test
tehrengruber Jul 27, 2023
add57d7
Extend ITIR TraceShifts to support tuple_get, make_tuple, if on itera…
tehrengruber Jul 27, 2023
0258414
Fix comment
tehrengruber Jul 27, 2023
8693032
Small fix
tehrengruber Jul 27, 2023
18cfb44
Merge branch 'itir_type_inference_power_fix' into itir_temporaries
tehrengruber Jul 27, 2023
e81908a
Merge origin_tehrengruber/improve_countsymbolrefs_pass
tehrengruber Jul 27, 2023
e053261
Merge origin_tehrengruber/improve_trace_shifts2
tehrengruber Jul 27, 2023
b76cdba
Address reviewer comments
tehrengruber Aug 2, 2023
5889f77
Address reviewer comments
tehrengruber Aug 2, 2023
cf82884
Address reviewer comments
tehrengruber Aug 7, 2023
dff0a06
Address reviewer comments
tehrengruber Aug 7, 2023
f4477b5
Address reviewer comments
tehrengruber Aug 10, 2023
f6ff141
Address reviewer comments
tehrengruber Aug 10, 2023
020ff8d
Fix format
tehrengruber Aug 10, 2023
48be47e
Merge branch 'improve_trace_shifts2' into itir_temporaries
tehrengruber Aug 11, 2023
6e0adde
Merge branch 'improve_countsymbolrefs_pass' into itir_temporaries
tehrengruber Aug 11, 2023
b7271eb
Merge branch 'itir_type_inference_power_fix' into itir_temporaries
tehrengruber Aug 11, 2023
b6bde77
Merge branch 'refactor_cse_pass' into itir_temporaries
tehrengruber Aug 11, 2023
72bc7aa
Merge origin/main
tehrengruber Aug 11, 2023
6708f4c
Fix broken tests
tehrengruber Aug 11, 2023
3dbd6f4
Extend trace shifts
tehrengruber Aug 15, 2023
7791134
Fix naming
tehrengruber Aug 15, 2023
21270ce
Small cleanup
tehrengruber Aug 15, 2023
4a7eda8
Merge origin/main
tehrengruber Aug 15, 2023
d0faed9
Fix lambda
tehrengruber Aug 15, 2023
799fc03
Address reviewer comments
tehrengruber Aug 21, 2023
39909a7
Fix typing
tehrengruber Aug 22, 2023
a2ba2ce
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber Aug 23, 2023
85a0acf
Fix typo
tehrengruber Aug 23, 2023
ffae240
Fix tests
tehrengruber Aug 23, 2023
4878e31
Fix tests and remove CollectShifts pass
tehrengruber Aug 24, 2023
4f87b07
Fix format
tehrengruber Aug 24, 2023
1d1d207
Merge origin/main
tehrengruber Aug 24, 2023
c15a951
Fix typo
tehrengruber Aug 24, 2023
5b822a8
Fix removed wrong file
tehrengruber Aug 24, 2023
46fdd84
Add comment
tehrengruber Aug 24, 2023
0df33ec
Merge remote-tracking branch 'origin/main' into itir_temporaries
tehrengruber Aug 24, 2023
809606b
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Aug 24, 2023
fe639f0
Fix mypy
tehrengruber Aug 24, 2023
33d41b7
Merge origin/main
tehrengruber Aug 24, 2023
afdabbd
Merge branch 'small_performance_enhancements' into improve_trace_shifts3
tehrengruber Aug 24, 2023
350d0ac
Address reviewer comments
tehrengruber Aug 25, 2023
2f5a7b3
Address reviewer comments
tehrengruber Aug 25, 2023
ffbbb44
Address reviewer comments
tehrengruber Aug 29, 2023
43eecfb
Address reviewer comments
tehrengruber Aug 29, 2023
58a5cd8
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Aug 29, 2023
10af7e6
Merge origin/main
tehrengruber Sep 12, 2023
b0d35f7
Address reviewer comments
tehrengruber Sep 12, 2023
2da6a15
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Sep 12, 2023
68bf554
Fix broken test
tehrengruber Sep 12, 2023
8d6e036
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Sep 12, 2023
7a789b8
Add temporary extraction heuristics
tehrengruber Sep 12, 2023
ee6f5ae
Merge origin/main
tehrengruber Sep 15, 2023
b646bcb
Address reviewer comments
tehrengruber Sep 15, 2023
8291517
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Sep 15, 2023
fde21ae
Fix format
tehrengruber Sep 15, 2023
4d1bb28
Merge branch 'itir_temporaries' into improve_trace_shifts3
tehrengruber Sep 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -321,10 +321,6 @@ module = 'gt4py.next.type_system.type_translation'
ignore_errors = true
module = 'gt4py.next.iterator.runtime'

[[tool.mypy.overrides]]
ignore_errors = true
module = 'gt4py.next.iterator.transforms.global_tmps'

# -- pytest --
[tool.pytest]

Expand Down
13 changes: 11 additions & 2 deletions src/gt4py/next/ffront/decorator.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

from gt4py._core import definitions as core_defs
from gt4py.eve.extended_typing import Any, Optional
from gt4py.eve.utils import UIDGenerator
from gt4py.eve.utils import UIDGenerator, content_hash
from gt4py.next.common import Dimension, DimensionKind, GridType
from gt4py.next.ffront import (
dialect_ast_enums,
Expand Down Expand Up @@ -539,6 +539,7 @@ class FieldOperator(GTCallable, Generic[OperatorNodeT]):
definition: Optional[types.FunctionType] = None
backend: Optional[ppi.ProgramExecutor] = None
grid_type: Optional[GridType] = None
_program_cache: dict = dataclasses.field(default_factory=dict)

@classmethod
def from_function(
Expand Down Expand Up @@ -607,6 +608,13 @@ def as_program(
# of arg and kwarg types
# TODO(tehrengruber): check foast operator has no out argument that clashes
# with the out argument of the program we generate here.
hash_ = content_hash(
(tuple(arg_types), tuple((name, arg) for name, arg in kwarg_types.items()))
)
try:
return self._program_cache[hash_]
except KeyError:
pass

loc = self.foast_node.location
param_sym_uids = UIDGenerator() # use a new UID generator to allow caching
Expand Down Expand Up @@ -660,12 +668,13 @@ def as_program(
untyped_past_node = ProgramClosureVarTypeDeduction.apply(untyped_past_node, closure_vars)
past_node = ProgramTypeDeduction.apply(untyped_past_node)

return Program(
self._program_cache[hash_] = Program(
past_node=past_node,
closure_vars=closure_vars,
backend=self.backend,
grid_type=self.grid_type,
)
return self._program_cache[hash_]

def __call__(
self,
Expand Down
1 change: 1 addition & 0 deletions src/gt4py/next/iterator/pretty_printer.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

Inspired by P. Yelland, “A New Approach to Optimal Code Formatting”, 2015
"""
# TODO(tehrengruber): add support for printing the types of itir.Sym, itir.Literal nodes
from __future__ import annotations

from collections.abc import Iterable, Sequence
Expand Down
104 changes: 0 additions & 104 deletions src/gt4py/next/iterator/transforms/collect_shifts.py

This file was deleted.

26 changes: 26 additions & 0 deletions src/gt4py/next/iterator/transforms/common_pattern_matcher.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# GT4Py - GridTools Framework
#
# Copyright (c) 2014-2023, ETH Zurich
# All rights reserved.
#
# This file is part of the GT4Py project and the GridTools framework.
# GT4Py is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or any later
# version. See the LICENSE.txt file at the top-level directory of this
# distribution for a copy of the license or check <https://www.gnu.org/licenses/>.
#
# SPDX-License-Identifier: GPL-3.0-or-later
from typing import TypeGuard

from gt4py.next.iterator import ir as itir


def is_applied_lift(arg: itir.Node) -> TypeGuard[itir.FunCall]:
"""Match expressions of the form `lift(λ(...) → ...)(...)`."""
return (
isinstance(arg, itir.FunCall)
and isinstance(arg.fun, itir.FunCall)
and isinstance(arg.fun.fun, itir.SymRef)
and arg.fun.fun.id == "lift"
)
Loading