{"id":522,"date":"2024-12-15T07:19:12","date_gmt":"2024-12-15T07:19:12","guid":{"rendered":"https:\/\/datadandies.nl\/?p=522"},"modified":"2024-12-20T09:19:54","modified_gmt":"2024-12-20T09:19:54","slug":"lazy-evaluation-in-python-and-in-dax-variables","status":"publish","type":"post","link":"https:\/\/datadandies.nl\/index.php\/2024\/12\/15\/lazy-evaluation-in-python-and-in-dax-variables\/","title":{"rendered":"Lazy evaluation in Python and in Power BI DAX variables"},"content":{"rendered":"\n

I remember reading about lazy evaluation for DAX variables the first time several years back in the book \u201cThe Definitive Guide to DAX\u201d from Marco Russo & Alberto Ferrari. I also remember being very confused about the concept back then. It took some time and research in order to understand the concept a little better.<\/p>\n\n\n\n

If you find yourself struggling with understanding lazy evaluation as well, I invite you to read this post.<\/p>\n\n\n\n

In order to better visualize certain aspects that will assist in explaining lazy evaluation, I have chosen to use Python instead of DAX to explain the concept. At the end I will briefly get back to lazy evaluation for DAX variables.<\/p>\n\n\n\n

Let\u2019s start with why lazy evaluation is interesting. Imagine having a huge dataset and limited memory (which is actually always the case).<\/p>\n\n\n\n

You want to save values from this dataset that match a certain condition. However, you run into memory issues: you do not have enough memory in the tool you have available for computing the results.<\/p>\n\n\n\n

Instead of storing the results in a list, which is loaded completely into memory, you could opt for a generator object instead. A generator object only stores the metadata of the object which leads to a much smaller size. Below is an example of this:<\/p>\n\n\n\n

<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
import<\/span> sys<\/span><\/span>\n<\/span>\n# Create a range object<\/span><\/span>\nr <\/span>=<\/span> <\/span>range<\/span>(<\/span>1_000_000<\/span>)<\/span><\/span>\nprint<\/span>(<\/span>"<\/span>Size of range object:<\/span>"<\/span>,<\/span> sys<\/span>.<\/span>getsizeof<\/span>(<\/span>r<\/span>))<\/span>  <\/span># Small size, e.g., 48 bytes<\/span><\/span>\n<\/span>\n# Create a list from the range<\/span><\/span>\nl <\/span>=<\/span> <\/span>list<\/span>(<\/span>r<\/span>)<\/span><\/span>\nprint<\/span>(<\/span>"<\/span>Size of list object:<\/span>"<\/span>,<\/span> sys<\/span>.<\/span>getsizeof<\/span>(<\/span>l<\/span>))<\/span>  <\/span># Much larger size, 8.000.056 bytes<\/span><\/span>\n<\/span>\n# Create a generator object from the range<\/span><\/span>\ng <\/span>=<\/span> <\/span>(<\/span>a <\/span>for<\/span> a <\/span>in<\/span> r<\/span>)<\/span><\/span>\nprint<\/span>(<\/span>"<\/span>Size of list object:<\/span>"<\/span>,<\/span> sys<\/span>.<\/span>getsizeof<\/span>(<\/span>g<\/span>))<\/span>  <\/span># Small size, 192 bytes<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
The snippet of code below gives a more elaborate explanation of what exactly lazy evaluation does.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
import<\/span> re<\/span><\/span>\nimport<\/span> sys<\/span><\/span>\n<\/span>\n# So imagine you have this huge dataset containing sales and you only want to have sales with apples:<\/span><\/span>\nhuge_dataset <\/span>=<\/span> <\/span>[{<\/span>1<\/span>:<\/span> <\/span>'<\/span>10 apples<\/span>'<\/span>,<\/span> <\/span>2<\/span>:<\/span> <\/span>'<\/span>3 pears<\/span>'<\/span>,<\/span> <\/span>3<\/span>:<\/span> <\/span>'<\/span>30 apples<\/span>'<\/span>,<\/span> <\/span>4<\/span>:<\/span> <\/span>'<\/span>ad infinitum<\/span>'<\/span>}]<\/span><\/span>\n<\/span>\n# Create regex object in order to extract all values with 'apples' from the dictionary in the list below<\/span><\/span>\nregex_apples <\/span>=<\/span> re<\/span>.<\/span>compile<\/span>(<\/span>'<\/span>.*apples<\/span>'<\/span>,<\/span> re<\/span>.<\/span>IGNORECASE<\/span>)<\/span><\/span>\n<\/span>\n# Using list comprehension, you can filter out the apples and store them in a list:<\/span><\/span>\n# Variable apples_sales stores all the filtered results in memory<\/span><\/span>\napples_sales <\/span>=<\/span> <\/span>[<\/span>sale <\/span>for<\/span> sale <\/span>in<\/span> huge_dataset<\/span>[<\/span>0<\/span>].<\/span>values<\/span>()<\/span> <\/span>if<\/span> re<\/span>.<\/span>match<\/span>(<\/span>regex_apples<\/span>,<\/span> sale<\/span>)]<\/span> <\/span># re.match returns a Match object when it finds something, and returns None when nothing is found. When a match is found, a Match object will be returned and this object will be interpreted as 'truthy' in the if-statement. When there is no match, or with other words, when no Match object is returned (resulting in "None"), the condition wil be 'falsy' in the if-statement. <\/span><\/span>\n<\/span>\n# This would however, load the entire huge_dataset in memory. If you do not want abuse your memory in such a manner, you could opt for a Generator object instead. <\/span><\/span>\n# Generator objects use lazy evaluation which means that they do not load the entire content of the object into memory. <\/span><\/span>\n# Variable apples_sales_gen stores a Generator object which only contains the "metadata" in memory, meaning the condition (if-statement) and the iterable object<\/span><\/span>\napples_sales_gen <\/span>=<\/span> <\/span>(<\/span>sale <\/span>for<\/span> sale <\/span>in<\/span> huge_dataset<\/span>[<\/span>0<\/span>].<\/span>values<\/span>()<\/span> <\/span>if<\/span> re<\/span>.<\/span>match<\/span>(<\/span>regex_apples<\/span>,<\/span> sale<\/span>))<\/span><\/span>\n<\/span>\n# Processing only starts when you explicitly iterate over the Generator object<\/span><\/span>\nprint<\/span>(<\/span>next<\/span>(<\/span>apples_sales_gen<\/span>))<\/span>   <\/span># Only the first item of the Generator object is printed: 10 apples<\/span><\/span>\nprint<\/span>(<\/span>next<\/span>(<\/span>apples_sales_gen<\/span>))<\/span>   <\/span># Only the second item of the Generator object is printed: 30 apples<\/span><\/span>\n<\/span>\n# The list stores all the filtered results, so this will show the entire list:<\/span><\/span>\nprint<\/span>(<\/span>apples_sales<\/span>)<\/span>             <\/span># All items in the list apples_sales will be printed: ['10 apples', '30 apples']<\/span><\/span>\n<\/span>\n# You can see the generator's object reference (it has not been consumed yet):<\/span><\/span>\nprint<\/span>(<\/span>apples_sales_gen<\/span>)<\/span>         <\/span># Outputs something like: <generator object <genexpr> at 0x0000020D2DE935A0><\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
As for how to apply this knowledge to lazy evaluation and DAX variables: variables that are declared in DAX are only evaluated when used in the RETURN statement. <\/p>\n","protected":false},"excerpt":{"rendered":"
I remember reading about lazy evaluation for DAX variables the first time several years back in the book \u201cThe Definitive Guide to DAX\u201d from Marco Russo & Alberto Ferrari. I also remember being very confused about the concept back then. It took some time and research in order to understand the concept a little better.…<\/p>\n
Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[6,37],"class_list":["post-522","post","type-post","status-publish","format-standard","hentry","category-blog","tag-dax","tag-python"],"_links":{"self":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/522","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/comments?post=522"}],"version-history":[{"count":4,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/522\/revisions"}],"predecessor-version":[{"id":529,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/522\/revisions\/529"}],"wp:attachment":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/media?parent=522"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/categories?post=522"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/tags?post=522"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}