{"id":424,"date":"2024-06-21T08:30:03","date_gmt":"2024-06-21T08:30:03","guid":{"rendered":"https:\/\/datadandies.nl\/?p=424"},"modified":"2024-06-21T08:35:45","modified_gmt":"2024-06-21T08:35:45","slug":"strip-outer-array-from-a-json-in-snowflake","status":"publish","type":"post","link":"https:\/\/datadandies.nl\/index.php\/2024\/06\/21\/strip-outer-array-from-a-json-in-snowflake\/","title":{"rendered":"Strip outer array from a JSON in Snowflake"},"content":{"rendered":"\n<p>Imagine that you want to load semi-structured data in the form of JSON into Snowflake. If the JSON contains an outer array and if you did not strip it, the entire dataset would be loaded into a single row in the target table. <\/p>\n\n\n\n<p>Sometimes that is acceptable and you can process the data in a later step. Whenever the dataset results in a row that is larger than 16 mb however, you have a problem. A VARIANT (datatype that is used often when loading JSON data) can only contain maximum 16 mb per row which means the dataset would not fit in the case mentioned. <\/p>\n\n\n\n<p>STRIP_OUTER_ARRAY can help you out when loading the data in such a case. STRIP_OUTER_ARRAY will remove the outer array which will result in multiple rows instead of only one. This solves the problem encountered when trying to copy a dataset larger than 16 mb, because now the data will be divided over several rows instead of only one. <\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#2e3440ff\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"COPY INTO &lt;target table&gt;\n  FROM @~\/&lt;file&gt;.json\n  FILE_FORMAT = (TYPE = 'JSON' STRIP_OUTER_ARRAY = true);\" style=\"color:#d8dee9ff;display:none\" aria-label=\"Kopieer\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki nord\" style=\"background-color: #2e3440ff\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #81A1C1\">COPY<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #81A1C1\">INTO<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #81A1C1\">&lt;target<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #81A1C1\">table&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D8DEE9FF\">  <\/span><span style=\"color: #81A1C1\">FROM<\/span><span style=\"color: #D8DEE9FF\"> @~<\/span><span style=\"color: #81A1C1\">\/&lt;file&gt;<\/span><span style=\"color: #D8DEE9FF\">.<\/span><span style=\"color: #81A1C1\">json<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D8DEE9FF\">  <\/span><span style=\"color: #81A1C1\">FILE_FORMAT<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #81A1C1\">=<\/span><span style=\"color: #D8DEE9FF\"> (<\/span><span style=\"color: #81A1C1\">TYPE<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #81A1C1\">=<\/span><span style=\"color: #D8DEE9FF\"> <\/span><span style=\"color: #ECEFF4\">&#39;<\/span><span style=\"color: #A3BE8C\">JSON<\/span><span style=\"color: #ECEFF4\">&#39;<\/span><span style=\"color: #D8DEE9FF\"> STRIP_OUTER_ARRAY <\/span><span style=\"color: #81A1C1\">=<\/span><span style=\"color: #D8DEE9FF\"> true);<\/span><\/span><\/code><\/pre><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Imagine that you want to load semi-structured data in the form of JSON into Snowflake. If the JSON contains an outer array and if you did not strip it, the entire dataset would be loaded into a single row in the target table. Sometimes that is acceptable and you can process the data in a&hellip;<\/p>\n<p class=\"more-link\"><a href=\"https:\/\/datadandies.nl\/index.php\/2024\/06\/21\/strip-outer-array-from-a-json-in-snowflake\/\" class=\"themebutton\">Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[38,4],"class_list":["post-424","post","type-post","status-publish","format-standard","hentry","category-blog","tag-json","tag-sql"],"_links":{"self":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/424","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/comments?post=424"}],"version-history":[{"count":3,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/424\/revisions"}],"predecessor-version":[{"id":427,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/posts\/424\/revisions\/427"}],"wp:attachment":[{"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/media?parent=424"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/categories?post=424"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datadandies.nl\/index.php\/wp-json\/wp\/v2\/tags?post=424"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}