Polars has been on an extremely fast development path for the past couple of years. The development team impressed us with faster speed and performance. Now they’ve officially released polars version 1.0.0! Here are some of the new features I’m most excited about.
- Addition of a JSON normalize function
- Added SQL support for INTERSECT and EXCEPT
- Support for order by in window functions
Addition of json_normalize
When I first started learning Polars, I was excited about how fast it was. That said, there were some functionalities that I loved in Pandas that just weren’t available in Polars. json_normalize was one of those. This function is super helpful in flattening complex JSON structures and is extremely valuable when working with pulling data from APIs.
With the latest release of Polars, we now have a basic json_normalize function! I’m so excited by this news that I’m currently working on adding coverage of this function in my Data Analysis with Polars in Python course. If you want to get a general idea of what this will look like you can check out my video on how it works in Pandas: https://www.youtube.com/watch?v=5JyLCsQ2JNc
One final note here is that the new json_normalize function does not have as much functionality as the Pandas function. With future releases, its likely that more will be added to it to make it comparable. Nevertheless, I’m thrilled that we now have a simple version of it!
SQL Support for INTERSECT and EXCEPT
My native SQL language is Microsoft’s T-SQL. That’s why I was stoked to hear that Polars’ SQL capabilities would start including INTERSECT and EXCEPT. These are powerful keywords that allow you to compare two queries and return a result based on that comparison.
INTERSECT will return results that are the same in both tables. EXCEPT will take everything from the first table that doesn’t exist in the second. It’s a powerful functionality that simplifies how you get to these results. And now we have them within Polars!
Support for Order By in Window Functions
Window Functions are a cornerstone in data analysis. I utilize them frequently in my work as a Business Intelligence Professional. In SQL, window functions often give you the ability to PARTITION BY and ORDER BY. However, when first learning how to do these in Polars, I was surprised that there was no ability to add an order by condition.
But with the recent release, Polars added “order_by” as a parameter within the over function. This really brings the library inline with how most people understand window functions. It closes what was an apparent gap in windowing functionality and I’m excited to cover more of this in my courses!
Conclusion
These are just a few of the things that I’m most excited about with the release of polars version 1.0.0. There are so many more changes and improvements to be found. To view the official release, check it out here! If your interested in learning Polars, check out Polars Code Academy on YouTube!