Skip to content

Convert categorical operations from pandas to Polars#63

Open
meshackamadi wants to merge 40 commits into
aeturrell:mainfrom
datathink:migrate-categorical-to-polars
Open

Convert categorical operations from pandas to Polars#63
meshackamadi wants to merge 40 commits into
aeturrell:mainfrom
datathink:migrate-categorical-to-polars

Conversation

@meshackamadi

Copy link
Copy Markdown

Summary

This PR migrates categorical data examples from pandas to Polars and updates the accompanying explanations to better reflect Polars' API and behavior.

Changes

  • Replaced pandas categorical operations with Polars equivalents.
  • Updated examples to use pl.Enum for ordered categorical data.
  • Improved explanations of category operations in Polars.
  • Added examples demonstrating:
    • value_counts()
    • retrieving Enum categories with .dtype.categories
    • including zero-count categories using a join
    • mode() on Enum columns
  • Updated examples to follow current Polars idioms.
  • Corrected documentation to reflect Polars' stricter handling of Enum categories.

Notes

  • value_counts() only returns categories that are present in the data.
  • Added an example showing how to include unused categories by joining the Enum category list with observed counts.
  • Removed pandas-specific categorical behavior that does not have a direct Polars equivalent.

hathawayj and others added 30 commits April 14, 2026 15:26
Removed duplicate sentences and improved clarity of the introduction. Updated contributor list formatting.
WIP: Data Transformation
…pis.ipynb, updating relevant code and documentation accordingly.
…help.quarto_ipynb_1, and workflow-packages-and-environments.quarto_ipynb_1.
…onsistency

- Changed `quarto render` command in GitHub Actions from `--no-execute` to `--execute` for proper rendering.
- Enhanced readability in `data-transform.ipynb` by removing unnecessary line breaks and simplifying code expressions.
- Added missing import statement for `polars` in `data-visualise.ipynb`.
- Improved code formatting and consistency across various cells in both notebooks.
- Import `lets_plot` at the beginning for clarity.
- Adjust DataFrame column operations for better readability.
- Simplify plot creation by using parentheses for chaining.
- Remove unnecessary imports and whitespace for cleaner code.
…pis.ipynb, updating relevant code and documentation accordingly.
…help.quarto_ipynb_1, and workflow-packages-and-environments.quarto_ipynb_1.
…help.quarto_ipynb_1, and workflow-packages-and-environments.quarto_ipynb_1.
- Added newline characters to the end of several text blocks in `webscraping-and-apis.ipynb` to ensure proper formatting.
- Adjusted the structure of code snippets for better clarity and consistency.
- Removed unnecessary lines and streamlined code for better presentation.
Convert Import Section of Python4DS to Polars
…display for functions, and correct code snippets for clarity. Adjusted execution counts and fixed minor documentation errors.
…ith polars

- Updated import statements and function calls in data-import.ipynb to use polars instead of pandas.
- Adjusted text descriptions to reflect the changes from pandas to polars, including installation instructions and function usage.
- Cleaned up markdown formatting for better readability.
- Removed unnecessary sections and streamlined explanations in data-transform.ipynb.
ugohuche and others added 10 commits April 24, 2026 19:48
…ts, enhancing clarity and consistency in code presentation.
- Added execution counts to various code cells across multiple notebooks for better tracking of code execution.
- Enhanced output displays in `boolean-data.ipynb`, `categorical-data.ipynb`, `communicate-plots.ipynb`, `data-import.ipynb`, `data-tidy.ipynb`, `data-transform.ipynb`, and `data-visualise.ipynb` to provide clearer results and visualizations.
- Improved overall consistency in notebook formatting and presentation.
- Updated the command line chapter in `command-line.md` to improve formatting by replacing asterisks with underscores for emphasis.
- Modified output displays in `functions.ipynb` and `iteration.ipynb` to reflect new data and execution counts, ensuring consistency and clarity in results.
- Adjusted various execution counts across notebooks to maintain accurate tracking of code execution.
- Updated execution counts to null and cleared outputs in `boolean-data.ipynb`, `categorical-data.ipynb`, `communicate-plots.ipynb`, `data-import.ipynb`, `data-tidy.ipynb`, `data-transform.ipynb`, and other notebooks for a cleaner presentation.
- Enhanced overall consistency in notebook formatting by removing previous output displays, ensuring a more streamlined user experience.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants