Conversation
awkward-pandas is deprecated. Using PyArrow instead as described in https://awkward-array.org/doc/main/user-guide/how-to-convert-pandas.html . This does change the output data type slightly, see tests affected by this commit.
Author
|
Note: The predecessor of awkward-pandas is akimbo (https://github.com/intake/awkward-pandas even forwards to https://github.com/intake/akimbo). That is a library with the feature that as soon as you write |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
awkward-pandas is deprecated. This PR solves that by using PyArrow instead as described in https://awkward-array.org/doc/main/user-guide/how-to-convert-pandas.html . This does change the output data type slightly, see tests affected by this commit.
Some arrays in the test are no longer plain nested Python arrays but Python arrays of NumPy arrays, using
awkward.array_equalto compare them across types.I tried to reflect the changed behavior in the documentation. This does change the dependency on awkward-pandas to a dependency on PyArrow, which I however believe to be justified since the above linked Pandas guide itself suggests doing so.
Fixes #1706
[edit] Side note: I thought about whether to try and replicate the old behavior of awkward-pandas, but I believe it should be better to use the documented approaches. It is worth noting that having NumPy arrays instead of plain Python arrays or no longer having the
.akaccessor but instead a different code to get Awkward arrays does affect the usage of this library, so this commit could also be taggedBREAKING CHANGE:. Btw. Pandas nowadays also has a native ArrowDtype https://pandas.pydata.org/docs/reference/api/pandas.ArrowDtype.html, but that is still written as experimental. But going from the Awkward dtype of awkward-pandas to the ArrowDtype provided by Pandas would rely on experimental features and I do not know if one would gain much.