Skip to content

Provide a more ergonomic method to extract python data #1288

@ntjohnson1

Description

@ntjohnson1

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I perform various operations on dataframes but find I have to jump through hoops if I want to extract individual items.

Describe the solution you'd like
I'm not particular on the exact api but looking for something

>>> type(df.column("uuid"))
pa.ChunkedArray # or something workable

Describe alternatives you've considered
A few candidate examples
df.select('uuid').distinct().collect()[0][0]
df.to_arrow_table()["uuid"] # just jump out of dataframe early but has penalty of collecting things I don't need

My issue with collect is that is a series of record batches which is cumbersome to manage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions