Skip to content

Add Getting Data section with Ibis #104

@ncclementi

Description

@ncclementi

Hello, I work on Ibis and with the default backend (duckdb) we can query overture maps very easily.

Would you be open to a PR contribution to your docs, to add this to the "Getting Data Section"? I can work on multiple examples, but to start it would be something like this.

# install instructions
$ pip install 'ibis-framework[duckdb,geospatial]'
import ibis
from ibis import _

# by default it gets duckdb backend
con = ibis.get_backend()

# example of how to get infrastructure data of `subtype="power"` for a `bbox` around Washington DC 
# and selecting only a few columns

url = "s3://overturemaps-us-west-2/release/2024-07-22.0/theme=base/type=infrastructure/*"
t = con.read_parquet(url, table_name="infra-power-dc")

# so far no execution 
expr = t.filter(_.bbox.xmin > -77.119795,
                _.bbox.xmax < -76.909366,
                _.bbox.ymin > 38.791631,
                _.bbox.ymax < 38.995968,
                _.subtype=="power"
                ).select(["names",
                          "geometry", 
                          "sources", 
                          "class"])

con.to_parquet(expr, "infra-power-dc-ibis.parquet")

t_power_dc = con.read_parquet("infra-power-dc-ibis.parquet") 

# turn interactive mode to preview data
ibis.options.interactive = True   

# inspect data where `class=="power_line", and select only few columns

t_infra.filter(t_infra["class"]=="power_line")["names", "geometry","class"]

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ namesgeometryclass      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ struct<primary: string, common: map<string, string>, rules: array<struct<varian… │ geospatial:geometrystring     │
├──────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┼────────────┤
│ NULL<LINESTRING (-77.113 38.805, -77.113 38.805, -77.112 38.806, -77.111 38.806,...>power_line │
│ NULL<LINESTRING (-77.11 38.818, -77.11 38.818, -77.111 38.818, -77.112 38.819, -...>power_line │
│ NULL<LINESTRING (-77.098 38.848, -77.097 38.848, -77.097 38.848, -77.097 38.848,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.097 38.848, -77.096 38.848, -77.095 38.847, -77.094 38.847,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.059 38.796, -77.061 38.796)>power_line │
│ NULL<LINESTRING (-77.06 38.799, -77.06 38.799, -77.061 38.797, -77.061 38.796)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.057 38.795, -77.059 38.796)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.055 38.797, -77.055 38.796, -77.055 38.796, -77.055 38.795,...>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.055 38.797, -77.055 38.796, -77.055 38.795, -77.057 38.795)>power_line │
│ {'primary': '230kV transmission line', 'common': None, ... +1}                   │ <LINESTRING (-77.087 38.844, -77.086 38.844, -77.085 38.844, -77.084 38.844,...>power_line │
│ …                                                                                │ …                                                                                │ …          │
└──────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┴────────────┘                     

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions