Roadmap for this repository

[bitsondatadev]

This repository was initially a bed where I placed my Trino experiments on the Trino Community Broadcast as I looked into the intersection of core Trino query engine abstractions, overlapping tech in Trino's ecosystem, and new features in Trino.

That said, I am reaching a point where I'd like narrow the focus on maintaining common areas of the code that seem to be the most valuable tutorials, and have a community section where I will keep older examples available for reference and depend on community contributions to keep them up-to-date.

I see the value and do have some time to update this repository, but there's honestly so many examples that require some specific tuning and testing every new version that to me it's better to prioritize few over many, while leaving space to add one-off examples that may or may not be maintained to keep the guarantees clear here.

My initial thoughts are to maintain the following:

• [❗HIGH] Trino + Iceberg + REST/Polaris + MinIO
  • See this issue: Failing trino-minio setup #46
  • Tutorial: Run through initial setup and REST internals.
• [⚠️MED] Trino + Iceberg/Delta + REST/Unity + MinIO
  • Tutorial: Using Unity to collaborate bettween Iceberg and Delta formats.
• Add SeaweedFS Move hdfs to community and make storage and catalogs modular #55 (partial)
• [❗HIGH] Trino + Iceberg/Hive + Hive Metastore + MinIO Move hdfs to community and make storage and catalogs modular #55
  • Refactored: See this issues: trino-minio_hive-metastore_1 container exited #48, Test Hive Metastore 4.x and see if we can archive my HMS image #43
  • Tutorial: Run through initial setup through Hive.
  • Tutorial: Cover Hive Metastore internals.
  • Tutorial: Run migration to Iceberg.
• [❗HIGH] Trino + Postgres
  • Tutorial: Run through a tutorial on why add a query engine to a database?
• [ℹ️LOW] Trino + Trino Gateway + Kubernetes
  • Tutorial: How to handle multiple Trinos for multiple areas of focus
  • Trino Clusters:
    1. (Small) Trino + Postgres
    2. (Med) Trino + Iceberg
    3. (Big) Trino + Iceberg (Fault Tolerant)
• [❗HIGH] Testing framework and some basic tests.
• [[⚠️MED] Automate releases triggered on new Trino versions (Run tests with new version if.
  • Run tests with the new version, if it passes, auto-update the scripts, otherwise keep current version and add Issue.
• [❗HIGH] Migrate other tutorials to a community folder.
  Issue: Migrate other tutorials to a community folder. #53
• [ℹ️ LOW] Change this repository to an awesome list

[metadaddy]

I'm happy to (continue to) maintain trino-getting-started/hive/trino-b2!

I also just noticed trino-getting-started/iceberg/trino-iceberg-minio - I'll see if I can create a trino-iceberg-b2 variant. Would that live in the new community folder?

[bitsondatadev]

I'm happy to (continue to) maintain trino-getting-started/hive/trino-b2!

I also just noticed trino-getting-started/iceberg/trino-iceberg-minio - I'll see if I can create a trino-iceberg-b2 variant. Would that live in the new community folder?

Yeah, that will be community but I almost want a way to indicate the ones that are actually getting love.

Another option would be to migrate that variant if you'd like to fork and I'll be happy to link to your repo from the readme. That way I'm not a bottleneck and it gets the same visibility.

WDYT?

[metadaddy]

It could live on https://github.com/backblaze-b2-samples, for sure. I'm thinking I could create a trino-getting-started-b2 (or something like that) repository, containing hive/trino-b2 and, at some point in the future, iceberg/trino-iceberg-v2. It seems sensible to keep the same directory structure.

As far as I remember, those samples are self-contained, right - I wouldn't need to drag anything else across?

[bitsondatadev]

It could live on https://github.com/backblaze-b2-samples, for sure. I'm thinking I could create a trino-getting-started-b2 (or something like that) repository, containing hive/trino-b2 and, at some point in the future, iceberg/trino-iceberg-v2. It seems sensible to keep the same directory structure.

As far as I remember, those samples are self-contained, right - I wouldn't need to drag anything else across?

That's correct, all self-contained and Docker reliant. Hopefully the Docker reliant part will eventually go away: trinodb/trino#23344

[metadaddy]

@bitsondatadev - here you go: https://github.com/backblaze-b2-samples/trino-getting-started-b2

I deliberately didn't make it a fork, since that's not the purpose, but I did keep all the git history.