Should we migrate away?

[obibring]

We love apex and want to stick with it but have been running up against issues. I've outlined them below, please let me know if you think Apex is the right solution for us at the moment or if it makes sense for us to migrate away from it.

Background
Our product is a browser extension that executes on web pages we don't control. Consequently, we log relatively large payloads to get a better idea of what was happening in the browser when the log was issued.

Problems we've faced
We initially saw exceptions thrown when our message strings exceeded the limit in structured data mode (#4 - thanks for your help with that). We created a new project in plain-text mode, and now the apex web interfaces freezes often. We often have to refresh the page to several times before a query produces data. We were also billed for $488 in charges last month for "BigQuery Analysis" (see screenshot below). This doesn't seem expected.

Conclusion
Given our use case, is it possible to run apex at low cost and with good performance?

Thanks in advance!

======================================================

[tj]

Holy that sounds far too high for BQ usage, unless you're running tons of queries against many months of data perhaps, it does depend on the usage, I spend about $30/mo for roughly 70gb/mo for comparison. How much data does your product listing show? 188TB sounds way too high — so far in my billing usage reporting I'm still the largest Apex Logs customer.

I did design Apex Logs more for the "collect huge amounts of logs, query (relatively) infrequently" use-case, but I wouldn't expect to see that kind of cost until you're up in 300gb+ and doing many queries against the entire dataset. It's partitioned by the hour, so if you query the past 6 hours or day etc it should be a small subset of the logs.

To be completely transparent though I think I'm going to be selling and/or shutting down the company next year unfortunately, so it may not make sense long-term. I should have an announcement in a month or two once I figure out the details, but I'm going to open-source as much as I can, if I can get Logs working well as OSS then it'll be free aside from the GCP bill.

In either case I'm happy to cover those surprise costs if you'd like, that doesn't sound quite right.

[obibring]

Many thanks for the straightforward feedback. I'll discuss with my team and see what they think. And much appreciated with regards to the costs but your attentive replies are more than enough!