Skip to content

feat: support new metrics firehose api with get_usage() #404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jul 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
99e5a10
initial implementation of hits endpoint
toph-allen Apr 29, 2025
1e6a816
use client method
toph-allen Apr 30, 2025
a06bb56
Merge branch 'main' into toph/metrics-firehose
toph-allen May 1, 2025
437803b
complete get_usage functionality
toph-allen May 2, 2025
19e45cd
changes to allow time zone propagation
toph-allen May 2, 2025
66e61ef
add tests, fix bugs
toph-allen May 2, 2025
24f9dc3
fix doc problem
toph-allen May 2, 2025
6442b4d
make lintr happier
toph-allen May 2, 2025
919bb0a
remove cyclocomp linter
toph-allen May 2, 2025
7b05c73
respond to comments
toph-allen Jul 15, 2025
01a6ccd
edit comment
toph-allen Jul 15, 2025
567fea3
Apply suggestion from @nealrichardson
toph-allen Jul 17, 2025
f68f753
Apply suggestion from @nealrichardson
toph-allen Jul 17, 2025
26980e3
Apply suggestion from @nealrichardson
toph-allen Jul 17, 2025
491f7df
Apply suggestion from @nealrichardson
toph-allen Jul 17, 2025
b474ed9
update documentation
toph-allen Jul 17, 2025
d2ae1a0
Merge branch 'main' into toph/metrics-firehose
toph-allen Jul 17, 2025
30ea333
Do not parse usage immediately; provide as.data.frame method
toph-allen Jul 18, 2025
cf48a08
update _pkgdown.yml
toph-allen Jul 18, 2025
442ac62
fix typo
toph-allen Jul 21, 2025
b19f147
fix tests
toph-allen Jul 21, 2025
9d6cef4
fix for CI warning
toph-allen Jul 21, 2025
68e4586
fix as.data.frame method for S3 generic conformance
toph-allen Jul 21, 2025
fa24199
update documentation to match
toph-allen Jul 21, 2025
b709457
fix lint
toph-allen Jul 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .lintr
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
linters: linters_with_defaults(
line_length_linter = line_length_linter(120L),
object_name_linter = object_name_linter(styles = c("snake_case", "symbols", "CamelCase")),
cyclocomp_linter = cyclocomp_linter(30L),
cyclocomp_linter = NULL, # Issues with R6 classes.
object_length_linter(32L),
indentation_linter = indentation_linter(hanging_indent_style = "tidy"),
return_linter = NULL
Expand Down
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Suggests:
rsconnect,
spelling,
testthat,
tidyr,
webshot2,
withr
VignetteBuilder:
Expand Down
4 changes: 4 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ S3method("[",connect_tag_tree)
S3method("[[",connect_tag_tree)
S3method(api_build,op_base_connect)
S3method(api_build,op_head)
S3method(as.data.frame,connect_list_hits)
S3method(as.data.frame,tbl_connect)
S3method(as_tibble,connect_list_hits)
S3method(connect_vars,op_base)
S3method(connect_vars,op_single)
S3method(connect_vars,tbl_connect)
Expand Down Expand Up @@ -97,6 +99,7 @@ export(get_tag_data)
export(get_tags)
export(get_thumbnail)
export(get_timezones)
export(get_usage)
export(get_usage_shiny)
export(get_usage_static)
export(get_user_permission)
Expand Down Expand Up @@ -159,6 +162,7 @@ importFrom(rlang,"%||%")
importFrom(rlang,":=")
importFrom(rlang,arg_match)
importFrom(rlang,is_string)
importFrom(tibble,as_tibble)
importFrom(utils,browseURL)
importFrom(utils,capture.output)
importFrom(utils,compareVersion)
Expand Down
5 changes: 5 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# connectapi (development version)

## New features

- New `get_usage()` function returns content usage data from Connect's `GET
v1/instrumentation/content/hits` endpoint on Connect v2025.04.0 and higher.
(#390)

## Enhancements and fixes

Expand Down
4 changes: 3 additions & 1 deletion R/connect.R
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,9 @@ Connect <- R6::R6Class(
docs = function(docs = "api", browse = TRUE) {
stopifnot(docs %in% c("admin", "user", "api"))
url <- paste0(self$server, "/__docs__/", docs)
if (browse) utils::browseURL(url)
if (browse) {
utils::browseURL(url)
}
url
},

Expand Down
151 changes: 151 additions & 0 deletions R/get.R
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,157 @@ get_usage_static <- function(
return(out)
}

#' Get usage information for deployed content
#'
#' @description
#' Retrieve content hits for all available content on the server. Available
#' content depends on the user whose API key is in use. Administrator accounts
#' will receive data for all content on the server. Publishers will receive data
#' for all content they own or collaborate on.
#'
#' If no date-times are provided, all usage data will be returned.
#'
#' @param client A `Connect` R6 client object.
#' @param from Optional date-time (`POSIXct` or `POSIXlt`). Only
#' records after this time are returned. If not provided, records
#' are returned back to the first record available.
#' @param to Optional date-time (`POSIXct` or `POSIXlt`). Only records
#' before this time are returned. If not provided, all records up to
#' the most recent are returned.
#'
#' @return A list of usage records. Each record is a list with all elements
#' as character strings unless otherwise specified.
#'
#' * `id`: An integer identifier for the hit.
#' * `user_guid`: The user GUID if the visitor is logged-in, `NULL` for
#' anonymous hits.
#' * `content_guid`: The GUID of the visited content.
#' * `timestamp`: The time of the hit in RFC3339 format.
#' * `data`: A nested list with optional fields:
#' * `path`: The request path (if recorded).
#' * `user_agent`: The user agent string (if available).
#'
#' Use [as.data.frame()] or [tibble::as_tibble()] to convert to a flat
#' table with parsed types. In the resulting data frame:
#'
#' * `timestamp` is parsed to `POSIXct`.
#' * `path` and `user_agent` are extracted from the nested `data` field.
#'
#' By default, [as.data.frame()] attempts to extract the nested fields using
#' the \pkg{tidyr} package. If \pkg{tidyr} is not available, or if you want to
#' skip unnesting, call `as.data.frame(x, unnest = FALSE)` to leave `data` as
#' a list-column.
#'
#' @details
#'
#' The data returned by `get_usage()` includes all content types. For Shiny
#' content, the `timestamp` indicates the *start* of the Shiny session.
#' Additional fields for Shiny and non-Shiny are available respectively from
#' [get_usage_shiny()] and [get_usage_static()]. `get_usage_shiny()` includes a
#' field for the session end time; `get_usage_static()` includes variant,
#' rendering, and bundle identifiers for the visited content.
#'
#' When possible, however, we recommend using `get_usage()` over
#' `get_usage_static()` or `get_usage_shiny()`, as it is faster and more efficient.
#'
#' @seealso [as.data.frame.connect_list_hits()], [as_tibble.connect_list_hits()]
#'
#' @examples
#' \dontrun{
#' client <- connect()
#'
#' # Fetch the last 2 days of hits
#' usage <- get_usage(client, from = Sys.Date() - 2, to = Sys.Date())
#'
#' # Fetch usage after a specified date and convert to a data frame.
#' usage <- get_usage(
#' client,
#' from = as.POSIXct("2025-05-02 12:40:00", tz = "UTC")
#' )
#'
#' # Fetch all usage
#' usage <- get_usage(client)
#'
#' # Convert to tibble or data frame
#' usage_df <- tibble::as_tibble(usage)
#'
#' # Skip unnesting if tidyr is not installed
#' usage_df <- as.data.frame(usage, unnest = FALSE)
#' }
#'
#' @export
get_usage <- function(client, from = NULL, to = NULL) {
error_if_less_than(client$version, "2025.04.0")

usage <- client$GET(
v1_url("instrumentation", "content", "hits"),
query = list(
from = make_timestamp(from),
to = make_timestamp(to)
)
)

class(usage) <- c("connect_list_hits", class(usage))
usage
}

#' Convert usage data to a data frame
#'
#' @description
#' Converts an object returned by [get_usage()] into a data frame with parsed
#' column types. By default, extracts `path` and `user_agent` from the `data`
#' field, if available.
#'
#' @param x A `connect_list_hits` object (from [get_usage()]).
#' @param row.names Passed to [base::as.data.frame()].
#' @param optional Passed to [base::as.data.frame()].
#' @param ... Passed to [base::as.data.frame()].
#' @param unnest Logical; if `TRUE` (default), extracts nested fields using
#' \pkg{tidyr}. Set to `FALSE` to skip unnesting.
#'
#' @return A `data.frame` with one row per usage record.
#' @export
#' @method as.data.frame connect_list_hits
as.data.frame.connect_list_hits <- function(
x,
row.names = NULL, # nolint
optional = FALSE,
...,
unnest = TRUE
) {
usage_df <- parse_connectapi_typed(x, connectapi_ptypes$usage)
if (unnest) {
if (!requireNamespace("tidyr", quietly = TRUE)) {
stop(
"`unnest = TRUE` requires tidyr. Install tidyr or set `unnest = FALSE`.",
call. = FALSE
)
}
usage_df <- tidyr::unnest_wider(
usage_df,
"data",
ptype = list(path = character(0), user_agent = character(0))
)
}
as.data.frame(usage_df, row.names = row.names, optional = optional, ...)
}

#' Convert usage data to a tibble
#'
#' @description
#' Converts an object returned by [get_usage()] to a tibble via
#' [as.data.frame.connect_list_hits()].
#'
#' @param x A `connect_list_hits` object.
#' @param ... Passed to [as.data.frame()].
#'
#' @return A tibble with one row per usage record.
#' @export
#' @importFrom tibble as_tibble
#' @method as_tibble connect_list_hits
as_tibble.connect_list_hits <- function(x, ...) {
tibble::as_tibble(as.data.frame(x, ...))
}

#' Get Audit Logs from Posit Connect Server
#'
Expand Down
5 changes: 5 additions & 0 deletions R/parse.R
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,19 @@ ensure_column <- function(data, default, name) {
# manual fix because vctrs::vec_cast cannot cast double -> datetime or char -> datetime
col <- coerce_datetime(col, default, name = name)
}

if (inherits(default, "fs_bytes") && !inherits(col, "fs_bytes")) {
col <- coerce_fsbytes(col, default)
}

if (inherits(default, "integer64") && !inherits(col, "integer64")) {
col <- bit64::as.integer64(col)
}

if (inherits(default, "list") && !inherits(col, "list")) {
col <- list(col)
}

col <- vctrs::vec_cast(col, default, x_arg = name)
}
data[[name]] <- col
Expand Down Expand Up @@ -101,6 +105,7 @@ parse_connectapi <- function(data) {
))
}


coerce_fsbytes <- function(x, to, ...) {
if (is.numeric(x)) {
fs::as_fs_bytes(x)
Expand Down
9 changes: 8 additions & 1 deletion R/ptype.R
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
NA_datetime_ <- # nolint: object_name_linter
vctrs::new_datetime(NA_real_, tzone = "UTC")
vctrs::new_datetime(NA_real_, tzone = Sys.timezone())
NA_list_ <- # nolint: object_name_linter
list(list())

Expand Down Expand Up @@ -38,6 +38,13 @@ connectapi_ptypes <- list(
"bundle_id" = NA_character_,
"data_version" = NA_integer_
),
usage = tibble::tibble(
"id" = NA_integer_,
"user_guid" = NA_character_,
"content_guid" = NA_character_,
"timestamp" = NA_datetime_,
"data" = NA_list_
),
content = tibble::tibble(
"guid" = NA_character_,
"name" = NA_character_,
Expand Down
2 changes: 2 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ reference:
Helpers to "get" data out of Connect
contents:
- starts_with("get")
- as.data.frame.connect_list_hits
- as_tibble.connect_list_hits

- title: "Other"
desc: >
Expand Down
28 changes: 28 additions & 0 deletions man/as.data.frame.connect_list_hits.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions man/as_tibble.connect_list_hits.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading