More receiving yards cases (#117)

* add a receiving yardage case

* docs: Created return tag for internal functions for CRAN

* chore: Update drive and live play columns, fix tests

Removed deprecated labels for elapsed time columns in cfbd_drives and added new columns to cfbd_live_plays documentation and tests. Updated test expectations to use expect_in instead of expect_setequal for column checks.

* feat: Enhance cfbd_play_stats_player output and docs

Added new columns to cfbd_play_stats_player output, improved sack player aggregation, handled NULL values, and updated documentation and examples to reflect changes. Also updated cfbd_live_plays documentation to include new columns for average start yard line and deserve to win metrics.

* fix: Substitute timeouts in cfbd_pbp_data when missing

Changed cfbd_pbp_data to assign 3 timeouts per half for offense and defense when timeout data is missing from the API. Updated documentation and examples to reflect this behavior.

* fix: Specify .groups argument in summarise call

Added .groups = "drop" to the dplyr::summarise call in add_play_counts to control grouping behavior and prevent potential warnings in future dplyr versions.

* docs: Update cfbd_drives return documentation

Removed the specific count of variables from the return value description in both R and Rd files to improve maintainability and accuracy as the data frame structure may change.

* fix: use `dplyr::distinct()` over `dplyr::distinct_all()` spacing and function usage in play data functions

Corrected spacing and replaced superseded `dplyr::distinct_all()` with `dplyr::distinct()`, and standardized assignment spacing for improved code readability and consistency.

* add more cases

* not sure these are necessary but sure

* matching parse logic

* chore: Fix argument order in expect_in for scoreboard tests

* fix: Skip games with insufficient plays in play-by-play data

Added a check to filter out games with fewer than 20 plays in the play-by-play data processing. This helps avoid issues with EPA/WPA models and improves data validation.

* chore: Bump version to 2.1.0 and update release notes

Update package version to 2.1.0. Add release notes for bug fixes in `cfbd_pbp_data()` and improvements to `add_yardage()` handling missing yardage values. Update cran-comments to reflect minor release and changes.

* chore: Normalize column names and update tests

Added normalization for 'seasonType' to 'season_type' in cfbd_stats_game_advanced. Updated tests to check for column inclusion with expect_in instead of expect_setequal, and added team ID columns to betting lines test.

---------

Co-authored-by: saiemgilani <saiem.gilani@gmail.com>

Author: akeaswaran

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: cfbfastR
 Title: Access College Football Play by Play Data
-Version: 2.0.0
+Version: 2.1.0
 Authors@R: c(
     person("Saiem", "Gilani", , "saiem.gilani@gmail.com", role = c("cre", "aut")),
     person("Akshay", "Easwaran", , "akeaswaran@me.com", role = "aut"),

NEWS.md

@@ -1,3 +1,9 @@
+# **cfbfastR v2.1.0**
+
+* Fixes a bug in `cfbd_pbp_data()` where play-by-play data for some games were not as expected.
+* Improves `add_yardage()` where plays with missing yardage values were not being handled correctly.
+
+
 # **cfbfastR v2.0.0**
 ### Breaking Changes to Loading Functions
 

R/cfbd_pbp_data.R

@@ -586,8 +586,15 @@ cfbd_pbp_data <- function(year,
     play_df <- purrr::map_dfr(
       g_ids,
       function(x){
-        play_df <- play_df %>%
-          dplyr::filter(.data$game_id == x) %>%
+        # Note: this should be changed to a complete data validation test in the future
+        # filter out games with less than 10 plays to avoid issues with EPA/WPA models
+        game_plays <- play_df %>%
+          dplyr::filter(.data$game_id == x)
+        if (nrow(game_plays) < 20) {
+          cli::cli_alert_danger(glue::glue("Skipping game_id {x} with only {nrow(game_plays)} plays"))
+          return(NULL)
+        }
+        game_plays <- game_plays %>%
           penalty_detection() %>%
           add_play_counts() %>%
           clean_pbp_dat() %>%
@@ -599,7 +606,7 @@ cfbd_pbp_data <- function(year,
           # create_wpa_betting() %>%
           create_wpa_naive(wp_model = wp_model)
         p(sprintf("x=%s", as.integer(x)))
-        return(play_df)
+        return(game_plays)
       }, ...)
     # } else{
     #   play_df <- purrr::map_dfr(

R/cfbd_stats.R

@@ -266,6 +266,8 @@ cfbd_stats_game_advanced <- function(year,
       colnames(df) <- gsub("_Start", "_start", colnames(df))
       colnames(df) <- gsub(".db", "_db", colnames(df))
       colnames(df) <- gsub("Id", "_id", colnames(df))
+      colnames(df) <- gsub("seasonType", "season_type", colnames(df))
+
 
 
       df <- df %>%

R/helper_pbp_add_yardage.R

@@ -85,12 +85,24 @@ add_yardage <- function(play_df) {
         -1 * as.numeric(stringr::str_extract(
           stringi::stri_extract_first_regex(.data$play_text, "(?<= for a loss of)[^,]+"), "\\d+"
         )),
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("pass to", ignore_case = TRUE)) &
+          stringr::str_detect(.data$play_text, regex("for a loss of", ignore_case = TRUE)) ~
+          -1 * as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for a loss of)[^,]+"), "\\d+"
+          )),
         .data$pass == 1 &
           stringr::str_detect(.data$play_text, regex("pass complete to", ignore_case = TRUE)) &
           stringr::str_detect(.data$play_text, regex(" for \\d+ y\\w*ds?", ignore_case = TRUE)) ~
         as.numeric(stringr::str_extract(
           stringi::stri_extract_first_regex(.data$play_text, "(?<= for)[^,]+"), "\\d+"
         )),
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("pass to", ignore_case = TRUE)) &
+          stringr::str_detect(.data$play_text, regex(" for \\d+ y\\w*ds?", ignore_case = TRUE)) ~
+          as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for)[^,]+"), "\\d+"
+          )),
         .data$pass == 1 &
           stringr::str_detect(.data$play_text, regex("Yd pass", ignore_case = TRUE)) ~
           as.numeric(stringr::str_extract(
@@ -99,6 +111,32 @@ add_yardage <- function(play_df) {
         .data$pass == 1 &
           stringr::str_detect(.data$play_text, regex("pass complete to", ignore_case = TRUE)) ~
           yards_gained, # 2024 has games that don't have yards in the PBP text but do have them in the yards_gained field.
+
+        # 2025 has some plays list "PASSER pass" at the very end of the play_text
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("pass \\(\\w", ignore_case = TRUE)) &
+          stringr::str_detect(.data$play_text, regex("^to ", ignore_case = FALSE)) ~ as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for)[^,]+"), "\\d+"
+          )),
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("pass$", ignore_case = TRUE)) &
+          stringr::str_detect(.data$play_text, regex("^to ", ignore_case = FALSE)) ~ as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for)[^,]+"), "\\d+"
+          )),
+        # 2025 has some plays that have yards in the PBP but no listed passer. the format is the same though
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("^to ", ignore_case = FALSE)) ~ as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for)[^,]+"), "\\d+"
+          )),
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("^to ", ignore_case = FALSE)) &
+          stringr::str_detect(.data$play_text, regex("for a loss of", ignore_case = TRUE)) ~
+          -1 * as.numeric(stringr::str_extract(
+            stringi::stri_extract_first_regex(.data$play_text, "(?<= for a loss of)[^,]+"), "\\d+"
+          )),
+        .data$pass == 1 &
+          stringr::str_detect(.data$play_text, regex("^to ", ignore_case = FALSE)) &
+          stringr::str_detect(.data$play_text, regex("for no gain", ignore_case = TRUE)) ~ 0,
         TRUE ~ NA_real_
       )
     )

cran-comments.md

@@ -1,30 +1,10 @@
 ## Release summary
 
-This is a major release that:
-
-* Addresses the noted documentation issues which caused the previous release to be archived by CRAN.
-* Addresses the error from the previous CRAN submission today though does not change official version number.
-* Addresses the missing documentation for the `update_cfb_pbp()` function noted in the previous CRAN response.
-* Addresses minor functionality issues in `cfbd_play_stats_player()` function.
-* Updates the `cfbd_*()` functions to use the new College Football Data API v2. 
-* Addresses the most recent CRAN comments from the previous submission.
-
-The following functions were added:
-  * `cfbd_metrics_fg_ep()`
-  * `cfbd_metrics_wepa_team_season()`
-  * `cfbd_metrics_wepa_players_passing()`
-  * `cfbd_metrics_wepa_players_rushing()`
-  * `cfbd_metrics_wepa_players_kicking()`
-  * `cfbd_ratings_fpi()`
-  * `cfbd_live_scoreboard()`
-  * `cfbd_live_plays()`
-  * `cfbd_api_key_info()`
-
-There are minor changes to the existing `cfbd_*()` functions under the hood. See `NEWS.md` for more details.
-
-While I believe I updated all twitter links in the `README.md` to non-redirecting links, they do give status 403
-when you try to access them without authentication. If this behavior is too problematic and against policy, please let me know and I will
-make the changes to the `README.md`.
+This is a minor release that:
+
+* Fixes a bug in `cfbd_pbp_data()` where play-by-play data for some games were not as expected.
+* Improves `add_yardage()` where plays with missing yardage values were not being handled correctly.
+
 
 ## R CMD check results
 

tests/testthat/test-cfbd_betting_lines.R

@@ -19,16 +19,16 @@ test_that("CFB Betting Lines", {
   cols <- c(
     "game_id", "season", "season_type", "week",
     "start_date",
-    "home_team", "home_conference", "home_classification", "home_score",
-    "away_team", "away_conference", "away_classification", "away_score",
+    "home_team_id", "home_team", "home_conference", "home_classification", "home_score",
+    "away_team_id", "away_team", "away_conference", "away_classification", "away_score",
     "provider", "spread", "formatted_spread",
     "spread_open", "over_under", "over_under_open",
     "home_moneyline", "away_moneyline"
   )
   expect_equal(nrow(x), 4)
   expect_equal(nrow(y), 4)
-  expect_setequal(colnames(x), cols)
-  expect_setequal(colnames(y), cols)
+  expect_in(cols, colnames(x))
+  expect_in(cols, colnames(y))
   expect_s3_class(x, "data.frame")
   expect_s3_class(y, "data.frame")
 })

tests/testthat/test-cfbd_live_scoreboard.R

@@ -51,8 +51,8 @@ test_that("CFB Live Scoreboard", {
   x <- cfbd_live_scoreboard(division='fbs', conference = "B12")
 
   y <- cfbd_live_scoreboard(division='fbs')
-  expect_in(colnames(x), cols)
-  expect_in(colnames(y), cols)
+  expect_in(cols, colnames(x))
+  expect_in(cols, colnames(y))
   expect_s3_class(x, "data.frame")
   expect_s3_class(y, "data.frame")
 })

tests/testthat/test-cfbd_stats_game_advanced.R

@@ -33,9 +33,9 @@ test_that("CFB Stats Game - Advanced", {
   y <- cfbd_stats_game_advanced(2019, team = "LSU")
 
   z <- cfbd_stats_game_advanced(2013, team = "Florida State")
-  expect_setequal(colnames(x), cols)
-  expect_setequal(colnames(y), cols)
-  expect_setequal(colnames(z), cols)
+  expect_in(cols, colnames(x))
+  expect_in(cols, colnames(y))
+  expect_in(cols, colnames(z))
   expect_s3_class(x, "data.frame")
   expect_s3_class(y, "data.frame")
   expect_s3_class(z, "data.frame")