Skip to content

Added REMOTE_S3_DELETE to allow s3 sync command to delete files #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Added REMOTE_S3_DELETE to allow s3 sync command to delete files #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Craft-Scripts Changelog

## 1.2.14 - 2023.10.16
### Added
* Added `REMOTE_S3_DELETE` variable to enable an option in `sync_backups_to_s3.sh` to allow for deleting files in the S3 bucket that aren't in the local backup directory.

## 1.2.13 - 2020.10.20
### Changed
* Removed the MySQL-specific `--set-gtid-purged=OFF` to the `common_mysql.sh` from the mysqldump command options
Expand Down Expand Up @@ -44,7 +48,7 @@

## 1.2.5 - 2017.11.14
### Changed
* Fixed an issue with the backup path for the `backup_dir.sh` script
* Fixed an issue with the backup path for the `backup_dir.sh` script

## 1.2.4 - 2017.10.28
### Added
Expand Down
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ All settings that are prefaced with `LOCAL_` refer to the local environment wher

`LOCAL_BACKUPS_PATH` is the absolute path to the directory where local backups should be stored. For database backups, a sub-directory `LOCAL_DB_NAME/db` will be created inside the `LOCAL_BACKUPS_PATH` directory to store the database backups. Paths should always have a trailing `/`

`LOCAL_AWS_PROFILE` is an [AWS named profile](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) you can set to determine which profile to connect to S3 with.
`LOCAL_AWS_PROFILE` is an [AWS named profile](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) you can set to determine which profile to connect to S3 with.

##### Using mysql within a local docker container

Expand Down Expand Up @@ -268,6 +268,8 @@ All settings that are prefaced with `REMOTE_` refer to the remote environment wh

`REMOTE_S3_PATH` is a optional path relative to the Amazon S3 bucket where the `sync_backups_to_s3.sh` script will contain the backups if specified

`REMOTE_S3_DELETE`, if set to `yes`, a `--delete` flag will be passed to the `aws sync` command so that the command will delete files from the Amazon S3 bucket backup directory that are not present in the local backup directory.

### Setting up SSH Keys

Normally when you `ssh` into a remote server (as some of the `craft-scripts` do), you have to enter your password. Best practices from a security POV is to not allow for password-based logins, but instead use [SSH Keys](https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2).
Expand Down
2 changes: 1 addition & 1 deletion composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "nystudio107/craft-scripts",
"description": "Shell scripts to manage database backups, asset backups, file permissions, asset syncing, cache clearing, and database syncing between Craft CMS environments",
"version": "1.2.13",
"version": "1.2.14",
"keywords": [
"craft",
"craftcms",
Expand Down
6 changes: 6 additions & 0 deletions scripts/craft2-example.env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,9 @@ REMOTE_S3_BUCKET="REPLACE_ME"

# Optional subfolder relative to the S3 bucket root; paths should always have a trailing /
REMOTE_S3_PATH=""

# The Amazon S3 sync command can optionally add a --delete flag to delete files in the bucket that aren't in the local backup directory.
# By default, this is set to "no". No files will be deleted, leaving a complete backup history in your S3 bucket in an ever growing archive.
# If you'd like to enable the --delete flag, set it to "yes", and the sync command will maintain the same number of files in your S3 bucket
# directory as are in your local backup directory.
REMOTE_S3_DELETE="no"
6 changes: 6 additions & 0 deletions scripts/craft3-example.env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,9 @@ REMOTE_S3_BUCKET="REPLACE_ME"

# Optional subfolder relative to the S3 bucket root; paths should always have a trailing /
REMOTE_S3_PATH=""

# The Amazon S3 sync command can optionally add a --delete flag to delete files in the bucket that aren't in the local backup directory.
# By default, this is set to "no". No files will be deleted, leaving a complete backup history in your S3 bucket in an ever growing archive.
# If you'd like to enable the --delete flag, set it to "yes", and the sync command will maintain the same number of files in your S3 bucket
# directory as are in your local backup directory.
REMOTE_S3_DELETE="no"
15 changes: 11 additions & 4 deletions scripts/sync_backups_to_s3.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# @copyright Copyright (c) 2017 nystudio107
# @link https://nystudio107.com/
# @package craft-scripts
# @since 1.1.2
# @since 1.1.3
# @license MIT

# Get the directory of the currently executing script
Expand All @@ -33,14 +33,21 @@ done
echo "Ensuring asset directory exists at '${LOCAL_BACKUPS_PATH}'"
mkdir -p "${LOCAL_BACKUPS_PATH}"

# Initialize delete_flag variable
delete_flag=""

# Check if REMOTE_S3_DELETE is set to 'yes'
if [ "${REMOTE_S3_DELETE}" = "yes" ]; then
delete_flag="--delete"
fi

# Sync the local backups to the Amazon S3 bucket
if [ -z "${LOCAL_AWS_PROFILE}" ] ; then
aws s3 sync "${LOCAL_BACKUPS_PATH}" s3://"${REMOTE_S3_BUCKET}"/"${REMOTE_S3_PATH}"
aws s3 sync "${LOCAL_BACKUPS_PATH}" s3://"${REMOTE_S3_BUCKET}"/"${REMOTE_S3_PATH}" ${delete_flag}
else
aws --profile="${LOCAL_AWS_PROFILE}" s3 sync "${LOCAL_BACKUPS_PATH}" s3://"${REMOTE_S3_BUCKET}"/"${REMOTE_S3_PATH}"
aws --profile="${LOCAL_AWS_PROFILE}" s3 sync "${LOCAL_BACKUPS_PATH}" s3://"${REMOTE_S3_BUCKET}"/"${REMOTE_S3_PATH}" ${delete_flag}
fi
echo "*** Synced backups to ${REMOTE_S3_BUCKET}"

# Normal exit
exit 0