Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Validate any SQL migration scripts are the most recent#6652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
withinfocus wants to merge5 commits intomain
base:main
Choose a base branch
Loading
fromdb-validation
Open
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 175 additions & 0 deletions.github/workflows/test-database.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -262,3 +262,178 @@ jobs:
working-directory: "dev"
run: docker compose down
shell: pwsh

validate-migration-naming:
name: Validate new migration naming and order
runs-on: ubuntu-22.04

steps:
- name: Check out repo
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
persist-credentials: false

- name: Validate new migrations for pull request
if: github.event_name == 'pull_request'
run: |
# Use C locale for consistent, locale-independent string comparison
export LC_ALL=C

# Get list of migrations from main branch
git fetch origin main:main
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Finding 1 continued: This fetch command should have error handling. If it fails, the script will continue with a stale or non-existent localmain branch.

Suggestion:

if! git fetch origin main:main;thenecho"ERROR: Failed to fetch main branch"exit 1fi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

⚠️Missing error handling: Ifgit fetch fails, the script continues with potentially stale or missing data, leading to incorrect validation results.

Suggested change
git fetch origin main:main
if !git fetch origin main:main; then
echo "ERROR:Failed to fetch main branch"
exit 1
fi

MAIN_MIGRATIONS=$(git ls-tree -r --name-only main -- util/Migrator/DbScripts/*.sql | sort)

# Get list of migrations from current branch
CURRENT_MIGRATIONS=$(git ls-tree -r --name-only HEAD -- util/Migrator/DbScripts/*.sql | sort)

# Save to temporary files for comparison
echo "$MAIN_MIGRATIONS" > /tmp/main_migrations.txt
echo "$CURRENT_MIGRATIONS" > /tmp/current_migrations.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

♻️Finding 4 - Cleanup: Consider adding cleanup for temporary files at the end of the script:

rm -f /tmp/main_migrations.txt /tmp/current_migrations.txt


# Find added migrations
ADDED_MIGRATIONS=$(comm -13 /tmp/main_migrations.txt /tmp/current_migrations.txt)

if [ -z "$ADDED_MIGRATIONS" ]; then
echo "No new migration files added."
exit 0
fi

echo "New migration files detected:"
echo "$ADDED_MIGRATIONS"
echo ""

# Check that all added migrations come after all existing migrations
LAST_MAIN_MIGRATION=$(echo "$MAIN_MIGRATIONS" | tail -n 1 | xargs basename)
echo "Last migration in main branch: $LAST_MAIN_MIGRATION"
echo ""

VALIDATION_FAILED=0
while IFS= read -r migration; do
MIGRATION_NAME=$(basename "$migration")

# Validate NEW migration filename format - enforce strict format for new migrations
# Required format: YYYY-MM-DD_NN_Description.sql (with leading zeros in date and sequence)
if ! [[ "$MIGRATION_NAME" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}_.+\.sql$ ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Finding 1 - Format enforcement issue: This regex requires leading zeros ([0-9]{2} for month and day), but the repository already contains at least one migration that violates this:

2025-10-3_00_AddOrganizationIntegration_ReadByTeamsConfigurationTenantIdTeamIdStoredProcedure.sql

This will cause validation failures on PRs that rebase against main if the problematic migration gets included.

Options:

  1. Rename existing non-compliant migrations before merging this PR (preferred for consistency)
  2. Make regex more permissive:^[0-9]{4}-[0-9]{1,2}-[0-9]{1,2}_[0-9]{2}_.+\.sql$

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Critical: Regex allows non-compliant migration that breaks sorting

The repository contains2025-10-3_00_AddOrganizationIntegration_ReadByTeamsConfigurationTenantIdTeamIdStoredProcedure.sql (note:10-3 not10-03).

Problem: Lexicographic string comparison produces incorrect ordering:

  • "2025-10-3" sortsAFTER"2025-10-30" in ASCII comparison
  • At character position 9:'3' (from"10-3_") vs'0' (from"10-30")
  • Character'3' has higher ASCII value than'0', so"10-3" >"10-30"

Impact:

  • New migrations dated 2025-10-30 through 2025-11-03 will incorrectly fail validation
  • Chronological order is violated in the migration list
  • Sorting doesn't reflect actual date order

Fix required before merging: Rename the non-compliant migration:

git mv util/Migrator/DbScripts/2025-10-3_00_AddOrganizationIntegration_ReadByTeamsConfigurationTenantIdTeamIdStoredProcedure.sql \     util/Migrator/DbScripts/2025-10-03_00_AddOrganizationIntegration_ReadByTeamsConfigurationTenantIdTeamIdStoredProcedure.sql

Then restore strict regex validation:

Suggested change
if ! [[ "$MIGRATION_NAME" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}_.+\.sql$ ]]; then
if ! [[ "$MIGRATION_NAME" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}_.+\.sql$ ]]; then

echo "ERROR: Migration '$MIGRATION_NAME' does not match required format"
echo "Required format: YYYY-MM-DD_NN_Description.sql"
Comment on lines +318 to +319
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

👏 Also a nice improvement here.

echo " - YYYY: 4-digit year"
echo " - MM: 2-digit month with leading zero (01-12)"
echo " - DD: 2-digit day with leading zero (01-31)"
echo " - NN: 2-digit sequence number (00, 01, 02, etc.)"
echo "Example: 2025-01-15_00_MyMigration.sql"
VALIDATION_FAILED=1
continue
fi

# Compare migration name with last main migration
if [[ "$MIGRATION_NAME" < "$LAST_MAIN_MIGRATION" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

💭Finding 5 - Multiple new migrations: When a PR adds multiple migrations (e.g.,2025-11-27_00_A.sql and2025-11-26_00_B.sql), this only validates each againstLAST_MAIN_MIGRATION, not against each other.

The out-of-order new migrations would be caught (B would fail since it's < A alphabetically), but the error message might be confusing since it references the main branch rather than the other new migration.

Is this the intended behavior, or should we also validate ordering within the set of new migrations?

echo "ERROR: New migration '$MIGRATION_NAME' is not chronologically after '$LAST_MAIN_MIGRATION'"
VALIDATION_FAILED=1
else
echo "OK: '$MIGRATION_NAME' is chronologically after '$LAST_MAIN_MIGRATION'"
fi
done <<< "$ADDED_MIGRATIONS"

echo ""
if [ $VALIDATION_FAILED -eq 1 ]; then
echo "FAILED: One or more migrations are incorrectly named or not in chronological order"
echo ""
echo "All new migration files must:"
echo " 1. Follow the naming format: YYYY-MM-DD_NN_Description.sql"
echo " 2. Use leading zeros in dates (e.g., 2025-01-05, not 2025-1-5)"
echo " 3. Include a 2-digit sequence number (e.g., _00, _01)"
echo " 4. Have a filename that sorts after the last migration in main"
echo ""
echo "To fix this issue:"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

🎨Finding 6 - Enhanced error message: Consider adding the specific git command to help developers fix the issue:

echo"To fix this issue:"echo"  1. Rename using git:"echo"     git mv util/Migrator/DbScripts/$MIGRATION_NAME util/Migrator/DbScripts/YYYY-MM-DD_NN_NewName.sql"echo"  2. Choose a date after$LAST_MAIN_MIGRATION"echo"  3. Commit the change: git commit -m 'Rename migration to fix ordering'"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

🎨Enhancement: Add specific git commands to help developers fix issues:

Suggested change
echo "To fix this issue:"
echo "To fix this issue:"
echo " 1. Rename using git:"
echo " git mv util/Migrator/DbScripts/$MIGRATION_NAME util/Migrator/DbScripts/YYYY-MM-DD_NN_NewName.sql"
echo " 2. Choose a date after $LAST_MAIN_MIGRATION"
echo " 3. Commit:git commit -m 'Rename migration to fix chronological ordering'"
echo ""

echo " 1. Locate your migration file(s) in util/Migrator/DbScripts/"
echo " 2. Rename to follow format: YYYY-MM-DD_NN_Description.sql"
echo " 3. Ensure the date is after $LAST_MAIN_MIGRATION"
echo ""
echo "Example: 2025-01-15_00_AddNewFeature.sql"
exit 1
fi

echo "SUCCESS: All new migrations are correctly named and in chronological order"
shell: bash

- name: Validate new migrations for push
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I assume these 2 jobs are identical except for the source of the current migrations? I suggest that the comparison logic could be moved into a script for reuse, e.g.

// job-specific logic to get the migrations and write them to temp files// then run the comparison/validationsh verify-migrations.sh /tmp/prev_migrations.txt /tmp/current_migrations.txt

if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
run: |
# Use C locale for consistent, locale-independent string comparison
export LC_ALL=C

# Get list of migrations from previous commit
PREV_MIGRATIONS=$(git ls-tree -r --name-only HEAD~1 -- util/Migrator/DbScripts/*.sql 2>/dev/null | sort || echo "")

# Get list of migrations from current commit
CURRENT_MIGRATIONS=$(git ls-tree -r --name-only HEAD -- util/Migrator/DbScripts/*.sql | sort)

# Save to temporary files for comparison
echo "$PREV_MIGRATIONS" > /tmp/prev_migrations.txt
echo "$CURRENT_MIGRATIONS" > /tmp/current_migrations.txt

# Find added migrations
ADDED_MIGRATIONS=$(comm -13 /tmp/prev_migrations.txt /tmp/current_migrations.txt)

if [ -z "$ADDED_MIGRATIONS" ]; then
echo "No new migration files added in this commit."
exit 0
fi

echo "New migration files detected:"
echo "$ADDED_MIGRATIONS"
echo ""

# Check that all added migrations come after all existing migrations
if [ -n "$PREV_MIGRATIONS" ]; then
LAST_PREV_MIGRATION=$(echo "$PREV_MIGRATIONS" | tail -n 1 | xargs basename)
echo "Last migration in previous commit: $LAST_PREV_MIGRATION"
echo ""

VALIDATION_FAILED=0
while IFS= read -r migration; do
MIGRATION_NAME=$(basename "$migration")

# Validate NEW migration filename format - enforce strict format for new migrations
# Required format: YYYY-MM-DD_NN_Description.sql (with leading zeros in date and sequence)
if ! [[ "$MIGRATION_NAME" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}_.+\.sql$ ]]; then
echo "ERROR: Migration '$MIGRATION_NAME' does not match required format"
echo "Required format: YYYY-MM-DD_NN_Description.sql"
echo " - YYYY: 4-digit year"
echo " - MM: 2-digit month with leading zero (01-12)"
echo " - DD: 2-digit day with leading zero (01-31)"
echo " - NN: 2-digit sequence number (00, 01, 02, etc.)"
echo "Example: 2025-01-15_00_MyMigration.sql"
VALIDATION_FAILED=1
continue
fi

# Compare migration name with last previous migration
if [[ "$MIGRATION_NAME" < "$LAST_PREV_MIGRATION" ]]; then
echo "ERROR: New migration '$MIGRATION_NAME' is not chronologically after '$LAST_PREV_MIGRATION'"
VALIDATION_FAILED=1
else
echo "OK: '$MIGRATION_NAME' is chronologically after '$LAST_PREV_MIGRATION'"
fi
done <<< "$ADDED_MIGRATIONS"

echo ""
if [ $VALIDATION_FAILED -eq 1 ]; then
echo "FAILED: One or more migrations are incorrectly named or not in chronological order"
echo ""
echo "All new migration files must:"
echo " 1. Follow the naming format: YYYY-MM-DD_NN_Description.sql"
echo " 2. Use leading zeros in dates (e.g., 2025-01-05, not 2025-1-5)"
echo " 3. Include a 2-digit sequence number (e.g., _00, _01)"
echo " 4. Have a filename that sorts after the last migration"
echo ""
echo "Migration files are located in: util/Migrator/DbScripts/"
exit 1
fi
else
echo "No previous migrations found (initial commit?). Skipping validation."
fi

echo "SUCCESS: All new migrations are correctly named and in chronological order"
shell: bash
Loading

[8]ページ先頭

©2009-2025 Movatter.jp