Repository files navigation

Migration script used to migrate Challenges, Resources, and Resource Roles from Informix to DynamoDB.It runs on a scheduled basis and also on-demand by exposing an API allowing admins to manually trigger the migration.

• Data migration script to move v4/Informix to v5 Dynamo/ES

• Challenge API
• Resources API

• NodeJS
• Elasticsearch(v6.3.1)
• DynamoDB
• Informix
• Docker(CE 17+)
• Docker Compose

Seeconfig/default.js. Most of them is self explain there.

• PORT: API server port; default to3001
• API_VERSION: API version; default tov5
• SCHEDULE_INTERVAL: the interval of schedule; default to5(minutes)
• CHALLENGE_TYPE_API_URL Challenge v4 api url from which challenge types data are fetched.
• CHALLENGE_TIMELINE_API_URL Challenge v5 api url from which challenge timelines are fetched.
• CREATED_DATE_BEGIN A filter; if set, only records in informix created after the date are migrated.
• BATCH_SIZE Maximum legacy will be load at 1 query
• ERROR_LOG_FILENAME Filename for data that error to migrate.
• RESOURCE_ROLE List of resource role to be included in migration

To install foreman follow thislinkTo know how to use foreman follow thislink

If you run something withnf run npm run <something> instead ofnpm run <something>, it'll load in a.env file from the root of the project

To simplify deployment, we're using docker. To build the imagesor run the container:

cd <legacy-challenge-migration-cli>/dockerdocker-compose up

This will automatically build the image if have not done this before.After container has been run, go to container shell and install dependencies:

docker exec -ti legacy-challenge-migration-cli bashnpm i

• Inside the docker container, start the express server:npm start

There are two parts need to be updated for local migrations,

• ./src/models/challenge.jsthroughput: 'ON_DEMAND', should be updated tothroughput:{ read: 4, write: 2 },
• ./config/default.jsTwo aws config should be uncommented

and env variableIS_LOCAL_DB should be set to true before you continue to the next steps.

• To run a migration, the command should be:MIGRATION=<name-of-migration-file> nf run npm run migrateExample:MIGRATION=001-migrate-taskData nf run npm run migrate

• TBD

• TBD

• TBD