Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Automation that scrapes USEPA github and provides that metadata for code.gov

License

NotificationsYou must be signed in to change notification settings

USEPA/code-json-generator

 
 

Daily code.json update

This application generates a machine-readable software inventory compliantwithCode.gov Metadata Schema version 2.0.0. To accomplish this goal,the application connects to a series of Git hosting API "endpoints",inspects the configured projects, and generates an inventory. This application has been enhanced to automate the generation of theAgency's code.json file through GitHub Actions.

The application configuration supports internal/external hosting platformswith or without authentication (for private repositories). It currently workswith GitHub and GitLab REST APIs but could easily be extended for other hostingplatform APIs as well.

For each project, the application will attempt to use a developer-provided"code.json" snippet which is a JSON file containing the "releases" arrayfor the project as defined by theCode.gov Metadata Schema version 2.0.0.

If a "code.json" snippet file is not provided by the project maintainer, theapplication will attempt to infer sufficient/required properties from thetarget project and generate the best possible metadata for inclusion in theagency inventory.

Agency code.json daily update through GitHub Actions

GitHub Actions is being used to run a scheduled event to create the Agency's code.json file daily.

As mentioned earlier, if a "code.json" snippet file is not provided by the project maintainer, theapplication will attempt to infer sufficient/required properties from thetarget project and generate the best possible metadata for inclusion in theagency inventory.

The example EPA Agency code.json for the github.com/USEPA organization can befound here.

Generating the code.json file

The code.json file can be generated by any of the GitHub Actions:

  1. Thedaily.yml generates the code.json file to this repository'soutput branch. This workflow is enabled.

  2. Thedaily-deploy.yml generates the code.json file to this repository'smaster branch. This workflow is disabled.

  3. Thego-push (replacing Build and Deploy) generates the code.json file to this repository'smaster branch on the git push command. This workflow is disabled.

Branch Directory

  1. output - includes the code.json file generated by thedaily.yml.
  2. cloudgov-deployment - provides intial code-json-generator GitHub Action workflow and removes 4000+ commit history generating the code.json file and deploying to cloud.gov.
  3. including-history and history-cloud.gov-deployment - provides intial code-json-generator GitHub Action workflow and includes 4000+ commit history generating the code.json file and deploying to cloud.gov.

Getting Started

This section describes the process to acquire and run the code-json-generatorapplication locally, GitHub, and in cloud.gov.

Prerequisites

Follow these prerequisites for running code.json generator locally, in GitHub, and for deploying to cloud.gov.

This application is a command line Node application. Node must be installedon the system prior to running this application.

https://nodejs.org/

Install the application

On the command line:

$ npm install -g code-json-generator

Configure the application runtime

The application requires a configuration file to dictate which repositoriesto include in the generated inventory. Anexample configuration fileis provided to help get started.

  1. Update the configuration file with your Agency's respective endpoints.
  2. Delete the existing create-agency-inventory-config.js file in the bin folder.
  3. Move the new configuration file from the etc folder into the bin folder.
  4. Rename the configuration file to create-agency-inventory-config.js.

Run the application locally

Running the application locally will help identify the specific repositories with missing or invalid code.json files. After all repositories have a valid code.json file, running the application will generate the Agency code.json locally.

The application provides command-line usage syntax help via the--help switch.

$ create-inventory.js --help  Usage: create-inventory [options]  Options:    -V, --version            output the version number    -c, --configFile <file>  Configuration file    -h, --help               output usage information

To run the application locally, you need to:

  1. Change the working directory to the bin folder.

  2. To generate an inventory, run the application and provide a configuration fileusing the--configFile <file> switch. The generated inventory will, bydefault, be printed to STDOUT. This output can be redirected to a filefor persistence.

$ create-inventory.js --configFile ./create-agency-inventory.config.json > code.json

Note: This example assumes a file containing proper configuration is locatedin the current working directory and is calledcreate-agency-inventory.config.json.Adjust this usage to suit actual work environment.

Run with GitHub Actions

To run the application with GitHub Actions, you need to:

  1. Change the working directory to the .github/workflows folder.

  2. Open the daily.yml file.

  3. Update theschedule event per the desired frequency.

  4. Save, commit and push your changes.

Deployment to Cloud.gov with GitHub Actions

You can deploy the code.json generator to Cloud.gov with GitHub Actions.

Prerequisites

To do this you will need a Cloud.gov account and theCloud Foundry Command Line Tools installed on your machine.

  1. Login to cloud.gov
  2. Navigate to your organization's space which the app will be deployed to.
  3. Follow the documentation to create a cloud.gov service account plan:space deployer for continuous deployment to Cloud.gov.
  4. Add the service key username as the CG_USERNAME secret in the GitHub Repository's Secrets.
  5. Add the service key username as the CG_USERNAME secret in the GitHub Repository's Secrets.

Deploying to cloud.gov

You need to establish the name of your application before deploying it to cloud.gov.

  1. Change the working directory to the etc/manifests folder.

  2. Open the frontend.yml file.

  3. Update the applications "name:" to whatever subdomain you wish to use (e.g., your-app-name).

  4. Save, commit and push your changes.

  5. You will see the code.json file located at: your-app-name.app.cloud.gov/code.json

Update frequency of the Agency's code.json file

You can modify how frequently the Agency's code.json file is generated and published to cloud.gov.

  1. Change the working directory to the .github/workflows folder.

  2. Open the daily-deploy.yml file.

  3. Update theschedule event per the desired frequency.

  4. Save, commit and push your changes.

Development

To develop and possibly contribute to this project please review thecode of conduct andcontributing guidelines.

Obtain the source code

Potentially fork this repository and then clone the fork to obtain the sourcecode.

$ git clone <fork_urn>/code-json-generator.git$ cd code-json-generator

Develop in a feature branch

Updates should be developed in a feature branch on the local clone of a fork.

$ git checkout -b feature-X

Submit pull requests for review and integration

Commit changes to feature branches and push feature branches to the forkedremote. Submit a pull request back to this upstream repository for reviewand integration. If the pull request fixes an open issue on this repository,include the textfixes #N (whereN is the issue number fixed) in thepull request title or description.

$ git commit -am 'Implemented feature'$ git push origin feature-X

Credits

This repository reused material from theUSGS code-json-generator repository.

Disclaimer

The United States Environmental Protection Agency (EPA) GitHub project code is provided on an "as is" basis and the user assumes responsibility for its use.  EPA has relinquished control of the information and no longer has responsibility to protect the integrity , confidentiality, or availability of the information.  Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by EPA.  The EPA seal and logo shall not be used in any manner to imply endorsement of any commercial product or activity by EPA or the United States Government.

About

Automation that scrapes USEPA github and provides that metadata for code.gov

Topics

Resources

License

Code of conduct

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript93.1%
  • Shell6.9%

[8]ページ先頭

©2009-2025 Movatter.jp