Repository files navigation

A best practices guide for submitting spark applications, integration with hive metastore, security, storage options, debugging options and performance considerations.

Return toLive Docs.

The documentation is made available under the Creative Commons Attribution-ShareAlike 4.0 International License. See the LICENSE file.

The sample code within this documentation is made available under the MIT-0 license. See the LICENSE-SAMPLECODE file.

1. Fork the repository
2. Make your change and double-check themkdocs.yml is updated accordingly.
3. Install the MkDocs command tool and material theme if needed:

pip install mkdocspip install mkdocs-material# material theme

4. MkDocs comes with a built-in dev-server that lets you preview your documentation as you work on it. Make sure you're in the same directory as themkdocs.yml configuration file, then run the command:

mkdocs serve

5. Open uphttp://127.0.0.1:8000/ in your browser, and you'll see the best practice website being displayed locally.
6. Adjust your document changes in real time.
7. When everything looks good and you're ready to deploy the change, run the command to build/compile the website content:

mkdocs build

8. This will refresh the directory./site. Take a look inside the directory and make sure your changes are included.

ls site

9. Commit change to github and send us apull request.

10. With a repo admin permission, we can merge the pull request into the main branch.

11. Most importantly, as a repo admin, we must run the deploy command to copy the './site' content to 'gh-pages' branch and pushing to GitHub. Without this step, the website content won't be refreshed.

mkdocs gh-deploy