Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Documentation on how to use this library using AWS lambda#78

Discussion options

When i was trying to use this using lambda, i had to remove following deps from the zip. Have put in the reasons for the same

  • botocore
  • boto3
  • lxml
    • Not using active directory features inside lambda (Ref)
    • Occupies ~10 MB space
    • Has native C libraries packaged with it and this is platform dependent and a direct zip and upload wont work (i am using a macosx for development and lambda uses amazon linux)

I also see pytz, however as of now i am not sure of the impact of removing the same

You must be logged in to vote

Thank you for this feedback. Regarding the dependency on lxml, this package is uses for some of our IdP (Identity provider plugin) code along with beautifulsoup. In looking at the beautifulsoup docs I see there is a HTML parser by default, from Python standard libraryref.

I think it'd be worth while to investigate if the parser included by default would be sufficient here (we haven't tested this), as it'd be nice to reduce the number of dependencies redshift_connector has. I will take a look into this option and follow up with my findings.

While lxml is not applicable to your use case, there are methods for downloading Python modules for a platform other than the one currently in use.se…

Replies: 1 comment

Comment options

Thank you for this feedback. Regarding the dependency on lxml, this package is uses for some of our IdP (Identity provider plugin) code along with beautifulsoup. In looking at the beautifulsoup docs I see there is a HTML parser by default, from Python standard libraryref.

I think it'd be worth while to investigate if the parser included by default would be sufficient here (we haven't tested this), as it'd be nice to reduce the number of dependencies redshift_connector has. I will take a look into this option and follow up with my findings.

While lxml is not applicable to your use case, there are methods for downloading Python modules for a platform other than the one currently in use.see this stackoverflow post for example. I had no clue this was an option, but it's pretty cool :)

As for the dependencies on boto3 and botocore, these are needed for use of IAM and IDP auth. Though needing to remove them before deploying your lambda is not ideal as they take up a good bit of space... ~67MB or so. Aside from taking up space, the lambda docs imply this shouldn't break anything:

Note: Lambda periodically updates the Boto3 libraries to enable the latest set of features and security updates. To have full control of the dependencies your function uses, package all of your dependencies with your deployment package.ref

As for documentation re: using redshift_connector with lambda, I've used the official AWS docs forDeploy Python Lambda functions with .zip file archives in the past.

Have you found gaps specific to redshift_connector in these docs that you'd like to see addressed in our docs? If so, please let us know and we can add some documentation :)

You must be logged in to vote
0 replies
Answer selected byBrooke-white
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Category
Ideas
Labels
None yet
2 participants
@kishaningithub@Brooke-white

[8]ページ先頭

©2009-2025 Movatter.jp