Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
/ddevPublic
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

docs: Explain corp vpn and proxy, fixup FAQ, fixes #7048#7061

Draft
rfay wants to merge9 commits intoddev:main
base:main
Choose a base branch
Loading
fromrfay:20250309_rfay_docs_vpn_proxy

Conversation

rfay
Copy link
Member

@rfayrfay commentedMar 10, 2025
edited
Loading

The Issue

We occasionally have people with VPN or proxy situations that need help and reference about those

How This PR Solves The Issue

  • Add VPN and Proxy info
  • Add more on DNS problems
  • Addname info to FAQ

TODO

Manual Testing Instructions

Review (mostly) athttps://ddev--7061.org.readthedocs.build/en/7061/users/usage/networking/

FAQ is athttps://ddev--7061.org.readthedocs.build/en/7061/users/usage/faq/#i-want-to-use-the-same-code-with-various-project-names

Automated Testing Overview

Release/Deployment Notes

@rfayrfay requested review fromstasadev andtyler36March 10, 2025 01:53
@rfayrfay marked this pull request as ready for reviewMarch 10, 2025 01:55
@rfayrfay requested review froma team ascode ownersMarch 10, 2025 01:55
Copy link
Member

@stasadevstasadev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks good to me.

Comment on lines +11 to +31
To fix this so that applications inside the web container (or other containers) can access the internet, the web image must be adjusted to trust the alternate CA that the VPN provides, so the intermediate system is not rejected as invalid.

Several specific ways to sort this out are listed in the related [Stack Overflow](https://stackoverflow.com/questions/71595327/corporate-network-vpn-ddev-composer-create-results-in-ssl-certificate-proble) question, but the basic answer is:

1. Obtain the CA `.crt` files from your IT department, vendor, or other source.
2. Place the `.crt` files in your `.ddev/web-build` directory.
3. Use a `.ddev/web-build/Dockerfile.vpn` to install the `.crt` files, as shown in this example `.ddev/web-build/Dockerfile.vpn`:

```Dockerfile
COPY <yourcert>*.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates --fresh
```

4. To test for success,

```bash
ddev restart
ddev exec curl -I https://www.google.com # Or any URL you need
```

and you expect a "200 OK" response.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This assumes you were able to download images in the first place by running Docker without Zscaler. Otherwise, with Colima on MacOS, you must update the CA certificates in the Docker virtual machine or no images will download. Once I did that I did not have to add the CA certificate to the web container. Haven't tried other Docker servers yet.
https://darren.oh.name/node/81

@rfay
Copy link
MemberAuthor

This still needs a bit of work and community review:

  • Test out proxying again. Can it work with image pulls using the add-on or not?
  • Community review of Zscaler has been requested, and@darrenoh has given his recent experience.

@rfayrfay marked this pull request as draftMarch 11, 2025 13:32
@stasadevstasadevforce-pushed the20250309_rfay_docs_vpn_proxy branch fromc2b2f12 to92e89adCompareMarch 14, 2025 15:29
@stasadev
Copy link
Member

Rebased to pick up buildkite skip for docs:

@rfay
Copy link
MemberAuthor

Here is@shelane writeup.

Adding Custom CA Certificates to Your DDEV Project

When working with DDEV in environments that require custom CA certificates (e.g., internal networks, VPNs, or corporate proxies), it is essential to properly configure these certificates to ensure secure communication. This guide provides step-by-step instructions for obtaining, converting, and installing custom CA certificates in your DDEV project.

Steps to Add Custom CA Certificates

  1. Obtain the CA Certificates

    Obtain the .crt files for your custom Certificate Authorities (CAs) from your IT department, vendor, or other trusted source.
    Ensure the certificates are provided in .crt format. If they are in a different format (e.g., .der), you will need to convert them to the .crt (PEM) format. See Step 2 for details.

  2. Verify and Convert Certificates to PEM Format

To ensure compatibility, all CA certificates must be in PEM format. PEM format certificates are human-readable and start with-----BEGIN CERTIFICATE-----.

  • Check if a certificate is in PEM format: Open the .crt file in a text editor (e.g., nano or vim). If the file starts with -----BEGIN CERTIFICATE-----, it is already in PEM format, and no conversion is needed.

  • Convert DER to PEM format: If the certificate is in DER format (binary), you can convert it to PEM format using OpenSSL:

openssl x509 -inform der -in <input-file>.crt -out <output-file>.crt

Replace .crt with the path to your DER certificate and .crt with the desired output file name.
3. Place the Certificates in the.ddev/web-build Directory

  • Copy all the PEM-formatted .crt files into your DDEV project's.ddev/web-build directory. For example:
my-ddev-project/.ddev/web-build/├── my-custom-cert1.crt├── my-custom-cert2.crt
  1. Create a Custom Dockerfile for Certificate Installation
  • In the .ddev/web-build directory, create a file named Dockerfile.vpn (or a similarly descriptive name).
  • Add the following content to theDockerfile.vpn to copy the certificates into the container and update the CA store:
# Copy custom certificates into the container's CA directoryCOPY *.crt /usr/local/share/ca-certificates/# Update the CA certificates storeRUN update-ca-certificates --fresh
  1. Restart DDEV to Apply Changes
  • Restart your DDEV project to rebuild the container and apply the custom certificates:
ddev restart
  1. Test the Certificates
  • Verify that the certificates have been installed correctly by running a curl command inside the DDEV container:
ddev exec curl -I https://www.google.com

Replacehttps://www.google.com with a URL that requires the custom CA certificates. A successful response will include a 200 OK status.

Troubleshooting

Common Issues

  1. Certificate Not in PEM Format:
  • If the certificate is not in PEM format, you will see errors when updating the CA certificates. Ensure all certificates are properly converted to PEM format using the openssl command in Step 2.
  1. Certificate Not Recognized:
  • Double-check that the .crt files are correctly placed in the.ddev/web-build directory and that theDockerfile.vpn is properly configured.
  1. Connection Issues Persist:
  • Verify that the CA certificates provided are correct and up-to-date. Contact your IT department or vendor if you suspect the certificates are outdated or invalid.

Debugging Tips

  • To check if the certificates were installed in the container, you can list the installed certificates:
ddev exec ls /etc/ssl/certs
  • To inspect the CA certificate store for your custom certificates:
ddev exec cat /etc/ssl/certs/ca-certificates.crt | grep "Your-Certificate-Name"

Example Directory Structure

After completing the steps above, your .ddev/web-build directory should look like this:

.ddev/├── config.yaml├── web-build/│   ├── Dockerfile.vpn│   ├── my-custom-cert1.crt│   ├── my-custom-cert2.crt

Summary

By following this guide, you can successfully add and configure custom CA certificates in your DDEV project. Ensuring the certificates are in PEM format and properly installed in the container is critical for secure communication with internal or external services.

rpkoller reacted with thumbs up emoji

@maciazek
Copy link

maciazek commentedMar 15, 2025
edited by rfay
Loading

Hello,

just my 5 cents here as I was working with DDEV behind corporate proxy for last few months, struggling with no previous experience with proxies at all (long story short: proxy info inside/etc/environment,/etc/apt/apt.conf,~/.bashrc,/etc/systemd/system/docker.service.d/http-proxy.conf combined withddev/ddev-proxy-support addon was enough to run MariaDB-based project).
But recently I've encountered startup connection errors (during installation of APT packages) with PostgreSQL-based project and after some research here on GitHub I figured out that I need to set up proxy in~/.docker/config.json (and downgrade DDEV to 1.23.5). After that I was able to run both MariaDB- and PostgreSQL-based projects, but only IP access was working - I couldn't open it via domain name (myapp.ddev.site).This comment was very helpful - after addingddev-myapp-web (andddev-myapp-phpmyadmin) tonoProxy in~/.docker/config.json, I was finally able to run both my projects without problems (and without addon).

  • I'm writing this because I see that you're going to add info about~/.docker/config.json in documentation, so maybe these container names innoProxy should also be included? *

I was doing all this research this week in my office, but today I was able to reproduce same situation on my personal PC with two VirtualBox machines (one Debian with Squid as proxy, and one Linux Mint with DDEV, connected together via 'internal' network).

@rfay
Copy link
MemberAuthor

@maciazek thanks for checking in!

your comment is about a proxy only, right, not a VPN situation?

Was your problem with the ddev-router not becoming ready as in the issue you link to?

If so, have you tested DDEV HEAD with the fix mentioned in that issue?https://ddev.readthedocs.io/en/stable/developers/building-contributing/#testing-latest-commits-on-head

If you have suggestions for this PR based on your work (and using DDEV HEAD) they would be very welcome!

@maciazek
Copy link

Yes, I'm talking about proxy only, not VPN as I'm working on-site.
We need to set up proxy config to access internet (in OS settings, in Firefox etc.)
Also we need to authenticate through proxy, so our proxy config looks like this:

http://username:password@192.168.1.10:3128

If there is some special character, it has to be url-encoded (with double %):

http://username:p%%40ssword@192.168.1.10:3128

And yes, after setting up my proxy correctly (in~/.docker/config.json) ddev-router couldn't start because of traefik healthcheck errors (that's why I downgraded to DDEV 1.23.5).

ddev-router failed to become ready; log=, err=health check timed out after 1m0s

Today I upgraded to 1.24.3 and ddev-router couldn't start again. Then I tried HEAD version (ddev version v1.24.3-25-g419560521) as described in docs and now ddev-router is starting without problems (I was testing in my local reproduced environment). This week I'll try to test it at my workplace, but I assume it will behave the same way.

So, in my opinion there are 2 things that should be added to docs (from my environment's perspective, of course):

  1. Proxy config in/etc/systemd/system/docker.service.d/http-proxy.conf - as without this, Docker couldn't pull any image (or maybe simply include links to docker documentation as proxy config is describedhere andhere). Both CLI (~/.docker/config.json) and daemon (/etc/systemd/system/docker.service.d/http-proxy.conf) config files are required in my case to make it all work right.

  2. Adding container names innoProxy in~/.docker/config.json as described in my previous post. BTW I'm not sure why I can't use wildcards (ddev-*-web,ddev-*-phpmyadmin) - I need to setup names for every project, so my~/.docker/config.json looks like this:

{"proxies": {"default": {"httpProxy":"http://username:p%%40ssword@192.168.1.10:3128","httpsProxy":"http://username:p%%40ssword@192.168.1.10:3128","noProxy":"localhost,127.0.0.1/8,::1,*.ddev.site,ddev-myapp1-web,ddev-myapp1-phpmyadmin,ddev-myapp2-web,ddev-myapp2-phpmyadmin"    }  }}

Maybe there is some simpler way to set these container names once for all projects?

@rfay
Copy link
MemberAuthor

Thanks@maciazek -

Proxy config in /etc/systemd/system/docker.service.d/http-proxy.conf

I've had varying approaches to this, and am a bit confused about it. You're sure that the /etc/systemd stuff is required, as opposed to just putting it in the user's .docker, true?

Adding container names in noProxy in ~/.docker/config.json

That seems enormously awkward and wrong of course.

One note: I've seen a mention that*.<domain> is sometimes invalid, and.domain is correct. However, I haven't confirmed that and don't know that Docker doesn't handle both.

Thanks so much for your careful work and help on this!

@maciazek
Copy link

I've had varying approaches to this, and am a bit confused about it. You're sure that the /etc/systemd stuff is required, as opposed to just putting it in the user's .docker, true?

I deleted file/etc/systemd/system/docker.service.d/http-proxy.conf, then daemon-reload & restart docker, thenddev delete images --all and when I triedddev start got this:

Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: server misbehavingUnable to pull Docker images: exit status 1

I have HTTP_PROXY env vars set in~/.bashrc and/etc/environment all the time.

I also tried "Daemon configuration" approachdescribed in the docs, but I was able to pull images only when I set up/etc/docker/daemon.json config.

Config set via both 'rootless modes' described in docs (~/.config/docker/daemon.json or~/.config/systemd/user/docker.service.d/http-proxy.conf) where giving me the same error mentioned above. So one way or another, I need to set up proxy in /etc... (/etc/systemd/system/docker.service.d/http-proxy.conf or/etc/docker/daemon.json) to be able to pull images. Or maybe there is some another method? Unfortunately I'm not very familiar with Docker (that's why I started to work with DDEV 😊) But to answer your question: proxy config inside~/docker/config.json was not sufficient for me.


That seems enormously awkward and wrong of course.

One note: I've seen a mention that *. is sometimes invalid, and .domain is correct. However, I haven't confirmed that and don't know that Docker doesn't handle both.

I did some more tests (in~/docker/config.json):

"noProxy": "127.0.0.0/8,::1,localhost,.ddev.site,.ddev.local"
I couldn't access both my app and phpmyadmin through domain

"noProxy": "127.0.0.0/8,::1,localhost,*.ddev.site,*.ddev.local,.ddev.site,.ddev.local,ddev-,ddev-*,ddev-*-web,ddev-*-phpmyadmin"
Same as above

"noProxy": "127.0.0.0/8,::1,localhost,*.ddev.site,*.ddev.local,.ddev.site,.ddev.local,ddev-,ddev-*,ddev-*-web,ddev-*-phpmyadmin,ddev-myapp-phpmyadmin"
I could only access phpmyadmin via domain - my app was still not accessible.

Errors are coming from Squid ((111) Connection refused) - for me it looks like Traefik uses proxy data from~/docker/config.json and tries to access these containers via my proxy (when i usedocker logs ddev-router I can see these container names there):

172.18.0.1 - - [16/Mar/2025:17:11:26 +0000] "GET / HTTP/2.0" 503 4348 "-" "-" 1 "myapp-web-80-https@file" "http://ddev-myapp-web:80" 92ms172.18.0.1 - - [16/Mar/2025:17:11:28 +0000] "GET / HTTP/2.0" 503 4373 "-" "-" 2 "myapp-phpmyadmin-80-https@file" "http://ddev-myapp-phpmyadmin:80" 2ms
rfay reacted with thumbs up emoji

@rfay
Copy link
MemberAuthor

Thanks.

Just a note that DDEV does not support rootless docker.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@darrenohdarrenohdarrenoh left review comments

@tyler36tyler36tyler36 approved these changes

@stasadevstasadevstasadev approved these changes

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

5 participants
@rfay@stasadev@maciazek@darrenoh@tyler36

[8]ページ先頭

©2009-2025 Movatter.jp