squid-deb-proxy ou archive.ubuntu.com, un problème?

Question

Bonjour, j'ai un problème dans ces 6 mois en essayant de mettre à jour mes systèmes de bureau.
Désolé, je ne peux pas dire s’il s’agit d’un problème temporaire du serveur archive.ubuntu.com ou d’un problème sur mon paquet squid-deb-proxy.

en regardant /var/log/squid-deb-proxy/access.log et la sortie de mise à niveau d'apt, je vois:

depuis la sortie d'apt:

Ign:42 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 libudev1 AMD64 229-4ubuntu21.1 Ign:41 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 libpam-systemd AMD64 229-4ubuntu21.1 Ign:44 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 udev AMD64 229-4ubuntu21.1 Ign:43 http://archive.ubuntu.com/ubuntu xenial-security/main i386 libudev1 i386 229-4ubuntu21.1 Ign:46 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 libsystemd0 AMD64 229-4ubuntu21.1 Ign:45 http://archive.ubuntu.com/ubuntu xenial-security/main i386 libsystemd0 i386 229-4ubuntu21.1 Ign:48 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 systemd-sysv AMD64 229-4ubuntu21.1 Ign:47 http://archive.ubuntu.com/ubuntu xenial-security/main AMD64 systemd AMD64 229-4ubuntu21.1 Ign:50 http://archive.ubuntu.com/ubuntu xenial-security/universe AMD64 chromium-codecs-ffmpeg-extra AMD64 64.0.3282.140-0ubuntu0.16.04.1 Ign:49 http://archive.ubuntu.com/ubuntu xenial-proposed/main AMD64 libapparmor1 AMD64 2.10.95-0ubuntu2.8 Err:52 http://archive.ubuntu.com/ubuntu xenial-proposed/main i386 gcc-5-base i386 5.4.0-6ubuntu1~16.04.9 Connessione terminata Err:51 http://151.99.86.73:80/data/0189f472fb16f092/archive.ubuntu.com/ubuntu xenial-proposed/main AMD64 gcc-5-base AMD64 5.4.0-6ubuntu1~16.04.9 403 Forbidden

à partir de /var/log/squid-deb-proxy/access.log:

1518428930.420 96 192.168.1.107 TCP_REFRESH_MODIFIED/302 445 GET http://archive.ubuntu.com/ubuntu/pool/main/s/systemd/libudev1_229-4ubuntu21.1_i386.deb - HIER_DIRECT/91.189.88.149 text/html 1518428931.389 32 192.168.1.107 TCP_DENIED/403 4233 GET http://151.99.86.73/data/028979686fecdd00/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libpam-systemd_229-4ubuntu21.1_AMD64.deb - HIER_NONE/- text/html 1518429050.519 0 192.168.1.107 TCP_DENIED/403 4212 GET http://151.99.86.73/data/00894e6ad5c1296b/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libudev1_229-4ubuntu21.1_i386.deb - HIER_NONE/- text/html 1518429050.577 57 192.168.1.107 TCP_REFRESH_MODIFIED/302 448 GET http://archive.ubuntu.com/ubuntu/pool/main/s/systemd/libsystemd0_229-4ubuntu21.1_i386.deb - HIER_DIRECT/91.189.88.149 text/html 1518429170.721 68 192.168.1.107 TCP_REFRESH_MODIFIED/302 445 GET http://archive.ubuntu.com/ubuntu/pool/main/s/systemd/systemd_229-4ubuntu21.1_AMD64.deb - HIER_DIRECT/91.189.88.152 text/html 1518429176.717 33 192.168.1.107 TCP_DENIED/403 4221 GET http://151.99.86.75/data/0089936cbd9662c9/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libsystemd0_229-4ubuntu21.1_i386.deb - HIER_NONE/- text/html 1518429290.819 0 192.168.1.107 TCP_DENIED/403 4212 GET http://151.99.86.73/data/01891f6e2a6c5119/archive.ubuntu.com/ubuntu/pool/main/s/systemd/systemd_229-4ubuntu21.1_AMD64.deb - HIER_NONE/- text/html 1518429290.875 56 192.168.1.107 TCP_REFRESH_MODIFIED/302 454 GET http://archive.ubuntu.com/ubuntu/pool/main/a/apparmor/libapparmor1_2.10.95-0ubuntu2.8_AMD64.deb - HIER_DIRECT/91.189.88.152 text/html 1518429410.878 0 192.168.1.107 TCP_DENIED/403 4239 GET http://151.99.86.73/data/0189e8705841ea73/archive.ubuntu.com/ubuntu/pool/main/a/apparmor/libapparmor1_2.10.95-0ubuntu2.8_AMD64.deb - HIER_NONE/- text/html 1518429410.969 91 192.168.1.107 TCP_REFRESH_MODIFIED/302 455 GET http://archive.ubuntu.com/ubuntu/pool/main/g/gcc-5/gcc-5-base_5.4.0-6ubuntu1%7e16.04.9_AMD64.deb - HIER_DIRECT/91.189.88.152 text/html 1518429531.019 0 192.168.1.107 TCP_DENIED/403 4244 GET http://151.99.86.73/data/0189f472fb16f092/archive.ubuntu.com/ubuntu/pool/main/g/gcc-5/gcc-5-base_5.4.0-6ubuntu1%7e16.04.9_AMD64.deb - HIER_NONE/- text/html

Le système se bloque en attente de données. Est-ce un problème bien connu?

Ma configuration: à partir de /etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default

.dl.google.com .ppa.launchpad.net .downloads-distro.mongodb.org # tutti questi sono per getdeb .getdeb.net .mirror.lagoon.nc .ftp.heanet.ie .getdeb.mirror.garr.it .archive.getdeb.net .getdeb.megaglest.org .mirrors.dotsrc.org .zulu930.server4you.de .188.138.90.189 .ftp.yzu.edu.tw # fine getdeb .apt.puppetlabs.com .deb.theforeman.org # postgresql.org .postgresql.org #owncloud .download.owncloud.org .download.opensuse.org #tox .pkg.tox.chat #variety .151.99.86.77

à partir de /etc/squid-deb-proxy/squid-deb-proxy.conf

acl allowed_networks src "/etc/squid-deb-proxy/autogenerated/allowed-networks-src.acl" # this file contains the archive mirrors by default, # if you use a different mirror, add it there acl to_archive_mirrors dstdomain "/etc/squid-deb-proxy/autogenerated/mirror-dstdomain.acl" # this contains the package blacklist acl blockedpkgs urlpath_regex "/etc/squid-deb-proxy/autogenerated/pkg-blacklist-regexp.acl" # default to a different port than stock squid http_port 8000 # ------------------------------------------------- # settings below probably do not need customization # user visible name visible_hostname squid-deb-proxy # we need a big cache, some debs are huge # maximum_object_size 512 MB # leonardo maximum_object_size 2048 MB # use a different dir than stock squid and default to 40G # cache_dir aufs /var/cache/squid-deb-proxy 40000 16 256 # leonardo cache_dir aufs /var/cache/squid-deb-proxy 100000 16 256 # use different logs cache_access_log /var/log/squid-deb-proxy/access.log cache_log /var/log/squid-deb-proxy/cache.log cache_store_log /var/log/squid-deb-proxy/store.log # tweaks to speed things up cache_mem 200 MB maximum_object_size_in_memory 10240 KB # pid pid_filename /var/run/squid-deb-proxy.pid # refresh pattern for debs and udebs refresh_pattern deb$ 129600 100% 129600 refresh_pattern udeb$ 129600 100% 129600 refresh_pattern tar.gz$ 129600 100% 129600 refresh_pattern tar.xz$ 129600 100% 129600 refresh_pattern tar.bz2$ 129600 100% 129600 # always refresh Packages and Release files refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims refresh_pattern \/InRelease$ 0 0% 0 refresh-ims refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims # handle meta-release and changelogs.ubuntu.com special # (fine to have this on debian too) refresh_pattern changelogs.ubuntu.com\/.* 0 1% 1 # only allow connects to ports for http, https acl Safe_ports port 80 acl Safe_ports port 443 563 # only allow ports we trust http_access deny !Safe_ports # do not allow to download from the pkg blacklist http_access deny blockedpkgs # allow access only to official archive mirrors # uncomment the third and fouth line to permit any unlisted domain http_access deny !to_archive_mirrors #http_access allow !to_archive_mirrors # don't cache domains not listed in the mirrors file # uncomment the third and fourth line to cache any unlisted domains cache deny !to_archive_mirrors #cache allow !to_archive_mirrors # allow access from our network and localhost http_access allow allowed_networks # And finally deny all other access to this proxy http_access deny all # we don't want to clash with the squid netdb state file netdb_filename stdio:/var/log/squid-deb-proxy/netdb.state

de squid-deb-proxy/allowed-networks-src.acl:

# private networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 192.168.1.0/16 127.0.0.1 # IPv6 private addresses fe80::/64 ::1/128 # additional non-private networks can be added to the directory: # /etc/squid-deb-proxy/allowed-networks-src.acl.d

à partir de /etc/squid-deb-proxy/allowed-networks-src.acl.d/10-default

188.138.90.189/17

Je ne sais pas quoi faire.

Leonardo Saracini · Accepted Answer

Le problème est résolu mais la raison en est un mystère.

Sur mon /etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default je dois ajouter un ip:

151.99.86.73
151.99.86.75
151.99.86.77
151.99.86.79

La plupart du temps sont nécessaires car certains téléchargements sur archive.ubuntu.com résolvent en 151.99.86.x/data/<exanumber>/archive.ubuntu.com/ubuntu/...

ainsi, lorsqu'une nouvelle erreur signalant 151.99.86.x apparaît, j'ajoute une nouvelle adresse IP au fichier acl.

J'aime comprendre pourquoi une adresse de domaine n'est pas utilisée à moins que 151.99.86.x et j'aime comprendre s'il existe une meilleure solution que d'ajouter une IP au moment où le problème se pose.

cordialement, Leonardo

Olaf Dietsche · Answer

Les parties pertinentes semblent être liées à IP 151.99.86.73

Err:51 http://151.99.86.73:80/data/0189f472fb16f092/archive.ubuntu.com/ubuntu xenial-proposed/main AMD64 gcc-5-base AMD64 5.4.0-6ubuntu1~16.04.9 403 Forbidden

1518428931.389 32 192.168.1.107 TCP_DENIED/403 4233 GET http://151.99.86.73/data/028979686fecdd00/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libpam-systemd_229-4ubuntu21.1_AMD64.deb - HIER_NONE/- text/html 1518429050.519 0 192.168.1.107 TCP_DENIED/403 4212 GET http://151.99.86.73/data/00894e6ad5c1296b/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libudev1_229-4ubuntu21.1_i386.deb - HIER_NONE/- text/html 1518429176.717 33 192.168.1.107 TCP_DENIED/403 4221 GET http://151.99.86.75/data/0089936cbd9662c9/archive.ubuntu.com/ubuntu/pool/main/s/systemd/libsystemd0_229-4ubuntu21.1_i386.deb - HIER_NONE/- text/html 1518429290.819 0 192.168.1.107 TCP_DENIED/403 4212 GET http://151.99.86.73/data/01891f6e2a6c5119/archive.ubuntu.com/ubuntu/pool/main/s/systemd/systemd_229-4ubuntu21.1_AMD64.deb - HIER_NONE/- text/html 1518429410.878 0 192.168.1.107 TCP_DENIED/403 4239 GET http://151.99.86.73/data/0189e8705841ea73/archive.ubuntu.com/ubuntu/pool/main/a/apparmor/libapparmor1_2.10.95-0ubuntu2.8_AMD64.deb - HIER_NONE/- text/html 1518429531.019 0 192.168.1.107 TCP_DENIED/403 4244 GET http://151.99.86.73/data/0189f472fb16f092/archive.ubuntu.com/ubuntu/pool/main/g/gcc-5/gcc-5-base_5.4.0-6ubuntu1%7e16.04.9_AMD64.deb - HIER_NONE/- text/html

Ces demandes sont refusées, car la destination (151.99.86.73) n'est pas répertoriée comme domaine autorisé. Ajouter cette adresse IP à /etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default devrait résoudre le problème. Ajouter une ligne contenant

151.99.86.73

et tu as fini.

Leonardo Saracini · Answer

sur /etc/squid-deb-proxy/mirror-dstdomain.acl J'ai supprimé le point '.' devant le domaine.

comme cela .archive.ubuntu.com devenir -> archive.ubuntu.com

le premier test se passe bien.

J'essaie pendant un certain temps jusqu'à ce que je puisse voir si l'erreur se reproduit.