Sympl issueshttps://gitlab.com/sympl.io/sympl/-/issues2023-05-12T15:40:23Zhttps://gitlab.com/sympl.io/sympl/-/issues/327letsencrypt initialisation uses incorrect e-mail address2023-05-12T15:40:23ZPaul Cammishletsencrypt initialisation uses incorrect e-mail address# Summary
When letsencrypt is initialised, if a second website has already been created, that site's domain is used to register with letsencrypt rather than the system's hostname domain.
# Steps to reproduce
1. Automatically install ...# Summary
When letsencrypt is initialised, if a second website has already been created, that site's domain is used to register with letsencrypt rather than the system's hostname domain.
# Steps to reproduce
1. Automatically install sympl on Debian 11.
2. 'sympl web create banana.DOMAIN'
3. Follow wiki instructions to rename system from localhost.localdomain to apple.DOMAIN
4. 'echo "letsencrypt" > /srv/apple.DOMAIN/config/ssl-provider'
5. 'sudo sympl-ssl --verbose --force $newhost'
# What is the current bug behavior?
When letsencrypt is run for the first time, if a website other than the default one has already been created, the wrong domain is used to register with letsencrypt
# What is the expected correct behavior?
The system hostname domain should be used
# Relevant logs and/or screenshots
```
* Examining certificates for apple.DOMAIN
SSL set 0: The certificate subject is not valid for this domain apple.DOMAIN.
SSL set 0: The certificate subject is not valid for this domain apple.DOMAIN.
No valid certificate sets found.
Fetching a new certificate from LetsEncrypt.
Created new account with email address: root@banana.DOMAIN
Requesting verification for apple.DOMAIN from https://acme-v02.api.letsencrypt.org/directory
Successfully verified apple.DOMAIN
Requesting verification for www.apple.DOMAIN from https://acme-v02.api.letsencrypt.org/directory
!! Unable to verify www.apple.DOMAIN (status: invalid)
!! Check http://www.apple.DOMAIN/.well-known/acme-challenge/V45LrunGXuYPgAU8fnsLSvQDZReL0DemhcFc0Nf0APY works.
Successfully fetched new certificate and created set 1
Rolled over to SSL set 1
```
You can see that while the correct certificate is requested (apple.DOMAIN), the wrong e-mail address (root@banana.DOMAIN) is used to register with letsencrypt.
# Possible fixes
Sorry, no idea.
/cc @kelduumhttps://gitlab.com/sympl.io/sympl/-/issues/319multiple: 'tempfile is deprecated; consider using mktemp instead.'2022-03-28T10:02:58ZPaul Cammishmultiple: 'tempfile is deprecated; consider using mktemp instead.'Reported in https://forum.sympl.host/t/tempfile-is-deprecated-messages/245
Cron weekly (and likely others) report `WARNING: tempfile is deprecated; consider using mktemp instead. ` when running the jobs.
On investigation, `tempfile` is...Reported in https://forum.sympl.host/t/tempfile-is-deprecated-messages/245
Cron weekly (and likely others) report `WARNING: tempfile is deprecated; consider using mktemp instead. ` when running the jobs.
On investigation, `tempfile` is used in:
```list
core/lib/symbiosis/config_file.rb
core/test.d/tc_utils.rb
core/test.d/tc_config_file.rb
dns/lib/symbiosis/config_files/tinydns.rb
firewall/sbin/sympl-firewall-blacklist
firewall/sbin/sympl-firewall-whitelist
firewall/sbin/sympl-firewall
firewall/test.d/tc_blacklistdb.r
ftp/test.d/tc_ftp.rb
mail/sympl/test.d/tc_poppassd.rb
mail/sympl/test.d/tc_dict_handler.rb
web/lib/symbiosis/config_files/apache.rb
web/lib/symbiosis/config_files/webalizer.rb
web/test.d/tc_apache_logger.rb
web/test.d/tb_sympl_web_configure.rb
```
More investigation is probably needed as it looks to be originating with the ruby tempfile.rb library.https://gitlab.com/sympl.io/sympl/-/issues/312sympl-firewall: iptables-persistent conflict2022-03-28T10:01:35ZPaul Cammishsympl-firewall: iptables-persistent conflictIt looks like when iptables-persistent is installed with a reasonable standard config, it can prevent DNS lookups when there's no IPv4 resolvers, which leads to the sympl-firewall hook waiting indefinitely and eventually being killed, an...It looks like when iptables-persistent is installed with a reasonable standard config, it can prevent DNS lookups when there's no IPv4 resolvers, which leads to the sympl-firewall hook waiting indefinitely and eventually being killed, and therefore no IPv6 coming up (and therefore no DNS resolution) which leads to other oddities.
Likely fix: make sure the hook doesn't stall indefinitely and instead times out.https://gitlab.com/sympl.io/sympl/-/issues/320sympl-firewall: does not play nicely with iptables-persistent2021-12-06T19:55:39ZPaul Cammishsympl-firewall: does not play nicely with iptables-persistentYou can get in an odd state if you don't have any v4 DNS resolvers and have iptables-persistent installed, where it will eventually fail to bring up the IPv6 address on the server, after timing out, and sympl-fireall will fail in an odd ...You can get in an odd state if you don't have any v4 DNS resolvers and have iptables-persistent installed, where it will eventually fail to bring up the IPv6 address on the server, after timing out, and sympl-fireall will fail in an odd was, meaning the server acts unusually.
Adding iptables-persistent (and friends) to the conflicts list should prevent this.https://gitlab.com/sympl.io/sympl/-/issues/304sympl11 - Exim configuration uses tainting workaround2021-08-13T16:12:36ZPaul Cammishsympl11 - Exim configuration uses tainting workaroundThe configuration in Exim 4.94 has introduced the concept of training for user-submitted variables.
This causes some issues with the Sympl configuration as we need to be able to read the relevant information based on the input to route ...The configuration in Exim 4.94 has introduced the concept of training for user-submitted variables.
This causes some issues with the Sympl configuration as we need to be able to read the relevant information based on the input to route mail correctly.
A workaround has been applied to the relevant parts, but this should be removed before it leaves testing.https://gitlab.com/sympl.io/sympl/-/issues/313sympl-mail: Exim deny-unusual-characters acl is a little over-strict for outg...2021-07-01T13:14:18ZPaul Cammishsympl-mail: Exim deny-unusual-characters acl is a little over-strict for outgoing mail.Non-local domains deny `%` and `!` in email addresses, although they're valid, and it seems like Xero are using `!`'s in emails in some cases.
Replacing `local_parts = ^[./|] : ^.*[@%!] : ^.*/\\.\\./` with `local_parts = ^[./|] : ^.*@ :...Non-local domains deny `%` and `!` in email addresses, although they're valid, and it seems like Xero are using `!`'s in emails in some cases.
Replacing `local_parts = ^[./|] : ^.*[@%!] : ^.*/\\.\\./` with `local_parts = ^[./|] : ^.*@ : ^.*/\\.\\./` in https://gitlab.mythic-beasts.com/sympl/sympl/-/blob/buster/mail/exim4/sympl.d/10-acl/50-acl-check-rcpt/20-deny-unusual-characters should fix this.https://gitlab.com/sympl.io/sympl/-/issues/228sympl-ssl dies when config directory is mangled2021-02-12T18:08:31ZPaul Cammishsympl-ssl dies when config directory is mangledThis looks to be a problem in Symbiosis also, but only appeared when upgrading from Symbiosis to Sympl.
What's happening is that sympl-ssl is being run, but if it has no certs for a site and a mangled config directory, it will fail and ...This looks to be a problem in Symbiosis also, but only appeared when upgrading from Symbiosis to Sympl.
What's happening is that sympl-ssl is being run, but if it has no certs for a site and a mangled config directory, it will fail and prevent the package from being configured.https://gitlab.com/sympl.io/sympl/-/issues/253sympl-test: Race condition with certificate testing2021-02-12T18:08:31ZPaul Cammishsympl-test: Race condition with certificate testingIt looks like on occasion a self-signed cert is being created, but being tested before it's valid.
Job [#9899](https://gitlab.mythic-beasts.com/sympl/sympl/-/jobs/9899) failed for 80f6dd1c78f1401f5980105fc948fa74a2f01759:
```
=========...It looks like on occasion a self-signed cert is being created, but being tested before it's valid.
Job [#9899](https://gitlab.mythic-beasts.com/sympl/sympl/-/jobs/9899) failed for 80f6dd1c78f1401f5980105fc948fa74a2f01759:
```
===============================================================================
Failure:
Exception raised:
OpenSSL::X509::CertificateError(<Not valid for rcyexz5q3p.test -- certificate is not yet valid (9)>)
test_ssl_verify_with_root_ca(SSLTest)
/etc/sympl/test.d/tc_ssl.rb:562:in `test_ssl_verify_with_root_ca'
559: #
560: assert_nothing_raised{ @domain.ssl_x509_certificate_file = @domain.directory+"/config/ssl.combined" }
561: assert_nothing_raised{ @domain.ssl_key_file = @domain.directory+"/config/ssl.combined" }
=> 562: assert_nothing_raised{ @domain.ssl_verify(@domain.ssl_x509_certificate, @domain.ssl_key, @domain.ssl_certificate_store, true) }
563: end
564:
565: def test_ssl_verify_with_intermediate_ca
===============================================================================
```https://gitlab.com/sympl.io/sympl/-/issues/303sympl-firewall: Traffic on the local IPv6 network can trigger blacklisting of...2021-01-23T17:45:17ZPaul Cammishsympl-firewall: Traffic on the local IPv6 network can trigger blacklisting of the LANSympl will track IPv6 traffic at a /64 resolution, but this means if something on the same LAN is flagged and blacklisted, it will effectively disable IPv6 traffic from the same /64, which can interfere with monitoring or similar.
What ...Sympl will track IPv6 traffic at a /64 resolution, but this means if something on the same LAN is flagged and blacklisted, it will effectively disable IPv6 traffic from the same /64, which can interfere with monitoring or similar.
What should probably happen is that Sympl is a bit more granular with it's filtering of V6 addresses on the same /64, and instead only blocks individual IPs if it sees them acting suspicious.https://gitlab.com/sympl.io/sympl/-/issues/268DKIM signature covers the sender address, but should cover the FROM HEADER ad...2020-09-24T06:23:41ZPaul CammishDKIM signature covers the sender address, but should cover the FROM HEADER address.# Summary
DKIM signatures are based on the SMTP sender address, not the email FROM HEADER address, which is the wrong thing to do. When the FROM address is local, and there's a DKIM key to sign with, then that should be done.
If there...# Summary
DKIM signatures are based on the SMTP sender address, not the email FROM HEADER address, which is the wrong thing to do. When the FROM address is local, and there's a DKIM key to sign with, then that should be done.
If there's no key to sign with, then perhaps we should not be sending the email!?
# Steps to reproduce
Send an email with a FROM address that doesn't match the SMTP sender address. You should notice that the DKIM header doesn't cover the FROM address.
/cc @kelduumhttps://gitlab.com/sympl.io/sympl/-/issues/301sympl-firewall: "Another app is currently holding the xtables lock"2020-09-17T13:30:58ZPaul Cammishsympl-firewall: "Another app is currently holding the xtables lock"One user was reporting emails like this, coming from `/usr/sbin/sympl-firewall` and `/usr/sbin/sympl-firewall-blacklist` on two hosts.
```text
From: Cron Daemon <root@hostname.fqdn>
Date: Mon, 14 Sep 2020 at 19:00
Subject: Cron <root@ho...One user was reporting emails like this, coming from `/usr/sbin/sympl-firewall` and `/usr/sbin/sympl-firewall-blacklist` on two hosts.
```text
From: Cron Daemon <root@hostname.fqdn>
Date: Mon, 14 Sep 2020 at 19:00
Subject: Cron <root@hostname> [ -x /usr/sbin/sympl-firewall ] &&
/usr/sbin/sympl-firewall
To: <root@hostname.fqdn>
Another app is currently holding the xtables lock. Perhaps you want to use
the -w option?
sympl-firewall: Firewall script failed.
sympl-firewall: Flushing /sbin/iptables rules and chains.
sympl-firewall: Flushing /sbin/ip6tables rules and chains.
sympl-firewall: Restoring old iptables rules and chains.
sympl-firewall: Restoring old ip6tables rules and chains.
sympl-firewall: Left firewall script in
/tmp/user/0/sympl-firewall-20200914-1505-1srb1j3-saved for inspection.
```
The direct cause is unclear at the moment, and they don't happen all the time (once a day or so, apparently), so it may simply be a race condition.https://gitlab.com/sympl.io/sympl/-/issues/264Default IP confusion with other services2020-04-21T21:19:55ZPaul CammishDefault IP confusion with other services# What is the current bug behavior?
When adding extra IPs manually (such as an IPv6 address), Sympl can get confused as to which is the primary IP, in cases where the IPs are listed out-of order in the output of `ip a`
# What is the e...# What is the current bug behavior?
When adding extra IPs manually (such as an IPv6 address), Sympl can get confused as to which is the primary IP, in cases where the IPs are listed out-of order in the output of `ip a`
# What is the expected correct behavior?
Sympl should probably take the IP(s) of the default domain `/srv/$HOSTNAME` as the default IP, only using the `config/ip` file to override this.
/cc @kelduumhttps://gitlab.com/sympl.io/sympl/-/issues/288sympl-core: Man page broken for sympl-ssl2020-04-20T11:37:33ZPaul Cammishsympl-core: Man page broken for sympl-sslThis is likely as the sympl-ssl man pages are built from the ruby normally, which currently has a wrapper to fix IPv6 support.
This should be fixed soon if possible, otherwise it will be fixed as part of issue #278.This is likely as the sympl-ssl man pages are built from the ruby normally, which currently has a wrapper to fix IPv6 support.
This should be fixed soon if possible, otherwise it will be fixed as part of issue #278.https://gitlab.com/sympl.io/sympl/-/issues/287sympl-mail: Man pages broken for support scripts2020-04-20T11:35:18ZPaul Cammishsympl-mail: Man pages broken for support scriptsThe man pages for:
```text
sympl-mail-dict-proxy
sympl-mail-encrypt-passwords
sympl-mail-poppassd
```
Only contain the Groff header, and should be fixed.The man pages for:
```text
sympl-mail-dict-proxy
sympl-mail-encrypt-passwords
sympl-mail-poppassd
```
Only contain the Groff header, and should be fixed.https://gitlab.com/sympl.io/sympl/-/issues/258Occasional short-term failures reported by monitoring2019-07-31T17:52:47ZPaul CammishOccasional short-term failures reported by monitoringRecently received the following report from the automatic monitoring. It resolved itself a few minutes later.
[paste_1093477.txt](/uploads/b13f16c409a7c2c5791a95e3d7601585/paste_1093477.txt)
I've seen similar short-term failures a coup...Recently received the following report from the automatic monitoring. It resolved itself a few minutes later.
[paste_1093477.txt](/uploads/b13f16c409a7c2c5791a95e3d7601585/paste_1093477.txt)
I've seen similar short-term failures a couple of timeshttps://gitlab.com/sympl.io/sympl/-/issues/176Symbiosis: SSL symlinks broken on hostname change2019-07-17T15:54:09ZPaul CammishSymbiosis: SSL symlinks broken on hostname changeImported from https://www.github.com/BytemarkHosting/symbiosis/issues/42
When the hostname of a system running Symbiosis is changed the symbolic links for the self signed certificates in <code>/etc/ssl</code> are broken.
The symbolic l...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/42
When the hostname of a system running Symbiosis is changed the symbolic links for the self signed certificates in <code>/etc/ssl</code> are broken.
The symbolic links in <code>/etc/ssl</code> continue to point at <code>/srv/original-server-name/...</code>.
This then prevents the Apache service from starting as the SSL files are missing/invalid.Backloghttps://gitlab.com/sympl.io/sympl/-/issues/144Symbiosis: If an SSL cert is automatically disabled, Symbiosis won't use auto...2019-07-17T15:53:24ZPaul CammishSymbiosis: If an SSL cert is automatically disabled, Symbiosis won't use automatically it again if it becomes validImported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` an...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` and moving the `config/ssl directory` out of the way, `symbiosis-httpd-configure` will disable the specific SSL cert for the site, swapping it to self-signed.
This is fine, and to be expected, however it does this by removing the relevant symlink from `/etc/apache2/sites-enabled`, which has the effect of flagging the site as "manually disabled", dropping it back to mass hosting, if configured.
Restoring the SSL configuration (removing `ssl-provider` and restoring `config/ssl`) then re-running `symbiosis-httpd-configure --verbose` you get:
```
# symbiosis-httpd-configure --verbose
[ . . . ]
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
[ . . . ]
Configuration: under100words.com.conf
Configuration is up-to date.
!! Configuration has been manually disabled.
```
So, it's still thinking that the site was manually disabled, so even if it managed to create the individual config as there are valid SSL certs, it's not being symlinked.
A manual workaround is to run `symbiosis-httpd-configure` for the specific site:
```
# symbiosis-httpd-configure --verbose under100words.com
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
Configuration: under100words.com.conf
Configuration is up-to date.
Enabling configuration.
Reloading Apache
```
This instead enables the config anyway, and things work normally again.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/256sympl-firewall: Failed to acquire lock on /var/lock/sympl-firewall.lock2019-07-17T15:47:39ZPaul Cammishsympl-firewall: Failed to acquire lock on /var/lock/sympl-firewall.lockJob [#11106](https://gitlab.mythic-beasts.com/sympl/sympl/-/jobs/11106) failed for 89b35a9928e5c77aa5ea832fb4a0e851cc3cd601:
```
+ symbiosis-firewall --verbose
sympl-firewall: Failed to acquire lock on /var/lock/sympl-firewall.lock: No ...Job [#11106](https://gitlab.mythic-beasts.com/sympl/sympl/-/jobs/11106) failed for 89b35a9928e5c77aa5ea832fb4a0e851cc3cd601:
```
+ symbiosis-firewall --verbose
sympl-firewall: Failed to acquire lock on /var/lock/sympl-firewall.lock: No locks available - Unable to acquire lock -- Resource temporarily unavailable
run-parts: autotest/test.d/50-test-cli exited with return code 1
ERROR: Job failed: Process exited with: 1. Reason was: ()
```
This is the testing job accidentally aligning with a scheduled run - it's only a few seconds of window, but it happens more often than I'd like.https://gitlab.com/sympl.io/sympl/-/issues/184Symbiosis: symbiosis-httpd-configure breaks when no certificates are available.2019-06-07T14:36:24ZPaul CammishSymbiosis: symbiosis-httpd-configure breaks when no certificates are available.Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/112
symbiosis-httpd-configure breaks when no certificates are available.
Steps to reproduce:
1. Spin up a new Symbiosis 8 server at panel.bytemark.co.uk
2. remove t...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/112
symbiosis-httpd-configure breaks when no certificates are available.
Steps to reproduce:
1. Spin up a new Symbiosis 8 server at panel.bytemark.co.uk
2. remove the contents of /etc/ss
3. run "symbiosis-httpd-configure --force --verbose"
4. run "apachectl restart"
Expected behaviour:
* Apache restarts.
Observed behaviour:
* Apache fails to restart. An error message like "Failed to configure at least one certificate and key for symbiosis.foo.bar.uk0.bigv.io:443"
Workaround:
* remove /etc/apache2/sites-enabled/zz-mass-hosting.ssl.conf - Apache will now start, until symbiosis-httpd-configure is run again.
Suggested fix:
* Symbiosis-httpd-configure should test for the presence of some certificate before proceeding to enable zz-mass-hosting.ssl.confBackloghttps://gitlab.com/sympl.io/sympl/-/issues/166Symbiosis: reject-www-data rule unintentionally removed from ip6tables when f...2019-06-07T14:36:07ZPaul CammishSymbiosis: reject-www-data rule unintentionally removed from ip6tables when file contains only IPv4 addressesImported from https://www.github.com/BytemarkHosting/symbiosis/issues/76
When an IPv4 address is added to the reject-www-data rule, the rule is removed from ip6tables.
1. Run `ip6tables -L -v -n` and notice the reject-www-data table is...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/76
When an IPv4 address is added to the reject-www-data rule, the rule is removed from ip6tables.
1. Run `ip6tables -L -v -n` and notice the reject-www-data table is present
2. Add 10.0.0.1 to `/etc/symbiosis/firewall/outgoing.d/50-reject-www-data`
3. Run `ip6tables -L -v -n` and notice the reject-www-data table is *no longer* presentBacklog