Sympl issueshttps://gitlab.com/sympl.io/sympl/-/issues2019-04-14T20:58:39Zhttps://gitlab.com/sympl.io/sympl/-/issues/160Symbiosis: Optional symbiosis-httpd-logger package fails under load2019-04-14T20:58:39ZPaul CammishSymbiosis: Optional symbiosis-httpd-logger package fails under loadImported from https://www.github.com/BytemarkHosting/symbiosis/issues/113
Under high loads with a large number of sites (and therefore large number of instances of the logger), the symbiosis-httpd-logger process stalls and fails to writ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/113
Under high loads with a large number of sites (and therefore large number of instances of the logger), the symbiosis-httpd-logger process stalls and fails to write logs, apparently causing Apache to stall and spin up more processes to deal with the incoming traffic.
A workaround for this seems to be to be to adjust the Apache templates for HTTP/HTTPS sites to log directly to disk, saving the extra processor time and RAM, however this means the `logs` directories won't be automatically generated (should be fixable in symbiosis-httpd-configure or elsewhere, and log files will not be owned by admin:admin (which is not a huge problem, as they are cycled automatically, and still readable by admin).https://gitlab.com/sympl.io/sympl/-/issues/159Symbiosis: On Stretch, man pages for symbiosis-common scripts are empty2019-04-14T20:53:51ZPaul CammishSymbiosis: On Stretch, man pages for symbiosis-common scripts are emptyImported from https://www.github.com/BytemarkHosting/symbiosis/issues/117
Looks like a problem finding ruby libraries during generation of the packages.Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/117
Looks like a problem finding ruby libraries during generation of the packages.https://gitlab.com/sympl.io/sympl/-/issues/158Symbiosis: On Stretch, httpd.postinst doesn't correctly preserve `no-stats` s...2019-06-07T10:51:39ZPaul CammishSymbiosis: On Stretch, httpd.postinst doesn't correctly preserve `no-stats` settingsImported from https://www.github.com/BytemarkHosting/symbiosis/issues/124
This is what I think should happen:
1. If `no-stats` is present and not set to `false`: remove, as this is the default now.
2. If `no-stats` is present and set t...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/124
This is what I think should happen:
1. If `no-stats` is present and not set to `false`: remove, as this is the default now.
2. If `no-stats` is present and set to `false`: move to `stats` and truncate, ensuring stats are enabled.
3. If `no-stats` isn't present: create `stats`.
4. Otherwise do nothing.
Patrick advised that we can potentially not do (3) and just put in release notes that the default is now that stats are disabled by default, as we use webalizer which is old and clunky and potentially many customers don't use it.Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/157Symbiosis: Mysql user of admin with admin password for mysql access2019-06-20T13:21:36ZPaul CammishSymbiosis: Mysql user of admin with admin password for mysql accessImported from https://www.github.com/BytemarkHosting/symbiosis/issues/61
Just had a customer who processed a migration from symb6 to symb8 and managed to overwrite the mysql db in the process.
Wondered if there might be some traction i...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/61
Just had a customer who processed a migration from symb6 to symb8 and managed to overwrite the mysql db in the process.
Wondered if there might be some traction in having an admin user with the admin password as a mysql user that we can control access for a little more, no access to the mysql folder for example...
Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/155Symbiosis: monit: Use systemd timer to launch instead of cron2019-04-14T20:52:49ZPaul CammishSymbiosis: monit: Use systemd timer to launch instead of cronImported from https://www.github.com/BytemarkHosting/symbiosis/issues/55
This requires a .timer as well as a .service file](https://wiki.archlinux.org/index.php/Systemd/Timers). It also means that sysvinit isn't really supported, unles...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/55
This requires a .timer as well as a .service file](https://wiki.archlinux.org/index.php/Systemd/Timers). It also means that sysvinit isn't really supported, unless a check is put in place to remove the cron job or otherwise disable it.https://gitlab.com/sympl.io/sympl/-/issues/154Symbiosis: Missing error document handling for non-mass-hosted domains2019-04-16T22:18:11ZPaul CammishSymbiosis: Missing error document handling for non-mass-hosted domainsImported from https://www.github.com/BytemarkHosting/symbiosis/issues/79
Relating to https://www.github.com/BytemarkHosting/symbiosis/issues/63 - the default site is configured using ErrorDocument in zz-masshosting.conf and zz-masshosti...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/79
Relating to https://www.github.com/BytemarkHosting/symbiosis/issues/63 - the default site is configured using ErrorDocument in zz-masshosting.conf and zz-masshosting.ssl.conf, but is not in the template for other sites.
When I installed symbiosis-stretch on a new virtualmachine it created `/etc/apache2/sites-enabled/symbiosis-stretch.work.telyn.uk0.bigv.io.conf`. I suspect this is unintended and possibly to do with how we do SSL now?https://gitlab.com/sympl.io/sympl/-/issues/153Symbiosis: Migration assistant2019-04-14T20:37:18ZPaul CammishSymbiosis: Migration assistantImported from https://www.github.com/BytemarkHosting/symbiosis/issues/60
We spend a lot of time helping customers with migrations, or advising that they migrate to avoid a dist-upgrade.
It would be nice if Symbiosis Stretch included a ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/60
We spend a lot of time helping customers with migrations, or advising that they migrate to avoid a dist-upgrade.
It would be nice if Symbiosis Stretch included a script to migrate all domains from Symbiosis Jessie (and maybe Wheezy). Something based on this migration guide would be useful. https://forum.bytemark.co.uk/t/the-symbiosis-migration-guide/2172
It should do as much of that as possible, and carry suitable health warnings, perhaps.
Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/151Symbiosis: Masked potentially dovecot service break upgrades2019-04-17T20:26:34ZPaul CammishSymbiosis: Masked potentially dovecot service break upgradesImported from https://www.github.com/BytemarkHosting/symbiosis/issues/77
(Reading database ... 55746 files and directories currently installed.)
Preparing to unpack .../symbiosis-xmpp_2015%3a1026_all.deb ...
Unpacking symbio...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/77
(Reading database ... 55746 files and directories currently installed.)
Preparing to unpack .../symbiosis-xmpp_2015%3a1026_all.deb ...
Unpacking symbiosis-xmpp (2015:1026) over (2015:1024) ...
service dovecot restart
Failed to restart dovecot.service: Unit dovecot.service is masked.
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 1
dpkg: warning: subprocess old post-removal script returned error exit status 2
dpkg: trying script from the new package instead ...
service dovecot restart
It no longer needs to re-start Dovecot :)https://gitlab.com/sympl.io/sympl/-/issues/150Symbiosis: MariaDB breaks phpMyAdmin authentication in Stretch2019-04-14T20:44:29ZPaul CammishSymbiosis: MariaDB breaks phpMyAdmin authentication in StretchImported from https://www.github.com/BytemarkHosting/symbiosis/issues/126
MariaDB uses unix socket authentication by default, rather than username/password. As such, phpMyAdmin HTTP authentication won't accept the credentials for the `r...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/126
MariaDB uses unix socket authentication by default, rather than username/password. As such, phpMyAdmin HTTP authentication won't accept the credentials for the `root@localhost` MySQL user.
Likely fix would be to either disable unix socket auth, or create an additional MySQL user which uses username/password authentication.https://gitlab.com/sympl.io/sympl/-/issues/149Symbiosis: Logrotate cron error for prosody when it's not running2019-06-07T14:25:51ZPaul CammishSymbiosis: Logrotate cron error for prosody when it's not runningImported from https://www.github.com/BytemarkHosting/symbiosis/issues/131
The logrotate cron will email the following warning every week if prosody isn't active:
<pre>
/etc/cron.daily/logrotate:
error: error running shared postrotate s...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/131
The logrotate cron will email the following warning every week if prosody isn't active:
<pre>
/etc/cron.daily/logrotate:
error: error running shared postrotate script for
'/var/log/prosody/prosody.log /var/log/prosody/prosody.err '
run-parts: /etc/cron.daily/logrotate exited with return code 1
</pre>
It looks like this is because the postrotate tries to check for the existence of `/var/run/prosody/prosody.pid` which won't be there when prosody is disabled (by default):
<pre>
[ -e /var/run/prosody/prosody.pid ] && /etc/init.d/prosody reload > /dev/null
</pre>
We should be able to suppress that by changing this line to e.g
<pre>
/etc/init.d/prosody reload > /dev/null
</pre>Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/148Symbiosis: Log files not created in /srv/site.com/public/logs/2019-06-20T13:20:10ZPaul CammishSymbiosis: Log files not created in /srv/site.com/public/logs/Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/140
I've recently setup Symbiosis on a Digital Ocean droplet to test some things. The initial setup worked perfectly on the second attempt. But I have a few questions...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/140
I've recently setup Symbiosis on a Digital Ocean droplet to test some things. The initial setup worked perfectly on the second attempt. But I have a few questions:
1) Is there a recommended approach for handling the DNS side? I found that adding the server's IP into /srv/mysite.com/config/ip did the trick along with a real email address to ensure Letsencrypt works as expected.
2) Curious also that I don't see any log files.
/srv/mysite.com/public/logs hasn't even been created. EDIT: Seems like symbiosis-httpd-logger is not running ...
Can anyone illuminate?Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/147Symbiosis: It's too easy to break Exim by changing ssl certificate ownership.2019-06-20T13:19:51ZPaul CammishSymbiosis: It's too easy to break Exim by changing ssl certificate ownership.Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/47
Pretty much everything in /srv/ is owned by admin:admin, so it's tempting to run something like "chown -R admin:admin /srv". The problem is that Exim certificates ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/47
Pretty much everything in /srv/ is owned by admin:admin, so it's tempting to run something like "chown -R admin:admin /srv". The problem is that Exim certificates lie in /srv/<HOSTNAME>/config/ssl/sets and Debian-exim (the user that runs Exim) is not a member of the admin group, so this is an awkward fact to learn and remember.
It might be better if the certificates were managed in /etc/ssl - from where they are currently, and tortuously symlinked.
Alternatively, if issue 38 https://gitlab.bytemark.co.uk/open-source/symbiosis/issues/38 is implemented, then I've made a suggestion for managing these certs.Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/144Symbiosis: If an SSL cert is automatically disabled, Symbiosis won't use auto...2019-07-17T15:53:24ZPaul CammishSymbiosis: If an SSL cert is automatically disabled, Symbiosis won't use automatically it again if it becomes validImported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` an...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` and moving the `config/ssl directory` out of the way, `symbiosis-httpd-configure` will disable the specific SSL cert for the site, swapping it to self-signed.
This is fine, and to be expected, however it does this by removing the relevant symlink from `/etc/apache2/sites-enabled`, which has the effect of flagging the site as "manually disabled", dropping it back to mass hosting, if configured.
Restoring the SSL configuration (removing `ssl-provider` and restoring `config/ssl`) then re-running `symbiosis-httpd-configure --verbose` you get:
```
# symbiosis-httpd-configure --verbose
[ . . . ]
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
[ . . . ]
Configuration: under100words.com.conf
Configuration is up-to date.
!! Configuration has been manually disabled.
```
So, it's still thinking that the site was manually disabled, so even if it managed to create the individual config as there are valid SSL certs, it's not being symlinked.
A manual workaround is to run `symbiosis-httpd-configure` for the specific site:
```
# symbiosis-httpd-configure --verbose under100words.com
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
Configuration: under100words.com.conf
Configuration is up-to date.
Enabling configuration.
Reloading Apache
```
This instead enables the config anyway, and things work normally again.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/143Symbiosis: I want SSL only without HSTS2020-07-11T06:36:30ZPaul CammishSymbiosis: I want SSL only without HSTSImported from https://www.github.com/BytemarkHosting/symbiosis/issues/66
I want a back-out path from ssl-only. Currently, if I deploy SSL only HSTS headers get issued, which mean I have no way to back out if I have problems with certifi...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/66
I want a back-out path from ssl-only. Currently, if I deploy SSL only HSTS headers get issued, which mean I have no way to back out if I have problems with certificate renewal or spot a problem with the way the SSL site renders
So, maybe I could make a file `config/ssl-only-no-sts` to get ssl throughout the site, and when I'm confident that I can commit to this configuration, then deploy STS.Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/142Symbiosis: Frequently used packages aren't installed by default2019-06-20T13:42:39ZPaul CammishSymbiosis: Frequently used packages aren't installed by defaultImported from https://www.github.com/BytemarkHosting/symbiosis/issues/120
There are a number of packages which are typically installed manually on a newly created Symbiosis server as a first priority, since they're used so often. The fo...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/120
There are a number of packages which are typically installed manually on a newly created Symbiosis server as a first priority, since they're used so often. The following packages are sure bets to be installed in most cases:
`curl, iotop, less, lsof, psmisc, rsync, screen, smartmontools, telnet, vim, wget, htop, mtr-tiny, xfsprogs, tree, dnsutils`
It would be useful if we could add these packages as a Symbiosis dependency to ensure they're installed automatically.Sympl v9.0 (for Debian Stretch)https://gitlab.com/sympl.io/sympl/-/issues/141Symbiosis: Exim can't deliver to a virgin mailbox2019-06-07T14:36:02ZPaul CammishSymbiosis: Exim can't deliver to a virgin mailboxImported from https://www.github.com/BytemarkHosting/symbiosis/issues/132
Mostly, Exim hands off email to dovecot for delivery. BUT, it's possible to use an Exim filter file to ask Exim to deliver email directly to a mailbox.
If Exim t...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/132
Mostly, Exim hands off email to dovecot for delivery. BUT, it's possible to use an Exim filter file to ask Exim to deliver email directly to a mailbox.
If Exim tries to do that before Dovecot has delivered to that user, Exim will fail.
Dovecot lazily (ie, when it first delivers an email to a user) creates a quota file in the root of the user's mailbox. Exim can't deliver email to a user if that quota file is missing. And it can't create it either.
This only matters if the user doesn't get mail delivered by dovecot, which is kind of unusual. The simple work-around is just to send an unfiltered email to the user.
A better fix might be to have a cron job looking for missing quota files, and adding them where required. Or maybe there's an Exim option to ignore the missing file? Or something.Backloghttps://gitlab.com/sympl.io/sympl/-/issues/140Symbiosis: Exim "Warning: purging the environment" error on restart2019-04-14T20:30:56ZPaul CammishSymbiosis: Exim "Warning: purging the environment" error on restartImported from https://www.github.com/BytemarkHosting/symbiosis/issues/114
Exim prints the following warning message whenever the service is restarted:
`Jun 18 14:12:19 symbiosis2.default.aladlow.uk0.bigv.io exim4[573]: Starting MTA:201...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/114
Exim prints the following warning message whenever the service is restarted:
`Jun 18 14:12:19 symbiosis2.default.aladlow.uk0.bigv.io exim4[573]: Starting MTA:2018-06-18 14:12:19 Warning: purging the environment.`
`Jun 18 14:12:19 symbiosis2.default.aladlow.uk0.bigv.io exim4[573]: Suggested action: use keep_environment.`
This message is fairly harmless but can cause emails to be generated from `cron.daily`, for example:
```
/etc/cron.daily/exim4-base:
LOG: MAIN
Warning: purging the environment.
Suggested action: use keep_environment.
```
The solution should be to set `keep_environment =` in Exim's configuration.https://gitlab.com/sympl.io/sympl/-/issues/139Symbiosis: Event bus for configuration changes2019-04-14T20:18:28ZPaul CammishSymbiosis: Event bus for configuration changesImported from https://www.github.com/BytemarkHosting/symbiosis/issues/45
A common problem in developing Symbiosis functionality is the difficulty in detecting configuration changes. Currently Symbiosis is configured using SFTP, but has ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/45
A common problem in developing Symbiosis functionality is the difficulty in detecting configuration changes. Currently Symbiosis is configured using SFTP, but has no hooks into that system, so it's always going to be somewhat hobbled.
It would be good to have some sort of event bus to detect configuration changes, and act upon them at the time. This removes the need for polling which is wasteful and not very timely in some cases. Using a more event based approach would make configuration changes more or less instant, and maybe more reliable too.
One way of achieving this would be to have hooks into the SFTP subsystem. OpenSSH doesn't seem to have them, others do, ie https://www.npmjs.com/package/ssh
Whilst not advocating the addition of node into the stack, it would be good to have this functionality somehow, so it's open for debate.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/137Symbiosis: Don't crash if a password file is empty2019-06-07T14:31:05ZPaul CammishSymbiosis: Don't crash if a password file is emptyImported from https://www.github.com/BytemarkHosting/symbiosis/issues/110
As reported here:
* https://forum.bytemark.co.uk/t/empty-password-crashes-cron-job/2744
The following code reproduces the problem:
```ruby
#!/usr/bin/ruby
requ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/110
As reported here:
* https://forum.bytemark.co.uk/t/empty-password-crashes-cron-job/2744
The following code reproduces the problem:
```ruby
#!/usr/bin/ruby
require 'cracklib'
c = CrackLib::Fascist(nil)
if c.ok?
puts "OK"
end
```
The following patch is probably sufficient to resolve the problem, but requires a test-case:
```
--- a/common/sbin/symbiosis-password-test
+++ b/common/sbin/symbiosis-password-test
@@ -155,6 +155,7 @@ Symbiosis::Domains.each(prefix) do |domain|
end
ftp_users.each do |u|
+ next if c.nil?
c = CrackLib::Fascist(u.password)
if c.ok?
```Backloghttps://gitlab.com/sympl.io/sympl/-/issues/135Symbiosis: DNS service records not created even though mailbox folders are there2019-06-07T14:30:27ZPaul CammishSymbiosis: DNS service records not created even though mailbox folders are thereImported from https://www.github.com/BytemarkHosting/symbiosis/issues/133
DNS srv records are not being created by the symbiosis-dns-generate command, the template suggests these are created at the presence of a mailbox folder:
```
%if...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/133
DNS srv records are not being created by the symbiosis-dns-generate command, the template suggests these are created at the presence of a mailbox folder:
```
%if domain.respond_to?(:mailboxes) and domain.mailboxes.length > 0
#
# SRV records for various mail services
#
:_submission._tcp.<%= domain %>:33:<%= domain.srv_record_for(0,5,587, "mail."+domain) %>:<%= ttl %>
:_imap._tcp.<%= domain %>:33:<%= domain.srv_record_for(0,5,143, "mail."+domain) %>:<%= ttl %>
:_imaps._tcp.<%= domain %>:33:<%= domain.srv_record_for(0,5,993, "mail."+domain) %>:<%= ttl %>
:_pop3._tcp.<%= domain %>:33:<%= domain.srv_record_for(10,5,110, "mail."+domain) %>:<%= ttl %>
:_pop3s._tcp.<%= domain %>:33:<%= domain.srv_record_for(10,5,995, "mail."+domain) %>:<%= ttl %>
% end
```
These service records are not created. could this be removed from the template?Backlog