Sympl issueshttps://gitlab.com/sympl.io/sympl/-/issues2020-04-23T19:38:17Zhttps://gitlab.com/sympl.io/sympl/-/issues/12Add local DNS server with ability to allow AXFR to slaves2020-04-23T19:38:17ZPaul CammishAdd local DNS server with ability to allow AXFR to slavesFor sympl to work standalone, it needs an integrated DNS server. The should be automatically updated by the various DNS related operations, and provide for AXFR requests to be allowed to enable external servers to provide DNS for domains...For sympl to work standalone, it needs an integrated DNS server. The should be automatically updated by the various DNS related operations, and provide for AXFR requests to be allowed to enable external servers to provide DNS for domains hosted within sympl
Patrick Cherry has implemented something already, which might be a useful starting point:
https://gitlab.com/patch0/djbdns/wikis/homeFuture Planshttps://gitlab.com/sympl.io/sympl/-/issues/15monit requires outside library which is not packaged2019-06-07T11:00:55ZPaul Cammishmonit requires outside library which is not packagedThe source repo is included in the repo here, but this is sub-optimal.
This should probably be replaced with something like https://mmonit.com/monit/, or simply replaced with a more maintainable alternative, as it's not that complex.The source repo is included in the repo here, but this is sub-optimal.
This should probably be replaced with something like https://mmonit.com/monit/, or simply replaced with a more maintainable alternative, as it's not that complex.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/20octoDNS as interface to DNS providers2021-09-05T00:36:13ZPaul CammishoctoDNS as interface to DNS providersoctoDNS has support for a number of providers, so it may be worth looking at integrating it.
https://github.com/github/octodns#supported-providersoctoDNS has support for a number of providers, so it may be worth looking at integrating it.
https://github.com/github/octodns#supported-providershttps://gitlab.com/sympl.io/sympl/-/issues/122Symbiosis: `symbiosis-configure-ips` doesn't remove IPs it added once they ar...2019-06-06T11:08:42ZPaul CammishSymbiosis: `symbiosis-configure-ips` doesn't remove IPs it added once they are removed from /srv/*/config/ipImported from https://www.github.com/BytemarkHosting/symbiosis/issues/59
If an IP has been added to a machine via `/srv/*/config/ip`, then if removed, it won't be removed from the configuration until next reboot when it won't be re-adde...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/59
If an IP has been added to a machine via `/srv/*/config/ip`, then if removed, it won't be removed from the configuration until next reboot when it won't be re-added.
This is likely as it can't be determined that Symbiosis added the IP, so we should probably either:
1. Make it clear in the docs that removing an IP will need a reboot or manual change via `ip`.
2. Automatically remove any IPs not set somewhere in `/srv/*/config/ip` when running symbiosis-configure-ips.
3. Provide a `--force` switch (like the other Symbiosis apps) to make the config match what symbiosis-configure-ips is trying to do.
Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/123Symbiosis: Ability to add an email attachment file extension black/whitelist2019-04-17T20:27:02ZPaul CammishSymbiosis: Ability to add an email attachment file extension black/whitelistImported from https://www.github.com/BytemarkHosting/symbiosis/issues/41
Would be nice to be able to block certain attachments to email, in addition to the usual antivirus scanningImported from https://www.github.com/BytemarkHosting/symbiosis/issues/41
Would be nice to be able to block certain attachments to email, in addition to the usual antivirus scanningFuture Planshttps://gitlab.com/sympl.io/sympl/-/issues/139Symbiosis: Event bus for configuration changes2019-04-14T20:18:28ZPaul CammishSymbiosis: Event bus for configuration changesImported from https://www.github.com/BytemarkHosting/symbiosis/issues/45
A common problem in developing Symbiosis functionality is the difficulty in detecting configuration changes. Currently Symbiosis is configured using SFTP, but has ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/45
A common problem in developing Symbiosis functionality is the difficulty in detecting configuration changes. Currently Symbiosis is configured using SFTP, but has no hooks into that system, so it's always going to be somewhat hobbled.
It would be good to have some sort of event bus to detect configuration changes, and act upon them at the time. This removes the need for polling which is wasteful and not very timely in some cases. Using a more event based approach would make configuration changes more or less instant, and maybe more reliable too.
One way of achieving this would be to have hooks into the SFTP subsystem. OpenSSH doesn't seem to have them, others do, ie https://www.npmjs.com/package/ssh
Whilst not advocating the addition of node into the stack, it would be good to have this functionality somehow, so it's open for debate.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/144Symbiosis: If an SSL cert is automatically disabled, Symbiosis won't use auto...2019-07-17T15:53:24ZPaul CammishSymbiosis: If an SSL cert is automatically disabled, Symbiosis won't use automatically it again if it becomes validImported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` an...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/111
For example, if I have a site (https://under100words.com) and manually disable Let's Encrypt by placing `false` in `/srv/under100words.com/config/ssl-provider` and moving the `config/ssl directory` out of the way, `symbiosis-httpd-configure` will disable the specific SSL cert for the site, swapping it to self-signed.
This is fine, and to be expected, however it does this by removing the relevant symlink from `/etc/apache2/sites-enabled`, which has the effect of flagging the site as "manually disabled", dropping it back to mass hosting, if configured.
Restoring the SSL configuration (removing `ssl-provider` and restoring `config/ssl`) then re-running `symbiosis-httpd-configure --verbose` you get:
```
# symbiosis-httpd-configure --verbose
[ . . . ]
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
[ . . . ]
Configuration: under100words.com.conf
Configuration is up-to date.
!! Configuration has been manually disabled.
```
So, it's still thinking that the site was manually disabled, so even if it managed to create the individual config as there are valid SSL certs, it's not being symlinked.
A manual workaround is to run `symbiosis-httpd-configure` for the specific site:
```
# symbiosis-httpd-configure --verbose under100words.com
Domain: under100words.com
Current SSL set 1: signed by /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3, expires 2018-02-20 13:36:22 UTC
This site has SSL enabled, and is using the host's primary IPs -- continuing with SNI.
SSL is enabled -- using SSL template
Adding to configurations
Configuration: under100words.com.conf
Configuration is up-to date.
Enabling configuration.
Reloading Apache
```
This instead enables the config anyway, and things work normally again.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/153Symbiosis: Migration assistant2019-04-14T20:37:18ZPaul CammishSymbiosis: Migration assistantImported from https://www.github.com/BytemarkHosting/symbiosis/issues/60
We spend a lot of time helping customers with migrations, or advising that they migrate to avoid a dist-upgrade.
It would be nice if Symbiosis Stretch included a ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/60
We spend a lot of time helping customers with migrations, or advising that they migrate to avoid a dist-upgrade.
It would be nice if Symbiosis Stretch included a script to migrate all domains from Symbiosis Jessie (and maybe Wheezy). Something based on this migration guide would be useful. https://forum.bytemark.co.uk/t/the-symbiosis-migration-guide/2172
It should do as much of that as possible, and carry suitable health warnings, perhaps.
Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/163Symbiosis: PHP 7.1/7.2 support2020-09-15T21:33:38ZPaul CammishSymbiosis: PHP 7.1/7.2 supportImported from https://www.github.com/BytemarkHosting/symbiosis/issues/128
(https://forum.bytemark.co.uk/t/symbiosis-stretch-update/2848/30)
PHP7.0's EOL is 3/12/18, which is a little short, given that PHP7.2 is only 'officially' availa...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/128
(https://forum.bytemark.co.uk/t/symbiosis-stretch-update/2848/30)
PHP7.0's EOL is 3/12/18, which is a little short, given that PHP7.2 is only 'officially' available in Buster, and PHP7.1 in Sid.
We already run 7.1 and 7.2 in containers (Systemd-nspawn, Docker) for some customers using Symbiosis, but the setup procedure is mostly manual.
Ideally we'd want to be able to create a file e.g. `/srv/domain.com/config/php` containing `7.1<` or `7.2` to automatically use the appropriate Apache template for that domain.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/165Symbiosis: Publish CAA (DNS TXT) records to improve security2019-04-14T20:59:45ZPaul CammishSymbiosis: Publish CAA (DNS TXT) records to improve securityImported from https://www.github.com/BytemarkHosting/symbiosis/issues/134
Certification Authority Authorization (CAA), specified in RFC 6844 in 2013, is a proposal to improve the strength of the PKI ecosystem with a new control to restr...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/134
Certification Authority Authorization (CAA), specified in RFC 6844 in 2013, is a proposal to improve the strength of the PKI ecosystem with a new control to restrict which CAs can issue certificates for a particular domain name. It prevents bad people obtaining certificates from rogue or sloppy certification authorities.
It's a simple DNS text record to say, for example:
`example.org. CAA 128 issue "letsencrypt.org"`
At minimum, we could publish this record for a domain that's protected by a LetsEncrypt certificate.
https://blog.qualys.com/ssllabs/2017/03/13/caa-mandated-by-cabrowser-forumFuture Planshttps://gitlab.com/sympl.io/sympl/-/issues/173Symbiosis: Simple database creator2020-09-15T21:31:03ZPaul CammishSymbiosis: Simple database creatorImported from https://www.github.com/BytemarkHosting/symbiosis/issues/46
A typical usage scenario for Symbiosis is a user creating a new site, then uploading a PHP webapp and expecting it to work.
The problem is, once they've uploaded ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/46
A typical usage scenario for Symbiosis is a user creating a new site, then uploading a PHP webapp and expecting it to work.
The problem is, once they've uploaded the PHP content, they need to create a MySQL database, which can only currently be done over the command-line or PHPMyadmin.
An interesting feature would allow a user to create a "database" file in the /srv/domain.tld/config directory, containing a user and password. Symbiosis would detect the presence of this file, and if a database named after the domain didn't already exist, would create a new database, and GRANT the right permissions to it allowing local access with the username/password in the file. The user could then install say, Wordpress with *only* SFTP.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/175Symbiosis: Some thoughts about Symbiosis firewall management2019-04-17T20:20:11ZPaul CammishSymbiosis: Some thoughts about Symbiosis firewall managementImported from https://www.github.com/BytemarkHosting/symbiosis/issues/136
The Symbiosis firewall seems to work well, but I have issues with it, which may be caused by my lack of understanding of the nuances of ruby, I guess. Mostly I w...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/136
The Symbiosis firewall seems to work well, but I have issues with it, which may be caused by my lack of understanding of the nuances of ruby, I guess. Mostly I want to stop the bots that relentlessly look for logins and passwords, while at present I think that Exim is doing a reasonable job of keeping spam out.
This attempts to explain my understanding of what is happening, so please bear with me - and maybe you wlll see if I have it wrong.
**What's there**
I am interested in improving the action of /usr/sbin/symbiosis-firewall-blacklist which runs every 15 minutes (or so) and is responsible for creating and deleting files in /etc/symbiosis/firewall/blacklist.d. The files are named for IP addresses (and have .auto) appended to them when created by the script. IP files can be created here to put into the firewall, and are ignored by the system. The files contain port numbers, 'all' or be empty (which implies 'all') - the port numbers are loaded into iptables and block attempts to connect from the named IP to the appropriate port.
The symbiosis-firewall-blacklist script has two distinct phases:
1) It creates a list of candidate IPs (and ports) using the files in /etc/symbiosis/firewall/patterns.d to scan log files. Once it has a list of candidates it creates the files in blacklist.d.
2) It then looks for files to delete.
So let's look at phase 1:
Each file in the patterns.d directory contains the name of a log file. This file is scanned looking for a regular expression match to find likely candidates for blocking. The script uses an sqlite3 database (in /var/lib/symbiosis/firewall-blacklist-logtail.db) to remember where it was in the file the last time it looked at that file. (I suspect that this is the cause of some people's uncertainty about this system. If you add a pattern which you've found in a log file, then it won't be activated until the problem happens again and re-appears in the log file so it can be seen.) However doing this makes sense for efficiency reasons.
The script uses pattern.rb to scan the log file. When matches are found it creates a two dimensional hash
`results[ip][port] = hit (actual code is results[ip.to_s][port] += 1)`
so for each port in the pattern.d file, the ip is awarded a single hit. My guess is that this method was used because different pattern files may locate the same bad IP and may also repeat ports. However it means that total hit counts are multiplied by the number of ports in the matched pattern file.
So it's found some lines in a logfile matching the regular expression. Now it wants to decide if any of these entries are worthy of being blacklisted.
This happens if any of the hit counts are greater than 20 (by default), or the value supplied by the -a (--block-after). While it's doing this, it's summing all the hits for the ip and writes them to another sqlite3 database (/var/lib/symbiosis/firewall-blacklist-count.db) along with the IP and a timestamp.
It now looks to see if this IP has been really bad, and is worth blocking completely. It creates a sum of the hits recorded for this IP in the last 24 hours and if this is over 100 (default) or the value determined from -b (--block-all-after) then this IP is set up to be blocked completely for all ports.
So now if it has some candidates, it will create files with the ip address as the names and attach .auto.
Phase 2 is reasonably simple. It looks for the modification time of the file, and if it's expired it's removed.
**Thoughts**
The Phase 1 algorithm really needs improving and I would say that this is really an urgently needed change. In many cases the system is not reactive enough - and I see that people are using other systems because it's not good enough.
When looking for candidates, the system only 'knows' about the matches it has found in the current slice of the log file that it's inspecting for that pass. For my site, this is typically 3 or 6. My system is not busy so not a lot happens every 15 minutes. The window for selecting candidates is much too small - and is essentially avoidable by delaying attacks.
Second, the system completely ignores the history that it has carefully stored away in the sqlite3 file. Mostly I find that these robots come back from the same IP address often several days later. The system should make much better use of the history that it has gathered.
Third, the current system will theoretically generate port specific matches in iptables. But it's not clear how often this will happen, and frankly I don't care, I'd be happy to completely block every bad site from everything. As an aside it would be good if the code making the firewall understood about 'multiport dports' to make less lines in the filters.
The current selection code is:
```
results.each do |ip, ports|
#
# tot up on a per-ip basis
#
total_for_ip = 0
ports.each do |port, hits|
total_for_ip += hits
blacklist[ip] << port if hits > @block_after
end
#
# Record our count
#
@count_db.set_count_for(ip, total_for_ip, timestamp)
#
# Get the hits for the last 24 hours
#
total_for_ip = @count_db.get_count_for(ip, timestamp - 86400)
#
# If an IP has exceeded the number of matches, block it from all ports.
#
if total_for_ip > @block_all_after
blacklist[ip] = %w(all)
end
end
```
I am no ruby programmer - so this is a syntactic guess but maybe this will be an improvement - without changing patterns.rb.
If you want to correct the syntax for me please do.
```
results.each do |ip, ports|
#
# get history for this ip for the last day
# and this ought to be a parameter to the script
# because I'd like to change this
# (@block_period defaults to 1 I think)
#
historysecs = timestamp - 3600*@block_period
past_for_ip = @count_db.get_count_for(ip, historysecs)
#
# tot up on a per-ip basis
#
total_for_ip = 0
ports.each do |port, hits|
total_for_ip += hits
# notice change here to include history
blacklist[ip] << port if hits + past_for_ip > @block_after
end
#
# Record our count
#
@count_db.set_count_for(ip, total_for_ip, timestamp)
#
# Get the hits for the some period, which can also be the epoch
# and again should be a parameter - block_all_period can perhap default to 3
#
historysecs = 0
if @block_all_period > 0
historysecs = timestamp - 3600*@block_all_period
end
total_for_ip = @count_db.get_count_for(ip, historysecs)
#
# If an IP has exceeded the number of matches, block it from all ports.
#
if total_for_ip > @block_all_after
blacklist[ip] = %w(all)
end
end
```
So this:
a) includes the history for some period when working out the bad guys
b) includes the history for some longer period when evaluating really bad guys
c) Now has an extra lookup for each ip,
This may cause problems if this hits ip addresses that should be in use. There are no tools for removing good guys from the history file. Maybe this can be supplied as a small tool. Perhaps when an address is placed in the whitelist directory, the address should be removed from the blacklist database if it's there.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/182Symbiosis: symbiosis-firewall 99-reject file is blank by default2019-04-15T09:14:34ZPaul CammishSymbiosis: symbiosis-firewall 99-reject file is blank by defaultImported from https://www.github.com/BytemarkHosting/symbiosis/issues/137
Symbiosis' firewall contains a `/etc/symbiosis/firewall/incoming.d/99-reject` rule by default, which will block connections from `0.0.0.0/0` (anywhere).
If we a...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/137
Symbiosis' firewall contains a `/etc/symbiosis/firewall/incoming.d/99-reject` rule by default, which will block connections from `0.0.0.0/0` (anywhere).
If we add an IP to this 99-reject file, connections will only be blocked from this IP and allowed from everywhere else, which isn't usually what we want to happen.
It would be safer if the 99-reject file contained `0.0.0.0/0` by default to avoid allowing more through the firewall than what was intended. This could still be removed from the file if needed.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/189Symbiosis: Update dns.rb to make default dmarc policy safer2019-06-07T14:36:29ZPaul CammishSymbiosis: Update dns.rb to make default dmarc policy saferImported from https://www.github.com/BytemarkHosting/symbiosis/issues/130
The current dmarc policy is dangerous, in that it says to quarantine 100% of unaligned email from the main domain (but oddly not subdomains). This policy is much ...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/130
The current dmarc policy is dangerous, in that it says to quarantine 100% of unaligned email from the main domain (but oddly not subdomains). This policy is much safer for an initial dmarc policy, as recommended for example, by Google at https://support.google.com/a/answer/2466563?hl=en&ref_topic=2759254Backloghttps://gitlab.com/sympl.io/sympl/-/issues/196Symbiosis: Using `/srv/<domains/mailboxes/<user>/forward` with an external de...2019-06-20T15:02:39ZPaul CammishSymbiosis: Using `/srv/<domains/mailboxes/<user>/forward` with an external destinations fails SPF and DKIM checksImported from https://www.github.com/BytemarkHosting/symbiosis/issues/54
As `/srv/<domains/mailboxes/<user>/forward` simply relays the mail, it often causes SPF and DKIM signed mail to fail. This is particularly bad as the more importan...Imported from https://www.github.com/BytemarkHosting/symbiosis/issues/54
As `/srv/<domains/mailboxes/<user>/forward` simply relays the mail, it often causes SPF and DKIM signed mail to fail. This is particularly bad as the more important the mail (banks etc) the more tight their SPK/DKIM will be, and the destination mail server may simply drop invalid mail, or in many cases (see [SNDS](https://wiki2.bytemark.co.uk/Microsoft_Smart_Network_Data_Service)) simply flag all mail coming in that route as spam.
This can be worked around by using a basic Sieve filter (created in a mail client or Roundcube, etc) instead of a basic forward, which generates a header in the final mailbox like this:
```
Delivered-To: under100words@example.net
Received: by 10.157.19.41 with SMTP id f38csp1124363ote;
Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
X-Received: by 10.28.157.150 with SMTP id g144mr1955711wme.89.1490347549347;
Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
Return-Path: <admin@carth.ebonhawk.example.uk0.bigv.io>
Received: from carth.ebonhawk.example.uk0.bigv.io (under100words.com. [2001:41c8:51:7a3::163])
by mx.google.com with ESMTPS id x4si1960991wmx.113.2017.03.24.02.25.48
for <under100words@example.net>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Fri, 24 Mar 2017 02:25:49 -0700 (PDT)
Received-SPF: neutral (google.com: 2001:41c8:51:7a3::163 is neither permitted nor denied by best guess record for domain of admin@carth.ebonhawk.example.uk0.bigv.io) client-ip=2001:41c8:51:7a3::163;
Authentication-Results: mx.google.com;
dkim=pass header.i=@bytemark.co.uk;
spf=neutral (google.com: 2001:41c8:51:7a3::163 is neither permitted nor denied by best guess record for domain of admin@carth.ebonhawk.example.uk0.bigv.io) smtp.mailfrom=admin@carth.ebonhawk.example.uk0.bigv.io;
dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytemark.co.uk
Received: from admin by carth.ebonhawk.example.uk0.bigv.io with local (Exim 4.84_2) (envelope-from <admin@carth.ebonhawk.example.uk0.bigv.io>) id 1crLTg-0006yS-L8 for under100words@example.net; Fri, 24 Mar 2017 09:25:48 +0000
X-Sieve: Pigeonhole Sieve 0.4.2
X-Sieve-Redirected-From: nobody@under100words.com
Received: from yrk.mx.bytemark.co.uk ([2001:41c9:0:1019::81:84]) by carth.ebonhawk.example.uk0.bigv.io with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <paul.cammish@bytemark.co.uk>) id 1crLTf-0006yJ-0J for nobody@under100words.com; Fri, 24 Mar 2017 09:25:48 +0000
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bytemark.co.uk; s=20131115; h=Content-Type:MIME-Version:Date:Message-ID:Subject:From:To; bh=0UTMmBvQCD1aDtGQ0LM/wKwl+36AyMeb9pEvAf/Lve8=; b=jCmTawHtj3wZvPA+DnmtoyYpJYmznfAbAR2ZZkF0WmkbB9MD0BdivSofH1ww4DoYM3tH3BtjFt02Bn0NpKpNeNX74c8l5lVQ6QB2uVc2dDBVhma/MLbceDvrjAsOXGnomgWIFPZNLzOAcXF/mS+5ctw7nRGg44nxxlXMKAymbzTvzOGiiyA0rX2DgKhrn3SDhBNNC2dW8AsiZ6lqs9jZz99bqJlMwzhGz709MfPfzdvnIKsnt9sBc/hL+KAu0lUH1l0ySSNeLqyU6CTU91B3ULnVkQoX9EMh6Rim3430thOQ0VAfon9kp3l7lVbaXfDOkcK2PLxRBYSqnmD5uVR3dQ==;
Received: from ratatoskr.bytemark.co.uk ([2001:41c9:0:1017::48] helo=ratatoskr.dh.bytemark.co.uk) by yrk.mx.bytemark.co.uk with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <paul.cammish@bytemark.co.uk>) id 1crLTe-0006jU-Kd for nobody@under100words.com; Fri, 24 Mar 2017 09:25:46 +0000
Received: from [2001:41c8:3:1::186] by ratatoskr.dh.bytemark.co.uk with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <paul.cammish@bytemark.co.uk>) id 1crLTe-0005hF-I3 for nobody@under100words.com; Fri, 24 Mar 2017 09:25:46 +0000
To: nobody@under100words.com
From: Paul <paul@example.co.uk>
Subject: Test forward
Organization: Bytemark Hosting
Message-ID: <bdf891dd-fb0c-f7ca-3f89-7d7a00ff49ee@bytemark.co.uk>
Date: Fri, 24 Mar 2017 09:25:44 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="1kPfKMM3kgqtkMjBuqBIR3uSp4MpjjNV9"
Sender: Symbiosis Administrator <admin@carth.ebonhawk.example.uk0.bigv.io>
```
This works with a simple sieve rule like:
```
# rule:[Redirect all mail.]
if true
{
redirect "under100words@example.net";
stop;
}
```
It would be great if the forward file generated the relevant sieve rule, and massively improve deliverability.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/200There should be a simpler way of selecting the default site2019-04-22T17:53:40ZPaul CammishThere should be a simpler way of selecting the default siteThis would replace the zz-mass-hosting configuration (which can cause odd issues with subdomains as it matches more than it needs) and replace the need for `/srv/$(hostname)` and simply set the SSL certs to the default site for things li...This would replace the zz-mass-hosting configuration (which can cause odd issues with subdomains as it matches more than it needs) and replace the need for `/srv/$(hostname)` and simply set the SSL certs to the default site for things like FTP, and use a simple Apache redirect to handle it properly.
Something like a symlink of `/srv/default_site` -> `/srv/example.com` would do the job, although this change would need to be detected and the relevant services reloaded/restarted as needed.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/227Sympl parser2019-07-05T12:19:35ZPaul CammishSympl parserA basic version of the Sympl parser should be created, covering the most common things:
Creating domains, sites, mailboxes, ftp accounts, etc.
This can then be used in the new documentation, and expanded on further.A basic version of the Sympl parser should be created, covering the most common things:
Creating domains, sites, mailboxes, ftp accounts, etc.
This can then be used in the new documentation, and expanded on further.BacklogPaul CammishPaul Cammishhttps://gitlab.com/sympl.io/sympl/-/issues/202Apache should support the PROXY protocol2021-05-14T14:44:13ZPaul CammishApache should support the PROXY protocolTo support reverse proxies passing through the originating source IP (for things like diagnostic logging, anti abuse and so on) Sympl should ideally support the PROXY protocol.
See https://www.haproxy.org/download/1.8/doc/proxy-protocol...To support reverse proxies passing through the originating source IP (for things like diagnostic logging, anti abuse and so on) Sympl should ideally support the PROXY protocol.
See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt and mod_remoteip.
This may have to be a per-IP configuration looking at the Apache docs, or an overall system configuration. It would be nice however if it could be transparently configured.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/207There are no build tests for webmail2019-05-28T11:23:23ZPaul CammishThere are no build tests for webmailThere aren't currently any tests to ensure webmail (ie: roundcube) is functional.
This is less of an issue at the moment as IMAP is tested, so this can be put off for now.
We should however create a test to ensure webmail can be logged...There aren't currently any tests to ensure webmail (ie: roundcube) is functional.
This is less of an issue at the moment as IMAP is tested, so this can be put off for now.
We should however create a test to ensure webmail can be logged into (via cURLing the local site, etc.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/218sympl-all-crontabs.c should be rewritten in something more portable2019-07-17T15:48:41ZPaul Cammishsympl-all-crontabs.c should be rewritten in something more portable```text
* A wrapper script which will do some simple permission and file-presence
* checks, then launch the sympl-crontab command for each domain which
* is present.
*
* The way this script works is pretty simple:
*
* 1. Iterate over e...```text
* A wrapper script which will do some simple permission and file-presence
* checks, then launch the sympl-crontab command for each domain which
* is present.
*
* The way this script works is pretty simple:
*
* 1. Iterate over every entry beneath /srv
* - Ignoring dotfiles.
* - Ignoring entries that do not contain /srv/$name/config/crontab
*
* 2. Once a valid entry has been found ensure that the owner of
* /srv/$name and /srv/$name/config/crontab matches.
*
* 3. Invoke our ruby wrapper as the appropriate user, via /bin/su.
```
This should really be rewritten in something more portable (to ease install on non-amd64 platforms), or simply use bash instead as there's nothing particularly fancy here.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/226common: Check new passwords against https://haveibeenpwned.com/API/v2#PwnedPa...2019-06-10T14:29:22ZPaul Cammishcommon: Check new passwords against https://haveibeenpwned.com/API/v2#PwnedPasswordsThe API at https://haveibeenpwned.com/API/v2#PwnedPasswords provides an API of compromised passwords.
This would be a good thing to check against when a user changes their password along with cracklib.The API at https://haveibeenpwned.com/API/v2#PwnedPasswords provides an API of compromised passwords.
This would be a good thing to check against when a user changes their password along with cracklib.Backloghttps://gitlab.com/sympl.io/sympl/-/issues/257Sympl should automatically update it's configuration near-instantly2020-01-28T13:33:20ZPaul CammishSympl should automatically update it's configuration near-instantlyWhen changes are made, typically it can take up to an hour to a day for everything to have run.
It would be nice if Sympl used [incrond](https://linux.die.net/man/8/incrond) (currently used by sympl-firewall) to detect changes to the co...When changes are made, typically it can take up to an hour to a day for everything to have run.
It would be nice if Sympl used [incrond](https://linux.die.net/man/8/incrond) (currently used by sympl-firewall) to detect changes to the configuration and update as needed, adding to incrond's config where needed as domains are added/removed.
This would make configuration practically instant, so would need some kind of logging/admin notification so you can see what's actually going on.Future Planshttps://gitlab.com/sympl.io/sympl/-/issues/269SNI for mail only works with 'bare' domain name (or www.domain.name for dovecot)2019-11-13T13:39:05ZPaul CammishSNI for mail only works with 'bare' domain name (or www.domain.name for dovecot)# Summary
You can't use mail.domain.name to access email securely
# Steps to reproduce
Use an SNI mail client to try to fetch / send mail using mail.domain.name as the host
# What is the current bug behavior?
The certificate retur...# Summary
You can't use mail.domain.name to access email securely
# Steps to reproduce
Use an SNI mail client to try to fetch / send mail using mail.domain.name as the host
# What is the current bug behavior?
The certificate returned is the default for the server.
# What is the expected correct behavior?
The certificate returned should be for the correct domain
# Possible fixes
When generating certificates for a domain, ensure one if requested for mail.domain.name. Then add an SNI section for Dovecot to reference this. Exim looks a little trickier, as it goes directly to /srv/$tls_in_sni/config/ssl/current/ssl.combined to get the certificate.
/cc @kelduumhttps://gitlab.com/sympl.io/sympl/-/issues/277sympl-mail: add Autoconfigure functionality2020-01-31T09:23:05ZPaul Cammishsympl-mail: add Autoconfigure functionalityAutoConfigure for email is fairly simple, and only requires an XML file at a specific path.
Adding functionality for this should be fairly easy to accomplish.
https://forum.sympl.host/t/configure-auto-discover-for-mail-setup/94?u=kelduumAutoConfigure for email is fairly simple, and only requires an XML file at a specific path.
Adding functionality for this should be fairly easy to accomplish.
https://forum.sympl.host/t/configure-auto-discover-for-mail-setup/94?u=kelduumhttps://gitlab.com/sympl.io/sympl/-/issues/278sympl-ssl: Reimplmentation2021-02-12T18:08:30ZPaul Cammishsympl-ssl: ReimplmentationComplete reimplementation of sympl-ssl in Python, maintaining all the existing functionality and resolving long-standing issues.Complete reimplementation of sympl-ssl in Python, maintaining all the existing functionality and resolving long-standing issues.Paul CammishPaul Cammishhttps://gitlab.com/sympl.io/sympl/-/issues/282sympl-mail: Enhancement - Expand blacklist functionality2020-04-20T11:22:27ZPaul Cammishsympl-mail: Enhancement - Expand blacklist functionalityAdding functionality for the b.barracudacentral.org and bl.spamcop.net RBLs alongside the current Spamhaus ones may be useful, however as they are more sensitive relevant warnings should be added to the documentation.Adding functionality for the b.barracudacentral.org and bl.spamcop.net RBLs alongside the current Spamhaus ones may be useful, however as they are more sensitive relevant warnings should be added to the documentation.https://gitlab.com/sympl.io/sympl/-/issues/283sympl-mail: Enhancement - Improve Exim logging2020-04-20T11:25:36ZPaul Cammishsympl-mail: Enhancement - Improve Exim loggingIt's been suggested that moving the `log_selector` configuration out of `00-main/50-tls-options` into it's own separate file would be useful (due to a limitation on how many instances there can be), and likely adding the `+smtp_protocol_...It's been suggested that moving the `log_selector` configuration out of `00-main/50-tls-options` into it's own separate file would be useful (due to a limitation on how many instances there can be), and likely adding the `+smtp_protocol_error` option to it which will improve data for blacklisting with `sympl-firewall`.https://gitlab.com/sympl.io/sympl/-/issues/284sympl-mail: Enhancement - Add a whitelist for Exim2020-04-20T11:22:27ZPaul Cammishsympl-mail: Enhancement - Add a whitelist for EximIf you're potentially blocking more mail due to the enhancement in issue #282, the ability to whitelist hosts/IPs, would be useful to skip all spam filtering and consider the source as trusted.
This would likely need to be configured on...If you're potentially blocking more mail due to the enhancement in issue #282, the ability to whitelist hosts/IPs, would be useful to skip all spam filtering and consider the source as trusted.
This would likely need to be configured on a whole-host basis, and maybe should be located in /etc/sympl rather than the Exim config directory.
Maybe something which can be done automatically based on a previous good reputation, similar to the blacklists used by the firewall?https://gitlab.com/sympl.io/sympl/-/issues/285sympl-mail: Enhancement - Reject abusive hosts in Exim2020-04-20T11:25:36ZPaul Cammishsympl-mail: Enhancement - Reject abusive hosts in EximAssuming we are logging more data from issue #283, we can then blacklist (or greylist?) abusive hosts using sympl-firewall, which should deal with attempts to brute-force account details.Assuming we are logging more data from issue #283, we can then blacklist (or greylist?) abusive hosts using sympl-firewall, which should deal with attempts to brute-force account details.https://gitlab.com/sympl.io/sympl/-/issues/286sympl-mail: Review Exim configuration2020-04-20T11:28:55ZPaul Cammishsympl-mail: Review Exim configurationThe Exim configuration has been inherited from older versions of Symbiosis, and has diverged a fair bit from the default Debian configuration. It's worth a full review of the config to bring it more in line and avoid issues later on.
On...The Exim configuration has been inherited from older versions of Symbiosis, and has diverged a fair bit from the default Debian configuration. It's worth a full review of the config to bring it more in line and avoid issues later on.
One specific mention was that we should
> Comment out the rfc_1413 lines in 00-main/60-general-options and add a separate file with an ‘official’ exim4 recipe for turning ident off.
...which seems like a good start.https://gitlab.com/sympl.io/sympl/-/issues/289sympl-firewall: The firewall shouldn't destroy other chains, and should be le...2020-04-20T14:05:48ZPaul Cammishsympl-firewall: The firewall shouldn't destroy other chains, and should be less ambiguous.This would be a change to existing operation, but Sympl shouldn't wipe out the all other iptables chains when it runs, and only modify rules it created itself (ie: comments).
Similarly, the ambiguously named blacklist and whitelist shou...This would be a change to existing operation, but Sympl shouldn't wipe out the all other iptables chains when it runs, and only modify rules it created itself (ie: comments).
Similarly, the ambiguously named blacklist and whitelist should have names referencing Sympl.https://gitlab.com/sympl.io/sympl/-/issues/313sympl-mail: Exim deny-unusual-characters acl is a little over-strict for outg...2021-07-01T13:14:18ZPaul Cammishsympl-mail: Exim deny-unusual-characters acl is a little over-strict for outgoing mail.Non-local domains deny `%` and `!` in email addresses, although they're valid, and it seems like Xero are using `!`'s in emails in some cases.
Replacing `local_parts = ^[./|] : ^.*[@%!] : ^.*/\\.\\./` with `local_parts = ^[./|] : ^.*@ :...Non-local domains deny `%` and `!` in email addresses, although they're valid, and it seems like Xero are using `!`'s in emails in some cases.
Replacing `local_parts = ^[./|] : ^.*[@%!] : ^.*/\\.\\./` with `local_parts = ^[./|] : ^.*@ : ^.*/\\.\\./` in https://gitlab.mythic-beasts.com/sympl/sympl/-/blob/buster/mail/exim4/sympl.d/10-acl/50-acl-check-rcpt/20-deny-unusual-characters should fix this.https://gitlab.com/sympl.io/sympl/-/issues/321Add DNS records without preventing automatic generation2023-03-16T12:58:49ZPaul CammishAdd DNS records without preventing automatic generationI have my domain sign my emails with DKIM, using the host name as a selector, but I can also use an external SMTP server for some things, which has given me a public key to add to DNS. I guess in this case, I want to be able to add recor...I have my domain sign my emails with DKIM, using the host name as a selector, but I can also use an external SMTP server for some things, which has given me a public key to add to DNS. I guess in this case, I want to be able to add records to the DNS for the domain, but if I edit the DNS file, all other records will stop being updated. It would be good if there could be a different file for additional records so that the automatic file would still match its checksum.https://gitlab.com/sympl.io/sympl/-/issues/324FTP logs should be written to /var/log/pure-ftp/connection.log or similar2022-04-25T11:58:29ZPaul CammishFTP logs should be written to /var/log/pure-ftp/connection.log or similarAt the moment they only get written to `/var/log/messages`, which isn't that logical as there's also a `/var/log/pure-ftpd/` directory, where you'd expect to find them.
Also, we shouldn't be logging the RDNS for connections without the ...At the moment they only get written to `/var/log/messages`, which isn't that logical as there's also a `/var/log/pure-ftpd/` directory, where you'd expect to find them.
Also, we shouldn't be logging the RDNS for connections without the IP where at all possible, as it's trivial to fake.https://gitlab.com/sympl.io/sympl/-/issues/326sympl-web: sympl-web-rotate-logs doen't use an efficent naming convention.2022-08-05T14:02:45ZPaul Cammishsympl-web: sympl-web-rotate-logs doen't use an efficent naming convention.`sympl-web-rotate-logs` uses what is basically the worst case for backup efficiently in the logging, although this works like logrotate.
You get ~30 days of old logs, each named `.[1-3]?[0-9]` the older ones of which are gzipped. Each t...`sympl-web-rotate-logs` uses what is basically the worst case for backup efficiently in the logging, although this works like logrotate.
You get ~30 days of old logs, each named `.[1-3]?[0-9]` the older ones of which are gzipped. Each time it rotates, the highest number is dropped, and everything is moved up a number.
This isn't terrible for finding the old data, but it's not ideal, and it means each time you run a backup, *all* of the logs have changed, so even a quiet site ends up with all the logs being backed up again.
The logs should be datestamped, and then the oldest one(s) removed, that way each day's logs don't end up getting backed up over and over again for a month.https://gitlab.com/sympl.io/sympl/-/issues/330sympl-webmail: Webmail should discourage over-use of the To: field2023-03-16T12:40:32ZPaul Cammishsympl-webmail: Webmail should discourage over-use of the To: fieldAs mentioned in https://forum.sympl.io/t/roundcube-max-disclosed-recipients/320, its possible to have Roundcube ask if you really want to send to lots of disclosed recipients.
This would be nice to add to the default configuration, alth...As mentioned in https://forum.sympl.io/t/roundcube-max-disclosed-recipients/320, its possible to have Roundcube ask if you really want to send to lots of disclosed recipients.
This would be nice to add to the default configuration, although with a reasonably high number.