1654 Commits

Author SHA1 Message Date
Ad Schellevis
9a33aeb07e system/trust - link location could be a file or link, but when it exists, it should be removed. fixes:
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
Traceback (most recent call last):
  File "/usr/local/opnsense/scripts/system/certctl.py", line 195, in <module>
    cmds[sys.argv[1]]()
  File "/usr/local/opnsense/scripts/system/certctl.py", line 152, in cmd_rehash
    os.symlink(src_filename, dst_filename)
FileExistsError: [Errno 17] File exists:
2024-10-07 19:47:25 +02:00
Ad Schellevis
e648aefb76 Firewall: Automation: Filter - add gateway groups to the list, closes https://github.com/opnsense/core/issues/7625 2024-10-05 18:22:30 +02:00
Franco Fichtner
b563c223bf system: closelog() has timing issues
Not sure where this comes from but the concensus is that
closelog() is relatively useless and testing verifies that
it works as expected without it.

Now we just have to make sure that each component using
openlog() will reopen the actual GUI log so that it can
keep logging correctly.
2024-09-30 13:19:26 +02:00
Franco Fichtner
f3a8c28c0e firmware: remove escaped slashes workaround, works since 24.7.5 2024-09-27 10:25:47 +02:00
Ad Schellevis
429b8e38b2 System: Trust - optimize certctl.py to prevent unneeded disk writes and flush certificates to ports openssl location (/usr/local/openssl/certs) 2024-09-25 19:55:02 +02:00
Ad Schellevis
b709232e44 System: Trust: Authorities - finish crl fetch script for installed certificates and hook into form and cron updates.
According to rfc5280 an CRL update should be issued before "next update", but does not seem to define a validity. Various sources seem to indicate updates should be fetched at least every couple of hours, in which case an hourly update sounds reasonable.

To avoid excessive writes, we compare each CRL offered with the last one received before flushing it to disk and keep track of actual changes. When nothing changes, no rehash is neede, which the configd action takes care of.

Distribution points could either be HTTP or LDAP, LDAPS and HTTPS may not be used according to the RFC (CAs SHOULD NOT include URIs that specify https, ldaps, or similar schemes in extensions.)
2024-09-24 20:58:04 +02:00
Ad Schellevis
6f79579537 System: Trust: Authorities - (work in progress) crl fetch script for installed certificates.
The overal structure is there, but needs the final bits and pieces added to it. committing it now as a backup
2024-09-24 18:38:53 +02:00
Franco Fichtner
61e47d7f9a interfaces: fix undefined function error in Monit CARP script
PR: https://www.reddit.com/r/opnsense/comments/1fobode/daily_errors/
2024-09-24 15:01:59 +02:00
iain MacDonnell
2a1ccae959
System/Logging - handle lines with no timestamp (#7889)
Recent feature addition to apply time constraint fails in cases where
a log line does not contain a timestamp, as in the case of multi-line
log entries. This change just moves on when a line contains no
timestamp, as we will eventually find the line that contains the
start of the log entry (and the timestamp).

fixes #7888
2024-09-23 17:54:11 +02:00
Franco Fichtner
89f84780ff interfaces: full-cirle on vpn_map vs. vpn event
We actually only want the vpn_map event in the stable branch
and move to a backwards compatible format for 25.1 but as a
new coalesced (mapped) caller.
2024-09-18 12:10:46 +02:00
Ad Schellevis
922907a931 System: Trust: silence cryptography deprecation warnings 2024-09-13 20:08:53 +02:00
Ad Schellevis
0378c650d4
Add System: Trust: Settings page (#7854)
* System: Trust: Settings - add boilerplate and move existing store_intermediate_certs setting to new module including migration

* System: Trust: Settings - add configuration constraints and glue to flush CRL's to local trust store when requested.

This implements the following options:

* Enable/Disable legacy providers (enabled by default, which is the current default)
* Option to write specific configuration constraints, when enabled, CipherString, Ciphersuites and MinProtocol[DTS] can be configured

One last piece of the puzzle is the "crl" event action, which should deploy to the local trust store as well ehen requested.

* Update src/opnsense/mvc/app/models/OPNsense/Core/Menu/Menu.xml

Co-authored-by: Franco Fichtner <franco@opnsense.org>

* System: Trust: Settings - process review comments https://github.com/opnsense/core/pull/7854

* System: Trust: Settings - flush CRL's when "Store CRL's" is selected

---------

Co-authored-by: Franco Fichtner <franco@opnsense.org>
2024-09-10 21:15:11 +02:00
Ad Schellevis
c4d0dd6154 core / trust - add support for certificate bundles in our certctl.py script.
Since we deploy ca_root_nss in a directory with less priority, we can exclude the certificates already part of the base install.
When ca_root_nss contains a certificate not yet known in base, it will copy the certificate from the bundle in the "hashed" file, which also gurantees our packed bundles contain the same content as openssl would use by default.

While here, also include /usr/share/certs/untrusted, which contains the base blacklisted certs.
2024-09-09 20:14:07 +02:00
Ad Schellevis
ebddde95f4 System: Trust - temporary disable validation of multiple certs in a ca, as discussed in 76228b0a45 (r146422317) with @fichtner 2024-09-09 12:27:05 +02:00
Franco Fichtner
d08069699f openvpn|wireguard: close-on-exec 2024-09-06 08:34:32 +02:00
Ad Schellevis
033402265e core / trust - missed a spot in 76228b0a45 (not removing crl's) 2024-09-05 11:28:19 +02:00
Ad Schellevis
76228b0a45 core / trust - cleanup certctl.py script and move to python cryptography for generic functionality.
This commit adds support for deployment of CRL's, as the original implementation also does (but these files don't exist at our end at the moment).
Secondly it fixes a minor omission where cert files with more than one cert in them where not omitted.
2024-09-04 18:34:17 +02:00
Franco Fichtner
4421914442 system: update comment on temperature stats collection 2024-09-03 20:57:09 +02:00
Franco Fichtner
6f21218193 system: remove RRD temp read from temperature.sh 2024-09-03 14:48:07 +02:00
Franco Fichtner
2dbd793dfe reporting: add executable flag for lint reasons 2024-09-03 08:01:33 +02:00
Franco Fichtner
3476d11a0a reporting: scrub rrd.inc use 2024-09-02 21:27:26 +02:00
Franco Fichtner
fcd54e4549 shell: configuration is static here, no need to restart cron/rrd 2024-09-02 21:10:16 +02:00
Franco Fichtner
ecf987820d reporting: style sweep 2024-09-02 21:06:03 +02:00
Ad Schellevis
895e58ff25
Reporting / rrd - refactor existing code (#7836)
Add RRD package with a simple factory class and a basic construct to define different rrd output types we support.
This package contains the following:

* RRD/Types -- Output definitions, responsible for generating RRD structures and feeding data
* RRD/Stats -- Statistics gathering classes
* Factory -- binds types and statistics together. 

On my end on a simple test this is roughly 40% faster than running /var/db/rrd/updaterrd.sh, which makes caching of metadata (config access) less relevant.

The new script should be able to replace all existing rrd cruft and supports a debug mode to find discrepanties between defined outputs in types and collected data in stats.

```
Usage: updaterrd.php [-h] [-d]
            -d debug mode, output errors to stdout
```
2024-09-02 16:40:47 +02:00
Franco Fichtner
4c07de7e34 interfaces: remove deprecated "vpn" event
Everything has been converted and plugins do not ship with a
"vpn" event.  "newwanip" is similar and the plugins have been
ensured to be moved to the new one in 24.7.3 already.
2024-09-02 09:25:58 +02:00
Franco Fichtner
919fc15fe7 interfaces: rename this correctly, strictly part of pf(sync) 2024-09-01 10:54:46 +02:00
Franco Fichtner
ab3bf029ca interfaces: another cleanup 2024-09-01 10:51:23 +02:00
Franco Fichtner
ba72ab76d3 system: remove (inconsistent) prefix from default snapshot names
Also want to avoid confusion about "BE" meaning business edition.
2024-08-29 08:47:39 +02:00
Franco Fichtner
8b7510a3a6 firmware: use full plugin names now and make sure to zap whitespace; closes #7818 2024-08-27 12:10:41 +02:00
Franco Fichtner
d1c7ed00ab firmware: collect plugin conflicts and remove them #7818
Also let conficts generate with its sibilings so we can remove
all of that code from the register script.  Most things are
fixed automatically now.
2024-08-27 11:01:48 +02:00
Franco Fichtner
ccac4779b7 interfaces: introduce vpn_map event
There appear to be no callers in plugins.  We could argue
that the vpn/vpn_map event isn't really needed as it brings
just OpenVPN, IPsec and WireGuard to the table, but we can
decide on this later.
2024-08-26 23:29:32 +02:00
Franco Fichtner
9fb4149530 interfaces: unify PPP linkup/linkdown scripting 2024-08-26 11:05:57 +02:00
Ad Schellevis
0adece8d3e System/Logging - add log search time constraint (valid_from) to limit searches when only a few lines match.
When searching large log files for messages that do not frequently occur, there is a large risk of reading all collected lines before returning the first results.
In most cases recent items are required, in which case going back for days of logs might not make sense.

This commit adds a simple "history" selection which translates into a "valid_from" filter on the log data. When timestamps are not parseable for whatever reason, the filter is ignored.

Only small downside is that we do need to translate timestamp again, when needed we could improve performance a bit by storing the original datetime value in NewBaseLogFormat so we don't have to parse it twice.
2024-08-24 20:55:12 +02:00
Ad Schellevis
798170b612 System/Logging - unify how search clauses work, when multiple clauses are specified, we should perform a lowercase compare on each clause (AND).
With this change in place, search behavior should be equal to model searches and searchRecordsetBase() consumers.
2024-08-24 18:12:25 +02:00
Franco Fichtner
7ce814346b interfaces: add logging to PPP link scripts to check for overlap
Currently we neither see when we spawn a new mpd5 daemon nor when
the PPPoE starts or stops working or if there is any overlap.
2024-08-22 14:36:09 +02:00
Franco Fichtner
49aa78594f rc: fix banner HTTPS fingerprint 2024-08-21 21:22:03 +02:00
Franco Fichtner
7837c74920 firmware: fix behaviour of register/resync a bit more 2024-08-21 08:04:23 +02:00
Franco Fichtner
689cf50cdf firmware: move cleanups to opnsense-update "-F"
This probably needs more engineering later on which makes
it a little harder to ship but in reality we want people
to run the portable command easily in order to unbreak their
environments as pkg-upgrade tripping over its own temporary
files and directories keeps happening in user reports.

While here use "-s" while at it to speed up the script call.
This also pertains to "-e" which is its sibling.
2024-08-20 11:28:13 +02:00
Franco Fichtner
61cea387c0 firmware: fix logic in previous 2024-08-20 08:14:32 +02:00
Stephan de Wit
44a1ea3096 syslog: regression in c522ecac65, wrong variable assignment. Fixes https://github.com/opnsense/core/issues/7778 2024-08-16 10:59:57 +02:00
Ad Schellevis
f477fa13b8 VPN: WireGuard - support carp vhid reuse on different interfaces, closes https://github.com/opnsense/core/issues/7773
Although all our examples always use vhid as a unique key per firewall, it is possible to add the same vhid to different interfaces.
When "disable preempt" is not selected, eventually all of them will switch between master/backup at the same time anyway, so we can assume all virtual ips switch simultaneously.

If preempt is disabled, our vhid matching might not be perfect, but likely better than before.
2024-08-14 21:44:45 +02:00
Franco Fichtner
ef5a59f5e4 firmware: add upgrade test, maybe later also a size check; closes #7769 2024-08-14 14:57:35 +02:00
Franco Fichtner
c054a2b94b firmware: add a comment here 2024-08-14 08:38:15 +02:00
Franco Fichtner
c3af228910 src: style et al 2024-08-13 20:21:36 +02:00
Sam Sheridan
7118a82a05 system: add snapshots (boot environments) GUI support #7749
This pull request introduces a new feature to the OPNsense web interface,
allowing users to manage FreeBSD boot environments directly within OPNsense.

This integration provides an intuitive and seamless way for users to create,
manage, and switch between boot environments, enhancing system management
and recovery options.

Renamed the menu item to "Snapshots" in an attempt to explain the feature
to non-FreeBSD users.
2024-08-13 20:15:08 +02:00
Franco Fichtner
346e913323 firmware: remove known plugins added to cor; closes #7195 2024-08-13 16:19:22 +02:00
Franco Fichtner
805120ea78 interfaces: decouple starting PPP from configured IP modes #7446
PR: https://forum.opnsense.org/index.php?topic=42008.0
2024-08-09 09:30:52 +02:00
Franco Fichtner
7c9ba6d785 interfaces: style in previous #7446 2024-08-08 23:31:36 +02:00
Franco Fichtner
58bfdecefa interfaces: backwards compat move of PPP IPv6 to linkup #7446 2024-08-08 23:10:13 +02:00
Franco Fichtner
4e9c75240d shell: minor cleanup 2024-08-08 14:25:56 +02:00