1308 Commits

Author SHA1 Message Date
Ad Schellevis
c20fd9a47a Services: Unbound DNS - minor start/stop changes
o make sure we are not going to exit the start script if rm fails for whatever reason
o add stop action which stops the service and unmounts volumes mounted by start, the current state only kills unbounds primary pid (without optional dhcp integration or mountpoints)
2022-10-20 09:08:13 +02:00
Franco Fichtner
5d62b395d1 unbound: fix bail on python glue chown (read-only error) 2022-10-17 11:04:51 +02:00
Stephan de Wit
d14ffae466
unbound: rework DNSBL implementation to python module (#6083)
Replaces the current blocklist implementation to use python instead of relying on unbound-control. The latter had the drawback of a very long execution time to administrate the local-data entries both locally and in Unbound. The memory footprint was also considerably larger due to unbound internals, while the python module keeps it all in memory in a simple dictionary - reducing the total amount of memory consumption by more than a factor of 10. A drawback is a potential decrease in performance of ~15%, although most setups shouldn't be affected by this as most hardware which is capable of running this should be scaled towards its intended use.

The option of returning NXDOMAIN has also been added (fixes #6027), which in this implementation is a lot easier than what we would have to do if local-data were to be used.
2022-10-14 17:02:23 +02:00
Franco Fichtner
8d50193182 firmware: remove faulty changelog to force a clean refetch 2022-10-12 10:01:45 +02:00
Ad Schellevis
0adf843ff6
VPN / IPsec - migrate ipsec.conf to swanctl.conf (#6076)
* VPN/IPsec - work in progress refactoring ipsec.conf to swanctl.conf for https://github.com/opnsense/core/issues/5636

Wrap the following blocks in functions:

o generation of strongswan.conf
o write and cleanup IPsec's CA database
o write certificates used by IPsec
o write RSA Key Pairs
o construct ipsec.secrets for swanctl, combining different sources, phase1 PSK's, certificates, keypairs
o replace VPN: IPsec: Lease Status with new mvc version that uses swanctl underneath

Remap ipsec.conf to swanctl.conf attributes using https://wiki.strongswan.org/projects/strongswan/wiki/Fromipsecconf

Remove lifetime defaults from phase 1 and 2 so new connections will use Strongswan's defaults and move the option in phase above Margintime/Rekeyfuzz as these combined are being migrated (see margintime in Fromipsecconf doc).

Restructure used path directives so we can use the default structure as descried in https://docs.strongswan.org/docs/5.9/swanctl/swanctlDir.html and installed via ports.

IPsec status overview needs a small fix as well (passthrough network seems to be returned by vici, but doesn't contain a class (which crashes list_status.py).)

While here, also remove some php warnings.
2022-10-10 19:52:36 +02:00
Ad Schellevis
385d182798 Firewall: Aliases - support Maxmind's unclassified EU (region, country unknown) as country selector EU
6255148,en,EU,Europe,,,0

Although the same applies for Asia, we can't use AS as country code here, so we do have to skip this for now until we find a better alternative.

6255147,en,AS,Asia,,,0

closes https://github.com/opnsense/core/issues/6063
2022-10-10 18:24:32 +02:00
Franco Fichtner
26ccc7839f system: remove _defaultgw(v6) file handling; closes #5900
Try to avoid touching default routes in dhclient-script which we
already partially do now.  PPPoE linkdown certainly doesn't need
to handle the file as well to remove the default route then.
2022-10-06 09:29:12 +02:00
Stephan de Wit
37fc19bfed unbound: move the removal of pluggable files above the configuration check
if bad input files are given, the root.key repair is triggered, which doesn't
make a lot of sense so just let unbound fail when this happens.

also remove the redirect to /dev/null to prevent possible file permission issues.
2022-10-04 12:10:00 +02:00
Franco Fichtner
b4979dc5eb unbound: stop testing staged changes
Not a lot of users for this manual testing script.  Only print
errors for current configuration.
2022-10-04 11:58:04 +02:00
Stephan de Wit
9c9a15b9ec unbound: relocate preloading of blocklist cache to start.sh
It doesn't really make sense to let unbound.inc handle this, instead it should
be part of the staging script.
2022-10-04 11:27:23 +02:00
Ad Schellevis
dbba3223f7 Interfaces: Diagnostics: Packet Capture - don't pass 0 to tcp dump (zero equals no limit) 2022-09-27 19:21:25 +02:00
Franco Fichtner
dd42c8785a firmware: restructure license information handling
opnsense-update -K will handle subscription key return if found.
Fetch the license metadata if it exists, move it to core version
meta file and handle the contents dymanically if it exists.

The only issue for later is that changing the mirror away from
business we have a lingering license file until the next check
for updates is done.
2022-09-27 09:13:45 +02:00
Franco Fichtner
0798b98a16 src: whitespace sweep 2022-09-23 20:31:42 +02:00
Ad Schellevis
797764c10a Core/Firmware - display license validity when applicable
Extends check.sh to fetch license json file when a uuid is found and flush this to /usr/local/opnsense/license.json.
The firmware page adds a license validity row, which is hidden by default and only shown when product.php returns 'product_license_valid_to'.

For the dashboard widget we'll add the "licensed until" string to the core product string when available.
2022-09-21 15:37:17 +02:00
kulikov-a
98f21ed601
unbound: reduce blocklist read timeout (#6030) 2022-09-18 10:14:12 +02:00
Ad Schellevis
f22c05ec61 Firewall/Diagnostics/Sessions: parse pftop internal data conversion (minor cleanups for https://github.com/opnsense/core/pull/6020) 2022-09-14 11:13:55 +02:00
kulikov-a
0f84667e37 parse pftop internal data conversion 2022-09-13 17:53:24 +03:00
Ad Schellevis
1548ae0eb7 Interfaces / Diagnostics / ARP Table - small cleanups for https://github.com/opnsense/core/pull/6016 2022-09-13 11:55:05 +02:00
moi
5f4cd3306f ARP table: IP addresses natural sort 2022-09-12 23:12:15 +02:00
moi
d5d837487f Resolves DNS only when -r arg is set + use Arp Internal DNS resolution 2022-09-12 23:03:51 +02:00
soif
de07e8dcf1 ARP Table: hostname uses reverse DNS lookup as fallback 2022-09-12 12:30:33 +02:00
Ad Schellevis
e1d8b471d0 Interfaces/Other Types/VLAN - pass proto to vlan interface for 802.1ad (QinQ) interface types.
Currently this doesn't work (yet), likely needs https://reviews.freebsd.org/D35848 so vlanproto can be set after the interface creation phase (now it's only possible to set proto there).
If `ifconfig XXXX vlanproto 802.1ad` sets the property, we do need this code to make sure we set these on QinQ parent vlan tags. As a workaround we could pass proto in legacy_interface_create() as well to make this (only) work on boot, but looking at the upstream fix, this might be something easy to pull in the near future.

for https://github.com/opnsense/core/issues/5893
2022-09-03 15:31:25 +02:00
Ad Schellevis
d8bf03d2e9 Interfaces/Diagnostics/Packet Capture - replace legacy version with new mvc variant. closes ehttps://github.com/opnsense/core/issues/5981 2022-08-30 17:35:45 +02:00
Ad Schellevis
4070ad0e2e Interfaces/Diagnostics/Packet Capture - work in progress for https://github.com/opnsense/core/issues/5981
o host field logic validator
o add remove job action
o add view action (initial version)
2022-08-29 21:16:49 +02:00
Ad Schellevis
21232cc4bd Interfaces/Diagnostics/Packet Capture - boilerplace for https://github.com/opnsense/core/issues/5981
o add basic workflow (start/stop/list)
o basic configd actions + implementation in capture.py script
o grid without actions to show current capture jobs
2022-08-28 15:45:42 +02:00
Ad Schellevis
bf11949841 Interfaces/Diagnostics/Packet Capture - boilerplace for https://github.com/opnsense/core/issues/5981 2022-08-27 16:16:45 +02:00
Franco Fichtner
38f70745eb system: another warning 2022-08-25 12:31:05 +02:00
Franco Fichtner
61dd38ce2e interfaces: null route for delegated prefix #3304
It's still debatable if ifctl is a tool to record data
for interfaces and act on it or if the latter part should
be handled by more authorative (interface) code.

Inspired by: https://github.com/opnsense/core/pull/5992
2022-08-25 10:04:27 +02:00
Franco Fichtner
a785c129a2 interfaces: web GUI locale sorts files differently #5862
Make suffix match explicit, give it lower priority and prevent
suffix match in plain inteface match afterwards to repair the
result return logic.
2022-08-23 09:31:18 +02:00
Franco Fichtner
20a60050ee system: run monitor reload inside system_routing_configure() #5956
Some call flows require this, others don't and on 22.7 we seem to miss
one that did.  Instead of adding more monitor reloads in the possible
spots move the ones that are shared into the general routing reload since
the two are almost always clustered together.

Also use the $interface argument to figure out which monitors require
reloading.  This will avoid quite a few spurious reloads on larger
setups.

Boot is a little special, but easy enough to ensure we don't call monitor
reload twice.
2022-08-19 11:03:31 +02:00
Franco Fichtner
5a5b1c9831 system: pretty print since we want users to inspect this 2022-08-18 09:04:22 +02:00
Franco Fichtner
d2e7cbade5 system: improve debugging capability of dns static routes
We are progressing steadly here, but now we need more visibility
of the sources of DNS routes to summarize servers and sources.

Also try not to deduplicate routes prematurely so that dynamic
hosts get priority over config-based ones like the override
setting actually implies as currently the last one won.
2022-08-18 08:44:44 +02:00
Franco Fichtner
eec0fbb96b firmware: improve previous a little more 2022-08-18 08:12:07 +02:00
Franco Fichtner
cf25dd91cb firmware: pkgs was still unknown 2022-08-17 15:37:49 +02:00
Franco Fichtner
76278022f8 interfaces: wait for DAD to complete #5933 2022-08-16 10:40:20 +02:00
Ad Schellevis
fcbdc44f63 Unbound - dhcp watcher : when the same fqdn is being reinjected on a different address, it would be removed previously. This commit aims to keep a consistent state when fqdn's move from one host to the other.
for https://github.com/opnsense/core/issues/5952

Test data:

lease 172.23.128.214 {
  starts 5 2022/08/12 18:17:31;
  ends 5 2022/12/12 18:27:31;
  tstp 5 2022/08/12 18:52:31;
  tsfp 5 2022/08/12 18:52:31;
  atsfp 5 2022/08/12 18:52:31;
  cltt 5 2022/08/12 18:17:31;
  binding state active;
  next binding state expired;
  hardware ethernet xx:xx:xx:xx:xx:xx;
  uid "\001h\327\232\206,\361";
  set vendor-class-identifier = "ubnt";
  client-hostname "MyDHCPClient";
}
lease 172.23.128.215 {
  starts 5 2022/08/12 18:17:31;
  ends 5 2022/08/12 18:47:31;
  tstp 5 2022/08/12 18:52:31;
  tsfp 5 2022/08/12 19:02:31;
  atsfp 5 2022/08/12 19:02:31;
  cltt 5 2022/08/12 18:17:31;
  binding state active;
  next binding state expired;
  hardware ethernet xx:xx:xx:xx:xx:xx;
  uid "\001h\327\232\206,\361";
  set vendor-class-identifier = "ubnt";
  client-hostname "MyDHCPClient";
}

When 172.23.128.215's end date surpasses "now", a removal after "end" will remove both 172.23.128.215 and 172.23.128.214
2022-08-15 14:48:00 +02:00
Franco Fichtner
489ae2f3c6 interfaces: DHCP is our problem child #5933 2022-08-15 13:20:41 +02:00
Franco Fichtner
e6ef454fa8 interfaces: signal address flush through rc.newwanip #5933 2022-08-15 12:14:48 +02:00
Franco Fichtner
72b04f0904 interfaces: fix interface name split for ifctl; closes #5929
Weird stuff such as "wififoo_wlan0" is now properly handled.
2022-08-12 12:07:42 +02:00
Franco Fichtner
85b223d00c interfaces: improve banner address return
primary will move to tracking interfaces or pick up any
SLAAC related address even though that is not being
actively configured.  Similar to the dashboard only print
the scoped address without the need to do the logic in
the caller.

Maybe we could even return the assigned prefix here for
clarity, but then the prefix isn't a reachable address.
2022-08-12 10:07:14 +02:00
Franco Fichtner
6043b5b125 interfaces: make rc.newwanip the same as "dns reload"... #5933
... a.k.a. rc.resolv_conf_generate.  While here protect more
code with the IP change guard and clear the IP address if the
address lookup came up empty.
2022-08-11 13:53:04 +02:00
Stephan de Wit
966e54011e
unbound: do not start DHCP watcher immediately after daemonizing unbound (#5920) 2022-08-02 10:33:59 +02:00
Franco Fichtner
bb4743a732 interfaces: stop DHCP from calling rc.newwanip when no changes are being done
This is a partial revert of 797c1864194 which has good intentions but rc.newwanip(v6)
isn't equipped with being called a lot of times yet.  We want to improve this for 23.1.
2022-08-02 09:04:12 +02:00
Franco Fichtner
f9ea49ae31 interfaces: manual cache invalidate for previous
Purge the file on all known dynamic spots that run a deconfigure
of some sort.  We probably need something for a forced reload as
well but for now let's see if this works in general.
2022-08-01 13:45:18 +02:00
Franco Fichtner
eebc1674a2 firewall: fix permission of script 2022-08-01 09:55:08 +02:00
Ad Schellevis
b6f95cdea4 Firewall: Aliases - add BGP ASN type. https://github.com/opnsense/core/issues/5913 2022-07-30 02:26:38 +02:00
Ad Schellevis
c21cafa924 config events, fix side affect of 3b7453ca2b. 2022-07-29 16:11:04 +02:00
Franco Fichtner
ef0d238ba1 interfaces: typo in script invoke 2022-07-28 21:09:04 +02:00
Stephan de Wit
f8650c76aa
System Status: replace old notices system with a global one (#5875)
* MVC / System status: first draft for backend implementation

* fix copyright

* fix permissions

* MVC / System Status: modify backend implementation and setup front-end

* MVC / System Status: minor cleanup, ACL check and fix reporting in production mode

* MVC / System Status: copy status sytem to legacy as well, remove the notices system, finish up front-end work

* MVC / System Status: remove useless constructor

* fix plist

* System Status: shorten previous

* System Status: add ACL check

* System Status: also remove legacy part

* System Status: also clean up on legacy page

* System Status: ACL check on dismiss action as well

* System Status: add readonly privilege check to dismiss action

* System Status: do not trust input

* System Status: address security concerns

* add default return

* System Status: move js code to separate script and make sure a logLocation is always provided

* System Status: clean up callout in both legacy page and volt template

After dismissing a message and closing the dialog, the old message was still bound to the dialog instance. re-registering the onclick callback solves this.

* System Status: adjust log location and kick off status system on alias errors

* System Status: let's hold off on throwing notifications for aliases for now

* System Status: add ACL entry for the dismiss API call, adjust to new deployment situation

Without the ACL entry, /api/core/system/dismissStatus calls are rejected for non-root users even when they should have rights to dismiss.

Also do a minor consistency improvement and also adjust to the new situation of production & development deployment types. We need to account for the possibility of 'deployment' being empty in the configuration, therefore a direct check of the 'development' type seems most fitting as this is unlikely to change or be subjected to any additions.

* System Status: also error out when unable to write new rules

* System Status: account for users without permissions

* System Status: name collision in FirmwareController

* System Status: replace old notices system with a global one (https://github.com/opnsense/core/pull/5875)

Review feedback / modifications in this commit:

o filter.inc

-- remove wedged message, when locked during parallel reloads it likely doesn't help to disable/enable
-- flush message to error trigger file

o SystemStatus.php
-- str_contains --> strpos; eases testing on OPNsense 22.1.x as str_contains is php 8 only

o Status collectors
-- simplify logic and propagate messages received from status file

o CrashReporterStatus
- the existence of a /tmp/PHP_errors.log  file should be enough to know some process signaled the crash reporter
- remove shell exec

o FirewallStatus
-- as only /tmp/rules.error remains, remove loop to read for non existing files

o opnsense_status.js
-- add opn-status-group class to container and point css modifications in that single direction to prevent other objects from being affected by our status popup modifications
-- windows file endings replaced (^M)

o css:
keep menu_messages container to ease migration for theme developers

NOTE : ** get_crash_report() could likely be simplified as well

* filter: change to mwexec, redirection is implied

* System Status: handle potential undefined array key

seems to be a one-off error: Exception: Error at /usr/local/opnsense/mvc/app/controllers/OPNsense/Core/Api/SystemController.php:90 - Undefined array key 0 (errno=2) in /usr/local/opnsense/mvc/app/controllers/OPNsense/Base/ApiControllerBase.php:159

Co-authored-by: Ad Schellevis <ad@opnsense.org>
2022-07-28 09:50:06 +02:00
Franco Fichtner
e12b4ab36e interfaces: same issue when -i not given
This is only for command line use but we never know how that
might change in the future.
2022-07-27 09:22:56 +02:00