1047 Commits

Author SHA1 Message Date
Franco Fichtner
a137d96af8 firmware: in case of fs integrity issues try not to break upgrades
File is always packaged, but we cannot trust the file system.
2021-10-14 14:49:11 +02:00
Franco Fichtner
f129c4682b interfaces: deprecate *up(v6) files, PPP is only user
Create a more distinguished file _uptime which could be used later
for other things.  Adjust scripting and remove unneeded cruft.
2021-10-05 10:09:12 +02:00
Franco Fichtner
7063dc9e02 firmware: shift away from old-style firmware-xxx files
opnsense-update can read the upgrade hint itself.  We may have
to stash an ABI in there to reach to a different location without
the need to publish a symbolic link.

Move the firmware message to a data location for cleanliness.
2021-09-28 07:38:21 +02:00
Franco Fichtner
f0aeb0eff1 interfaces: add all sorts of stuff to interfaces_addresses() #5086
It allows us to do post-processing on returned addresses for
e.g. #5086 alias parsing for unwanted automatic bind mode.
2021-09-22 11:12:12 +02:00
Ad Schellevis
c96e5f88b4 Firewall / Aliases - minor bugfix in "filter diag table_size" (caf4439cf0) 2021-09-21 16:34:03 +02:00
Ad Schellevis
caf4439cf0 Firewall / Aliases - extend "filter diag table_size" command to include details as well. 2021-09-20 21:59:00 +02:00
Franco Fichtner
9e44d9a1a9 ipsec: rewrite netmask calculation #5201
We need to make sure both the local and the remote IP belong to
the same CIDR range, which might not be the case if we just
calculate the subnet size required by their direct distance.

Rewrite find_smallest_cidr() to take an array of IPs to calculate
their smallest shared subnet mask.  Code is actually pretty simple
and fast.  However, we are not going to account for network and
broadcast address reservation unless that turns out to be an issue.

In the IPv6 case assume that /64 is a good approximation of the
result.

Remove code cruft in utilities while at it also replacing a simple
function only called once in setaddr.sh.
2021-09-16 09:54:33 +02:00
Franco Fichtner
239a6e5dba firmware: masking vulnerability urls in FreeBSD due to UUID use 2021-09-01 14:36:49 +02:00
Ad Schellevis
19a927ce2c firmware: a major upgrade logically also expects a reboot.
The other side could check if the action would be a major upgrade, but it feels more clear to always communicate the correct intention.
If this doesn't have side affects, we could change 1077e0a2a8/src/opnsense/mvc/app/views/OPNsense/Core/firmware.volt (L223) as well

cc @fichtner
2021-08-25 18:07:17 +02:00
Ad Schellevis
6cd8a97182 Squashed commit of the following:
commit 46e0383625acfa59e723c390d0b5b2feed8a53aa
Author: Ad Schellevis <ad@opnsense.org>
Date:   Mon Aug 23 11:05:53 2021 +0200

    Firewall / Log - Live log : support rfc5424 format for https://github.com/opnsense/core/pull/5175

commit 0cf3030724d02181991436b324fe5fc70118d4d5
Author: Ad Schellevis <ad@opnsense.org>
Date:   Sat Aug 21 09:36:56 2021 +0200

    System logging - switch local logging to rfc5424 format.

    Allow custom destinations to choose for rfc5424. closes https://github.com/opnsense/core/issues/4911

commit a46e39bcecca6dab1d5a68a0e7f481ea42c16034
Author: Ad Schellevis <ad@opnsense.org>
Date:   Fri Aug 20 18:23:22 2021 +0200

    System logging - switch local logging to rfc5424 format.

    add severity filter to api and log pages

commit b0f38003d7745a01202ffca5e7b5b697ff211f1e
Author: Ad Schellevis <ad@opnsense.org>
Date:   Fri Aug 20 15:42:58 2021 +0200

    System logging - switch local logging to rfc5424 format.

    Split BaseLogFormat into a generic LogFormat and a child NewBaseLogFormat to be able to support both property and method based parsers to keep (external) plugins compatible.

commit 264513f148185faf4a4509bc19aa35739c9844f9
Author: Ad Schellevis <ad@opnsense.org>
Date:   Thu Aug 19 19:56:19 2021 +0200

    System logging - switch local logging to rfc5424 format.

    Refactor log parser (BaseLogFormat), since the plugins use this as well we might have to undo a bit or refactor those as well.
2021-08-23 13:15:51 +02:00
Ad Schellevis
d0630be3b0 Firewall / Aliases - less excessive logging for async dns resolve (76b8ae4490) 2021-08-19 16:12:01 +02:00
Ad Schellevis
76b8ae4490 Firewall / Aliases - improve resolve performance by implementing async dns lookups. ref https://github.com/opnsense/core/issues/5117
This will need a new version of py-dnspython (py-dnspython2 in ports) for dns.asyncresolver support. Some additional log messages have been added to gain more insights into the resolving process via the general log.
Intermediate results aren't saved to disk anymore, which also simplifies the resolve() function in the Alias class. An address parser can queue hostname lookups for later retrieval (see _parse_address()) so we can batch process the list of hostnames to be collected.
2021-08-19 15:53:25 +02:00
Ad Schellevis
4414ffd81b Authentication / LDAP (+TOTP), allow automatic user creation when configured. closes https://github.com/opnsense/core/issues/5116
o add "Automatic user creation" option in System/Access/Servers (for ldap + derivatives)
o simple detached flow, updatePolicies() calculates differences between local and remote group membership, when there is something to sync (remote groups exists) and a local user doesn't exist a configd signal is send to create a new empty user without rights and a random password.

The user_dn field isn't populated, although this will cost additional queries to the remote host, it might be worth the flexibility of allowing to move users to different auth scopes.
2021-08-17 19:33:25 +02:00
James Golovich
f98f4e4f1e
Allow DNS resolver to skip entry on EmptyLabel (#4560)
* Allow DNS resolver to skip entry on EmptyLabel

A name like '.example.com' is not a valid name, but should be handled like a non existant name instead of throwing an exception
2021-08-16 19:10:07 +02:00
kulikov-a
3f73088673
rulecache.py: skip empty metadata (#5148) 2021-08-08 11:12:48 +02:00
kulikov-a
27fdfe49c3
Aliases progress bar: small suggestions (#5149)
* alias.volt: change a bit
* pftablecount.py: count all tables from -vvsT
2021-08-08 10:47:17 +02:00
Ad Schellevis
9b31fad9ed Squashed commit of the following:
commit 9158a0618f7c7943a10fbcdf7b8a4950844d6133
Author: Ad Schellevis <ad@opnsense.org>
Date:   Sat Aug 7 22:35:00 2021 +0200

    Firewall / aliases - simplify and change https://github.com/opnsense/core/pull/5142 a bit

commit 72ad80b072e11e8ad58a82f59a395c9961eef386
Author: kulikov-a <36099472+kulikov-a@users.noreply.github.com>
Date:   Wed Aug 4 23:29:09 2021 +0300

    alias.volt: add limits progress bar

commit 6ba824471d9fc6186775dc6bdab815b4d1af0ce9
Author: kulikov-a <36099472+kulikov-a@users.noreply.github.com>
Date:   Wed Aug 4 23:09:17 2021 +0300

    AliasController.php: add getRecordsAction

commit 76de5668ff1f1225f32bd3abfc76f91c3500d753
Author: kulikov-a <36099472+kulikov-a@users.noreply.github.com>
Date:   Wed Aug 4 23:02:23 2021 +0300

    pfrecords.py: show tables load stats

commit 9719a8d31fd127a621c1cea219c9cd6c42bd852b
Author: kulikov-a <36099472+kulikov-a@users.noreply.github.com>
Date:   Wed Aug 4 23:00:02 2021 +0300

    actions_filter.conf: add pfrecords.py action

commit f106c8db128b3cdbdfb4ec79e766b84f120d4510
Author: kulikov-a <36099472+kulikov-a@users.noreply.github.com>
Date:   Wed Aug 4 22:40:11 2021 +0300

    update_tables.py: send error to syslog, stats file
2021-08-07 22:41:11 +02:00
Franco Fichtner
9506f4c1c6 firmware: now that we use TEE we should define it #5136 2021-08-05 14:27:30 +02:00
Franco Fichtner
af7543e79b firmware: visibility issue on #5136 2021-08-05 13:05:23 +02:00
Franco Fichtner
1c895ed435 firmware: also check plugins for up to date core pkg #5136 2021-08-05 10:05:23 +02:00
Franco Fichtner
f9f9dd28c1 firmware: remove spurious grep 2021-08-03 12:30:10 +02:00
Franco Fichtner
16a3559747 firmware: replace php with pkg version compare
As a small bonus now also corectly check if an upgrade
for opnsense-update is available.  If not, we are now
able to ignore the downgrade.
2021-08-03 12:23:54 +02:00
Franco Fichtner
5969d20641 firmware: backend now supports reinstall like opnsense-bootstrap -q 2021-08-03 11:49:20 +02:00
Franco Fichtner
4472a23596 firmware: switch to FreeBSD where appropriate 2021-08-02 21:48:29 +02:00
Franco Fichtner
eb72ff6d8a unbound: reject invalid cache data 2021-08-02 09:53:58 +02:00
Franco Fichtner
7165b665eb firmware: sync plugins in console update; closes #5136 2021-08-01 11:27:05 +02:00
Ad Schellevis
2909e86088 unbound - dhcp watcher, support configurable domains per ip range, for https://github.com/opnsense/core/issues/5118 2021-07-30 23:01:55 +02:00
Ad Schellevis
41b311ecc2 Revert "Firewall / Aliases - append most likely cause for pfctl error (Invalid argument isn't very explanatory by itself). closes https://github.com/opnsense/core/issues/5127"
This reverts commit 2ea90801290856f453126b1e6d63e66c79a5ca6f.
2021-07-30 18:45:40 +02:00
Ad Schellevis
2ea9080129 Firewall / Aliases - append most likely cause for pfctl error (Invalid argument isn't very explanatory by itself). closes https://github.com/opnsense/core/issues/5127 2021-07-30 17:47:18 +02:00
Franco Fichtner
4ae1555e07 firewall: change "proto" to "protonum" to avoid ambiguity 2021-07-23 09:02:22 +02:00
Franco Fichtner
538e2f04c2 firewall: update filterlog reader slightly
CARP in IPv4 and IPv6 uses the IP header's "ttl" or "hoplimit" so
that we use the same name for reading it to avoid duplication.
The values are the same in any case.

Change "flowlabel" to "flow" to avoid confusion with "label".

Change IP "version" to "ipversion" and consolidate CARP "version(2)"
into "version".
2021-07-23 08:14:36 +02:00
Franco Fichtner
1aeb520631 firewall: parse ridentifier as rid if != '0' #5016 2021-07-22 17:34:02 +02:00
Franco Fichtner
5e860053e1 firmware: mask subscription in GUI output 2021-07-22 10:16:04 +02:00
Ad Schellevis
be90cf00a6 Firewall / Diagnostics / States, Sessions - fix some minor glitches.
o mark unsortable columns
o fix type validations (avg=* isn't an int).
o better to return an empty structure than [], avoid JS errors in the frontend
2021-07-21 18:49:10 +02:00
Ad Schellevis
e4aa3e4f5e Firewall / Diagnostics - time to kill legacy diag_system_pftop.php and replace it with "Sessions", which offers insights into the top sessions on the firewall related to rules and network segments. 2021-07-20 21:53:33 +02:00
Franco Fichtner
72e6ca2fee unbound: more refactoring, less pages, blacklist -> blocklist
PR: https://github.com/opnsense/core/issues/4327
2021-07-20 11:00:00 +02:00
Franco Fichtner
dac9a74479 interfaces: if it looks like SLAAC get rid of it
PR: http://forum.opnsense.org/index.php?topic=8985.0
2021-07-20 09:27:54 +02:00
Franco Fichtner
83cd07eda4 firmware: add version/date header here as well 2021-07-14 09:22:46 +02:00
Franco Fichtner
b1241aeb74 shell: fix IPv4 /31 assignment
PR: https://forum.opnsense.org/index.php?topic=23878.0
2021-07-13 13:56:44 +02:00
Franco Fichtner
11b5fe6e9c interfaces: deprecate SLAAC addresses on linkdown; closes #4929 2021-07-13 12:45:05 +02:00
Franco Fichtner
e8d7876d21 unbound: add 'unbound check' target for advanced options #4327 2021-07-13 12:27:14 +02:00
Franco Fichtner
4a1bc9f8b5 unbound: allow to retain cache; closes #2750 2021-07-13 11:21:01 +02:00
Franco Fichtner
09f510fba2 unbound: fix /var MFS dilemma for DNSBL after boot; closes #4938
o Create a persistent directory to feed the unbound includes
o Move runtime data to /tmp directory

After reboot we use the old persistent list still available.

I am not sure if it's clear enough that the blacklists cannot
auto-update and therefore require a cron job or manual update
from the respective GUI page ("apply" essentially is "download
and apply").
2021-07-09 13:57:07 +02:00
Franco Fichtner
28fa452a47 firmware: fix comment on previous 2021-06-29 14:42:43 +02:00
Franco Fichtner
38578176e4 firmware: comment on compexity avoidance in shell menu
Fix plist while here.
2021-06-29 14:39:14 +02:00
kulikov-a
97643a3edb
read_log.py: set label for obsolete rule in log (#5075) 2021-06-27 13:34:32 +02:00
Franco Fichtner
35aa9bf568 interfaces: back out this change until we have feedback #4929 2021-06-23 11:41:51 +02:00
Ad Schellevis
caed6e2504 Firewall: Diagnostics: States Dump - refactor to MVC.
o a rudimentary column sort
2021-06-22 11:19:39 +02:00
Ad Schellevis
43b7f1eb7f Firewall: Diagnostics: States Dump - refactor to MVC.
o fix minor regression duplicating records (only flush on last content line)
o add link to firewall / rules page
2021-06-21 19:37:27 +02:00
Ad Schellevis
63bdff8cf3 Firewall: Diagnostics: States Dump - refactor to MVC.
o add api endpoint and ui to kill states using filter (kill by host or network as well)
o show ruleids in service control spot to filter states for a specific (auto-generated) rule
o support passing a ruleid to the ui page to limit selection, e.g. /ui/diagnostics/firewall/states#d0953c4424f27d5249027086b4599999
2021-06-21 18:27:21 +02:00