Franco Fichtner
a419b9e32b
src: style sweep
2022-12-19 09:27:51 +01:00
Stephan de Wit
a4ae646327
unbound / overview: migrate to duckdb ( #6182 )
...
* unbound / overview: migrate to duckdb
* unbound: make ajax calls concurrent, no need to wait on each other
* unbound / overview: adjust to DataFrame optimization
While previous insert statements with SQLite had to be rate limited to prevent blocking on the
side of the logger, this limitation can be ditched entirely with the approach of appending DataFrames.
Also, the client chart expects timestamps for the entirety of the dataset, so make use of
the NaN value.
* unbound / overview: connection can be none
* unbound / overview: include DNSSEC and timing information and refactor logger
The logger was lacking some error handling due to it being daemonized. On a restart the process
will still produce a core dump likely associated with daemonization, but this will be addressed in a separate issue
To accomodate future additions, DNSSEC and RTT information is now also included in the data set.
* remove leftover syslog message
* unbound / overview: add UUID as well
* unbound / overview: account for null return_msg objects
* unbound / overview: Move DNS statistics to reporting->dns, add detailed grid
This also introduces the option to clear the DNS data. Also restructure the data to a more sensible flow representation.
2022-12-15 10:45:16 +01:00
Ad Schellevis
868c9531cd
system: remove last bits of clog (circular logging) support, closes https://github.com/opnsense/core/issues/5892
2022-07-19 20:13:00 +02:00
Gareth Owen
3ca57fae8b
DHCP: Update DNS with hostname only static entries ( #4689 )
...
DHCP static mappings that have a hostname configured but no IP address,
now update the Outbound DNS entry with the IP assigned from the pool
2021-08-12 16:52:45 +02:00
Ad Schellevis
94a8906ef9
Generic log handling: ignore garbled data in log lines, closes https://github.com/opnsense/core/issues/4830
2021-03-15 19:45:31 +01:00
Ad Schellevis
8cf9c4fe08
fix previous, related to https://github.com/opnsense/core/issues/3831
2019-11-28 16:10:42 +01:00
Ad Schellevis
0844a5a27b
log, minor bug in reverse log reader, when reading clog files, the beginning of the file isn't detected. found while working on https://github.com/opnsense/core/issues/3831
2019-11-28 15:56:57 +01:00
Ad Schellevis
5c6d780f8c
logging, throw an exception when logfile doesn't look like a clog file, makes it easier to auto-detect clog/plain logs
2019-11-26 21:44:46 +01:00
Ad Schellevis
4531ceea4d
Flowd aggregator: python 3 exception doesn't have a "message" property, cast error to string and check for "malformed"
...
ref : https://forum.opnsense.org/index.php?topic=13711.0
2019-08-16 12:12:51 +02:00
Ad Schellevis
459da410eb
update copyright in previous while here.
2019-06-02 10:51:50 +02:00
Ad Schellevis
40bd0c5dc8
dhcpd leases watcher, file rotation issue. closes https://github.com/opnsense/core/issues/3478
2019-06-02 10:43:37 +02:00
Ad Schellevis
64186cca48
python3, minor compatibility fix in sqlite3_helper.py
2019-05-13 18:43:27 +02:00
Ad Schellevis
74f0022626
python, log_helper.py python 3 compatibility (supports 2 and 3 now)
2019-04-05 12:09:53 +02:00
Ad Schellevis
8f060d4a7a
Netflow/insight and the endless UFS instability... in some cases sqlite can't figure out database inconsistencies by itself. In these cases it is practical to force a repair (rebuild) of the underlaying files manually.
...
This commit adds an option in reporting_settings.php to request a dump/import from https://github.com/opnsense/core/blob/18.1.9/src/opnsense/site-python/sqlite3_helper.py#L69
2018-06-15 10:31:00 +02:00
Ad Schellevis
4cbc8e1dd1
python, improve performance of reverse_log_reader() by avoiding string copies while parsing. Tried the new method using a 50MB suricata log file, which was parsed about 30% quicker then using the previous version.
2018-05-20 18:45:13 +02:00
Franco Fichtner
342b898218
src: a bit of python code style
2018-04-06 05:37:31 +00:00
Franco Fichtner
345bbec96b
src: a bit of python code style
2018-04-03 06:53:55 +00:00
Ad Schellevis
407e4f1ded
add clog reader to log_helper module
2017-10-16 21:48:25 +02:00
Ad Schellevis
f2599d3ae6
support stream input in reverse_log_reader()
2017-10-16 21:42:25 +02:00
Ad Schellevis
d567d90012
IPS, cleanup. move libs to shared location
2017-10-16 16:12:59 +02:00
Ad Schellevis
9666ba5b48
better sqlite3 repair and corruption detection for network insight
2017-08-11 19:18:35 +02:00
Franco Fichtner
0c2fefab03
src: transform Ad's copyrights, add e-mail everywhere
2017-08-06 11:19:35 +02:00
Ad Schellevis
1474f77ddc
dhcpd watcher (unbound) cleanup mac address, remove ;
2017-04-04 20:15:20 +02:00
Ad Schellevis
e7137c1ec3
dhcpd.leases using wrong timezone, closes https://github.com/opnsense/core/issues/1521
2017-04-04 19:45:42 +02:00
Ad Schellevis
151e633d09
(dhcpd/unbound) don't try to handle empty section, closes https://github.com/opnsense/core/issues/1320
2016-12-31 16:25:50 +01:00
Ad Schellevis
b0aeb04b3f
fix typo
2016-08-07 18:51:17 +02:00
Ad Schellevis
5ec2101ac2
(insight) check database integrity before start, repair if broken
2016-08-07 16:10:19 +02:00
Franco Fichtner
a8006f272e
src: align a few headers and sweep whitespace
2016-03-07 07:30:02 +01:00
Franco Fichtner
2a1afe9ec3
site-python: move to private opnsense dir
...
Discussed with: @adschellevis
2016-02-29 20:34:28 +01:00
