1301 Commits

Author SHA1 Message Date
Ad Schellevis
f086f8c48d flowd aggregate, minor bug in commit counter, leading to single row commits after row # 100k 2019-05-20 11:42:00 +02:00
Ad Schellevis
28ed5741f4 netflow, make sure get_timeserie_data() returns string type objects in stead of bytes, to prevent 17e4e9c0fc (commitcomment-33587137)
we might find some other small type interpretation differences, but the bumpiest seems to be gone now.
2019-05-18 09:23:25 +02:00
Ad Schellevis
2bdc74b8a1 netflow, switch dump_log.py script to python3 2019-05-18 09:22:29 +02:00
Ad Schellevis
3d1617b2bd netflow, minor type issue, parameter received as bytes in get_timeseries.py 2019-05-17 16:50:32 +02:00
Ad Schellevis
31982e2872 netflow, migrate flowd aggregator to python 3 2019-05-17 16:41:08 +02:00
Ad Schellevis
17e4e9c0fc netflow, aggregator replace flowd with our new implementation 2019-05-16 18:32:55 +02:00
Ad Schellevis
8aecf38f2e netflow, log parser. for backwards compatibility add copy of recv_sec field in recv 2019-05-16 18:29:16 +02:00
Ad Schellevis
5eef7248a8 netflow, flow log parser improvements
- faster / cleaner ipv4 conversion
- ipv6 conversion in compressed format, which equals flowd previous output
- unpack source and dest ports
2019-05-16 16:20:34 +02:00
Ad Schellevis
0bdbdf7d21 netflow, improve log parser performance, cache some calculations.
In theory we could probably increase performance even more by limitting the number of calls to struct.unpack, but this will make it more difficult to read.
This version is about 30% faster then the original C version shipped with flowd.

Some more testing todo, but preliminary results look good.
2019-05-16 13:54:24 +02:00
Ad Schellevis
93f7cff3d8 netflow, flowparser.py performance improvement in parsing ipv4/ipv6 addresses. 2019-05-15 22:01:27 +02:00
Ad Schellevis
a99d1ecf8e netflow, minor type issue in previous 2019-05-14 22:08:52 +02:00
Ad Schellevis
7e55ef5218 netflow, work in progress pure python replacement to parse /var/log/flowd.log files 2019-05-14 22:06:53 +02:00
Ad Schellevis
1dfc0e250a CaptivePortal, convert scripts from python 2 --> 3 2019-05-13 19:29:06 +02:00
Ad Schellevis
2ee18896f2 CaptivePortal, remove unused listARPtable.py script 2019-05-13 18:42:56 +02:00
Ad Schellevis
a06ad6a5d0 netflow, convert parts that don't depend on flowd to python 3 2019-05-13 15:48:33 +02:00
Ad Schellevis
645fdcffd6 proxy, remote access control convert to python 3 2019-05-13 15:30:12 +02:00
Ad Schellevis
df63625418 list ssl ciphers script, python 2 -> 3 conversion 2019-05-13 14:10:01 +02:00
Ad Schellevis
e41aa9a74b status interfaces, list_interrupts.py convert python 2 -> 3 2019-05-13 14:03:18 +02:00
Ad Schellevis
71cdb25d96 Routing/status, python 2-->3 conversion 2019-05-13 13:57:32 +02:00
Ad Schellevis
a4a70fdfbc python 2->3 ipsec scripts 2019-05-13 13:42:35 +02:00
Ad Schellevis
ad7d6df09c unbound:dhcp helper. convert to python3, fix startup read known addresses in the process.
It seems that unbound_known_addresses() wasn't working, since output_stream was ignored. only init with /dev/null when None.
2019-05-13 13:27:23 +02:00
Ad Schellevis
d756b13f52 python 2->3 scripts/dhcp/get_leases.py 2019-05-13 13:17:48 +02:00
Ad Schellevis
11d64579ce python3, use symlink in already converted scripts 2019-05-13 13:00:49 +02:00
Ad Schellevis
0af2c26c80 python3, use symlink (rename python3.6 to python3) for the files already in stable/19.1 2019-05-13 12:33:01 +02:00
Ad Schellevis
404f796eb8 python2->3 scripts in src/opnsense/scripts/interfaces 2019-05-13 11:37:01 +02:00
Ad Schellevis
df4fe1c683 IDS, missing cleanup after rule installation, for https://github.com/opnsense/core/issues/3472 2019-05-08 11:47:54 +02:00
Franco Fichtner
32ff471bf5 firmware: package manager not responding fixes; closes #3172 2019-05-03 17:09:18 +02:00
Franco Fichtner
6018681623 dhcpd: more conversion, only services_dhcpd_configure() left 2019-04-29 08:05:15 +02:00
Franco Fichtner
ba0c82dd54 dhcpd: naming convention, plugins_configure() use 2019-04-29 07:49:26 +02:00
Franco Fichtner
04d4f50cca interfaces: separate dhcpd from services.inc 2019-04-28 15:07:10 +02:00
Franco Fichtner
8d0072e775 system: only use rrd where needed 2019-04-23 17:18:53 +02:00
Franco Fichtner
168be32071 system: provided by legacy bindings are facilities...
... which are bound to MVC code.  Move remaining "pure" gateway
functions to the legacy_bindings and use plugins_run() stub to
get to the dpinger data...  The data could later be coalesced
or split up, but for now that seems reasonable enough (provided
it works ok and @adschellevis more or less likes this direction).
2019-04-21 19:27:45 +02:00
Franco Fichtner
f3b5c0e8f7 system: hide dpinger configuration inside new plugins_configure() call
Some back and forth between explicit and implicit requires while here.
The code is helplessly glued together and no plugin facility to get
data from a function call currently exists.
2019-04-21 19:04:15 +02:00
Ad Schellevis
2940f6adc8 Filter, live-log. python 3 conversion, encoding issue. not 100% this is the fix, but lets try to open the file in utf-8 first. for https://github.com/opnsense/core/issues/3434 2019-04-21 18:02:31 +02:00
Ad Schellevis
3dc5f71503 Filter, alias. calculate uniqueid() using all of its contents +minor cleanup (this alias is always in _known_aliases). closes https://github.com/opnsense/core/issues/3432 2019-04-21 12:00:17 +02:00
Ad Schellevis
e784027b73 filter, aliases. add alias cleanup. previously a delete of an alias would result in the old one still being there. closes https://github.com/opnsense/core/issues/3431
Since all aliases are written to /usr/local/etc/filter_tables.conf, it should be safe to drop all for which a file in aliastables exists but no entry is available in filter_tables.conf anymore.
For debug purposes we log the entries which we delete, although this probably wouldn't be absolutely necessary, it might help to debug potential issues.
2019-04-19 14:32:03 +02:00
Ad Schellevis
a193e1e397 Captive portal, in case someone execs listClients before service start, make sure the directory exists. for https://github.com/opnsense/core/issues/3425 2019-04-17 08:45:42 +02:00
Franco Fichtner
9f54891523 dpinger: reshuffle some related code 2019-04-17 07:06:26 +02:00
Franco Fichtner
494ba0b7b5 system: move gwlb.inc to dpinger.inc 2019-04-16 19:55:17 +02:00
Ad Schellevis
b5adb0c312 Routing, gateways. move get_gateway_error() from gwlb.inc to gateway_alert script, since only used there now. for https://github.com/opnsense/core/issues/2279 2019-04-16 14:19:42 +02:00
Ad Schellevis
507523fe70 Routing, gateways. ditch return_gateways_array() in route selection. for https://github.com/opnsense/core/issues/2279 2019-04-11 13:32:32 +02:00
Franco Fichtner
608a563710 firewall: missing newline 2019-04-10 07:49:13 +02:00
Franco Fichtner
a0f234f761 src: style and whitespace sweep 2019-04-09 07:48:04 +02:00
Ad Schellevis
03f0fa2826 Filter, python scripts conversion 2 --> 3 2019-04-05 16:48:23 +02:00
Ad Schellevis
a71d32808d IDS, convert python scripts from 2 to 3 2019-04-05 14:56:02 +02:00
Ad Schellevis
6de00487f2 IDS, skip flowbits:noalert rules in "filter_frop", for https://github.com/opnsense/core/issues/3386 2019-04-05 10:53:25 +02:00
Ad Schellevis
bccf7f3065 bootstrap 3.4.1 for captive portal template, closes https://github.com/opnsense/core/issues/3377 2019-04-02 10:13:15 +02:00
Franco Fichtner
be9a1c94ab interfaces: small tweaks in PPP link scripts for #2267 2019-03-26 21:16:54 +01:00
Franco Fichtner
0e95b0f407 interfaces: remove rc.kill_states
There's no real world evidence this works or is needed today.
We do have to other state kills / flushes now, one works and
the other may be defunct.  To progress this remove this arcane
script and if the need arises rewrite it as something portable.

See:      https://redmine.pfsense.org/issues/1629
Also see: https://redmine.pfsense.org/issues/2887
2019-03-21 18:33:18 +01:00
Franco Fichtner
7b937e878d firmware: we use grep, catch both versions of the error 2019-03-17 10:25:09 +01:00