Harden alert commands, upgrades and dashboards, and fix review findings

.github/workflows/plugin-ci-workflow.yml

@@ -220,6 +220,8 @@ jobs:
         php tests/regression/threshold_recovery_database_test.php
         php tests/regression/quality_must_fix_database_test.php
         php tests/regression/incoming_preservation_database_test.php
+        php tests/regression/review_fixes_database_test.php
+        php tests/regression/review_fixes_paths_database_test.php
       working-directory: ${{ github.workspace }}/cacti/plugins/syslog
       env:
         SYSLOG_TEST_MYSQL_DSN: 'mysql:host=127.0.0.1;port=3306'
@@ -311,6 +313,25 @@ jobs:
           cat log/cacti.log
           exit 1
         fi
+        # The populated alert rule (id 1) must have been evaluated and have fired
+        # for the host "bob": the processor logs it and stores it in syslog_logs.
+        if ! grep -q "The Syslog Alert 'interface down alert' with Severity 'Warning', has been Triggered on Host 'bob'" log/cacti.log; then
+          echo "The populated alert rule did not fire"
+          cat log/cacti.log
+          exit 1
+        fi
+        alerts=$(mysql --defaults-file="$HOME/.my.cnf" cacti -N -e "SELECT COUNT(*) FROM syslog_logs WHERE alert_id = 1 AND host = 'bob'")
+        if [ "$alerts" -lt 1 ]; then
+          echo "The populated alert rule left no alert log row"
+          exit 1
+        fi
+
+    # Runs the real processor over queued messages and reads the results back.
+    # The script removes its rows and restores cacti.log to its starting size.
+    - name: Run Syslog live pipeline checks
+      run: |
+        cd ${{ github.workspace }}/cacti
+        sudo -u www-data env SYSLOG_TEST_LIVE=1 php plugins/syslog/tests/regression/live_pipeline_test.php
 
     - name: View Cacti Logs
       if: always()
@@ -324,14 +345,17 @@ jobs:
       if: always()
       run: |
         if [ -f ${{ github.workspace }}/cacti/log/cacti.log ]; then
-          if sudo grep -E "FATAL:|ERROR:|WARNING:" "${{ github.workspace }}/cacti/log/cacti.log"; then
+          status=0
+          matches=$(sudo grep -E "FATAL:|ERROR:|WARNING:" "${{ github.workspace }}/cacti/log/cacti.log") || status=$?
+          if [ "$status" -gt 1 ]; then
+            echo "Unable to inspect Cacti log"
+            exit "$status"
+          fi
+          # The only entry allowed is the alert the populated rule is meant to raise.
+          unexpected=$(printf '%s\n' "$matches" | grep -v -F "The Syslog Alert 'interface down alert' with Severity 'Warning', has been Triggered on Host 'bob'" | grep -v '^$') || true
+          if [ -n "$unexpected" ]; then
+            printf '%s\n' "$unexpected"
             echo "Found FATAL/ERROR/WARNING entries in Cacti log"
             exit 1
-          else
-            status=$?
-            if [ "$status" -ne 1 ]; then
-              echo "Unable to inspect Cacti log"
-              exit "$status"
-            fi
           fi
         fi

CHANGELOG.md

@@ -2,6 +2,50 @@
 
 --- develop ---
 
+* test: Extend the browser E2E run with import, reprocess, dashboard and long owner name paths, and make its login work on Cacti 1.2.x and develop
+
+* test: Add an optional Xdebug dump and merge report that list which changed lines the real entry points executed
+
+* test: Run the real processor over alert command, removal rule and invalid alert rule cases against an installed Cacti in CI
+
+* test: Add real-database checks for each upgrade choice, every dashboard record type and dimension, every removal rule type and method, alert evaluation with invalid rules, owner column limits, hostname lookup budget and rule imports
+
+* bug: Reject imported alert rules whose match type produces no query
+
+* test: Extend the browser E2E run with dashboard, archiving removal rule, alert rule isolation, import validation and single-rule reprocess checks
+
+* test: Load functions extracted from plugin pages through temporary files instead of evaluating source strings
+
+* test: Add database regression checks for the dashboard queries, archiving removal rules, rule reprocessing, alert rule isolation and upgrade truncate scope
+
+* bug: Validate imported saved search templates like edited ones
+
+* bug: Stop hostname lookups after a time budget per processing run so senders cannot stall the poller
+
+* bug: Widen rule owner columns to the 50 character Cacti username limit
+
+* bug: Reprocess only the selected removal rule instead of every enabled rule
+
+* bug: Archive host and program removal rules without an ambiguous column error
+
+* bug: Group the All records dashboard queries per log table so panels show every host, program and time bucket
+
+* bug: Validate imported alert rules like edited ones and ignore row ids in import files
+
+* bug: Apply the Max Report Records setting to threshold alert messages
+
+* bug: Evaluate the remaining alert rules when one rule fails to compile instead of silencing all of them
+
+* bug: Keep asterisks inside a host-level alert rule expression intact when the grouped query is built
+
+* bug: Keep alert, removal and report rules when the upgrade type Truncate is chosen; it now discards stored messages only
+
+* security: Substitute alert command tokens in a single pass so values containing a token name are not expanded again
+
+* test: Insert a valid alert rule in the CI test data and assert in the workflow that the processor evaluates and fires it
+
+* bug: Check sender names with A and AAAA records, accept IPv4 and IPv6 addresses without a lookup and store IPv6 addresses in one canonical form
+
 * issue#317: Expire partitions by their actual UTC upper boundaries instead of partition count, preserving cutoff-spanning partitions and supporting legacy TO_DAYS layouts.
 
 * issue#317: Defer automatic partition reorganization when overflow estimates exceed 100,000 rows or 128 MiB, or cannot be read; preserve the safety partition and report the reason through maintenance health.

README.md

@@ -355,6 +355,13 @@ If you are upgrading to version 2.5 from an earlier version, make sure that you
 update this template format and restart rsyslog.  You may loose some syslog
 data, but doing this in a timely fashion, will minimize data loss.
 
+Senders that only have an IPv6 address are supported.  With hostname resolution
+enabled the plugin accepts an IPv4 or IPv6 address as it is, and stores an IPv6
+address in its compressed lower case form (for example `2001:db8::1`) so one
+address is kept under one device name.  Names are looked up with A and AAAA
+records.  The `host` columns hold 64 characters, which is enough for any IPv6
+address.
+
 Ensure you restart rsyslog after these changes are completed.  Other logging
 servers such as Syslog-NG are also supported with this plugin.  Please see some
 additional documentation here: [Cacti Documentation