Repository navigation
Harden the slow log import and pages, and fix review findings - #60
Open
somethingwithproof wants to merge 45 commits into
Open
somethingwithproof wants to merge 45 commits into
somethingwithproof wants to merge 45 commits into
Conversation
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
… run Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
This was referenced Oct 24, 2026
* refactor: separate Slowlog import stages and bound statistics memory Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: refresh profiling against the latest import fixes Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
somethingwithproof
enabled auto-merge (squash)
October 24, 2026 15:15
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



fix: harden the import, the views and the schema upgrade (2.7)
Description
This PR fixes a set of defects found in a read-through of the whole plugin. It contains security-relevant fixes: a SQL injection in the import path, an unescaped value in the Details filter, and a delete confirmation whose No button deleted the selected logs. The first two need a signed-in user who holds the Slowlog realm; none is reachable without authentication. The details are below so reviewers can judge severity themselves.
The change is 42 small commits, one logical change each, so it can be reviewed and reverted commit by commit. The plugin version goes from 2.6 to 2.7 so the column widening in
includes/database.phpruns once through the existing version-drift upgrade path.Related issue
No open issue covers these findings. The nearest are #14 (security audit) and #11 (prepared statements); this PR does not claim to close either.
Motivation and context
Per finding. "Needs" says who can trigger it.
Query_time,Lock_time,Rows_*,Bytes_sent,Thread_id) were concatenated unquoted into the bulk INSERT. A value such as(SELECT ...)ran as SQL with Cacti's database account.slowlog_log_number()keeps plain decimals and stores 0 for anything else.<option value="...">. Names come from tokenized slow log SQL, so a quoted identifier could break out of the attribute.html_escape()on the value.selected_itemsfield as Yes, so No deleted the selected logs.cactiReturnTobuttons, as in Cacti core.strpos()header matching: a query containingSET timestamp=was read as a header and dropped.Id:suffix ended up in the address and IPv6 addresses were lost.slowlog_parse_user_host()with one pattern.--or#comment hid the rest of the statement from table detection.oquery), falling back toquery.get_table_associations()read every column of every row of a log in one SELECT.logentry, query, oquery.substr()could cut a multibyte character; a Max Query Length of 0 or less stored empty or truncated queries.slowlog_truncate_query(): character boundary, ignores values below 1.max_allowed_packet) still finished as a complete import.SHOW TABLES FROM <db>with an unquoted name failed for names such asmy-db.user/schema20,ip_address15, table names 45,import_text_status40 are shorter than real data. Cacti's session clears strict SQL mode, so the values were silently truncated and distinct users or tables merged (in strict mode the INSERT fails).slowlog_widen_columns(),db_update_table()).plugin_config.versionwas written before the schema upgrade ran, so a failed ALTER was never retried.slowlog_upgrade_tables()succeeds; failure is logged and retried on the next page load.user_auth_realmdirectly (missing group grants) and cached the answer in the session.is_realm_allowed(); only the realm id is cached.rowsandpagewere validated only as integers: a huge page size loaded every row, including both query columns, into memory; a negative value was a SQL error.slowlog_page_bounds(): 1 to 1000 rows (30 when not positive), page at least 1.encodeURIComponent, so&,#or+truncated them.date()in PHP's time zone and written into atimestampcolumn that the database reads in its own session zone, so with PHP and the database in different zones every stored time was shifted (5 hours with PHP in America/New_York and a UTC database). Digit-padded MariaDB# Time:headers also lost the time of day.SET timestamp=line is stored throughFROM_UNIXTIME(); a MySQL 8 ISO# Time:withZor an offset is parsed with its zone; padded headers keep the time of day; out-of-range values are clamped.TimeExpressionTestruns under four PHP time zones.Not in this PR (needs a decision or is larger than a fix):
memory_limitis still set to -1 for everyslowlog.phprequest. The upload form describes that override to the user. With the page size bounded, the view that could use unbounded memory is limited; removing the override means changing that screen and its warnings too.How has this been tested?
tests/Unit,tests/Integrationandtests/Security, using the repo's stubbed-database bootstrap. Run against the pre-change source (main), 48 of them fail, covering each finding above; all pass on this branch.tests/E2Erun against an installed Cacti and a real MariaDB:ImportFlowTest: imports a sample log throughimport_log.php, checks the rows, the tables and stats, the failed-insert path (a trigger stands in for a server side failure), the multibyte length cap, and the 2.6 to 2.7 upgrade of an old schema. The upgrade runs against a scratch copy of the plugin directory, because the upgrade prunestests/from the directory it runs in; the test also checks that the real tree is intact.DeleteConfirmationFlowTest: signs in over HTTP on PHP's built-in server, selects two logs, opens the confirmation page, checks that No is a plain button and nothing is deleted, confirms with Yes, then revokes the realm and checks the page is denied.DriverGuardTest: the driver scripts undertests/E2Edo nothing (exit 404 / print nothing) unless started from the command line withSLOWLOG_E2E=1and a temp directory from the environment. They take no path from a request.SLOWLOG_E2E=1on the Pest step; without it the E2E tests are skipped. A new step before it makescacti/logwritable for the runner user (the sudo install leavescacti.logroot-owned and Cacti's CLI bootstrap refuses to start then); the tests skip with that reason if the log is still not writable, and cutcacti.logback to its earlier length after each driver. The drivers move Cacti's log file to a temp directory while they run, so the deliberate failures do not reach the job's final scan ofcacti.log.tests/bin/patch-coverage.phpagainstmain: 100 percent of the 131 changed measured lines (slowlog.phpis allowlisted there, as before; its changes are covered by source-shape tests and the E2E tests).tests/bin/check-i18n-pot.php mainpasses (one new string added tolocales/po/cacti.pot);tests/bin/validate-manifest.phppasses.plugin_manage.php --plugin=slowlog --install --enablesucceeds and creates the 8 tables; norequiresentry or dependency was added.What I could not run locally: the CI matrix itself (Cacti 1.2.x with PHP 8.2 to 8.4). Local runs used a Cacti 1.3 development tree and PHP 8.4. The E2E drivers rely on the 1.2.x login form and
user_authcolumns being the same as in that tree; if they differ the CI run will show it.Performance
Fixture: a synthetic MariaDB-style slow log of 200,000 entries (75 MB, 30 table names, joins, multi-line queries with comments), imported with
import_logfile()in table mode "all" (inline post-processing), producing 200,000 detail rows and 266,667 table associations. 3 runs per case, alternatingmainand this branch, medians reported (run values in brackets). Environment: PHP 8.4.27 CLI (Xdebug module loaded, no coverage mode) in one Docker container, MariaDB 11.8.9 in another on the same host, a laptop running other containers at the same time, so timings are noisy at the few-percent level.Full import (parse, insert, method and table classification, stats cache):
get_table_associations()alone on the same log:What this shows: peak memory no longer grows with the size of the log (about 7 times lower for the whole import and 30 times lower for table association on this fixture). Wall and CPU time did not change measurably; the import is 4 percent slower in the median, which is inside the spread of the individual runs, so I am not claiming either direction. The cost is 166 more statements (about 10 percent for the import): one SELECT per 2000 rows plus a smaller bulk INSERT at the end of each chunk.
Not measured: database server CPU and time, the Details and chart views (the page bound adds one function call and removes the unbounded case; no timing taken), the tab check (one
is_realm_allowed()call per page instead of a cached session value), the 2.6 to 2.7ALTER TABLEon a largeplugin_slowlog_detailstable, and Xdebug cachegrind profiles.Risk
db_update_table()altersplugin_slowlog_details(user,ip_address,schema) and four smallALTER TABLE ... MODIFY COLUMNstatements widen the other columns. On a very large details table the first alter rebuilds the table; do it in a quiet period. If any step fails, the recorded version stays at 2.6, a warning is written to the Cacti log, and the upgrade is retried on the next load ofplugins.phporslowlog.php.import_log.php --reprocess=Nto refresh an existing log).compatfloor inINFOis unchanged (1.2.29). The new code uses features available in PHP 7.1 and later;mb_strcut()is used only when mbstring is loaded.Rollback
Revert the commits or the merge. Nothing needs undoing in the database: the widened columns hold the same data and are compatible with the 2.6 code. If
plugin_config.versionalready says 2.7 after a rollback, run the plugin's upgrade or set it back to 2.6 in the Plugin Management page; no schema step depends on it.Types of changes
Checklist