Bug #3926
closed
Rule page display time increasing lineary with the number of reports in DB
Description
We have a case where the displaying of rule page time increase enormously with the number of report in DB:
The metrics are as follow:
- number of nodes: ~75
- number of rules: ~45
(clearly not much)
And the timing for rule page:
- reports for ~12h (DB: 2Go) => 6s
- reports for ~24h (DB: 4Go) => 10s
- report for ~48h => 19s
- report for ~3 days => 35s
- report for ~4 days => 1m5s
- after, timeout of the server
Note that even for really few reports, the display time is already 6s!
Moreover, in that case, the database process seems to be CPU bounded, not I/O bounded: the i/o wait is almost 0, but CPU (when the page is rendering) is 100% for the postgres process.
The index are well touch (96%).
Java and Slapd do nothing during that time.
Other pages, like node list, are fast (in order of 1s) to display.
So:
- there seems to be a linear complexity somewhere in the query we are doing for rules.
- the index are well used, but they may be not sufficient
Updated by François ARMAND over 11 years ago
Yesterdays, we tried a lot of things (update to last available linux kernel, update of postgres, removing compression on ZFS, switching from ZFS to ext4, twicking grsec, reindexing, removing a lot of reports...)
We ended with only reports for the last 12h or so, and by putting back the non composite index (see #3674)
Today, Coredumb reported that the rule display time didn't increased even if the number of reports doubled.
So it seems that the solution was to adding back the non-composite index.
We will see in some days how things go.
Updated by François ARMAND over 11 years ago
- Status changed from 8 to Rejected
Coredumb reported that with both indexes (as describe in #3674), the rule page loading time is constant.
So that problem was a sequel of the missing index on node only.