Architecture #9582
openQuery to get config of last run can be pretty expensive when there are a lot of new nodes runs
Description
The query to get last nodes runs configuration is ok, but when we have a significant number of nodes to filter on, it gets pretty slow (from 50ms to 2s with 900 nodes)
There is a threashold where it is cheaper to simply take all nodes, and filter in scala (as taking all nodes is more about 1s on my test system)
(note: this is also true in 3.1 implementation to get expected reports)
This needs more testing, obviously, but it could improve dashboard loading at web interface start, or when all nodes are answering at the same time
Updated by Nicolas CHARLES about 8 years ago
limit seems to be at about 400 nodes in the query out of 900 total node (a bit less than 50%) with a very large DB (8 days worth of reports and config)
with smaller (16 GB) dataset, difference seems even bigger
more test should be done with more and less nodes than 900, but it seems we can assume than more than 500 nodes in the query=> postfilter the whole list of nodes
Updated by Alexis Mousset about 8 years ago
- Target version changed from 321 to 4.0.1
Updated by Vincent MEMBRÉ about 8 years ago
- Target version changed from 4.0.1 to 4.0.2
Updated by Vincent MEMBRÉ about 8 years ago
- Target version changed from 4.0.2 to 4.0.3
Updated by Vincent MEMBRÉ almost 8 years ago
- Target version changed from 4.0.3 to 4.0.4
Updated by Benoît PECCATTE over 7 years ago
- Severity set to Minor - inconvenience | misleading | easy workaround
- User visibility set to Operational - other Techniques | Technique editor | Rudder settings
- Priority set to 16
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.0.4 to 4.0.5
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.0.5 to 4.0.6
- Priority changed from 16 to 15
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.0.6 to 4.0.7
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.0.7 to 357
Updated by Alexis Mousset over 7 years ago
- Target version changed from 357 to 4.1.6
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.1.6 to 4.1.7
- Priority changed from 29 to 28
Updated by Vincent MEMBRÉ over 7 years ago
- Target version changed from 4.1.7 to 4.1.8
Updated by Vincent MEMBRÉ about 7 years ago
- Target version changed from 4.1.8 to 4.1.9
- Priority changed from 28 to 27
Updated by Vincent MEMBRÉ about 7 years ago
- Target version changed from 4.1.9 to 4.1.10
Updated by Vincent MEMBRÉ almost 7 years ago
- Target version changed from 4.1.10 to 4.1.11
Updated by Vincent MEMBRÉ over 6 years ago
- Target version changed from 4.1.11 to 4.1.12
Updated by Vincent MEMBRÉ over 6 years ago
- Target version changed from 4.1.12 to 4.1.13
Updated by Vincent MEMBRÉ over 6 years ago
- Target version changed from 4.1.13 to 4.1.14
Updated by Benoît PECCATTE over 6 years ago
- Target version changed from 4.1.14 to 4.1.15
Updated by Vincent MEMBRÉ about 6 years ago
- Target version changed from 4.1.15 to 4.1.16
Updated by Vincent MEMBRÉ about 6 years ago
- Target version changed from 4.1.16 to 4.1.17
Updated by Vincent MEMBRÉ about 6 years ago
- Target version changed from 4.1.17 to 4.1.18
- Priority changed from 27 to 0
Updated by Vincent MEMBRÉ almost 6 years ago
- Target version changed from 4.1.18 to 4.1.19
Updated by Alexis Mousset almost 6 years ago
- Target version changed from 4.1.19 to 4.1.20
Updated by François ARMAND almost 6 years ago
- Target version changed from 4.1.20 to 4.1.21
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 4.1.21 to 4.1.22
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 4.1.22 to 4.1.23
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 4.1.23 to 4.1.24
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 4.1.24 to 588
Updated by François ARMAND over 5 years ago
- Category changed from Web - Compliance & node report to Performance and scalability
- Target version changed from 588 to 5.0.13
- User visibility changed from Operational - other Techniques | Technique editor | Rudder settings to Operational - other Techniques | Rudder settings | Plugins
- Priority changed from 0 to 27
@ncharles: it would be interesting to benchmark that on the load testing env.
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 5.0.13 to 5.0.14
Updated by Vincent MEMBRÉ about 5 years ago
- Target version changed from 5.0.14 to 5.0.15
Updated by Vincent MEMBRÉ about 5 years ago
- Target version changed from 5.0.15 to 5.0.16
Updated by Alexis Mousset almost 5 years ago
- Target version changed from 5.0.16 to 5.0.17
- Priority changed from 27 to 28
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 5.0.17 to 5.0.18
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 5.0.18 to 5.0.19
- Priority changed from 28 to 29
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 5.0.19 to 5.0.20
Updated by Vincent MEMBRÉ about 4 years ago
- Target version changed from 5.0.20 to 797
- Priority changed from 29 to 30
Updated by Benoît PECCATTE over 3 years ago
- Target version changed from 797 to 6.1.14
- Priority changed from 30 to 32
Updated by Vincent MEMBRÉ over 3 years ago
- Target version changed from 6.1.14 to 6.1.15
Updated by Vincent MEMBRÉ over 3 years ago
- Target version changed from 6.1.15 to 6.1.16
Updated by Vincent MEMBRÉ over 3 years ago
- Target version changed from 6.1.16 to 6.1.17
Updated by Vincent MEMBRÉ about 3 years ago
- Target version changed from 6.1.17 to 6.1.18
Updated by Vincent MEMBRÉ about 3 years ago
- Target version changed from 6.1.18 to 6.1.19
Updated by François ARMAND almost 3 years ago
- Tracker changed from Bug to Architecture
- Severity deleted (
Minor - inconvenience | misleading | easy workaround) - User visibility deleted (
Operational - other Techniques | Rudder settings | Plugins) - Priority deleted (
32)
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.19 to 6.1.20
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.20 to 6.1.21
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.21 to old 6.1 issues to relocate
Updated by Alexis Mousset almost 2 years ago
- Target version changed from old 6.1 issues to relocate to 8.0.0~alpha1
Updated by Vincent MEMBRÉ over 1 year ago
- Target version changed from 8.0.0~alpha1 to 8.0.0~beta1
Updated by Alexis Mousset over 1 year ago
- Target version changed from 8.0.0~beta1 to 8.1.0~alpha1
Updated by Vincent MEMBRÉ 11 months ago
- Target version changed from 8.1.0~alpha1 to 8.1.0~beta1
Updated by Vincent MEMBRÉ 10 months ago
- Target version changed from 8.1.0~beta1 to 8.1.0~beta2
Updated by Vincent MEMBRÉ 9 months ago
- Target version changed from 8.1.0~beta2 to 8.1.0~rc1
Updated by Vincent MEMBRÉ 9 months ago
- Target version changed from 8.1.0~rc1 to 8.1.0
Updated by Vincent MEMBRÉ 9 months ago
- Target version changed from 8.1.0 to 8.1.1
Updated by Vincent MEMBRÉ 8 months ago
- Target version changed from 8.1.1 to 8.1.2
Updated by Vincent MEMBRÉ 8 months ago
- Target version changed from 8.1.2 to 8.1.3
Updated by Vincent MEMBRÉ 7 months ago
- Target version changed from 8.1.3 to 8.1.4
Updated by Vincent MEMBRÉ 6 months ago
- Target version changed from 8.1.4 to 8.1.5
Updated by Vincent MEMBRÉ 6 months ago
- Target version changed from 8.1.5 to 8.1.6
Updated by Vincent MEMBRÉ 5 months ago
- Target version changed from 8.1.6 to 8.1.7
Updated by Vincent MEMBRÉ 3 months ago
- Target version changed from 8.1.7 to 8.1.8
Updated by Vincent MEMBRÉ about 1 month ago
- Target version changed from 8.1.8 to 8.1.9
Updated by Vincent MEMBRÉ 25 days ago
- Target version changed from 8.1.9 to 8.1.10
Updated by Vincent MEMBRÉ 20 days ago
- Target version changed from 8.1.10 to 8.1.11