Project

General

Profile

Actions
Copy link

Question #17772

closed

"nodes" table is very big

Added by Laurent Luyckx over 4 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
N/A
Assignee:
-
Category:
Performance and scalability
Target version:
old 6.1 issues to relocate
Regression:
No

Description

Hi,

Question is similar to another posted here https://issues.rudder.io/issues/15436

I'm also seeing a huge nodes table:

           Table            |    Size    | Relation size | External Size 
----------------------------+------------+---------------+---------------
 nodes                      | 44 GB      | 13 GB         | 30 GB
 groups                     | 3123 MB    | 1088 MB       | 2035 MB
 directives                 | 625 MB     | 438 MB        | 187 MB
 eventlog                   | 437 MB     | 281 MB        | 156 MB
 rules                      | 291 MB     | 205 MB        | 86 MB
 reportsexecution           | 106 MB     | 4040 kB       | 102 MB

The number of entries on the "nodes" table is very large:

rudder=# select count(*) from nodes;
   count   
-----------
 133025851
(1 row)

My question is merely the same. Can do clean old entries without breaking anything?
Is there something else I can do?

I'm nearly running out of disk space so a vacuum full will not work.

Thank you for your help

Related issues 1 (0 open — 1 closed)

Related to Rudder - Bug #17778: table nodes contains on entry per node per generation, which is too muchReleasedFrançois ARMANDActions
Actions
Copy link
 #1

Updated by Laurent Luyckx over 4 years ago

  • Target version set to 6.0.7
Actions
Copy link
 #2

Updated by Nicolas CHARLES over 4 years ago

Hi Laurent,

As in related issue, you can run

DELETE FROM nodes WHERE endtime < (now() - interval '2 months');

to remove most data, and then 
vacuum full nodes

impact will be that individual events on nodes won't be able to display node details, but that's probably something you can live with

Actions
Copy link
 #3

Updated by Nicolas CHARLES over 4 years ago

  • Related to Bug #17778: table nodes contains on entry per node per generation, which is too much added
Actions
Copy link
 #4

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.7 to 6.0.8
Actions
Copy link
 #5

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.8 to 6.0.9
Actions
Copy link
 #6

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.9 to 6.0.10
Actions
Copy link
 #7

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.10 to 798
Actions
Copy link
 #8

Updated by Benoît PECCATTE over 3 years ago

  • Target version changed from 798 to 6.1.14
Actions
Copy link
 #9

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.14 to 6.1.15
Actions
Copy link
 #10

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.15 to 6.1.16
Actions
Copy link
 #11

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.16 to 6.1.17
Actions
Copy link
 #12

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.17 to 6.1.18
Actions
Copy link
 #13

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.18 to 6.1.19
Actions
Copy link
 #14

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.19 to 6.1.20
Actions
Copy link
 #15

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.20 to 6.1.21
Actions
Copy link
 #16

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.21 to old 6.1 issues to relocate
Actions
Copy link
 #17

Updated by Alexis Mousset almost 2 years ago

  • Status changed from New to Resolved
  • Regression set to No

closing as a workaround was provided

Actions
Copy link

Also available in: Atom PDF