Project

General

Profile

Actions

Bug #14004

closed

Wasted disk space due to inefficient columns ordering

Added by Nicolas CHARLES almost 6 years ago. Updated almost 6 years ago.

Status:
Rejected
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Fix check:
Regression:

Description

According to https://blog.2ndquadrant.com/on-rocks-and-sand/ proper column ordering in Postgresql can save up to 10-20% of disk space
Given that Rudder uses quite a lot of free space, it would be relevant to benefit from this

Note that Postresql is so wonderful that it advertise which ordering is the best for a table:

rudder=> SELECT a.attname, t.typname, t.typalign, t.typlen
rudder->   FROM pg_class c
rudder->   JOIN pg_attribute a ON (a.attrelid = c.oid)
rudder->   JOIN pg_type t ON (t.oid = a.atttypid)
rudder->  WHERE c.relname = 'nodecompliancelevels'   AND a.attnum >= 0
rudder->  ORDER BY t.typlen DESC;
      attname       |   typname   | typalign | typlen 
--------------------+-------------+----------+--------
 runtimestamp       | timestamptz | d        |      8
 badpolicymode      | int4        | i        |      4
 auditnotapplicable | int4        | i        |      4
 noncompliant       | int4        | i        |      4
 auditerror         | int4        | i        |      4
 pending            | int4        | i        |      4
 success            | int4        | i        |      4
 repaired           | int4        | i        |      4
 error              | int4        | i        |      4
 unexpected         | int4        | i        |      4
 missing            | int4        | i        |      4
 noanswer           | int4        | i        |      4
 notapplicable      | int4        | i        |      4
 reportsdisabled    | int4        | i        |      4
 compliant          | int4        | i        |      4
 ruleid             | text        | i        |     -1
 directiveid        | text        | i        |     -1
 nodeid             | text        | i        |     -1
(18 lignes)

this ticket will only update table creation and migration - we won't change existing tables

Actions #1

Updated by Nicolas CHARLES almost 6 years ago

  • Status changed from New to In progress
Actions #2

Updated by Nicolas CHARLES almost 6 years ago

  • Target version changed from 4.3.8 to 6.0.0~beta1

gain is not that obvious ...

  oid  | table_schema |      table_name       | row_estimate | total_bytes | index_bytes | toast_bytes | table_bytes | total  | index  |   toast    | table  
-------+--------------+-----------------------+--------------+-------------+-------------+-------------+-------------+--------+--------+------------+--------
 16524 | public       | nodecompliancelevels  |         1398 |      589824 |      368640 |        8192 |      212992 | 576 kB | 360 kB | 8192 bytes | 208 kB
 25947 | public       | nodecompliancelevels2 |         1398 |      589824 |      368640 |        8192 |      212992 | 576 kB | 360 kB | 8192 bytes | 208 kB
Actions #3

Updated by Nicolas CHARLES almost 6 years ago

ok, with RudderSysEvents we do gain a bit (2%)

  oid  | table_schema |    table_name    | row_estimate | total_bytes | index_bytes | toast_bytes | table_bytes |  total  |  index  |   toast    |  table  
-------+--------------+------------------+--------------+-------------+-------------+-------------+-------------+---------+---------+------------+---------
 16412 | public       | ruddersysevents  |        17258 |     7921664 |     3801088 |        8192 |     4112384 | 7736 kB | 3712 kB | 8192 bytes | 4016 kB
 26008 | public       | ruddersysevents2 |        17258 |     7790592 |     3768320 |        8192 |     4014080 | 7608 kB | 3680 kB | 8192 bytes | 3920 kB

Actions #4

Updated by Nicolas CHARLES almost 6 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2099
Actions #5

Updated by Nicolas CHARLES almost 6 years ago

  • Status changed from Pending technical review to Rejected
Actions

Also available in: Atom PDF