Project

General

Profile

Bug #14014

A lot of space is wasted on table nodecompliance and nodeconfigurations due to pretty jsonification

Added by Nicolas CHARLES 9 months ago. Updated 8 months ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
User visibility:
Effort required:
Priority:
0

Description

About 30% of the table size if use by spaces, even when accounting the compression within postgres
Creating the nodecompliance3 table, with the script
insert into nodecompliance3 (nodeid, runtimestamp, endoflife, runanalysis, summary, details) select nodeid, runtimestamp, endoflife, runanalysis, summary, regexp_replace(regexp_replace(regexp_replace(regexp_replace(details, '\s+"', '"', 'ig'), '\s+{', '{', 'ig'), '\s+}', '}', 'ig'), '\s+\]', ']', 'ig') from nodecompliance;

result in following disk usage, after vacuum full of both tables


  oid  | table_schema |      table_name      | row_estimate | total_bytes | index_bytes | toast_bytes | table_bytes |  total  | index  |   toast    | table  
-------+--------------+----------------------+--------------+-------------+-------------+-------------+-------------+---------+--------+------------+--------
 16497 | public       | nodecompliance       |          350 |     1089536 |       98304 |      761856 |      229376 | 1064 kB | 96 kB  | 744 kB     | 224 kB
 25435 | public       | nodecompliance3      |          350 |      794624 |       98304 |      483328 |      212992 | 776 kB  | 96 kB  | 472 kB     | 208 kB

Associated revisions

Revision 30b14f18 (diff)
Added by Nicolas CHARLES 9 months ago

Fixes #14014: A lot of space is wasted on table nodecompliance due to pretty jsonification

History

#1

Updated by Nicolas CHARLES 9 months ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
#2

Updated by Nicolas CHARLES 9 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2100
#3

Updated by Nicolas CHARLES 9 months ago

  • Status changed from Pending technical review to Pending release
#4

Updated by Nicolas CHARLES 9 months ago

  • Subject changed from A lot of space is wasted on table nodecompliance due to pretty jsonification to A lot of space is wasted on table nodecompliance and nodeconfigurations due to pretty jsonification
#5

Updated by Vincent MEMBRÉ 8 months ago

  • Status changed from Pending release to Released
This bug has been fixed in Rudder 4.3.8 and 5.0.4 which were released today.
Changelog
Changelog

Also available in: Atom PDF