SMIC: Semantic Multi-Item Compression based on CLIP dictionary

Tom Bachard, Thomas Maugey · 2024

Read SMIC: Semantic Multi-Item Compression based on CLIP dictionary by Tom Bachard, Thomas Maugey (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05035.txt.

Description

Tom Bachard and Thomas Maugey compress whole photo collections by meaning in a 2024 paper. Semantic compression judges loss by whether the meaning survives, not by pixel error. Most such schemes lean on the CLIP model, whose space describes images in words and pictures together.

Collections repeat themselves, and that redundancy is the target. The authors show CLIP's space supports easy semantic adding and subtracting. On that fact they build a shared dictionary codec for many images at once. It beat a leading generative codec on rate, near a hundred-thousandth of a bit per pixel per image, with fidelity kept. The learned dictionary itself reads as a semantic projector of image content.

Catalog record

CREATORTom Bachard, Thomas Maugey
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-05035

Browse the full library · About the Tower of Babel