i scraped all media from a bot that posts a random image from a set every hour and found out that from 2187 pictures there were only 99 unique ones

deduplicating decreased the folder containing the pictures from 140 MB to 6.5 MB